TWI847276B - Encoding/decoding method, apparatus, device, storage medium, and computer program product - Google Patents

Abstract

Embodiments of the present application disclose an encoding/decoding method, an apparatus, a device, a storage medium, and a computer program product, which belongs to the field of audio processing technologies. In the method, an HOA signal of an audio frame is encoded/decoded by combining a coding/decoding scheme selected based on a virtual loudspeaker and a coding/decoding scheme based on directional audio coding, that is, an appropriate coding/decoding scheme is selected for different audio frames, so that a compression rate of an audio signal can be improved. In addition, in order to achieve a smooth transition of auditory quality when switching between different encoding/decoding schemes, for some audio frames, a new encoding and decoding scheme is used to encode/decode these audio frames instead of directly using either of the foregoing two encoding/decoding schemes. That is, signals of a specified channel in the HOA signals of these audio frames are encoded into a bit stream, that is, encoding/decoding is performed by using a compromise solution, so that hearing quality after the HOA signals recovered by decoding are rendered and played can be smoothly transitioned.

Description

Coding and decoding method, device, equipment, storage medium and computer program product

The present application embodiment relates to the field of audio processing technology, and more particularly to a coding and decoding method, device, equipment, storage medium and computer program product.

As a three-dimensional audio technology, higher order ambisonics (HOA) technology has received widespread attention because of its higher flexibility in three-dimensional audio playback. In order to achieve better auditory effects, HOA technology requires a large amount of data to record detailed sound scene information. However, as the HOA order increases, more data will be generated, and a large amount of data will cause difficulties in transmission and storage. Therefore, how to encode and decode HOA signals has become a key issue of concern at present.

Related technologies propose two schemes for encoding and decoding HOA signals. One of the schemes is a coding and decoding scheme based on directional audio coding (DirAC). In this scheme, the encoder extracts the core layer signal and spatial parameters from the HOA signal of the current frame and encodes the extracted core layer signal and spatial parameters into the bitstream. The decoder uses a decoding method symmetrical to the encoding to reconstruct the HOA signal of the current frame from the bitstream. The other scheme is a coding and decoding scheme based on virtual speaker selection. In this scheme, the encoder selects a target virtual speaker that matches the HOA signal of the current frame from a set of virtual speakers based on the match-projection (MP) algorithm, determines the virtual speaker signal based on the HOA signal of the current frame and the target virtual speaker, determines the residual signal based on the HOA signal of the current frame and the virtual speaker signal, and encodes the virtual speaker signal and the residual signal into the bitstream. The decoder uses a decoding method symmetric to the encoding to reconstruct the HOA signal of the current frame from the bitstream.

However, when there are fewer heterogeneous sound sources in the sound field, the compression rate of the codec scheme selected based on the virtual speaker is higher. When there are more heterogeneous sound sources in the sound field, the compression rate of the codec scheme based on DirAC is higher. Among them, heterogeneous sound sources refer to point sound sources with different positions and/or directions. The sound field types of different audio frames (related to the heterogeneous sound sources in the sound field) may be different. If you want to satisfy the high compression rate of audio frames under different sound field types at the same time, you need to select a suitable codec scheme for the corresponding audio frame according to the sound field type of each audio frame, so you need to switch between different codec schemes. However, the HOA signals reconstructed based on different coding and decoding schemes have different auditory qualities after rendering and replay. When switching between different coding and decoding schemes, how to ensure a smooth transition of auditory quality is a problem that needs to be considered at present.

The present application embodiment provides a coding method, device, apparatus, storage medium and computer program product, which can ensure smooth transition of auditory quality when switching between different coding schemes. The technical solution is as follows:

In a first aspect, a coding method is provided, the method comprising: determining a coding scheme of a current frame according to an HOA signal of the current frame, the coding scheme of the current frame being one of a first coding scheme, a second coding scheme and a third coding scheme; wherein the first coding scheme is an HOA coding scheme based on directional audio coding (i.e., a DirAC decoding scheme), the second coding scheme is an HOA coding scheme based on virtual speaker selection (which can be simply referred to as an MP-based HOA decoding scheme), and the third coding scheme is a hybrid coding scheme; if the coding scheme of the current frame is the third coding scheme, a signal of a specified channel in the HOA signal is encoded into a bitstream, the specified channel being some of all channels of the HOA signal. Among them, the hybrid coding scheme uses both technical means related to the first coding scheme (i.e., the DirAC coding scheme) and technical means related to the second coding scheme (the MP-based HOA coding scheme) during the coding process, so it is called a hybrid coding scheme.

In the embodiment of the present application, a suitable coding scheme is selected for different audio frames, which can improve the compression rate of the audio signal. At the same time, for some audio frames, instead of directly adopting any of the first coding scheme and the second coding scheme, a new coding scheme is adopted to encode and decode these audio frames, that is, the signals of the specified channels in the HOA signals of these audio frames are encoded into the bitstream, that is, a compromise scheme is adopted for coding and decoding, so that the auditory quality of the HOA signal after rendering and playback after decoding and recovery can be smoothly transitioned.

Optionally, the signal of the designated channel includes a first-order ambisonics (FOA) signal, and the FOA signal includes an omnidirectional W signal, and directional X signal, Y signal, and Z signal.

Optionally, encoding the signal of a designated channel in the HOA signal into a bitstream includes: determining a virtual speaker signal and a residual signal based on the W signal, the X signal, the Y signal, and the Z signal; and encoding the virtual speaker signal and the residual signal into the bitstream.

Optionally, determining the virtual speaker signal and the residual signal based on the W signal, the X signal, the Y signal, and the Z signal includes: determining the W signal as one virtual speaker signal; determining three residual signals based on the W signal, the X signal, the Y signal, and the Z signal, or determining the X signal, the Y signal, and the Z signal as three residual signals. Optionally, determining the difference signals between the X signal, the Y signal, and the Z signal and the W signal as three residual signals.

Optionally, encoding the virtual speaker signal and the residual signal into the bitstream includes: combining the virtual speaker signal with the first preset mono signal to obtain a stereo signal; combining the three residual signals with the second preset mono signal to obtain two stereo signals; and encoding the obtained three stereo signals into the bitstream respectively through a stereo encoder.

Optionally, the three channels of residual signals are combined with the second preset mono signal to obtain two channels of stereo sound signals, including: combining two channels of residual signals with the highest correlation among the three channels of residual signals to obtain one channel of the two channels of stereo sound signals; and combining a channel of residual signals other than the two channels of residual signals with the highest correlation among the three channels of residual signals with the second preset mono signal to obtain another channel of the two channels of stereo sound signals.

Optionally, the first preset mono signal is an all-zero signal or an all-one signal, the all-zero signal includes a signal whose sampling point values are all zero or a signal whose frequency values are all zero, and the all-one signal includes a signal whose sampling point values are all one or a signal whose frequency values are all one; the second preset mono signal is an all-zero signal or an all-one signal; the first preset mono signal is the same as or different from the second preset mono signal.

Optionally, encoding the virtual speaker signal and the residual signal into the bitstream includes: encoding the virtual speaker signal and each of the three residual signals into the bitstream respectively through a mono encoder.

Optionally, after determining the coding scheme of the current frame according to the HOA signal of the current frame, it further includes: if the coding scheme of the current frame is the first coding scheme, encoding the HOA signal into the bitstream according to the first coding scheme; if the coding scheme of the current frame is the second coding scheme, encoding the HOA signal into the bitstream according to the second coding scheme.

Optionally, determining the coding scheme of the current frame according to the high-order stereo reverberation HOA signal of the current frame includes: determining an initial coding scheme of the current frame according to the HOA signal, the initial coding scheme being the first coding scheme or the second coding scheme; if the initial coding scheme of the current frame is the same as the initial coding scheme of the previous frame of the current frame, determining the coding scheme of the current frame to be the initial coding scheme of the current frame; if the initial coding scheme of the current frame is the first coding scheme and the initial coding scheme of the previous frame of the current frame is the second coding scheme, or the initial coding scheme of the current frame is the second coding scheme and the initial coding scheme of the previous frame of the current frame is the first coding scheme, determining the coding scheme of the current frame to be the third coding scheme.

Optionally, after determining the initial coding scheme of the current frame according to the HOA signal, it further includes: encoding indication information of the initial coding scheme of the current frame into the bitstream.

Optionally, after determining the coding scheme of the current frame according to the high-order stereo reverberation HOA signal of the current frame, the method further includes: determining a value of a switching flag of the current frame, when the coding scheme of the current frame is the first coding scheme or the second coding scheme, the value of the switching flag of the current frame is the first value; when the coding scheme of the current frame is the third coding scheme, the value of the switching flag of the current frame is the second value; and encoding the value of the switching flag into the bitstream. That is, the switching flag is used to indicate whether the current frame is a switching frame.

Optionally, after determining the coding scheme of the current frame according to the HOA signal of the current frame, it further includes: encoding indication information of the coding scheme of the current frame into the bitstream.

Optionally, the designated channel is consistent with a transmission channel preset in the first coding scheme, so as to ensure that the auditory quality of the switching frame is similar to the auditory quality of the audio frame encoded by the first coding scheme.

In a second aspect, a decoding method is provided, the method comprising: obtaining a decoding scheme of a current frame based on a bit stream, the decoding scheme of the current frame being one of a first decoding scheme, a second decoding scheme and a third decoding scheme; wherein the first decoding scheme is a high-order stereo reverberation HOA decoding scheme based on directional audio decoding, the second decoding scheme is an HOA decoding scheme based on virtual speaker selection, and the third decoding scheme is a hybrid decoding scheme; if the decoding scheme of the current frame is the third decoding scheme, determining a signal of a specified channel in the HOA signal of the current frame based on the bit stream, the specified channel being a portion of all channels of the HOA signal; determining the HOA based on the signal of the specified channel The gain of one or more residual channels other than the designated channel in the signal; based on the signal of the designated channel and the gain of the one or more residual channels, the signal of each residual channel in the one or more residual channels is determined; based on the signal of the designated channel and the signal of the one or more residual channels, the reconstructed HOA signal of the current frame is obtained. Among them, the hybrid decoding scheme uses both the technical means related to the first decoding scheme (i.e., the DirAC decoding scheme) and the technical means related to the second decoding scheme (the MP-based HOA decoding scheme) during the decoding process, so it is called a hybrid decoding scheme.

In the embodiment of the present application, since the encoder encodes the HOA signal of the current frame using the third encoding scheme, the signal of the designated channel is encoded into the bitstream, then the decoder parses the signal of the designated channel from the bitstream, and then reconstructs the signals of the remaining channels based on the signal of the designated channel, and then reconstructs the HOA signal. That is, a compromise solution is adopted, so that the auditory quality of the HOA signal after rendering and playback after decoding and recovery can be smoothly transitioned.

Optionally, determining the signal of the designated channel in the HOA signal of the current frame based on the bitstream includes: determining a virtual speaker signal and a residual signal based on the bitstream; and determining the signal of the designated channel based on the virtual speaker signal and the residual signal.

Optionally, determining the virtual speaker signal and the residual signal based on the bit stream includes: decoding the bit stream by a stereo decoder to obtain three-way stereo signals; and determining one virtual speaker signal and three-way residual signals based on the three-way stereo signals.

Optionally, determining one virtual speaker signal and three residual signals based on the three stereo sound signals includes: determining one virtual speaker signal based on one stereo sound signal among the three stereo sound signals; and determining three residual signals based on the other two stereo sound signals among the three stereo sound signals.

Optionally, determining the virtual speaker signal and the residual signal based on the bit stream includes: decoding the bit stream by a mono decoder to obtain one virtual speaker signal and three residual signals.

Optionally, the signal of the designated channel includes a first-order stereo reverberation FOA signal, the FOA signal includes an omnidirectional W signal, and directional X, Y, and Z signals; determining the signal of the designated channel based on the virtual speaker signal and the residue signal includes: determining the W signal based on the virtual speaker signal; determining the X, Y, and Z signals based on the residue signal and the W signal, or determining the X, Y, and Z signals based on the residue signal.

Optionally, the method further includes: if the decoding scheme of the current frame is the first decoding scheme, then according to the first decoding scheme, the reconstructed HOA signal of the current frame is obtained according to the bit stream; if the decoding scheme of the current frame is the second decoding scheme, then according to the second decoding scheme, the reconstructed HOA signal of the current frame is obtained according to the bit stream.

Optionally, according to the second decoding scheme, the reconstructed HOA signal of the current frame is obtained according to the bit stream, including: according to the second decoding scheme, the initial HOA signal is obtained according to the bit stream; if the decoding scheme of the previous frame of the current frame is the third decoding scheme, the high-order part of the initial HOA signal is gain-adjusted according to the high-order gain of the previous frame of the current frame; based on the low-order part of the initial HOA signal and the gain-adjusted high-order part, the reconstructed HOA signal is obtained. That is, through the high-order gain adjustment, the auditory quality is further smoothed.

Optionally, obtaining a decoding scheme for the current frame based on the bitstream includes: parsing a value of a switching flag of the current frame from the bitstream; if the value of the switching flag is a first value, parsing indication information of the decoding scheme of the current frame from the bitstream, the indication information being used to indicate that the decoding scheme of the current frame is the first decoding scheme or the second decoding scheme; if the value of the switching flag is the second value, determining that the decoding scheme of the current frame is a third decoding scheme.

Optionally, obtaining a decoding scheme for a current frame based on a bitstream includes: parsing indication information of a decoding scheme for the current frame from the bitstream, wherein the indication information is used to indicate that the decoding scheme for the current frame is a first decoding scheme, a second decoding scheme, or a third decoding scheme.

Optionally, obtaining a decoding scheme for a current frame based on a bitstream includes: parsing an initial decoding scheme for the current frame from the bitstream, the initial decoding scheme being a first decoding scheme or a second decoding scheme; if the initial decoding scheme for the current frame is the same as an initial decoding scheme for a frame previous to the current frame, determining the decoding scheme for the current frame to be the initial decoding scheme for the current frame; if the initial decoding scheme for the current frame is the first decoding scheme and the initial decoding scheme for the frame previous to the current frame is the second decoding scheme, or the initial decoding scheme for the current frame is the second decoding scheme and the initial decoding scheme for the frame previous to the current frame is the first decoding scheme, determining the decoding scheme for the current frame to be a third decoding scheme.

A third party manufacturer provides a coding device, which has the function of implementing the coding method in the first aspect. The coding device includes one or more modules, and the one or more modules are used to implement the coding method provided in the first aspect.

That is, a coding device is provided, which includes: a first determination module, used to determine the coding scheme of the current frame according to the high-order stereo reverberation HOA signal of the current frame, the coding scheme of the current frame is one of the first coding scheme, the second coding scheme and the third coding scheme; wherein the first coding scheme is the HOA coding scheme based on directional audio coding, the second coding scheme is the HOA coding scheme based on virtual speaker selection, and the third coding scheme is a hybrid coding scheme; the first coding module is used to encode the signal of the specified channel in the HOA signal into the bitstream if the coding scheme of the current frame is the third coding scheme, and the specified channel is a part of all channels of the HOA signal.

Optionally, the signal of the designated channel includes a first-order stereo reverberation FOA signal, and the FOA signal includes an omnidirectional W signal, and directional X signal, Y signal, and Z signal.

Optionally, the first encoding module includes: a first determining submodule, used to determine the virtual speaker signal and the residual signal based on the W signal, the X signal, the Y signal and the Z signal; and a coding submodule, used to encode the virtual speaker signal and the residual signal into a bitstream.

Optionally, the first determining submodule is used to: determine the W signal as one virtual speaker signal; determine three residual signals based on the W signal, the X signal, the Y signal and the Z signal, or determine the X signal, the Y signal and the Z signal as three residual signals.

Optionally, the encoding submodule is used to: combine the virtual speaker signal with the first preset mono signal to obtain a stereo signal; combine the three residual signals with the second preset mono signal to obtain two stereo signals; and encode the obtained three stereo signals into bit streams respectively through the stereo encoder.

Optionally, the coding submodule is used to: combine the two most correlated residual signals among the three residual signals to obtain one stereo signal among the two stereo signals; and combine a residual signal other than the two most correlated residual signals among the three residual signals with the second preset mono signal to obtain the other stereo signal among the two stereo signals.

Optionally, the first preset mono signal is an all-zero signal or an all-one signal, the all-zero signal includes a signal whose sampling point values are all zero or a signal whose frequency values are all zero, and the all-one signal includes a signal whose sampling point values are all one or a signal whose frequency values are all one; the second preset mono signal is an all-zero signal or an all-one signal; the first preset mono signal is the same as or different from the second preset mono signal.

Optionally, the encoding submodule is used to encode the virtual speaker signal and each of the three residual signals into a bitstream respectively through a mono encoder.

Optionally, the device further includes: a second coding module, used to encode the HOA signal into the bitstream according to the first coding scheme if the coding scheme of the current frame is the first coding scheme; and a third coding module, used to encode the HOA signal into the bitstream according to the second coding scheme if the coding scheme of the current frame is the second coding scheme.

Optionally, the first determination module includes: a second determination submodule, used to determine an initial coding scheme of the current frame according to the HOA signal, the initial coding scheme being the first coding scheme or the second coding scheme; a third determination submodule, used to determine that the coding scheme of the current frame is the initial coding scheme of the current frame if the initial coding scheme of the current frame is the same as the initial coding scheme of the previous frame of the current frame; and a fourth determination submodule, used to determine that the coding scheme of the current frame is the third coding scheme if the initial coding scheme of the current frame is the first coding scheme and the initial coding scheme of the previous frame of the current frame is the second coding scheme, or the initial coding scheme of the current frame is the second coding scheme and the initial coding scheme of the previous frame of the current frame is the first coding scheme.

Optionally, the device further comprises: a fourth encoding module, used for encoding indication information of the initial encoding scheme of the current frame into the bitstream.

Optionally, the device further includes: a second determination module, used to determine the value of the switching flag of the current frame, when the coding scheme of the current frame is the first coding scheme or the second coding scheme, the value of the switching flag of the current frame is the first value; when the coding scheme of the current frame is the third coding scheme, the value of the switching flag of the current frame is the second value; a fifth coding module, used to encode the value of the switching flag into the bitstream.

Optionally, the device further comprises: a sixth encoding module, used for encoding indication information of the encoding scheme of the current frame into the bitstream.

Optionally, the designated channel is consistent with a transmission channel preset in the first coding scheme.

In a fourth aspect, a decoding device is provided, wherein the decoding device has the function of implementing the decoding method in the second aspect. The decoding device includes one or more modules, wherein the one or more modules are used to implement the decoding method provided in the second aspect.

That is, a decoding device is provided, which includes: a first acquisition module, which is used to obtain a decoding scheme of a current frame based on a bit stream, and the decoding scheme of the current frame is one of a first decoding scheme, a second decoding scheme and a third decoding scheme; wherein the first decoding scheme is a high-order stereo reverberation HOA decoding scheme based on directional audio decoding, the second decoding scheme is an HOA decoding scheme based on virtual speaker selection, and the third decoding scheme is a hybrid decoding scheme; a first determination module, which is used to determine a signal of a specified channel in the HOA signal of the current frame based on the bit stream if the decoding scheme of the current frame is the third decoding scheme, and the specified channel is a part of all channels of the HOA signal; a second determination module, which is used to determine the HOA based on the signal of the specified channel. The signal includes a gain of one or more residual channels other than the designated channel in the signal; a third determination module is used to determine the signal of each residual channel in the one or more residual channels based on the signal of the designated channel and the gain of the one or more residual channels; and a second acquisition module is used to obtain the reconstructed HOA signal of the current frame based on the signal of the designated channel and the signal of the one or more residual channels.

Optionally, the first determination module includes: a first determination submodule, used to determine a virtual speaker signal and a residual signal based on a bit stream; and a second determination submodule, used to determine a signal of a designated channel based on the virtual speaker signal and the residual signal.

Optionally, the first determination submodule is used to: decode the code stream through a stereo decoder to obtain three-way stereo signals; and determine one virtual speaker signal and three-way residual signals based on the three-way stereo signals.

Optionally, the first determination submodule is used to: determine a virtual speaker signal based on one of the three stereo sound signals; and determine three residual signals based on the other two of the three stereo sound signals.

Optionally, the first determination submodule is used to: decode the code stream through a mono decoder to obtain one virtual speaker signal and three residual signals.

Optionally, the signal of the designated channel includes a first-order stereo reverberation FOA signal, the FOA signal includes an omnidirectional W signal, and directional X, Y, and Z signals; the first determination submodule is used to: determine the W signal based on the virtual speaker signal; determine the X, Y, and Z signals based on the residual signal and the W signal, or determine the X, Y, and Z signals based on the residual signal.

Optionally, the device further includes: a first decoding module, which is used to obtain a reconstructed HOA signal of the current frame according to the bit stream according to the first decoding scheme if the decoding scheme of the current frame is the first decoding scheme; and a second decoding module, which is used to obtain a reconstructed HOA signal of the current frame according to the bit stream according to the second decoding scheme if the decoding scheme of the current frame is the second decoding scheme.

Optionally, the second decoding module includes: a first acquisition submodule, used to obtain an initial HOA signal according to the bit stream according to the second decoding scheme; a gain adjustment submodule, used to adjust the gain of the high-order part of the initial HOA signal according to the high-order gain of the previous frame of the current frame if the decoding scheme of the previous frame of the current frame is the third decoding scheme; and a second acquisition submodule, used to obtain a reconstructed HOA signal based on the low-order part of the initial HOA signal and the gain-adjusted high-order part.

Optionally, the first acquisition module includes: a first parsing submodule, used to parse the value of the switching flag of the current frame from the bit stream; a second parsing submodule, used to parse indication information of the decoding scheme of the current frame from the bit stream if the value of the switching flag is the first value, the indication information is used to indicate that the decoding scheme of the current frame is the first decoding scheme or the second decoding scheme; a third determination submodule, used to determine that the decoding scheme of the current frame is the third decoding scheme if the value of the switching flag is the second value.

Optionally, the first acquisition module includes: a third parsing submodule, used to parse out indication information of a decoding scheme of a current frame from the bit stream, the indication information being used to indicate whether the decoding scheme of the current frame is the first decoding scheme, the second decoding scheme or the third decoding scheme.

Optionally, the first acquisition module includes: a fourth parsing submodule, used to parse out an initial decoding scheme of the current frame from the bit stream, the initial decoding scheme being the first decoding scheme or the second decoding scheme; a fourth determining submodule, used to determine that the decoding scheme of the current frame is the initial decoding scheme of the current frame if the initial decoding scheme of the current frame is the same as the initial decoding scheme of the previous frame of the current frame; and a fifth determining submodule, used to determine that the decoding scheme of the current frame is the third decoding scheme if the initial decoding scheme of the current frame is the first decoding scheme and the initial decoding scheme of the previous frame of the current frame is the second decoding scheme, or the initial decoding scheme of the current frame is the second decoding scheme and the initial decoding scheme of the previous frame of the current frame is the first decoding scheme.

In a fifth aspect, a coding end device is provided, the coding end device includes a processor and a memory, the memory is used to store a program for executing the coding method provided in the first aspect, and to store data involved in implementing the coding method provided in the first aspect. The processor is configured to execute the program stored in the memory. The operating device of the storage device may also include a communication bus, which is used to establish a connection between the processor and the memory.

In a sixth aspect, a decoding end device is provided, the decoding end device includes a processor and a memory, the memory is used to store a program for executing the decoding method provided in the second aspect, and to store data involved in implementing the decoding method provided in the second aspect. The processor is configured to execute the program stored in the memory. The operating device of the storage device may also include a communication bus, which is used to establish a connection between the processor and the memory.

In a seventh aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the instructions are run on a computer, the computer executes the encoding method described in the first aspect or the decoding method described in the second aspect.

In an eighth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the encoding method described in the first aspect or the decoding method described in the second aspect.

The technical effects obtained by the above-mentioned third party manufacturers in the fourth aspect, the fifth aspect, the sixth aspect, the seventh aspect and the eighth aspect are similar to the technical effects obtained by the corresponding technical means in the first aspect or the second aspect, and will not be elaborated here.

The technical solution provided by the embodiment of the present application can at least bring the following beneficial effects: In the embodiment of the present application, two schemes (i.e., a coding and decoding scheme based on virtual speaker selection and a coding and decoding scheme based on directional audio coding) are combined to encode and decode the HOA signal of the audio frame, that is, appropriate coding and decoding schemes are selected for different audio frames, which can improve the compression rate of the audio signal. At the same time, in order to ensure a smooth transition of auditory quality when switching between different coding and decoding schemes, this scheme does not directly adopt any of the above two schemes for encoding and decoding for some audio frames, but adopts a new coding and decoding scheme to encode and decode these audio frames, that is, encodes the signals of the specified channels in the HOA signals of these audio frames into the bit stream, that is, adopts a compromise scheme for encoding and decoding, so that the auditory quality after rendering and playing the decoded and restored HOA signals can be smoothly transitioned.

In order to make the purpose, technical solutions and advantages of the embodiment of this application clearer, the embodiment of this application will be further described in detail below in conjunction with the accompanying drawings.

Before explaining the encoding and decoding method provided by the embodiment of the present application in detail, the implementation environment involved in the embodiment of the present application is first introduced.

Please refer to Figure 1, which is a schematic diagram of an implementation environment provided by an embodiment of the present application. The implementation environment includes a source device 10, a destination device 20, a link 30 and a storage device 40. Among them, the source device 10 can generate encoded media data. Therefore, the source device 10 can also be called a media data encoding device. The destination device 20 can decode the encoded media data generated by the source device 10. Therefore, the destination device 20 can also be called a media data decoding device. The link 30 can receive the encoded media data generated by the source device 10, and can transmit the encoded media data to the destination device 20. The storage device 40 can receive the encoded media data generated by the source device 10 and can store the encoded media data. Under such conditions, the destination device 20 can directly obtain the encoded media data from the storage device 40. Alternatively, the storage device 40 can correspond to a file server or another intermediate storage device that can save the encoded media data generated by the source device 10. Under such conditions, the destination device 20 can store the encoded media data stored in the storage device 40 via data streaming or downloading.

Both the source device 10 and the destination device 20 may include one or more processors and a memory coupled to the one or more processors, and the memory may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, any other medium that can be used to store desired program code in the form of instructions or data structures accessible by a computer, etc. For example, source device 10 and destination device 20 may each include a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a handheld phone such as a so-called "smart" phone, a television, a camera, a display device, a digital media player, a video game console, a car computer, or the like.

The link 30 may include one or more media or devices capable of transmitting the encoded media data from the source device 10 to the destination device 20. In one possible implementation, the link 30 may include one or more communication media that enable the source device 10 to directly send the encoded media data to the destination device 20 in real time. In the embodiment of the present application, the source device 10 may modulate the encoded media data based on a communication standard, which may be a wireless communication protocol, etc., and may send the modulated media data to the destination device 20. The one or more communication media may include wireless and/or wired communication media, for example, the one or more communication media may include a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, which may be a local area network, a wide area network, or a global network (e.g., the Internet), etc. The one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the source device 10 to the destination device 20, and the embodiment of the present application is not specifically limited to this.

In a possible implementation, the storage device 40 may store the received encoded media data sent by the source device 10, and the destination device 20 may directly obtain the encoded media data from the storage device 40. Under such conditions, the storage device 40 may include any one of a variety of distributed or locally accessed data storage media, for example, any one of the multiple distributed or locally accessed data storage media may be a hard disk drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc read-only memory (CD-ROM), a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded media data.

In one possible implementation, the storage device 40 may correspond to a file server or another intermediate storage device that can store the encoded media data generated by the source device 10, and the destination device 20 may stream or download the media data stored in the storage device 40. The file server may be any type of server that can store the encoded media data and send the encoded media data to the destination device 20. In one possible implementation, the file server may include a network server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive, etc. The destination device 20 may obtain the encoded media data through any standard data connection (including an Internet connection). Any standard data connection may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of the two suitable for obtaining encoded media data stored on a file server. The transmission of the encoded media data from the storage device 40 may be a data stream, a download transmission, or a combination of the two.

The implementation environment shown in FIG1 is only one possible implementation method, and the technology of the embodiment of the present application can be applied not only to the source device 10 that can encode media data and the destination device 20 that can decode the encoded media data shown in FIG1, but also to other devices that can encode media data and decode encoded media data, and the embodiment of the present application does not make specific limitations on this.

In the implementation environment shown in FIG1 , the source device 10 includes a data source 120, an encoder 100, and an output interface 140. In some embodiments, the output interface 140 may include a modem/demodulator (a modem) and/or a transmitter, wherein the transmitter may also be referred to as a transmitter. The data source 120 may include an image capture device (e.g., a camera, etc.), an archive containing previously captured media data, a feed interface for receiving media data from a media data content provider, and/or a computer graphics system for generating media data, or a combination of these sources of media data.

The data source 120 may send media data to the encoder 100, and the encoder 100 may encode the media data received from the data source 120 to obtain the encoded media data. The encoder may send the encoded media data to the output interface. In some embodiments, the source device 10 directly sends the encoded media data to the destination device 20 via the output interface 140. In other embodiments, the encoded media data may also be stored in the storage device 40 for the destination device 20 to obtain and use for decoding and/or display.

In the implementation environment shown in FIG. 1 , the destination device 20 includes an input interface 240, a decoder 200, and a display device 220. In some embodiments, the input interface 240 includes a receiver and/or a modem. The input interface 240 may receive the encoded media data via the link 30 and/or from the storage device 40, and then send it to the decoder 200, which may decode the received encoded media data to obtain decoded media data. The decoder may send the decoded media data to the display device 220. The display device 220 may be integrated with the destination device 20 or may be external to the destination device 20. Generally speaking, the display device 220 displays the decoded media data. The display device 220 may be any of a variety of types of display devices. For example, the display device 220 may be a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

Although not shown in FIG. 1 , in some aspects, encoder 100 and decoder 200 may be integrated with an encoder and decoder, respectively, and may include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software for encoding both audio and video in a common data flow or in separate data flows. In some embodiments, the MUX-DEMUX units may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP), if applicable.

The encoder 100 and the decoder 200 may each be any of the following circuits: one or more microprocessors, digital signal processing (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), discrete logic, hardware, or any combination thereof. If the technology of the embodiment of the present application is partially implemented in software, the device may store instructions for the software in a suitable non-volatile computer-readable storage medium, and may use one or more processors to execute the instructions in hardware to implement the technology of the embodiment of the present application. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered as one or more processors. Each of the encoder 100 and the decoder 200 may be included in one or more encoders or decoders, and any of the encoders or decoders may be integrated as part of a combined encoder/decoder (encoder-decoder) in a corresponding device.

The present application embodiments may generally refer to encoder 100 as "signaling" or "sending" certain information to another device, such as decoder 200. The term "signaling" or "sending" may generally refer to the transmission of syntax elements and/or other data used to decode compressed media data. This transmission may occur instantly or nearly instantly. Alternatively, this communication may occur over a period of time, such as when the syntax elements are stored in the encoded bit stream to a computer-readable storage medium during encoding, and the decoding device may then retrieve the syntax elements at any time after the syntax elements are stored to the medium.

The encoding and decoding method provided in the embodiment of the present application can be applied to a variety of scenarios. Next, taking the media data to be encoded as HOA signal as an example, several of the scenarios are introduced respectively.

Please refer to FIG. 2, which is a schematic diagram of an implementation environment of a coding and decoding method provided by an embodiment of the present application applied to a terminal scenario. The implementation environment includes a first terminal 101 and a second terminal 201, and the first terminal 101 is connected to the second terminal 201 for communication. The communication connection can be a wireless connection or a wired connection, and the embodiment of the present application does not limit this.

The first terminal 101 can be a sending device or a receiving device, and similarly, the second terminal 201 can be a receiving device or a sending device. For example, when the first terminal 101 is a sending device, the second terminal 201 is a receiving device, and when the first terminal 101 is a receiving device, the second terminal 201 is a sending device.

Next, the first terminal 101 is taken as a sending device and the second terminal 201 is taken as a receiving device as an example for introduction.

The first terminal 101 and the second terminal 201 both include an audio acquisition module, an audio playback module, a codec, a decoder, a channel coding module and a channel decoding module. In the present application embodiment, the codec is a three-dimensional audio codec, and the decoder is a three-dimensional audio decoder.

The audio acquisition module in the first terminal 101 collects the HOA signal and transmits it to the encoder. The encoder encodes the HOA signal using the encoding method provided in the embodiment of the present application. This encoding can be called source encoding. Afterwards, in order to realize the transmission of the HOA signal in the channel, the channel encoding module needs to perform channel encoding again, and then transmit the encoded code stream in the digital channel through the wireless or wired network communication equipment.

The second terminal 201 receives the code stream transmitted in the digital channel through a wireless or wired network communication device, the channel decoding module performs channel decoding on the code stream, and then the decoder uses the decoding method provided in the embodiment of the present application to decode to obtain the HOA signal, and then plays it through the audio replay module.

Among them, the first terminal 101 and the second terminal 201 can be any electronic product that can perform human-computer interaction with the user through one or more methods such as keyboard, touch pad, touch screen, remote control, voice interaction or handwriting device, such as personal computer (PC), mobile phone, smart phone, personal digital assistant (PDA), wearable device, handheld computer PPC (pocket PC), tablet computer, smart car device, smart TV, smart speaker, etc.

Those skilled in the art should understand that the above-mentioned terminals are merely examples, and other existing or future terminals that may appear, if applicable to the embodiments of this application, should also be included in the protection scope of the embodiments of this application and are incorporated herein by reference.

Please refer to Figure 3, which is a schematic diagram of an implementation environment of a coding and decoding method provided by an embodiment of the present application applied to a transcoding scenario of a wireless or core network device. The implementation environment includes a channel decoding module, an audio decoder, an audio encoder, and a channel encoding module. In the embodiment of the present application, the audio encoder is a three-dimensional audio encoder, and the audio decoder is a three-dimensional audio decoder.

The audio decoder may be a decoder using the decoding method provided in the embodiment of the present application, or may be a decoder using other decoding methods. The audio encoder may be a encoder using the encoding method provided in the embodiment of the present application, or may be a encoder using other encoding methods. In the case where the audio decoder is a decoder using the decoding method provided in the embodiment of the present application, the audio encoder is an encoder using other encoding methods; in the case where the audio decoder is a decoder using other decoding methods, the audio encoder is an encoder using the encoding method provided in the embodiment of the present application.

In the first case, the audio decoder is a decoder using the decoding method provided in the embodiment of the present application, and the audio encoder is an encoder using other encoding methods.

At this time, the channel decoding module is used to perform channel decoding on the received code stream, and then the audio decoder is used to perform source decoding using the decoding method provided in the embodiment of the present application, and then the audio encoder is used to perform encoding according to other encoding methods to achieve conversion from one format to another, i.e. transcoding. After that, it is sent after channel encoding.

In the second case, the audio decoder is a decoder using other decoding methods, and the audio encoder is an encoder using the encoding method provided in the embodiment of the present application.

At this time, the channel decoding module is used to perform channel decoding on the received code stream, and then the audio decoder is used to use other decoding methods to perform source decoding, and then the audio encoder uses the encoding method provided by the embodiment of the present application to perform encoding, so as to realize the conversion from one format to another format, i.e. transcoding. After that, it is sent after channel encoding.

The wireless device may be a wireless access point, a wireless router, a wireless connector, etc. The core network device may be a mobility management entity, a gateway, etc.

Those skilled in the art should understand that the above-mentioned wireless devices or core network devices are merely examples, and other existing or future wireless or core network devices that may be applicable to the embodiments of this application should also be included in the protection scope of the embodiments of this application and are incorporated herein by reference.

Please refer to FIG. 4, which is a schematic diagram of an implementation environment of a coding and decoding method provided by an embodiment of the present application applied to a broadcasting and television scene. The broadcasting and television scene is divided into a live broadcast scene and a post-production scene. For the live broadcast scene, the implementation environment includes a live program three-dimensional sound production module, a three-dimensional sound encoding module, a set-top box and a speaker group, and the set-top box includes a three-dimensional sound decoding module. For the post-production scene, the implementation environment includes a post-program three-dimensional sound production module, a three-dimensional sound encoding module, a network receiver, a mobile terminal, a headset, etc.

In the live broadcast scenario, the live program three-dimensional sound production module produces a three-dimensional sound signal (such as an HOA signal), and the three-dimensional sound signal is obtained by applying the encoding method of the embodiment of the present application to obtain a code stream, which is transmitted to the user side through the broadcasting network, and the three-dimensional sound decoding module in the set-top box uses the decoding method provided in the embodiment of the present application to decode the code stream, thereby reconstructing the three-dimensional sound signal, and replaying it by the speaker group. Alternatively, the code stream is transmitted to the user side via the Internet, and the three-dimensional sound decoding module in the network receiver uses the decoding method provided in the embodiment of the present application to decode the code stream, thereby reconstructing the three-dimensional sound signal, and replaying it by the speaker group. Alternatively, the code stream is transmitted to the user side via the Internet, and the 3D sound decoding module in the mobile terminal decodes the code stream using the decoding method provided in the embodiment of the present application, thereby reconstructing the 3D sound signal, which is replayed by the earphone.

In the post-production scenario, the post-program 3D sound production module produces a 3D sound signal, and the 3D sound signal is obtained by applying the encoding method of the embodiment of the present application to obtain a code stream, which is transmitted to the user side through the broadcasting network, and the 3D sound decoding module in the set-top box uses the decoding method provided by the embodiment of the present application to decode the code stream, thereby reconstructing the 3D sound signal, and replaying it by the speaker group. Alternatively, the code stream is transmitted to the user side via the Internet, and the 3D sound decoding module in the network receiver uses the decoding method provided by the embodiment of the present application to decode the code stream, thereby reconstructing the 3D sound signal, and replaying it by the speaker group. Alternatively, the code stream is transmitted to the user side via the Internet, and the 3D sound decoding module in the mobile terminal decodes the code stream using the decoding method provided in the embodiment of the present application, thereby reconstructing the 3D sound signal, which is replayed by the earphone.

Please refer to Figure 5, which is a schematic diagram of an implementation environment of a coding and decoding method provided by an embodiment of the present application applied to a virtual reality streaming scene. The implementation environment includes a coding end and a decoding end, the coding end includes a collection module, a pre-processing module, a coding module, a packaging module and a sending module, and the decoding end includes an unpacking module, a decoding module, a rendering module and a headset.

The acquisition module acquires the HOA signal, and then performs preprocessing on the HOA signal through the preprocessing module. The preprocessing operation includes filtering out the low-frequency part of the HOA signal, usually with 20Hz or 50Hz as the dividing point, extracting the azimuth information in the HOA signal, etc. Then, the encoding module performs encoding processing using the encoding method provided in the embodiment of the present application, and after encoding, the packaging module performs packaging, and then sends it to the decoding end through the sending module.

The unpacking module at the decoding end first unpacks the signal, and then decodes the signal by using the decoding method provided by the embodiment of the present application, and then performs binaural rendering processing on the decoded signal by the rendering module, and the rendered signal is mapped to the listener's earphone. The earphone can be an independent earphone or an earphone on a virtual reality-based eyewear device.

It should be noted that the system architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Ordinary technical personnel in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

Next, the encoding and decoding method provided by the embodiment of the present application is explained in detail. It should be noted that, in combination with the implementation environment shown in FIG. 1 , any encoding method described below can be executed by the encoder 100 in the source device 10. Any decoding method described below can be executed by the decoder 200 in the destination device 20.

FIG6 is a flow chart of a coding method provided by an embodiment of the present application, and the coding method is applied to a coding end. Referring to FIG6 , the method includes the following steps.

Step 601: Determine the coding scheme of the current frame according to the HOA signal of the current frame.

For the HOA signals of multiple audio frames to be encoded, the encoding end encodes frame by frame. Among them, the HOA signal of the audio frame is an audio signal obtained through the HOA acquisition technology. The HOA signal is a scene audio signal and also a three-dimensional audio signal. The HOA signal refers to the audio signal obtained by collecting the sound field at the position of the microphone in the space. The collected audio signal is called the original HOA signal. The HOA signal of the audio frame can also be an HOA signal obtained by converting three-dimensional audio signals in other formats. For example, a 5.1-channel signal is converted into an HOA signal, or a three-dimensional audio signal mixed with a 5.1-channel signal and object audio is converted into an HOA signal. Optionally, the HOA signal of the audio frame to be encoded is a time domain signal or a frequency domain signal, and may include all channels of the HOA signal or some channels of the HOA signal. Exemplarily, if the order of the HOA signal of the audio frame is 3, the number of channels of the HOA signal is 16, the frame length of the audio frame is 20ms, and the sampling rate is 48KHz, then the HOA signal of the audio frame to be encoded includes signals of 16 channels, and each channel includes 960 sampling points.

In order to reduce the computational complexity, if the HOA signal of the audio frame obtained by the encoder is the original HOA signal, and the number of sampling points or frequency points of the original HOA signal is large, then the encoder can downsample the original HOA signal to obtain the HOA signal of the audio frame to be encoded. For example, the encoder performs 1/Q downsampling on the original HOA signal to reduce the number of sampling points or frequency points of the HOA signal to be encoded. For example, in the embodiment of the present application, each channel of the original HOA signal contains 960 sampling points. After 1/120 downsampling, each channel of the HOA signal to be encoded contains 8 sampling points.

In the embodiment of the present application, the encoding method of the encoding end is introduced by taking the encoding of the current frame by the encoding end as an example. The current frame is an audio frame to be encoded. That is, the encoding end obtains the HOA signal of the current frame and encodes the HOA signal of the current frame using the encoding method provided in the embodiment of the present application.

It should be noted that in order to meet the requirement of having a high compression rate for audio frames under different sound field types, it is necessary to select a suitable coding and decoding scheme for the corresponding audio frame according to the sound field type of each audio frame. In the embodiment of the present application, the coding end first determines the initial coding scheme of the current frame according to the HOA signal of the current frame, and the initial coding scheme is the first coding scheme or the second coding scheme. The coding end then determines whether to use the first coding scheme, the second coding scheme or the third coding scheme to encode the HOA signal of the current frame by comparing whether the initial coding scheme of the current frame is the same as the initial coding scheme of the previous frame of the current frame. Among them, if the initial coding scheme of the current frame is the same as the initial coding scheme of the previous frame of the current frame, the coding end adopts the coding scheme consistent with the initial coding scheme of the current frame to encode the HOA signal of the current frame. If the initial coding scheme of the current frame is different from the initial coding scheme of the previous frame of the current frame, the coding end adopts the switching frame coding scheme to encode the HOA signal of the current frame.

In the embodiment of the present application, the coding scheme of the current frame is one of the first coding scheme, the second coding scheme and the third coding scheme. Among them, the first coding scheme is the HOA coding scheme based on DirAC, the second coding scheme is the HOA coding scheme based on virtual speaker selection, and the third coding scheme is a hybrid coding scheme. Optionally, the hybrid coding scheme is also called a switching frame coding scheme. The third coding scheme is a switching frame coding scheme provided in the embodiment of the present application. The third coding scheme is for a smooth transition of auditory quality when switching between different coding and decoding schemes. The embodiment of the present application will introduce these three coding schemes in detail below. In the embodiment of the present application, the HOA coding scheme based on virtual speaker selection is also called the MP-based HOA coding scheme.

In the embodiment of the present application, the coding end determines the initial coding scheme of the current frame according to the HOA signal of the current frame. Then, the coding end determines the coding scheme of the current frame based on the initial coding scheme of the current frame and the initial coding scheme of the previous frame of the current frame. It should be noted that the embodiment of the present application does not limit the implementation method of the coding end determining the initial coding scheme.

Optionally, the coding end performs a sound field type analysis on the HOA signal of the current frame to obtain a sound field classification result of the current frame, and determines an initial coding scheme for the current frame based on the sound field classification result of the current frame. It should be noted that the embodiment of the present application does not limit the method of sound field type analysis. For example, the coding end performs a sound field type analysis by performing a singular value decomposition on the HOA signal of the current frame, or performs other linear decomposition on the HOA signal to perform a sound field type analysis.

Optionally, the sound field classification result includes the number of dissimilar sound sources. Taking the example that the encoder directly performs sound field type analysis on the HOA signal of the current frame, one implementation method of the encoder performing sound field type analysis on the HOA signal of the current frame to obtain the sound field classification result of the current frame is: the encoder performs singular value decomposition on the HOA signal of the current frame to obtain M singular values. The encoder calculates the ratio of the i-th singular value to the i+1-th singular value among the M singular values to obtain M-1 sound field classification parameters. Wherein, i=1,2,…,M. The encoder determines the number of dissimilar sound sources corresponding to the current frame based on the M-1 sound field classification parameters. Where M=min(L,K), L represents the number of channels of the HOA signal of the current frame, K represents the number of signal points of each channel of the HOA signal of the current frame, and min represents the minimum value operation. If the HOA signal is a time domain signal, the number of signal points is the number of sampling points, and if the HOA signal is a frequency domain signal, the number of signal points is the number of frequency points.

Optionally, assuming that the M-1 sound field type parameters are temp[i], i=0,1,…,M-2, the encoder determines the number of dissimilar sound sources corresponding to the current frame based on the M-1 sound field classification parameters. One implementation method is: starting from i=0, the following processes are executed in sequence: determine whether temp[i] is greater than a preset dissimilar sound source determination threshold; if temp[i] is less than the dissimilar sound source determination threshold in this round of the process, update the value of i to i+1, and continue to execute the next round of the process; if temp[i] is greater than or equal to the dissimilar sound source determination threshold in this round of the process, determine that the number of dissimilar sound sources corresponding to the current frame is equal to i+1, and end the process. Optionally, the phase difference sound source determination threshold is 30, 80 or 100, etc. The phase difference sound source determination threshold is a preset value, which can be preset based on experience or through statistics.

Correspondingly, in one implementation, after determining the number of dissimilar sound sources corresponding to the current frame, if the number of dissimilar sound sources corresponding to the current frame is greater than the first threshold and less than the second threshold, the coding end determines that the initial coding scheme of the current frame is the second coding scheme. If the number of dissimilar sound sources corresponding to the current frame is not greater than the first threshold or not less than the second threshold, the coding end determines that the initial coding scheme of the current frame is the first coding scheme. The first threshold is less than the second threshold. Optionally, the first threshold is 0 or other values, and the second threshold is 3 or other values. The aforementioned first threshold and second threshold are preset values, which can be preset based on experience or through statistics.

For example, assume that the number of channels of the HOA signal of the current frame is L=16, the number of frequency points of each channel is K=8, and min(L,K)=8. Then, the encoder performs singular value decomposition on the HOA signal of the current frame to obtain singular values v[i], i=0,1,…,min(L,K)-1. The encoder calculates the ratio between adjacent singular values and uses the obtained ratio as the sound field classification result temp[i] of the current frame, temp[i]=v[i]/v[i+1], i=0,1,…,min(L,K)-2. Assuming that the threshold for determining the dissimilarity sound source is 100, the process of determining the number of dissimilarity sound sources n is as follows: starting from i=0, determine whether temp[i] is greater than or equal to 100. If temp[i] is greater than or equal to 100, that is, temp[i]≥100 is satisfied, then stop the determination; otherwise, i=i+1 and continue the determination. If the determination is stopped, the sequence number i at the time of stopping the determination plus 1 is equal to the number of dissimilarity sound sources n corresponding to the current frame. For example, when i=0, if temp[0]≥100, then stop judging, and the number of dissimilar sound sources n is equal to 1; otherwise, let i=1 and continue judging i=1; when i=1, temp[1]≥100, then stop judging, and the number of dissimilar sound sources n is equal to i+1=2. Assuming that the first threshold is 0 and the second threshold is 3, if the number of dissimilar sound sources n corresponding to the current frame satisfies 0＜n＜3, the encoder determines that the initial coding scheme of the current frame is the second coding scheme. If the number of dissimilar sound sources n corresponding to the current frame satisfies n=0 or n≥3, the encoder determines that the initial coding scheme of the current frame is the first coding scheme.

Optionally, the sound field classification result includes a sound field type, and the sound field type is divided into a diffuse sound field and a heterogeneous sound field. The sound field type can be determined based on the number of heterogeneous sound sources obtained by the aforementioned method, that is, the encoder determines the sound field type of the current frame based on the number of heterogeneous sound sources corresponding to the current frame. For example, if the number of heterogeneous sound sources corresponding to the current frame is greater than a first threshold and less than a second threshold, the encoder determines that the sound field type of the current frame is a heterogeneous sound field. If the number of heterogeneous sound sources corresponding to the current frame is not greater than the first threshold or not less than the second threshold, the encoder determines that the sound field type of the current frame is a diffuse sound field. Correspondingly, if the sound field type of the current frame is a heterogeneous sound field, the coding end determines that the initial coding scheme of the current frame is the second coding scheme, that is, the HOA coding scheme based on MP. If the sound field type of the current frame is a diffuse sound field type, the coding end determines that the initial coding scheme of the current frame is the first coding scheme, that is, the HOA coding scheme based on DirAC.

In some embodiments, after the initial coding scheme of each audio frame (including the current frame) is determined by the above implementation method, the initial coding scheme of each audio frame may be switched back and forth, that is, more switching frames need to be encoded in the end. Since the switching between coding schemes brings more problems, that is, more problems need to be solved, the problems caused by switching can be reduced by reducing the number of switching frames. In order to reduce the number of switching frames, the encoder can first determine the expected coding scheme of the current frame according to the sound field classification result of the current frame, that is, the encoder uses the initial coding scheme determined by the above method as the expected coding scheme. Then, the encoder adopts a sliding window method to update the initial coding scheme of the current frame based on the expected coding scheme, such as the encoder updates the initial coding scheme of the current frame through hangover processing.

Optionally, assuming that the length of the sliding window is N, the sliding window contains the expected coding scheme of the current frame and the updated initial coding scheme of the N-1 frames before the current frame. If the cumulative number of second coding schemes in the sliding window is not less than the first specified threshold, the coding end updates the initial coding scheme of the current frame to the second coding scheme. If the cumulative number of second coding schemes in the sliding window is less than the first specified threshold, the coding end updates the initial coding scheme of the current frame to the first coding scheme. Among them, the length N of the sliding window is 8, 10, 15, etc., and the first specified threshold is 5, 6, 7, etc. The embodiment of the present application does not limit the values of the length of the sliding window and the first specified threshold. Take an example as follows, assuming that the length of the sliding window is 10, the first specified threshold is 7, the sliding window contains the expected coding scheme of the current frame and the updated initial coding schemes of the previous 9 frames of the current frame. If the number of second coding schemes in the sliding window accumulates to no less than 7, the encoder determines the initial coding scheme of the current frame as the second coding scheme. If the number of second coding schemes in the sliding window accumulates to less than 7, the encoder updates the initial coding scheme of the current frame to the first coding scheme.

Alternatively, if the cumulative number of the first coding schemes in the sliding window is not less than the second specified threshold, the coding end updates the initial coding scheme of the current frame to the first coding scheme. If the cumulative number of the first coding schemes in the sliding window is less than the second specified threshold, the coding end updates the initial coding scheme of the current frame to the second coding scheme. The second specified threshold is 5, 6, 7, etc., and the embodiment of the present application does not limit the value of the second specified threshold. Optionally, the second specified threshold is different from or the same as the first specified threshold.

In addition to the implementations described above, the encoder may also use other methods to obtain the sound field classification result of the current frame, and the method for determining the initial coding scheme based on the sound field classification result may also use other methods, which are not limited in this embodiment of the present application.

In the embodiment of the present application, after the coding end determines the initial coding scheme of the current frame, if the initial coding scheme of the current frame is the same as the initial coding scheme of the previous frame of the current frame, the coding end determines the coding scheme of the current frame to be the initial coding scheme of the current frame. If the initial coding scheme of the current frame is different from the initial coding scheme of the previous frame of the current frame, the coding end determines the coding scheme of the current frame to be the third coding scheme. That is, if the initial coding scheme of the current frame is the same as the initial coding scheme of the previous frame of the current frame and is the first coding scheme, the coding end determines the coding scheme of the current frame to be the first coding scheme. If the initial coding scheme of the current frame is the same as the initial coding scheme of the previous frame of the current frame and is the second coding scheme, the coding end determines that the coding scheme of the current frame is the second coding scheme. If one of the initial coding scheme of the current frame and the initial coding scheme of the previous frame of the current frame is the first coding scheme and the other is the second coding scheme, the coding end determines that the coding scheme of the current frame is the third coding scheme. Among them, one of the initial coding scheme of the current frame and the initial coding scheme of the previous frame of the current frame is the first coding scheme and the other is the second coding scheme, that is, the initial coding scheme of the current frame is the first coding scheme and the initial coding scheme of the previous frame of the current frame is the second coding scheme, or the initial coding scheme of the current frame is the second coding scheme and the initial coding scheme of the previous frame of the current frame is the first coding scheme. That is, for the switching frame, the coding end does not use the first coding scheme or the second coding scheme to encode the HOA signal of the switching frame, but uses the switching frame coding scheme to encode the HOA signal of the switching frame. For the non-switching frame, the coding end will use the coding scheme consistent with the initial coding scheme of the non-switching frame to encode the HOA signal of the switching frame. Among them, the audio frame whose initial coding scheme is different from the initial coding scheme of the previous frame is the switching frame, and the audio frame whose initial coding scheme is the same as the initial coding scheme of the previous frame is the non-switching frame.

It should be noted that, in addition to determining the coding scheme of the current frame, the encoder also needs to encode information that can indicate the coding scheme of the current frame into the bitstream, so that the decoder can determine which decoding scheme to use to decode the bitstream of the current frame. In the embodiment of the present application, there are multiple ways to implement the encoder to encode the information that can indicate the coding scheme of the current frame into the bitstream, and three of them are introduced below.

First implementation, coding switching flag, and indication information of two coding schemes

In this implementation, the coding end needs to determine the value of the switching flag of the current frame and encode the value of the switching flag of the current frame into the bitstream. When the coding scheme of the current frame is the first coding scheme or the second coding scheme, the value of the switching flag of the current frame is the first value. When the coding scheme of the current frame is the third coding scheme, the value of the switching flag of the current frame is the second value. Optionally, the first value is "0" and the second value is "1", and the first value and the second value may also be other values.

In addition, the coding end encodes the indication information of the initial coding scheme of the current frame into the bitstream. Alternatively, if the value of the switching flag of the current frame is the first value, the coding end encodes the indication information of the initial coding scheme of the current frame into the bitstream, and if the value of the switching flag of the current frame is the second value, the coding end encodes the default indication information into the bitstream.

Optionally, the indication information of the initial coding scheme is represented by a coding mode corresponding to the initial coding scheme, that is, the coding mode is used as the indication information. For example, the coding mode corresponding to the initial coding scheme is the initial coding mode, and the initial coding mode is the first coding mode (i.e., DirAC mode) or the second coding mode (i.e., MP mode). Optionally, the default indication information is the default coding mode, and the default coding mode is the first coding mode or the second coding mode. In some other embodiments, the default indication information is other coding modes, that is, the indication information of the coding scheme for switching frames encoded into the bitstream is not limited.

That is, in the first implementation, the encoder indicates the switching frame with the switching flag, and the indication information of the coding scheme of the switching frame encoded into the bitstream may not be limited. The indication information of the coding scheme of the switching frame may be the initial coding mode, the default coding mode, or randomly selected from the first coding mode and the second coding mode, or other indication information. It should be noted that in this implementation, the switching flag is used to indicate whether the current frame is a switching frame, so that the decoder can directly determine whether the current frame is a switching frame by obtaining the switching flag in the bitstream.

Optionally, in the first implementation, the switching flag of the current frame and the indication information of the initial coding scheme each occupy one bit of the code stream. Exemplarily, the value of the switching flag of the current frame is "0" or "1", wherein the value of the switching flag of the current frame is "0" indicating that the current frame is not a switching frame, that is, the value of the switching flag of the current frame is a first value. The switching flag is "1" indicating that the current frame is a switching frame, that is, the value of the switching flag of the current frame is a second value. Optionally, the indication information of the initial coding scheme is "0" or "1", wherein "0" indicates DirAC mode (i.e., DirAC coding scheme), and "1" indicates MP mode (i.e., MP-based coding scheme).

In some other embodiments, if the initial coding scheme of the current frame is different from the initial coding scheme of the previous frame of the current frame, the coding end determines that the value of the switching flag of the current frame is the second value, and encodes the value of the switching flag of the current frame into the bitstream. That is, for the switching frame, since the switching flag in the bitstream can indicate the switching frame, there is no need to encode the indication information of the coding scheme of the switching frame.

Second implementation method: Encoding the indication information of two coding schemes

In this implementation, the coding end encodes the indication information of the initial coding scheme of the current frame into the bitstream. Taking the coding mode as the indication information as an example, the indication information encoded into the bitstream is essentially the coding mode consistent with the initial coding scheme, that is, the initial coding mode, and the initial coding mode is the first coding mode or the second coding mode. In addition, the coding end may not encode the switching flag.

Optionally, in the first implementation, the indication information of the initial coding scheme occupies one bit of the bitstream. Exemplarily, taking the coding mode as the indication information, the coding mode encoded into the bitstream is "0" or "1", wherein "0" indicates the DirAC mode, indicating that the initial coding scheme of the current frame is the first coding scheme, and "1" indicates the MP mode, indicating that the initial coding scheme of the current frame is the second coding scheme.

The third implementation method is to encode the indication information of three encoding schemes

In this implementation, the coding end encodes the indication information of the coding scheme of the current frame into the bitstream. Taking the coding mode as the indication information as an example, the indication information encoded into the bitstream is essentially the coding mode consistent with the coding scheme of the current frame, and the coding mode consistent with the coding scheme of the current frame is the actual coding mode, which is the first coding mode, the second coding mode or the third coding mode. Optionally, the third coding mode is the MP-W mode.

Optionally, in the third implementation, the indication information of the coding scheme of the current frame occupies two bits of the bitstream. Exemplarily, the indication information of the coding scheme of the current frame is "00", "01" or "10". Among them, "00" indicates that the coding scheme of the current frame is the first coding scheme, "01" indicates that the coding scheme of the current frame is the second coding scheme, and "10" indicates that the coding scheme of the current frame is the third coding scheme.

As can be seen from the above, in the first implementation, after the encoder determines the initial coding scheme of the current frame, it determines the value of the switching flag and encodes the value of the switching flag into the bitstream. In addition, the indication information of the initial coding scheme of the current frame is encoded into the bitstream, or, if the current frame is a switching frame, the encoder encodes the default indication information into the bitstream, and if the current frame is a non-switching frame, the encoder encodes the indication information of the initial coding scheme of the current frame into the bitstream. In the second implementation, after the encoder determines the initial coding scheme of the current frame, it directly encodes the indication information of the initial coding scheme of the current frame into the bitstream. In the third implementation method described above, after the coding end determines the initial coding scheme of the current frame, it determines the coding scheme of the current frame based on the initial coding scheme of the current frame and the initial coding scheme of the previous frame of the current frame, and encodes the indication information of the coding scheme of the current frame into the bitstream.

Step 602: If the coding scheme of the current frame is the third coding scheme, the signal of the designated channel in the HOA signal is encoded into the bitstream, and the designated channel is a part of all channels of the HOA signal.

In the embodiment of the present application, if the coding scheme of the current frame is the third coding scheme, indicating that the current frame is a switching frame, the coding end encodes the HOA signal of the current frame according to the third coding scheme (i.e., the hybrid coding scheme). Corresponding to the first implementation method in the above step 601, if the value of the switching flag of the current frame is the second value, it indicates that the current frame is a switching frame. Corresponding to the second implementation method in the above step 601, if the initial coding scheme of the current frame is different from the initial coding scheme of the previous frame of the current frame, it indicates that the current frame is a switching frame. Corresponding to the third implementation method in the above step 601, if the coding scheme of the current frame is the third coding scheme, the coding scheme of the current frame indicates that the current frame is a switching frame. For the switching frame, the encoder adopts the third coding scheme to encode the HOA signal of the current frame. The third coding scheme indicates that the signal of the specified channel in the HOA signal of the current frame is encoded into the bitstream, wherein the specified channel is a part of all channels of the HOA signal. That is, for the switching frame, the encoder encodes the signal of the specified channel in the HOA signal of the switching frame into the bitstream instead of using the first coding scheme or the second coding scheme to encode the switching frame. That is, this scheme adopts a compromise method to encode the switching frame in order to achieve a smooth transition of the auditory quality when the coding scheme is switched.

Optionally, the designated channel is consistent with the transmission channel preset in the first coding scheme, that is, the designated channel is the default channel. That is, under the premise that the third coding scheme is different from the second coding scheme, in order to make the coding effects of the third coding scheme close to those of the second coding scheme, the coding end encodes the signal of the channel that is the same as the transmission channel preset in the first coding scheme in the HOA signal of the switching frame into the bit stream, so that the auditory quality transitions as smoothly as possible. It should be noted that different transmission channels can be preset according to different coding bandwidths, bit rates, and even different application scenarios. Optionally, the preset transmission channels can also be the same under different coding bandwidths, bit rates or application scenarios.

Optionally, the signal of the designated channel includes a FOA signal, and the FOA signal includes an omnidirectional W signal, and directional X signal, Y signal, and Z signal. That is, the designated channel includes a FOA channel, and the signal of the FOA channel is a low-order signal, that is, if the current frame is a switching frame, the encoder encodes the low-order part of the HOA signal of the current frame into the bitstream, and the low-order part includes the W signal, X signal, Y signal, and Z signal of the FOA channel.

It should be noted that in the present application embodiment, there are many implementation methods for the encoding end to encode the signal of the designated channel in the HOA signal into the bitstream, and it is sufficient to encode the signal of the designated channel into the bitstream. Next, some of the implementation methods are introduced.

In the embodiment of the present application, if the designated channel includes the FOA channel, the encoding end determines the virtual speaker signal and the residual signal based on the W signal, the X signal, the Y signal and the Z signal, and encodes the virtual speaker signal and the residual signal into the bitstream.

Optionally, the coding end determines the W signal as a virtual speaker signal, determines three residual signals based on the W signal, the X signal, the Y signal and the Z signal, or determines the X signal, the Y signal and the Z signal as three residual signals. Optionally, the coding end determines the difference signals between any three signals of the W signal, the X signal, the Y signal and the Z signal and the remaining signal as three residual signals. For example, the coding end determines the difference signals between the X signal, the Y signal and the Z signal and the W signal as three residual signals. Exemplarily, the coding end uses the difference signals X', Y', and Z' obtained by X-W, Y-W, and Z-W as three residual signals.

If the encoder uses a core encoder to encode the current frame, the core encoder is a stereo encoder. Since the determined virtual speaker signal and the three residual signals are all mono signals, the encoder needs to first combine the stereo signals based on these mono signals, and then use the stereo encoder for encoding. Optionally, the encoder combines the virtual speaker signal with the first preset mono signal to obtain a stereo signal, and combines the three residual signals with the second preset mono signal to obtain two stereo signals. The encoder encodes the obtained three stereo signals into the bitstream respectively through the stereo encoder.

Among them, the embodiment of the present application does not limit the specific combination method of the encoder to combine the three-way residual signal with a preset mono signal to obtain two-way stereo signals. Optionally, the encoder combines the two most correlated residual signals among the three-way residual signals to obtain one stereo signal among the two-way stereo signals, and combines one residual signal other than the two most correlated residual signals among the three-way residual signals with a second preset mono signal to obtain another stereo signal among the two-way stereo signals. That is, the encoder combines the signals according to their correlation to obtain the stereo signal. In some other embodiments, the encoding end may also combine any two of the three residual signals to obtain one of the two stereo signals, and combine the remaining residual signal with the second preset mono signal to obtain the other of the two stereo signals.

Optionally, the first preset mono signal in the embodiment of the present application is an all-zero signal or an all-one signal, and the second preset mono signal is an all-zero signal or an all-one signal. Optionally, the first preset mono signal is the same as or different from the second preset mono signal, that is, the first preset mono signal and the second preset mono signal are all-zero signals or all-one signals, or the first preset mono signal is an all-zero signal and the second preset mono signal is an all-one signal, or the first preset mono signal is an all-one signal and the second preset mono signal is an all-zero signal. Among them, the all-zero signal includes a signal whose sampling point values are all zero or a signal whose frequency values are all zero, and the all-one signal includes a signal whose sampling point values are all one or a signal whose frequency values are all one. Wherein, if the HOA signal is a time domain signal, the all-zero signal includes a signal whose sampling point values are all zero, and the all-one signal includes a signal whose sampling point values are all one. If the HOA signal is a frequency domain signal, the all-zero signal includes a signal whose frequency point values are all zero, and the all-one signal includes a signal whose frequency point values are all one. In some other embodiments, the first preset mono signal and/or the second preset mono signal may also be other preset signals.

If the core codec used by the encoding end is a mono codec, the encoding end encodes the virtual speaker signal and each of the three residual signals into a bitstream through the mono codec.

FIG7 is a schematic diagram of a switching frame coding scheme provided by an embodiment of the present application. Referring to FIG7, the current frame to be coded is a switching frame, the coding end obtains the HOA signal of the current frame, uses the W signal in the HOA signal as a virtual speaker signal, and determines the residual signal according to the FOA signal in the HOA signal, such as determining the residual signal according to the X, Y, and Z signals in the HOA signal, or determining the residual signal according to the W signal and the X, Y, and Z signals. The coding end encodes the determined virtual speaker signal and the residual signal into the bitstream through the core encoder to obtain the bitstream of the switching frame.

Optionally, in other embodiments, the encoding end determines two signals among the W signal, the X signal, the Y signal, and the Z signal as two virtual speaker signals, and determines the remaining two signals as two residual signals. The encoding end combines the two virtual speaker signals to obtain one stereo signal, and combines the two residual signals to obtain another stereo signal. The encoding end encodes the obtained two stereo signals into bitstreams respectively through a stereo encoder.

Among them, the embodiment of the present application does not limit the specific combination method of the encoding end combining the W signal, the X signal, the Y signal and the Z signal in pairs to obtain two-way stereo signals. Optionally, the encoding end determines the W signal as a virtual speaker signal, and determines the signal with the highest correlation with the W signal among the X signal, the Y signal and the Z signal as another virtual speaker signal, that is, the W signal and the signal with the highest correlation with the W signal among the four signals included in the FOA channel are combined, and the remaining two signals are combined. Alternatively, the encoding end combines any two signals among the W signal, the X signal, the Y signal and the Z signal to obtain a stereo signal, and combines the remaining two signals to obtain another stereo signal.

It should be noted that the embodiment of the present application does not limit the specific implementation method of the encoding end using the core encoder to encode the virtual speaker signal and the residual signal, for example, it does not limit the number of encoding bits corresponding to the virtual speaker signal and the residual signal respectively.

The above describes the process of encoding the current frame by the encoder when the current frame is a switching frame, that is, the encoder encodes the signal of the specified channel in the HOA signal of the switching frame into the bitstream according to the third coding scheme, and the third coding scheme is the switching frame coding scheme. As can be seen from the above, in the embodiment of the present application, the signal of the specified channel may include a W signal, which is a core signal of the HOA signal. In this way, the switching frame coding scheme can also be called a coding scheme based on MP-W. Next, the process of encoding the current frame by the encoder when the current frame is a non-switching frame is described.

In the embodiment of the present application, if the coding scheme of the current frame is the first coding scheme, the coding end encodes the HOA signal of the current frame into the bitstream according to the first coding scheme. If the coding scheme of the current frame is the second coding scheme, the coding end encodes the HOA signal of the current frame into the bitstream according to the second coding scheme. That is, if the current frame is not a switching frame, the coding end uses the initial coding scheme of the current frame to encode the current frame.

Exemplarily, referring to FIG8 , the implementation process of the encoder encoding the HOA signal of the current frame into the bitstream according to the second coding scheme is as follows: the encoder selects a target virtual speaker that matches the HOA signal of the current frame from a virtual speaker set based on the MP algorithm, determines the virtual speaker signal through an MP-based spatial encoder based on the HOA signal of the current frame and the target virtual speaker, determines the residual signal through an MP-based spatial encoder based on the HOA signal of the current frame and the virtual speaker signal, and encodes the virtual speaker signal and the residual signal into the bitstream through a core encoder. It should be noted that the MP-based HOA coding scheme and the switching frame coding scheme have different principles and specific methods for determining the virtual speaker signal and the residual signal, and the virtual speaker signal and the residual signal determined by the two schemes are also different. For the same frame, the effective information encoded into the bitstream using the MP-based HOA coding scheme will be more than that using the switching frame coding scheme. On the premise that the switching frame coding scheme is different from the second coding scheme, in order to make the coding effects of the switching frame coding scheme and the second coding scheme close, the switching frame coding scheme also encodes the virtual speaker signal and the residual signal into the bitstream, so as to make the auditory quality transition as smooth as possible.

The implementation process of the encoder encoding the HOA signal of the current frame into the bitstream according to the first coding scheme is as follows: the encoder extracts the core layer signal and spatial parameters from the HOA signal of the current frame, and encodes the extracted core layer signal and spatial parameters into the bitstream. For example, referring to FIG9 , the encoder extracts the core layer signal from the HOA signal of the current frame through the core coding signal acquisition module, extracts the spatial parameters from the HOA signal of the current frame through the DirAC-based spatial parameter extraction module, encodes the core layer signal into the bitstream through the core encoder, and encodes the spatial parameters into the bitstream through the spatial parameter encoder. Among them, the channel corresponding to the core layer signal is consistent with the designated channel in this scheme. In addition, the first coding scheme not only encodes the core layer signal into the bitstream, but also encodes the extracted spatial parameters into the bitstream. The spatial parameters contain rich scene information, such as direction information. It can be seen that, for the same frame, the effective information encoded into the bitstream using the DirAC-based HOA coding scheme will be more than the effective information encoded into the bitstream using the switching frame coding scheme. Under the premise that the switching frame coding scheme is different from the first coding scheme, in order to make the coding effects of the switching frame coding scheme close to those of the first coding scheme, the switching frame coding scheme also encodes the signal of the transmission channel preset by the first coding scheme in the HOA signal into the bitstream, but will not encode more information in the HOA signal except the signal of the specified channel into the bitstream, that is, it will not extract spatial parameters, and will not encode spatial parameters into the bitstream, so as to make the transition of auditory quality as smooth as possible.

FIG10 is a flow chart of another coding method provided by the embodiment of the present application. Please refer to FIG10, and take encoding the indication information of the initial coding scheme of the current frame into the bitstream as an example to explain the coding method provided by the embodiment of the present application again. The coding end first obtains the HOA signal of the current frame to be encoded. Then, the coding end performs a sound field type analysis on the HOA signal to determine the initial coding scheme of the current frame, and the coding end encodes the indication information of the initial coding scheme of the current frame into the bitstream. The coding end determines whether the initial coding scheme of the current frame is the same as the initial coding scheme of the previous frame. If the initial coding scheme of the current frame is the same as the initial coding scheme of the previous frame, the coding end uses the initial coding scheme of the current frame to encode the HOA signal of the current frame to obtain the code stream of the current frame. If the initial coding scheme of the current frame is different from the initial coding scheme of the previous frame, the coding end uses the switching frame coding scheme to encode the HOA signal of the current frame to obtain the code stream of the current frame.

It should be noted that if the current frame is the first audio frame to be encoded, the initial coding scheme of the current frame is the first coding scheme or the second coding scheme, and the encoding end uses the initial coding scheme of the current frame to encode the HOA signal of the current frame into the bitstream.

In summary, in the embodiment of the present application, two schemes (i.e., a coding and decoding scheme based on virtual speaker selection and a coding and decoding scheme based on directional audio coding) are combined to encode and decode the HOA signal of the audio frame, that is, appropriate coding and decoding schemes are selected for different audio frames, which can improve the compression rate of the audio signal. At the same time, in order to ensure a smooth transition of auditory quality when switching between different coding and decoding schemes, this scheme does not directly adopt any of the above two schemes for encoding certain audio frames, but adopts a new coding and decoding scheme to encode and decode these audio frames, that is, encodes the signals of the specified channels in the HOA signals of these audio frames into the bit stream, that is, adopts a compromise scheme for encoding and decoding, so that the auditory quality after rendering and playing the decoded and restored HOA signals can be smoothly transitioned.

FIG11 is a flow chart of a decoding method provided by an embodiment of the present application, and the method is applied to a decoding end. It should be noted that the decoding method corresponds to the encoding method shown in FIG6. Referring to FIG11, the method includes the following steps.

Step 1101: Obtain a decoding solution for the current frame based on the bitstream.

The decoding scheme of the current frame is one of the first decoding scheme, the second decoding scheme and the third decoding scheme. The first decoding scheme is the HOA decoding scheme based on DirAC, the second decoding scheme is the HOA decoding scheme based on virtual speaker selection, and the third decoding scheme is a hybrid decoding scheme. Optionally, the hybrid decoding scheme is also called a switching frame decoding scheme.

It should be noted that since the encoder uses different coding schemes to encode different audio frames, the decoder also needs to use the corresponding decoding scheme to decode each audio frame.

Next, we will first introduce how the decoding end determines the coding scheme of the current frame. As mentioned above, in step 601 of the coding method shown in FIG6 , three implementation methods are introduced in which the coding end encodes information that can be used to indicate the coding scheme of the current frame into the bitstream. Correspondingly, there are also three corresponding implementation methods for the decoding end to determine the coding scheme of the current frame, which will be introduced next.

The first implementation encodes the switching flag and the indication information of the two encoding schemes.

The decoding end first parses the value of the switching flag of the current frame from the bitstream. If the value of the switching flag is the first value, the decoding end then parses the indication information of the decoding scheme of the current frame from the bitstream, and the indication information is used to indicate that the decoding scheme of the current frame is the first decoding scheme or the second decoding scheme. If the value of the switching flag is the second value, the decoding end determines that the decoding scheme of the current frame is the third decoding scheme. It should be noted that the indication information of the coding scheme encoded into the bitstream by the encoding end is the indication information of the decoding scheme parsed from the bitstream by the decoding end.

In other words, if the decoder parses the switching flag value of the current frame as the first value, it means that the current frame is a non-switching frame. The decoder then parses the indication information of the decoding scheme from the bitstream and determines the decoding scheme of the current frame based on the indication information. If the decoder parses the switching flag value of the current frame as the second value, it means that the current frame is a switching frame. Even if the bitstream contains indication information, the decoder does not need to decode the indication information.

It should be noted that if the value of the switching flag is the second value, the decoding end determines that the decoding scheme of the current frame is the switching frame decoding scheme, and the current frame is a switching frame. The switching frame decoding scheme is a decoding scheme different from the first decoding scheme and the second decoding scheme. The switching frame decoding scheme is for a smooth transition of auditory quality.

Optionally, in the first implementation, the indication information of the decoding scheme and the switching flag each occupy one bit of the code stream. Exemplarily, the decoding end first parses the value of the switching flag of the current frame from the code stream. If the value of the parsed switching flag is "0", that is, the value of the switching flag is the first value, the decoding end then parses the indication information of the decoding scheme of the current frame from the code stream. If the parsed indication information is "0", the decoding end determines that the decoding scheme of the current frame is the first decoding scheme. If the parsed indication information is "1", the decoding end determines that the decoding scheme of the current frame is the second decoding scheme. If the parsed switching flag is the value "1", the decoding end determines that the decoding scheme of the current frame is the switching frame decoding scheme (the third decoding scheme).

The second implementation encodes the indication information of the two coding schemes.

The decoding end parses the initial decoding scheme of the current frame from the bitstream, and the initial decoding scheme is the first decoding scheme or the second decoding scheme. If the initial decoding scheme of the current frame is the same as the initial decoding scheme of the previous frame of the current frame, the decoding scheme of the current frame is determined to be the initial decoding scheme of the current frame. If the initial decoding scheme of the current frame is different from the initial decoding scheme of the previous frame of the current frame, the decoding scheme of the current frame is determined to be the third decoding scheme, that is, the hybrid decoding scheme. The difference between the initial decoding scheme of the current frame and the initial decoding scheme of the previous frame of the current frame means that the initial decoding scheme of the current frame is the first decoding scheme and the initial decoding scheme of the previous frame of the current frame is the second decoding scheme, or the initial decoding scheme of the current frame is the second decoding scheme and the initial decoding scheme of the previous frame of the current frame is the first decoding scheme. That is, one of the initial decoding scheme of the current frame and the initial decoding scheme of the previous frame of the current frame is the first decoding scheme, and the other is the second decoding scheme.

Optionally, in the second implementation, the indication information for indicating the initial coding scheme occupies one bit of the code stream. For example, the coding mode is used as the indication information, and the coding mode in the code stream occupies one bit. Exemplarily, the decoding end parses the indication information of the initial coding scheme of the current frame from the code stream. If the parsed indication information is "0" and the indication information of the previous frame of the current frame is also "0", the decoding end determines that the decoding scheme of the current frame is the first decoding scheme. If the parsed indication information is "1" and the indication information of the previous frame of the current frame is also "1", the decoding end determines that the decoding scheme of the current frame is the second decoding scheme. If the parsed indication information is "0" and the indication information of the previous frame of the current frame is "1", or the parsed indication information is "1" and the indication information of the previous frame of the current frame is "0", the decoding end determines that the decoding scheme of the current frame is the switching frame decoding scheme.

Optionally, the indication information of the initial decoding scheme of the frame before the current frame is cached data. When decoding the current frame, the decoding end can obtain the indication information of the initial decoding scheme of the frame before the current frame from the cache.

The third implementation encodes the indication information of three coding schemes.

The decoding end parses the bitstream to obtain indication information of the decoding scheme of the current frame, where the indication information is used to indicate whether the decoding scheme of the current frame is the first decoding scheme, the second decoding scheme or the third decoding scheme.

Optionally, in the third implementation, the indication information of the decoding scheme occupies two bits of the bitstream. For example, assuming that the coding mode is used as the indication information, the coding mode of the current frame occupies two bits of the bitstream.

For example, the decoder parses the indication information of the decoding scheme of the current frame from the bitstream. If the indication information parsed is "00", the decoder determines that the decoding scheme of the current frame is the first decoding scheme. If the indication information parsed is "01", the decoder determines that the decoding scheme of the current frame is the second decoding scheme. If the indication information parsed is "10", the decoder determines that the decoding scheme of the current frame is the switching frame decoding scheme.

Step 1102: If the decoding scheme of the current frame is the third decoding scheme, the signal of the designated channel in the HOA signal of the current frame is determined based on the bit stream, and the designated channel is a part of all channels of the HOA signal.

In the embodiment of the present application, after the decoder obtains the decoding scheme of the current frame, if the decoding scheme of the current frame is the third coding scheme, indicating that the current frame is a switching frame, the decoder determines the signal of the designated channel in the HOA signal of the current frame based on the bitstream. That is, for the switching frame, the encoder encodes the signal of the designated channel into the bitstream, then the decoder adopts the switching frame decoding scheme to decode the switching frame, that is, it is necessary to parse the signal of the designated channel from the bitstream first.

Next, the implementation process of decoding the switching frame by the decoding end using the switching frame decoding scheme is introduced in detail, that is, the implementation process of the decoding end determining the signal of the specified channel in the HOA signal of the current frame based on the bit stream when the current frame is a switching frame.

It should be noted that the process of the decoder determining the signal of the specified channel in the HOA signal of the current frame based on the bitstream is symmetrical to the process of the encoder encoding the signal of the specified channel in the HOA signal of the current frame into the bitstream. In the above-mentioned embodiment of the encoding method, some implementation processes of encoding the signal of the specified channel into the bitstream are introduced, and the decoding process symmetrical to these implementation processes will be introduced at the decoder.

In the embodiment of the present application, if the encoding end first determines the virtual speaker signal and the residual signal based on the signal of the designated channel, and then encodes the virtual speaker signal and the residual signal into the bit stream, then, correspondingly, the decoding end first determines the virtual speaker signal and the residual signal based on the bit stream, and then determines the signal of the designated channel based on the virtual speaker signal and the residual signal.

Optionally, if the encoding end encodes three-way stereo signals obtained based on the combination of the virtual speaker signal and the residual signal into a bitstream through a stereo encoder, then the decoding end decodes the bitstream through the stereo decoder to obtain three-way stereo signals, and then determines one virtual speaker signal and three-way residual signals based on the three-way stereo signals. Optionally, the decoding end determines one virtual speaker signal based on one stereo signal among the three-way stereo signals, and determines three-way residual signals based on the other two stereo signals among the three-way stereo signals. That is, the decoding end first parses the three-way stereo sound signal from the bit stream, and then disassembles the three-way stereo sound signal to obtain one virtual speaker signal and three-way residual signals.

For example, the decoder parses three stereo signals from the bitstream, namely S1, S2 and S3, where S1 is obtained by combining one virtual speaker signal and one preset mono signal, S2 is obtained by combining two residual signals, and S3 is obtained by combining one remaining residual signal and one preset mono signal. The decoder disassembles S1 to obtain one virtual speaker signal, disassembles S2 to obtain two residual signals, and disassembles S3 to obtain one remaining residual signal.

Optionally, if the encoding end encodes four-channel mono signals determined based on the virtual speaker signal and the residual signal into the bit stream through a mono encoder, then the decoding end decodes the bit stream through a mono decoder to obtain one virtual speaker signal and three residual signals, and the four-channel mono signals include the one virtual speaker signal and the three residual signals.

Optionally, if the signal of the designated channel includes an FOA signal, and the FOA signal includes an omnidirectional W signal, and directional X signal, Y signal, and Z signal, then after the decoder determines the virtual speaker signal and the residual signal based on the bitstream, the decoder determines the W signal based on the virtual speaker signal. The decoder determines the X signal, the Y signal, and the Z signal based on the residual signal and the W signal, or the decoder determines the X signal, the Y signal, and the Z signal based on the residual signal. For example, when the decoder parses out three residual signals, the sum of the three residual signals and the W signal is determined as the X signal, the Y signal, and the Z signal, or the three residual signals are determined as the X signal, the Y signal, and the Z signal, respectively. If the encoder determines the difference signals between the X signal, the Y signal, and the Z signal and the W signal as three-way residual signals, the decoder determines the sum of the three-way residual signals and the W signal as the X signal, the Y signal, and the Z signal. If the encoder determines the X signal, the Y signal, and the Z signal as three-way residual signals, the decoder determines the three-way residual signals as the X signal, the Y signal, and the Z signal. That is, the decoding process of the decoder matches the encoding process of the encoder.

If the encoding end encodes two stereo signals determined based on the virtual speaker signal and the residual signal into a bitstream through a stereo encoder, then the decoding end decodes the bitstream through a stereo decoder to obtain the two stereo signals. The decoding end determines two virtual speaker signals based on one of the two stereo signals, and determines two residual signals based on the other of the two stereo signals. The two virtual speaker signals and the two residual signals include a W signal, an X signal, a Y signal, and a Z signal. Optionally, if the encoding end determines the W signal and the one signal with the highest correlation with the W signal among the X signal, the Y signal, and the Z signal as two virtual speaker signals, the two virtual speaker signals determined by the decoding end include the W signal and the one signal with the highest correlation with the W signal among the X signal, the Y signal, and the Z signal. Assuming that the one signal with the highest correlation with the W signal among the X signal, the Y signal, and the Z signal is the X signal, the two virtual speaker signals determined by the decoding end include the W signal and the X signal, and the two residual signals determined by the decoding end include the Y signal and the Z signal.

Step 1103: Based on the signal of the designated channel, determine the gain of one or more remaining channels other than the designated channel in the HOA signal of the current frame.

In the embodiment of the present application, after the decoding end determines the signal of the designated channel in the HOA signal of the current frame based on the code stream, it determines the gain of one or more residual channels in the HOA signal other than the designated channel based on the signal of the designated channel.

Exemplarily, assuming that the designated channel is the FOA channel, the FOA channel can be called a low-order channel, the signal of the FOA channel can be called the low-order part of the HOA signal, one or more remaining channels in the HOA signal except the designated channel are called high-order channels, and the signal of the high-order channel can be called the high-order part of the HOA signal. Then, the decoding end determines the high-order gain of the HOA signal, that is, the gain of the high-order channel, based on the low-order part of the HOA signal.

Optionally, the decoder first performs analysis filtering on the signal of the designated channel in the HOA signal to obtain the signal of the designated channel after analysis filtering, and determines the gain of the one or more remaining channels based on the signal of the designated channel after analysis filtering. For example, assuming that the signal of the designated channel is the low-order part of the HOA signal, the decoder first performs analysis filtering on the low-order part of the HOA signal to obtain the low-order part of the HOA signal after analysis filtering, and then estimates the high-order gain based on the low-order part of the HOA signal after analysis filtering. Optionally, in this solution, for the switching frame, the analysis filter used by the decoding end for analysis filtering processing is the same as the analysis filter used in the DirAC-based HOA decoding solution, so that the decoding delay of the switching frame can be consistent with the decoding delay of the DirAC-based HOA decoding solution, that is, the delay is aligned. It should be noted that the decoding delay mentioned in this article is the end-to-end encoding and decoding delay, and the decoding delay can also be called the encoding delay.

It should be noted that in the embodiment of the present application, the decoding end determines the gain of one or more residual channels other than the designated channel in the HOA signal based on the signal of the designated channel, that is, the process of estimating the gain of the residual channel based on the signal of the designated channel. The specific implementation method is the same as the residual channel gain estimation method in the DirAC-based coding and decoding scheme, and the embodiment of the present application is not described in detail. Exemplarily, in this scheme, for switching frames, the method for estimating the high-order gain based on the low-order part of the HOA signal by the decoding end is the same as the high-order gain estimation method in the DirAC-based coding and decoding scheme.

Step 1104: Determine the signal of each of the one or more residual channels based on the signal of the designated channel and the gain of the one or more residual channels.

In the embodiment of the present application, the decoder determines the signal of each residual channel in the one or more residual channels based on the signal of the designated channel and the gain of the one or more residual channels. Exemplarily, assuming that the signal of the designated channel is the low-order part in the HOA signal, and the gain of the one or more residual channels is the high-order gain, then the decoder can determine the high-order part in the HOA signal based on the W signal and the high-order gain in the low-order part. Alternatively, if the decoder performs analysis filtering on the low-order part of the HOA signal, then the decoder can determine the high-order part of the HOA signal after analysis filtering based on the W signal and the high-order gain in the low-order part of the HOA signal after analysis filtering.

Step 1105: Obtain a reconstructed HOA signal of the current frame based on the signal of the designated channel and the signals of the one or more remaining channels.

In the embodiment of the present application, after obtaining the signal of the designated channel and the signal of the one or more residual channels, the decoder obtains the reconstructed HOA signal of the current frame based on the signal of the designated channel and the signal of the one or more residual channels, that is, reconstructs the HOA signal of the current frame. Exemplarily, the decoder performs synthetic filtering processing on the signal of the designated channel and the signal of the one or more residual channels to obtain the reconstructed HOA signal of the current frame. For example, assuming that the signal of the designated channel is the low-order part of the HOA signal, and the signal of the one or more residual channels is the high-order part of the HOA signal, then the decoder performs synthetic filtering processing on the low-order part and the high-order part of the HOA signal to obtain the reconstructed HOA signal of the current frame. Alternatively, if the decoder performs analysis filtering on the low-order part of the HOA signal, the decoder performs synthesis filtering on the low-order part of the HOA signal after analysis filtering and the high-order part of the HOA signal after analysis filtering to obtain the reconstructed HOA signal of the current frame. Optionally, in this scheme, for the switching frame, the synthesis filter used by the decoder for synthesis filtering is the same as the synthesis filter used in the HOA encoding and decoding scheme based on DirAC, so that the decoding delay of the switching frame can be consistent with the decoding delay of the HOA decoding scheme based on DirAC, that is, the delay is aligned.

FIG12 is a schematic diagram of a switching frame decoding scheme provided by an embodiment of the present application. Referring to FIG12 , the current frame to be decoded is a switching frame. Assuming that the signal of the designated channel is the low-order part of the HOA signal, then, in the decoding process, the decoding end obtains the code stream of the current frame to be decoded, and performs core decoding on the code stream through the core decoder to reconstruct the low-order part of the HOA signal of the current frame, and adopts a method similar to that of determining the high-order part in the HOA decoding scheme based on DirAC, and estimates the high-order part based on the low-order part, that is, reconstructs the high-order part of the HOA signal. Afterwards, the decoding end reconstructs the HOA signal based on the decoded low-order part and the estimated high-order part.

The above describes the process of decoding the current frame at the decoder when the current frame is a switching frame, that is, the decoder uses a switching frame decoding scheme to decode the switching frame, that is, the decoder first decodes the signal of the specified channel in the HOA signal (such as the low-level part), and then reconstructs the signals of each remaining channel (such as reconstructing the high-level part). Next, the process of decoding the current frame at the decoder when the current frame is a non-switching frame is described.

In the embodiment of the present application, after the decoding end determines the decoding scheme of the current frame, if the decoding scheme of the current frame is the first decoding scheme, the decoding end obtains the reconstructed HOA signal of the current frame according to the bit stream according to the first decoding scheme. If the decoding scheme of the current frame is the second decoding scheme, the decoding end obtains the reconstructed HOA signal of the current frame according to the bit stream according to the second decoding scheme.

In the embodiment of the present application, referring to FIG13, the decoding end obtains the reconstructed HOA signal of the current frame according to the bitstream according to the second decoding scheme in the following process: the decoding end parses the virtual speaker signal and the residual signal from the bitstream through the core decoder, and sends the parsed virtual speaker signal and the residual signal to the MP-based spatial decoder to obtain the reconstructed HOA signal of the current frame. It should be noted that the decoding scheme shown in FIG13 corresponds to the encoding scheme shown in FIG8.

The decoding end obtains the reconstructed HOA signal of the current frame according to the bitstream in accordance with the first decoding scheme, and the implementation process is as follows: the decoding end parses the core layer signal and the spatial parameter from the bitstream, and reconstructs the HOA signal of the current frame based on the core layer signal and the spatial parameter. For example, referring to FIG14, the decoding end parses the core layer signal from the bitstream through the core decoder, and parses the spatial parameter from the bitstream through the spatial parameter decoder, and performs DirAC-based HOA signal synthesis processing based on the parsed core layer signal and spatial parameter to obtain the reconstructed HOA signal of the current frame. It should be noted that the decoding scheme shown in FIG14 corresponds to the coding scheme shown in FIG9.

Optionally, since the high-order part of the HOA signal has a greater impact on the auditory quality, in order to further achieve a smooth transition of the auditory quality when switching between different coding and decoding schemes, the decoder can also adjust the gain of the high-order part of the current frame in the process of obtaining the reconstructed HOA signal of the current frame according to the bit stream according to the second decoding scheme. For example, the decoder obtains the initial HOA signal according to the bit stream according to the second decoding scheme. If the decoding scheme of the previous frame of the current frame is the third decoding scheme, that is, the previous frame of the current frame is a switching frame, the decoder adjusts the gain of the high-order part of the initial HOA signal according to the high-order gain of the previous frame of the current frame. Then, the decoder obtains the reconstructed HOA signal of the current frame based on the low-order part of the initial HOA signal and the gain-adjusted high-order part.

It should be noted that if the previous frame of the current frame is a switching frame, the current frame uses the high-order gain of the previous frame to adjust the gain of the high-order part of the initial HOA signal of the current frame, so that the gain-adjusted high-order part of the current frame is similar to the high-order part of the previous frame, such as the gain adjustment makes the energy of the high-order part of the HOA signal of the two adjacent frames similar. In this way, in the process of rendering and playing each audio frame at the subsequent decoding end, the auditory quality of the switching frame and the auditory quality of the next frame of the switching frame can be smoothly transitioned.

Optionally, in addition to performing high-order gain adjustment on the audio frames whose decoding scheme is the second decoding scheme after the switching frame, for other audio frames whose decoding scheme is the second decoding scheme, the decoding end may also perform gain adjustment on the high-order part of the HOA signals of these audio frames, and the embodiment of the present application does not limit the specific implementation method of gain adjustment on the high-order part of the HOA signals of these audio frames. Optionally, in addition to performing gain adjustment on the high-order part, the decoding end may also perform gain adjustment on other parts of the HOA signals of these audio frames. That is, the embodiment of the present application does not limit which channels of the HOA signal are gain-adjusted. In other words, the decoding end can perform gain adjustment on the signal of any one or more channels in the HOA signal, and the one or more channels may include part or all of the high-order channels, or part or all of the remaining channels except the specified channel, or other channels.

FIG15 is a flow chart of another decoding method provided by an embodiment of the present application. Referring to FIG15 , taking the case where the encoder encodes the indication information of the initial coding scheme into the bitstream, and assuming that the switching flag is not encoded in the bitstream, during the decoding process, the decoder first parses the indication information of the initial decoding scheme of the current frame from the bitstream. Then, the decoder determines whether the initial decoding scheme of the current frame is the same as the initial decoding scheme of the previous frame. If the initial decoding scheme of the current frame is the same as the initial decoding scheme of the previous frame, indicating that the current frame is a non-switching frame, the decoder uses the initial decoding scheme of the current frame to decode the bitstream to obtain the reconstructed HOA signal of the current frame. If the initial decoding scheme of the current frame is different from the initial decoding scheme of the previous frame, it means that the current frame is a switching frame. Then the decoding end adopts the switching frame decoding scheme to decode the bit stream to obtain the reconstructed HOA signal of the current frame.

In summary, in the embodiment of the present application, two schemes (i.e., a coding and decoding scheme based on virtual speaker selection and a coding and decoding scheme based on directional audio coding) are combined to encode and decode the HOA signal of the audio frame, that is, appropriate coding and decoding schemes are selected for different audio frames, which can improve the compression rate of the audio signal. At the same time, in order to ensure a smooth transition of auditory quality when switching between different coding and decoding schemes, this scheme does not directly adopt any of the above two schemes for encoding and decoding for some audio frames, but adopts a new coding and decoding scheme to encode and decode these audio frames, that is, the signals of the specified channels in the HOA signals of these audio frames are encoded into the bit stream during encoding, that is, a compromise scheme is adopted for encoding and decoding, so that the auditory quality after rendering and playing the decoded and restored HOA signals can be smoothly transitioned.

FIG16 is a schematic diagram of the structure of a coding device 1600 provided in an embodiment of the present application. The coding device 1600 can be implemented by software, hardware, or a combination of both to form part or all of a coding end device, and the coding end device can be any coding end device in the aforementioned embodiments. Referring to FIG16 , the device 1600 includes: a first determination module 1601 and a first coding module 1602.

The first determination module 1601 is used to determine a coding scheme of the current frame according to the high-order stereo reverberation HOA signal of the current frame, and the coding scheme of the current frame is one of a first coding scheme, a second coding scheme and a third coding scheme; wherein the first coding scheme is an HOA coding scheme based on directional audio coding, the second coding scheme is an HOA coding scheme based on virtual speaker selection, and the third coding scheme is a hybrid coding scheme;

The first encoding module 1602 is used to encode the signal of the specified channel in the HOA signal into the bit stream if the encoding scheme of the current frame is the third encoding scheme, and the specified channel is a part of all channels of the HOA signal.

Optionally, the signal of the designated channel includes a first-order stereo reverberation FOA signal, and the FOA signal includes an omnidirectional W signal, and directional X signal, Y signal, and Z signal.

Optionally, the first encoding module 1602 includes: a first determining submodule, configured to determine a virtual speaker signal and a residual signal based on the W signal, the X signal, the Y signal, and the Z signal; and a coding submodule, configured to encode the virtual speaker signal and the residual signal into a bitstream.

Optionally, the first determining submodule is used to: determine the W signal as one virtual speaker signal; determine three residual signals based on the W signal, the X signal, the Y signal and the Z signal, or determine the X signal, the Y signal and the Z signal as three residual signals.

Optionally, the encoding submodule is used to: combine the virtual speaker signal with the first preset mono signal to obtain a stereo signal; combine the three residual signals with the second preset mono signal to obtain two stereo signals; and encode the obtained three stereo signals into bit streams respectively through the stereo encoder.

Optionally, the coding submodule is used to: combine the two most correlated residual signals among the three residual signals to obtain one stereo signal among the two stereo signals; and combine a residual signal other than the two most correlated residual signals among the three residual signals with the second preset mono signal to obtain the other stereo signal among the two stereo signals.

Optionally, the first preset mono signal is an all-zero signal or an all-one signal, the all-zero signal includes a signal whose sampling point values are all zero or a signal whose frequency values are all zero, and the all-one signal includes a signal whose sampling point values are all one or a signal whose frequency values are all one; the second preset mono signal is an all-zero signal or an all-one signal; the first preset mono signal is the same as or different from the second preset mono signal.

Optionally, the encoding submodule is used to encode the virtual speaker signal and each of the three residual signals into a bitstream respectively through a mono encoder.

Optionally, the device 1600 further includes: a second coding module, used to encode the HOA signal into the bitstream according to the first coding scheme if the coding scheme of the current frame is the first coding scheme; and a third coding module, used to encode the HOA signal into the bitstream according to the second coding scheme if the coding scheme of the current frame is the second coding scheme.

Optionally, the first determination module 1601 includes: a second determination submodule, used to determine an initial coding scheme of the current frame according to the HOA signal, the initial coding scheme being the first coding scheme or the second coding scheme; a third determination submodule, used to determine that the coding scheme of the current frame is the initial coding scheme of the current frame if the initial coding scheme of the current frame is the same as the initial coding scheme of the previous frame of the current frame; and a fourth determination submodule, used to determine that the coding scheme of the current frame is the third coding scheme if the initial coding scheme of the current frame is the first coding scheme and the initial coding scheme of the previous frame of the current frame is the second coding scheme, or the initial coding scheme of the current frame is the second coding scheme and the initial coding scheme of the previous frame of the current frame is the first coding scheme.

Optionally, the device 1600 further includes: a fourth encoding module, used to encode indication information of the initial encoding scheme of the current frame into the bitstream.

Optionally, the device 1600 further includes: a second determination module, used to determine the value of the switching flag of the current frame, when the coding scheme of the current frame is the first coding scheme or the second coding scheme, the value of the switching flag of the current frame is the first value; when the coding scheme of the current frame is the third coding scheme, the value of the switching flag of the current frame is the second value; a fifth coding module, used to encode the value of the switching flag into the bitstream.

Optionally, the device 1600 further includes: a sixth encoding module, used to encode indication information of the encoding scheme of the current frame into the bitstream.

Optionally, the designated channel is consistent with a transmission channel preset in the first coding scheme.

In the embodiment of the present application, two schemes (i.e., a coding and decoding scheme based on virtual speaker selection and a coding and decoding scheme based on directional audio coding) are combined to encode and decode the HOA signal of the audio frame, that is, appropriate coding and decoding schemes are selected for different audio frames, which can improve the compression rate of the audio signal. At the same time, in order to ensure a smooth transition of auditory quality when switching between different coding and decoding schemes, this scheme does not directly adopt any of the above two schemes for encoding and decoding for some audio frames, but adopts a new coding and decoding scheme to encode and decode these audio frames, that is, encodes the signals of the specified channels in the HOA signals of these audio frames into the bit stream, that is, adopts a compromise scheme for encoding and decoding, so that the auditory quality after rendering and playing the decoded and restored HOA signals can be smoothly transitioned.

It should be noted that: the coding device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate when encoding audio frames. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the coding device and the coding method embodiment provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

FIG17 is a schematic diagram of the structure of a decoding device 1700 provided in an embodiment of the present application. The decoding device 1700 can be implemented by software, hardware, or a combination of the two to become part or all of a decoding end device, and the decoding end device can be any encoding end device in the aforementioned embodiments. Referring to FIG17 , the decoding device 1700 includes: a first acquisition module 1701, a first determination module 1702, a second determination module 1703, a third determination module 1704, and a second acquisition module 1705.

The first acquisition module 1701 is used to obtain a decoding scheme of the current frame based on the bit stream, and the decoding scheme of the current frame is one of the first decoding scheme, the second decoding scheme and the third decoding scheme; wherein the first decoding scheme is a high-order stereo reverberation HOA decoding scheme based on directional audio decoding, the second decoding scheme is an HOA decoding scheme based on virtual speaker selection, and the third decoding scheme is a hybrid decoding scheme; the first determination module 1702 is used to determine the signal of a specified channel in the HOA signal of the current frame based on the bit stream if the decoding scheme of the current frame is the third decoding scheme, and the specified channel is a part of all channels of the HOA signal; the second determination module 1703 is used to determine the HOA based on the signal of the specified channel The signal is used to determine the gain of one or more residual channels other than the designated channel in the signal; a third determination module 1704 is used to determine the signal of each residual channel in the one or more residual channels based on the signal of the designated channel and the gain of the one or more residual channels; a second acquisition module 1705 is used to obtain the reconstructed HOA signal of the current frame based on the signal of the designated channel and the signal of the one or more residual channels.

Optionally, the first determination module 1702 includes: a first determination submodule, used to determine a virtual speaker signal and a residual signal based on a bit stream; and a second determination submodule, used to determine a signal of a designated channel based on the virtual speaker signal and the residual signal.

Optionally, the first determination submodule is used to: decode the code stream through a stereo decoder to obtain three-way stereo signals; and determine one virtual speaker signal and three-way residual signals based on the three-way stereo signals.

Optionally, the first determination submodule is used to: determine a virtual speaker signal based on one of the three stereo sound signals; and determine three residual signals based on the other two of the three stereo sound signals.

Optionally, the first determination submodule is used to: decode the code stream through a mono decoder to obtain one virtual speaker signal and three residual signals.

Optionally, the signal of the designated channel includes a first-order stereo reverberation FOA signal, the FOA signal includes an omnidirectional W signal, and directional X, Y, and Z signals; the first determination submodule is used to: determine the W signal based on the virtual speaker signal; determine the X, Y, and Z signals based on the residual signal and the W signal, or determine the X, Y, and Z signals based on the residual signal.

Optionally, the device 1700 further includes: a first decoding module, used to obtain a reconstructed HOA signal of the current frame according to the bit stream according to the first decoding scheme if the decoding scheme of the current frame is the first decoding scheme; and a second decoding module, used to obtain a reconstructed HOA signal of the current frame according to the bit stream according to the second decoding scheme if the decoding scheme of the current frame is the second decoding scheme.

Optionally, the second decoding module includes: a first acquisition submodule, used to obtain an initial HOA signal according to the bit stream according to the second decoding scheme; a gain adjustment submodule, used to adjust the gain of the high-order part of the initial HOA signal according to the high-order gain of the previous frame of the current frame if the decoding scheme of the previous frame of the current frame is the third decoding scheme; and a second acquisition submodule, used to obtain a reconstructed HOA signal based on the low-order part of the initial HOA signal and the gain-adjusted high-order part.

Optionally, the first acquisition module 1701 includes: a first parsing submodule, used to parse the value of the switching flag of the current frame from the bit stream; a second parsing submodule, used to parse the indication information of the decoding scheme of the current frame from the bit stream if the value of the switching flag is the first value, the indication information is used to indicate that the decoding scheme of the current frame is the first decoding scheme or the second decoding scheme; a third determination submodule, used to determine that the decoding scheme of the current frame is the third decoding scheme if the value of the switching flag is the second value.

Optionally, the first acquisition module 1701 includes: a third parsing submodule, used to parse out indication information of a decoding scheme of a current frame from the bit stream, wherein the indication information is used to indicate whether the decoding scheme of the current frame is the first decoding scheme, the second decoding scheme or the third decoding scheme.

Optionally, the first acquisition module 1701 includes: a fourth parsing submodule, used to parse out an initial decoding scheme of the current frame from the bit stream, the initial decoding scheme being the first decoding scheme or the second decoding scheme; a fourth determining submodule, used to determine that the decoding scheme of the current frame is the initial decoding scheme of the current frame if the initial decoding scheme of the current frame is the same as the initial decoding scheme of the previous frame of the current frame; and a fifth determining submodule, used to determine that the decoding scheme of the current frame is the third decoding scheme if the initial decoding scheme of the current frame is the first decoding scheme and the initial decoding scheme of the previous frame of the current frame is the second decoding scheme, or the initial decoding scheme of the current frame is the second decoding scheme and the initial decoding scheme of the previous frame of the current frame is the first decoding scheme.

In the embodiment of the present application, two schemes (i.e., a coding and decoding scheme based on virtual speaker selection and a coding and decoding scheme based on directional audio coding) are combined to encode and decode the HOA signal of the audio frame, that is, appropriate coding and decoding schemes are selected for different audio frames, which can improve the compression rate of the audio signal. At the same time, in order to ensure a smooth transition of auditory quality when switching between different coding and decoding schemes, this scheme does not directly adopt any of the above two schemes for encoding and decoding for some audio frames, but adopts a new coding and decoding scheme to encode and decode these audio frames, that is, encodes the signals of the specified channels in the HOA signals of these audio frames into the bit stream, that is, adopts a compromise scheme for encoding and decoding, so that the auditory quality after rendering and playing the decoded and restored HOA signals can be smoothly transitioned.

It should be noted that: the decoding device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate when decoding audio frames. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the decoding device and the decoding method embodiment provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

FIG18 is a schematic block diagram of a coding and decoding device 1800 used in an embodiment of the present application. The coding and decoding device 1800 may include a processor 1801, a memory 1802, and a bus system 1803. The processor 1801 and the memory 1802 are connected via the bus system 1803. The memory 1802 is used to store instructions, and the processor 1801 is used to execute the instructions stored in the memory 1802 to execute various coding or decoding methods described in the embodiment of the present application. To avoid repetition, it will not be described in detail here.

In the present application embodiment, the processor 1801 may be a central processing unit (CPU), or other general-purpose processors, DSP, ASIC, FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

The memory 1802 may include a ROM device or a RAM device. Any other suitable type of storage device may also be used as the memory 1802. The memory 1802 may include code and data 18021 accessed by the processor 1801 using a bus 1803. The memory 1802 may further include an operating system 18023 and an application 18022, which includes at least one program that allows the processor 1801 to execute the encoding or decoding method described in the embodiment of the present application. For example, the application 18022 may include applications 1 to N, which further include an encoding or decoding application (referred to as a codec application for short) that executes the encoding or decoding method described in the embodiment of the present application.

In addition to the data bus, the bus system 1803 may also include a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, all the buses are labeled as the bus system 1803 in the figure.

Optionally, the codec 1800 may also include one or more output devices, such as a display 1804. In one example, the display 1804 may be a touch display that combines a display with a touch unit operable to sense touch input. The display 1804 may be connected to the processor 1801 via a bus 1803.

It should be noted that the encoding and decoding device 1800 can execute the encoding method in the embodiment of this application, and can also execute the decoding method in the embodiment of this application.

Those skilled in the art will appreciate that the functions described in conjunction with the various illustrative logic blocks, modules, and algorithm steps disclosed herein may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions described in the various illustrative logic blocks, modules, and steps may be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or a communication medium including any medium that facilitates the transfer of a computer program from one place to another (e.g., based on a communication protocol). In this manner, computer-readable media generally may correspond to (1) non-transitory, tangible computer-readable storage media, or (2) communications media, such as signals or carrier waves. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementing the techniques described in this application. A computer program product may include computer-readable media.

By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, disk storage devices or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave are used to transmit instructions from a website, server, or other remote source, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. However, it should be understood that the computer-readable storage media and data storage media do not include connections, carriers, signals, or other transient media, but are actually directed to non-transient tangible storage media. As used herein, disks and discs include compact discs (CDs), laser discs, optical discs, DVDs, and Blu-ray discs, where disks typically reproduce data magnetically and optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor" as used herein may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functions described by the various illustrative logic blocks, modules, and steps described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated in a combined transcoder. Moreover, the techniques may be fully implemented in one or more circuits or logic elements. In one example, various illustrative logic blocks, units, and modules in the encoder 100 and the decoder 200 can be understood as corresponding circuit devices or logic elements.

The technology of the embodiments of the present application can be implemented in a variety of devices or equipment, including a wireless handset, an integrated circuit (IC) or a set of ICs (e.g., a chipset). The various components, modules or units described in the embodiments of the present application are to emphasize the functional aspects of the device for executing the disclosed technology, but they do not necessarily need to be implemented by different hardware units. In fact, as described above, the various units can be combined with appropriate software and/or firmware in a codec hardware unit, or provided by an interoperating hardware unit (including one or more processors as described above).

That is to say, in the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server, data center, etc. that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiment of the present application may be a non-volatile storage medium, in other words, a non-transient storage medium.

It should be understood that the "at least one" mentioned in this article refers to one or more, and "multiple" refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, "/" means or, for example, A/B can mean A or B; the "and/or" in this article is only a description of the association relationship between related objects, indicating that three relationships can exist, for example, A and/or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first" and "second" are used to distinguish the same or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not necessarily limit the differences.

The above description is an embodiment provided for this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application shall be included in the scope of protection of this application.

10: Source device 101: First terminal 120: Data source 100: Encoder 140: Output interface 20: Destination device 200: Decoder 201: Second terminal 220: Display device 240: Input interface 30: Link 40: Storage device 601, 602, 1101, 1102, 1103, 1104, 1105: Steps 1600: Encoding device 1601: First determination module 1602: First encoding module 1700: Decoding device 1701: First acquisition module 1702: First determination module 1703: Second determination module 1704: Third determination module 1705: Second acquisition module 1800: Codec device 1801: Processor 1802: Memory 18021: Data 18022: Application 18023: Operating system 1803: Bus system 1804: Display

FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application. FIG. 2 is a schematic diagram of an implementation environment of a terminal scenario provided by an embodiment of the present application. FIG. 3 is a schematic diagram of an implementation environment of a transcoding scenario of a wireless or core network device provided by an embodiment of the present application. FIG. 4 is a schematic diagram of an implementation environment of a broadcast television scenario provided by an embodiment of the present application. FIG. 5 is a schematic diagram of an implementation environment of a virtual reality streaming scenario provided by an embodiment of the present application. FIG. 6 is a flow chart of a coding method provided by an embodiment of the present application. FIG. 7 is a schematic diagram of a switching frame coding scheme provided by an embodiment of the present application. FIG. 8 is a schematic diagram of an HOA coding scheme based on virtual speaker selection provided by an embodiment of the present application. FIG. 9 is a schematic diagram of a HOA coding scheme based on DirAC provided by an embodiment of the present application. FIG. 10 is a flow chart of another coding method provided by an embodiment of the present application. FIG. 11 is a flow chart of a decoding method provided by an embodiment of the present application. FIG. 12 is a schematic diagram of a switching frame decoding scheme provided by an embodiment of the present application. FIG. 13 is a schematic diagram of an HOA decoding scheme based on virtual speaker selection provided by an embodiment of the present application. FIG. 14 is a schematic diagram of an HOA decoding scheme based on DirAC provided by an embodiment of the present application. FIG. 15 is a flow chart of another decoding method provided by an embodiment of the present application. FIG. 16 is a structural schematic diagram of a coding device provided by an embodiment of the present application. FIG. 17 is a structural schematic diagram of a decoding device provided by an embodiment of the present application. Figure 18 is a schematic block diagram of a coding and decoding device provided in an embodiment of the present application.

601, 602: Steps

Claims (48)

A coding method, comprising: determining a coding scheme of the current frame according to a high-order stereo reverberation HOA signal of the current frame, the coding scheme of the current frame being one of a first coding scheme, a second coding scheme and a third coding scheme, wherein the first coding scheme is an HOA coding scheme based on directional audio coding, the second coding scheme is an HOA coding scheme based on virtual speaker selection, and the third coding scheme is a hybrid coding scheme; and if the coding scheme of the current frame is the third coding scheme, encoding a signal of a designated channel in the HOA signal into a bitstream, the designated channel being a part of all channels of the HOA signal; wherein determining the coding scheme of the current frame according to the high-order stereo reverberation HOA signal of the current frame The coding scheme of the previous frame includes: determining the initial coding scheme of the current frame according to the HOA signal, the initial coding scheme being the first coding scheme or the second coding scheme; if the initial coding scheme of the current frame is the same as the initial coding scheme of the previous frame of the current frame, determining the coding scheme of the current frame to be the initial coding scheme of the current frame; and if the initial coding scheme of the current frame is the first coding scheme and the initial coding scheme of the previous frame of the current frame is the second coding scheme, or the initial coding scheme of the current frame is the second coding scheme and the initial coding scheme of the previous frame of the current frame is the first coding scheme, determining the coding scheme of the current frame to be the third coding scheme. The encoding method as described in claim 1, wherein the signal of the designated channel includes a first-order stereo reverberation FOA signal, and the FOA signal includes an omnidirectional W signal, and directional X signal, Y signal, and Z signal. The encoding method as described in claim 2, wherein encoding the signal of the specified channel in the HOA signal into the code stream includes: determining a virtual speaker signal and a residual signal based on the W signal, the X signal, the Y signal, and the Z signal; and encoding the virtual speaker signal and the residual signal into the code stream. The coding method as described in claim 3, wherein the determining of the virtual speaker signal and the residual signal based on the W signal, the X signal, the Y signal and the Z signal comprises: determining the W signal as one of the virtual speaker signals; and determining three of the residual signals based on the W signal, the X signal, the Y signal and the Z signal, or determining the X signal, the Y signal and the Z signal as three of the residual signals. The encoding method as described in claim 4, wherein the encoding of the virtual speaker signal and the residual signal into the bitstream comprises: combining the one virtual speaker signal with the first preset mono signal to obtain one stereo signal; combining the three residual signals with the second preset mono signal to obtain two stereo signals; and encoding the obtained three stereo signals into the bitstream respectively through the stereo encoder. The coding method as described in claim 5, wherein the combining the three-channel residual signal with the second preset mono signal to obtain two-channel stereo signals comprises: combining the two channels of residual signals with the highest correlation among the three-channel residual signals to obtain one channel of the two-channel stereo signals; and combining a channel of residual signal other than the two channels of residual signals with the highest correlation among the three-channel residual signal with the second preset mono signal to obtain another channel of the two-channel stereo signals. The encoding method as described in claim 5, wherein the first preset mono signal is an all-zero signal or an all-one signal, the all-zero signal includes a signal whose sampling point values are all zero or a signal whose frequency points are all zero, and the all-one signal includes a signal whose sampling point values are all one or a signal whose frequency points are all one; the second preset mono signal is an all-zero signal or an all-one signal; and the first preset mono signal is the same as or different from the second preset mono signal. The encoding method as described in claim 4, wherein encoding the virtual speaker signal and the residual signal into the bitstream comprises: encoding the one-channel virtual speaker signal and each of the three residual signals into the bitstream respectively through a mono encoder. The coding method as described in any one of claim items 1-8, wherein after determining the coding scheme of the current frame according to the high-order stereo reverberation HOA signal of the current frame, the coding method further comprises: if the coding scheme of the current frame is the first coding scheme, encoding the HOA signal into the bitstream according to the first coding scheme; and if the coding scheme of the current frame is the second coding scheme, encoding the HOA signal into the bitstream according to the second coding scheme. The coding method as described in claim 1, wherein after determining the initial coding scheme of the current frame according to the HOA signal, further comprises: encoding the indication information of the initial coding scheme of the current frame into the bitstream. The coding method as described in any one of claim items 1-8, wherein after determining the coding scheme of the current frame according to the high-order stereo reverberation HOA signal of the current frame, further comprises: determining the value of the switching flag of the current frame, when the coding scheme of the current frame is the first coding scheme or the second coding scheme, the value of the switching flag of the current frame is the first value; when the coding scheme of the current frame is the third coding scheme, the value of the switching flag of the current frame is the second value; and encoding the value of the switching flag into the bitstream. The coding method as described in any one of claim items 1-8, wherein after determining the coding scheme of the current frame according to the HOA signal of the current frame, further comprises: encoding the indication information of the coding scheme of the current frame into the bitstream. A coding method as described in any one of claim items 1-8, wherein the specified channel is consistent with the transmission channel preset in the first coding scheme. A decoding method, comprising: obtaining a decoding scheme of a current frame based on a bit stream, wherein the decoding scheme of the current frame is one of a first decoding scheme, a second decoding scheme and a third decoding scheme; wherein the first decoding scheme is a high-order stereo reverberation HOA decoding scheme based on directional audio decoding, the second decoding scheme is an HOA decoding scheme based on virtual speaker selection, and the third decoding scheme is a hybrid decoding scheme; if the decoding scheme of the current frame is the third decoding scheme, then determining a signal of a specified channel in the HOA signal of the current frame based on the bit stream, and the specified channel The channel is a part of all channels of the HOA signal; based on the signal of the designated channel, the gain of one or more residual channels in the HOA signal except the designated channel is determined; based on the signal of the designated channel and the gain of the one or more residual channels, the signal of each residual channel in the one or more residual channels is determined; and based on the signal of the designated channel and the signal of the one or more residual channels, the reconstructed HOA signal of the current frame is obtained; wherein the decoding scheme for obtaining the current frame based on the code stream includes: parsing out the the value of the switching flag of the current frame; if the value of the switching flag is the first value, parsing the indication information of the decoding scheme of the current frame from the code stream, the indication information is used to indicate that the decoding scheme of the current frame is the first decoding scheme or the second decoding scheme; and if the value of the switching flag is the second value, determining that the decoding scheme of the current frame is the third decoding scheme; wherein the decoding scheme of the current frame based on the code stream includes: parsing the initial decoding scheme of the current frame from the code stream, the initial decoding scheme is the first decoding scheme or the third decoding scheme the second decoding scheme; if the initial decoding scheme of the current frame is the same as the initial decoding scheme of the previous frame of the current frame, the decoding scheme of the current frame is determined to be the initial decoding scheme of the current frame; and if the initial decoding scheme of the current frame is the first decoding scheme and the initial decoding scheme of the previous frame of the current frame is the second decoding scheme, or the initial decoding scheme of the current frame is the second decoding scheme and the initial decoding scheme of the previous frame of the current frame is the first decoding scheme, the decoding scheme of the current frame is determined to be the third decoding scheme. The decoding method as described in claim 14, wherein the signal of the designated channel in the HOA signal of the current frame based on the code stream comprises: determining a virtual speaker signal and a residual signal based on the code stream; and determining the signal of the designated channel based on the virtual speaker signal and the residual signal. The decoding method as described in claim 15, wherein the determining of the virtual speaker signal and the residual signal based on the bit stream comprises: decoding the bit stream by a stereo decoder to obtain three-way stereo signals; and Based on the three-way stereo signals, determining one of the virtual speaker signals and three of the residual signals. The decoding method as described in claim 16, wherein the determining one of the virtual speaker signals and the three residual signals based on the three stereo signals comprises: determining the one of the virtual speaker signals based on one of the three stereo signals; and determining the three residual signals based on the other two of the three stereo signals. The decoding method as described in claim 15, wherein the determining of the virtual speaker signal and the residual signal based on the bit stream comprises: decoding the bit stream by a mono decoder to obtain one channel of the virtual speaker signal and three channels of the residual signal. The decoding method of claim 15, wherein the signal of the designated channel includes a first-order stereo reverberation FOA signal, and the FOA signal includes an omnidirectional W signal, and directional X signal, Y signal, and Z signal; the signal of the designated channel is determined based on the virtual speaker signal and the residual signal, including: determining the W signal based on the virtual speaker signal; and determining the X signal, the Y signal, and the Z signal based on the residual signal and the W signal, or determining the X signal, the Y signal, and the Z signal based on the residual signal. The decoding method as described in any one of claim items 14-19 further comprises: if the decoding scheme of the current frame is the first decoding scheme, then according to the first decoding scheme, the reconstructed HOA signal of the current frame is obtained according to the code stream; and if the decoding scheme of the current frame is the second decoding scheme, then according to the second decoding scheme, the reconstructed HOA signal of the current frame is obtained according to the code stream. The decoding method as described in claim 20, wherein the step of obtaining the reconstructed HOA signal of the current frame according to the bitstream according to the second decoding scheme comprises: obtaining the initial HOA signal according to the bitstream according to the second decoding scheme; if the decoding scheme of the previous frame of the current frame is the third decoding scheme, adjusting the gain of the high-order part of the initial HOA signal according to the high-order gain of the previous frame of the current frame; and obtaining the reconstructed HOA signal based on the low-order part and the gain-adjusted high-order part of the initial HOA signal. A decoding method as described in any one of claim items 14-19, wherein the decoding scheme for the current frame obtained based on the code stream includes: parsing indication information of the decoding scheme for the current frame from the code stream, wherein the indication information is used to indicate that the decoding scheme for the current frame is the first decoding scheme, the second decoding scheme, or the third decoding scheme. A coding device comprises: a first determining module, used for determining a coding scheme of the current frame according to a high-order stereo reverberation HOA signal of the current frame, wherein the coding scheme of the current frame is one of a first coding scheme, a second coding scheme and a third coding scheme, wherein the first coding scheme is an HOA coding scheme based on directional audio coding, the second coding scheme is an HOA coding scheme based on virtual speaker selection, and the third coding scheme is a hybrid coding scheme; and a first coding module, used for encoding a signal of a specified channel in the HOA signal into a bitstream if the coding scheme of the current frame is the third coding scheme, wherein the specified channel is a part of all channels of the HOA signal; wherein the first determining module comprises: a second determining submodule , used to determine the initial coding scheme of the current frame according to the HOA signal, the initial coding scheme being the first coding scheme or the second coding scheme; a third determination submodule, used to determine the coding scheme of the current frame as the initial coding scheme of the current frame if the initial coding scheme of the current frame is the same as the initial coding scheme of the previous frame of the current frame; and a fourth determination submodule, used to determine the coding scheme of the current frame as the third coding scheme if the initial coding scheme of the current frame is the first coding scheme and the initial coding scheme of the previous frame of the current frame is the second coding scheme, or the initial coding scheme of the current frame is the second coding scheme and the initial coding scheme of the previous frame of the current frame is the first coding scheme. The encoding device as described in claim 23, wherein the signal of the designated channel includes a first-order stereo reverberation FOA signal, and the FOA signal includes an omnidirectional W signal, and directional X signal, Y signal, and Z signal. The coding device as described in claim 24, wherein the first coding module includes: a first determination submodule for determining a virtual speaker signal and a residual signal based on the W signal, the X signal, the Y signal, and the Z signal; and a coding submodule for encoding the virtual speaker signal and the residual signal into the bitstream. The coding device as described in claim 25, wherein the first determination submodule is used to: determine the W signal as one of the virtual speaker signals; and determine the difference signals between the X signal, the Y signal, and the Z signal and the W signal as three of the residual signals, or determine the X signal, the Y signal, and the Z signal as three of the residual signals. The coding device as described in claim 26, wherein the coding submodule is used to: combine the one-way virtual speaker signal with the first-way preset mono signal to obtain one-way stereo signal; combine the three-way residual signal with the second-way preset mono signal to obtain two-way stereo signals; and encode the obtained three-way stereo signals into the code stream respectively through the stereo encoder. The coding device as claimed in claim 27, wherein the coding submodule is used to: combine the two residual signals with the highest correlation among the three residual signals to obtain one stereo signal among the two stereo signals; and combine one residual signal among the three residual signals except the two residual signals with the highest correlation with the second preset mono signal to obtain another stereo signal among the two stereo signals. The coding device as described in claim 27, wherein the first preset mono signal is an all-zero signal or an all-one signal, the all-zero signal includes a signal whose sampling point values are all zero or a signal whose frequency points are all zero, and the all-one signal includes a signal whose sampling point values are all one or a signal whose frequency points are all one; the second preset mono signal is an all-zero signal or an all-one signal; and the first preset mono signal is the same as or different from the second preset mono signal. The coding device as described in claim 26, wherein the coding submodule is used to: encode the one-channel virtual speaker signal and each of the three-channel residual signals into the bitstream respectively through a mono encoder. The coding device as described in any one of claim items 23-30 further comprises: a second coding module, for encoding the HOA signal into the bitstream according to the first coding scheme if the coding scheme of the current frame is the first coding scheme; and a third coding module, for encoding the HOA signal into the bitstream according to the second coding scheme if the coding scheme of the current frame is the second coding scheme. The coding device as described in claim 31 further comprises: a fourth coding module for encoding the indication information of the initial coding scheme of the current frame into the bitstream. The coding device as described in any one of claim 23-30 further comprises: a second determination module, used to determine the value of the switching flag of the current frame, when the coding scheme of the current frame is the first coding scheme or the second coding scheme, the value of the switching flag of the current frame is the first value; when the coding scheme of the current frame is the third coding scheme, the value of the switching flag of the current frame is the second value; and a fifth coding module, used to encode the value of the switching flag into the bitstream. The coding device as described in any one of claim 23-30 further comprises: a sixth coding module, used to encode the indication information of the coding scheme of the current frame into the bit stream. A coding device as described in any one of claim items 23-30, wherein the designated channel is consistent with the transmission channel preset in the first coding scheme. A decoding device comprises: a first acquisition module, used for obtaining a decoding scheme of a current frame based on a bit stream, wherein the decoding scheme of the current frame is one of a first decoding scheme, a second decoding scheme and a third decoding scheme; wherein the first decoding scheme is a high-order stereophonic HOA decoding scheme based on directional audio decoding, the second decoding scheme is an HOA decoding scheme based on virtual speaker selection, and the third decoding scheme is a hybrid decoding scheme; a first determination module, used for determining a signal of a specified channel in the HOA signal of the current frame based on a bit stream if the decoding scheme of the current frame is the third decoding scheme, wherein the specified channel is the specified channel. a first determining module for determining the gain of one or more residual channels in the HOA signal except the designated channel based on the signal of the designated channel; a third determining module for determining the signal of each residual channel in the one or more residual channels based on the signal of the designated channel and the gain of the one or more residual channels; and a second obtaining module for obtaining the reconstructed HOA signal of the current frame based on the signal of the designated channel and the signal of the one or more residual channels; wherein the first obtaining module includes: a first parsing submodule for obtaining the reconstructed HOA signal of the current frame from the bitstream; parsing the value of the switching flag of the current frame; a second parsing submodule, for parsing the indication information of the decoding scheme of the current frame from the code stream if the value of the switching flag is the first value, the indication information is used to indicate that the decoding scheme of the current frame is the first decoding scheme or the second decoding scheme; and a third determining submodule, for determining that the decoding scheme of the current frame is the third decoding scheme if the value of the switching flag is the second value; wherein the first obtaining module includes: a fourth parsing submodule, for parsing the initial decoding scheme of the current frame from the code stream, the initial decoding scheme is the first decoding scheme or the second decoding scheme; a fourth determination submodule, for determining the decoding scheme of the current frame to be the initial decoding scheme of the current frame if the initial decoding scheme of the current frame is the same as the initial decoding scheme of the previous frame of the current frame; and a fifth determination submodule, for determining the decoding scheme of the current frame to be the third decoding scheme if the initial decoding scheme of the current frame is the first decoding scheme and the initial decoding scheme of the previous frame of the current frame is the second decoding scheme, or the initial decoding scheme of the current frame is the second decoding scheme and the initial decoding scheme of the previous frame of the current frame is the first decoding scheme. A decoding device as described in claim 36, wherein the first determination module includes: a first determination submodule for determining a virtual speaker signal and a residual signal based on the bitstream; and a second determination submodule for determining the signal of the specified channel based on the virtual speaker signal and the residual signal. The decoding device as described in claim 37, wherein the first determination submodule is used to: decode the bit stream through a stereo decoder to obtain three-way stereo signals; and determine one of the virtual speaker signals and three of the residual signals based on the three-way stereo signals. A decoding device as described in claim 38, wherein the first determination submodule is used to: determine the one-way virtual speaker signal based on one of the three-way stereo signals; and determine the three-way residual signal based on the other two of the three-way stereo signals. The decoding device as described in claim 37, wherein the first determination submodule is used to: decode the code stream through a mono decoder to obtain one channel of the virtual speaker signal and three channels of the residual signal. A decoding device as described in any one of claim items 37-40, wherein the signal of the designated channel includes a first-order stereo reverberation FOA signal, and the FOA signal includes an omnidirectional W signal, and directional X signal, Y signal, and Z signal; the first determination submodule is used to: determine the W signal based on the virtual speaker signal; and determine the X signal, the Y signal, and the Z signal based on the residual signal and the W signal, or determine the X signal, the Y signal, and the Z signal based on the residual signal. The decoding device as described in any one of claim items 36-40 further comprises: a first decoding module, for obtaining the reconstructed HOA signal of the current frame according to the bit stream according to the first decoding scheme if the decoding scheme of the current frame is the first decoding scheme; and a second decoding module, for obtaining the reconstructed HOA signal of the current frame according to the bit stream according to the second decoding scheme if the decoding scheme of the current frame is the second decoding scheme. A decoding device as described in claim 42, wherein the second decoding module comprises: a first acquisition submodule, used to obtain an initial HOA signal according to the bit stream according to the second decoding scheme; a gain adjustment submodule, used to adjust the gain of the high-order part of the initial HOA signal according to the high-order gain of the previous frame of the current frame if the decoding scheme of the previous frame of the current frame is the third decoding scheme; and a second acquisition submodule, used to obtain the reconstructed HOA signal based on the low-order part and the gain-adjusted high-order part of the initial HOA signal. A decoding device as described in any one of claim items 36-40, wherein the first acquisition module includes: a third parsing submodule, used to parse the code stream to obtain indication information of the decoding scheme of the current frame, and the indication information is used to indicate that the decoding scheme of the current frame is the first decoding scheme, the second decoding scheme, or the third decoding scheme. A coding end device, wherein the coding end device includes a memory and a processor; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the coding method described in any one of claim items 1-13. A decoding end device, wherein the decoding end device includes a memory and a processor; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the decoding method described in any one of claim items 14-22. A computer-readable storage medium, wherein the storage medium stores instructions, and when the instructions are executed on the computer, the computer executes the steps of the method described in any one of request items 1-22. A computer program product, wherein the computer program product comprises instructions, which when executed by a processor implement the method described in any one of claims 1-22.