JP7389651B2 - Variable alphabet size in digital audio signals - Google Patents

Description

(Cross reference to related applications)
This application has priority to U.S. Patent Application No. 15/926,089, filed March 20, 2018, which claims the benefit of U.S. Provisional Patent Application No. 62/489,867, filed April 25, 2017. , the entire disclosures of which are incorporated herein by reference.

TECHNICAL FIELD This disclosure relates to encoding or decoding audio signals.

Audio codecs can encode time-domain audio signals into digital files or streams and decode digital files or streams into time-domain audio signals. Continuing efforts are being made to improve audio codecs, including reducing the size of encoded files or streams.

One embodiment of an encoding system includes a processor and a memory device storing instructions executable by the processor, the instructions being operable by the processor to perform a method for encoding an audio signal. a memory device executable, the method comprising: receiving a digital audio signal; and parsing the digital audio signal into a plurality of frames each including a specified number of audio samples. transforming the audio samples of each frame to generate a plurality of frequency domain coefficients for each frame; and dividing the plurality of frequency domain coefficients for each frame into a plurality of bands for each frame. each band having a reshaping parameter representing a time resolution and a frequency resolution; and encoding the digital audio signal into a bitstream including the reshaping parameters, the reshaping for the first band. The parameters are encoded using a first alphabet size, and the reshaping parameters for a second band different from the first band are encoded using a second alphabet size different from the first alphabet size. and outputting a bitstream.

One embodiment of a decoding system includes a processor and a memory device storing instructions executable by the processor, the instructions being operable by the processor to perform a method for decoding an encoded audio signal. and a memory device, the method comprising: receiving a bitstream including a plurality of frames each divided into a plurality of bands; extracting from the bitstream reconstruction parameters representative of temporal and frequency resolution for a first band, the reconstruction parameters for a first band being embedded in the bitstream using a first alphabet size; A reshaping parameter for a second band different from the first band is embedded in a bitstream using a second alphabet size different from the first alphabet size; decoding the stream to generate a decoded digital audio signal.

Another embodiment of the encoding system includes a receiver circuit for receiving a digital audio signal and a framer for parsing the digital audio signal into a plurality of frames each containing a specified number of audio samples. a transformer circuit for transforming the audio samples of each frame to generate a plurality of frequency domain coefficients for each frame and dividing the plurality of frequency domain coefficients for each frame into a plurality of bands for each frame; a frequency band divider circuit for encoding a digital audio signal, the frequency band divider circuit having a reshaping parameter representing a time resolution and a frequency resolution, each band having a reshaping parameter for each band; An encoder circuit for bitstreaming, wherein a reshaping parameter for a first band is encoded using a first alphabet size, and a reshaping parameter for a second band different from the first band is , an encoder circuit encoded using a second alphabet size different from the first alphabet size, and an output circuit for outputting a bitstream.

1 illustrates a block diagram of one embodiment of an encoding system according to some embodiments. FIG. FIG. 5 illustrates a block diagram of another embodiment of an encoding system according to some embodiments. 1 illustrates a block diagram of one embodiment of a decoding system according to some embodiments. FIG. FIG. 5 illustrates a block diagram of another embodiment of a decoding system according to some embodiments. 4 illustrates some of the quantities associated with encoding a digital audio signal, according to some embodiments. 1 illustrates a flowchart of one embodiment of a method for encoding an audio signal, according to some embodiments. 1 illustrates a flowchart of one embodiment of a method for decoding an encoded audio signal, according to some embodiments. 4 illustrates an example of pseudocode for encoding and decoding an audio signal, according to some embodiments. 4 illustrates an example of pseudocode for encoding and decoding an audio signal, according to some embodiments. 4 illustrates an example of pseudocode for encoding and decoding an audio signal, according to some embodiments. 4 illustrates an example of pseudocode for encoding and decoding an audio signal, according to some embodiments. 1 illustrates a block diagram of one embodiment of an encoding system according to some embodiments. FIG.

Corresponding reference numbers indicate corresponding elements throughout the several figures. The elements in the drawings are not necessarily drawn to scale. The configurations shown in the drawings should be construed as illustrative only and not as limiting the scope of the invention in any way.

In audio encoding and/or decoding systems, such as codecs, reshaping parameters in different bands can be encoded using alphabets with different sizes. Using different alphabet sizes may allow for more compact compression in a bitstream (eg, an encoded digital audio signal), as described in more detail below.

FIG. 1 depicts a block diagram of one embodiment of an encoding system 100, according to some embodiments. The configuration of FIG. 1 is only one example of an encoding system; other suitable configurations may also be used.

Encoding system 100 can receive digital audio signal 102 as input and output a bitstream 104. Input signal 102 and output signal 104 each include one or more separate files stored locally or on an accessible server and/or one or more files generated locally or on an accessible server. Can contain audio streams.

Encoding system 100 may include a processor 106. Encoding system 100 may further include a memory device 108 that stores instructions 110 executable by processor 106. Instructions 110 may be executed by processor 106 to perform a method for encoding an audio signal. One embodiment of such a method for encoding audio signals is described in detail below.

In the configuration of FIG. 1, the encoding is performed in software, typically by a processor in the computing device that can also perform additional tasks. Alternatively, the encoding can also be performed in hardware, such as by a dedicated chip or processor embedded in the hardware to perform the encoding. An example of such a hardware-based encoder is shown in FIG.

FIG. 2 depicts a block diagram of another embodiment of an encoding system 200, according to some embodiments. The configuration of FIG. 2 is only one example of an encoding system; other suitable configurations may also be used.

Encoding system 200 can receive a digital audio signal 202 as input and output a bitstream 204. Encoding system 200 may include a dedicated encoding processor 206, which may include a chip embedded in hardware to perform a particular encoding method. An example of such a method for encoding an audio signal is described in detail below.

The embodiments of FIGS. 1 and 2 illustrate encoding systems that can operate in software and hardware, respectively. Figures 3 and 4 below illustrate equivalent decoding systems that can operate in software and hardware, respectively.

FIG. 3 depicts a block diagram of one embodiment of a decoding system according to some embodiments. The configuration of FIG. 3 is only one example of a decoding system; other suitable configurations may also be used.

Decoding system 300 can receive a bitstream 302 as input and output a decoded digital audio signal 304. Input signal 302 and output signal 304 each include one or more separate files stored locally or on an accessible server and/or one or more files generated locally or on an accessible server. Can contain audio streams.

Decoding system 300 can include a processor 306. Decoding system 300 may further include a memory device 308 that stores instructions 310 executable by processor 306. Instructions 310 may be executed by processor 306 to perform a method for decoding an audio signal. An example of such a method for decoding an audio signal is described in detail below.

In the configuration of FIG. 3, decoding is performed in software, typically by a processor in the computing device that can also perform additional tasks. Alternatively, decoding can also be performed in hardware, such as by a dedicated chip or processor embedded in the hardware to perform the encoding. An example of such a hardware-based decoder is shown in FIG.

FIG. 4 depicts a block diagram of another embodiment of a decoding system 400, according to some embodiments. The configuration of FIG. 4 is only one example of a decoding system; other suitable configurations may also be used.

Decoding system 400 can receive a bitstream 402 as input and output a decoded digital audio signal 404. Decoding system 400 may include a dedicated decoding processor 406 and may include a chip embedded in hardware to perform a particular decoding method. An example of such a method for decoding an audio signal is described in detail below.

FIG. 5 illustrates some of the quantities associated with encoding a digital audio signal, according to some embodiments. Decoding a bitstream generally involves the same amount as encoding a bitstream, but with the mathematical operations performed in reverse. The amounts shown in FIG. 5 are only one example of such amounts; other suitable amounts can be used as well. Each of the quantities shown in FIG. 5 can be used with any of the encoders or decoders shown in FIGS. 1-4.

The encoder can receive a digital audio signal 502. Digital audio signal 502 is in the time domain and may include a series of integer or floating point numbers representing audio signal amplitude as it evolves over time. Digital audio signal 502 may be in the form of a stream (eg, without a designated start and/or end), such as a live broadcast from a studio. Alternatively, the digital audio signal 502 may be an individual file (e.g., with a start and end and a specified duration).

The encoder may parse the digital audio signal 502 into multiple frames 504, where each frame 504 includes a specified number of audio samples 506. For example, frame 504 may include 1024 samples 506 or another suitable value. In general, grouping digital audio signal 502 into frames 504 allows an encoder to efficiently apply encoder processing to a well-defined number of samples 506. In some embodiments, such processing may be different for each frame, such that each frame is processed independently of other frames.

The encoder may perform a transformation 508 of the audio samples 506 of each frame 504. In some embodiments, this transform may be a modified discrete cosine transform. Other suitable transforms such as Fourier, Laplace, etc. can be used. Transform 508 converts time domain quantities, such as samples 506 within frame 504, to frequency domain quantities, such as frequency domain coefficients 510 for frame 504. Transform 508 may generate multiple frequency domain coefficients 510 for each frame 504. In some examples, the number of frequency domain coefficients 510 produced by transform 508 may be equal to the number of samples 506 in the frame, such as 1024. Frequency domain coefficients 510 describe how much signals of a particular frequency are present within a frame.

In some embodiments, a time-domain frame can be further divided into subblocks of consecutive samples, and a transform can be applied to each subblock. For example, a frame of 1024 samples can be divided into 8 subblocks of 128 samples each, and each such subblock can be transformed into a block of 128 frequency coefficients. can. Transforms for instances where a frame is divided into subblocks are sometimes referred to as short transforms. For examples where the frame is not divided into subblocks, the transform may be referred to as a long transform.

The encoder may divide the frequency domain coefficients 510 for each frame 504 into multiple bands 512 for each frame 504. In some embodiments, there may be 22 bands 512 per frame, but other values can be used as well. Each band 512 represents a range of frequencies 510 within frame 504 such that all frequency ranges concatenated can include all frequencies represented within frame 504. In an example using a short transform, each resulting block of frequency coefficients can be divided into the same number of bands that can correspond one-to-one with the bands used for the long transform. In an example using a short transform, the number of coefficients in a given band within a block is proportionally less compared to the number of coefficients in that given band in the case of a long transform. For example, a frame can be divided into eight subblocks, where the band in the short transform block has one-eighth the number of coefficients in the corresponding band in the long transform. A band in the long transform can have 32 coefficients, and in the short transform the same band can have 4 coefficients in each of the 8 frequency blocks. A band in the short transform may relate to an 8x4 matrix with a resolution of 8 in the time domain and 4 in the frequency domain. A band in the long transform may relate to a 1x32 matrix with a resolution of 1 in the time domain and 32 in the frequency domain. Accordingly, each band 512 may include a reshaping parameter 518 representing a temporal resolution 514 and a frequency resolution 516. In some examples, the reshaping parameters 518 can represent the temporal resolution 514 and frequency resolution 516 by providing values for variations from default values of the temporal resolution 514 and frequency resolution 516.

In general, the goal of a codec is to create a frequency-domain representation of a particular frame, but a time-domain representation of this frame using a limited amount of data, governed by the particular data rate or bit rate of the encoded file. The goal is to ensure that it is represented as accurately as possible. For example, the data transfer rate may include 1411 kbps (kilobits per second), 320 kbps, 256 kbps, 192 kbps, 160 kbps, 128 kbps, or other values. Generally, the higher the data rate, the more accurate the representation of the frame.

In pursuing the goal of increasing accuracy using only limited data rates, codecs can trade off between time and frequency resolution for each band. For example, a codec may double the time resolution of a particular band while halving the frequency resolution of that band. Performing such operations (eg, exchanging time resolution for frequency resolution, or vice versa) may be referred to as reshaping the time-frequency structure of the band. In general, in the initial transformation, the time resolution of all bands may be the same, but after reshaping, the time-frequency structure of one band in a frame is independent of the time-frequency structure of other bands in this frame. Since each band can be reshaped independently of the other bands.

In some examples, each band can have a size equal to the product of the band's time resolution 514 and the band's frequency resolution 516. In some examples, the temporal resolution 514 of one band may be equal to eight audio samples and the temporal resolution 514 of another band may be equal to one audio sample. Other suitable temporal resolutions 514 can be used as well.

In some embodiments, the encoder adjusts the time resolution 514 and frequency resolution of each band of each frame without changing the size of the band (e.g., without changing the product of time resolution 514 and frequency resolution 516). 516 can be adjusted in a complementary manner. The encoder can quantify this adjustment using reshaping parameters.

The reshaping parameter can be a selected integer. For example, if the reconstruction parameter is 3, the time resolution can be multiplied by the quantity 2 ³ and the frequency resolution can be multiplied by the quantity 2 ^-3 . Other suitable integers can be used, including positive integers (meaning the time resolution 514 increases and frequency resolution 516 decreases), negative integers (meaning the time resolution decreases and the frequency resolution 516 decreases), and negative integers (meaning the time resolution 514 increases and the frequency resolution 516 decreases). (meaning increasing), and zero (meaning that the time resolution 514 and frequency resolution 516 do not change, e.g., multiplying by the amount ²⁰ ).

In some embodiments, the number of allowed reshaping parameter values may be limited to a finite number of integers. As a particular example, allowed reshaping parameter values may include 0, 1, 2, and 3, for a total of four integers. As another particular example, allowable reshaping parameter values may include 0, 1, 2, 3, and 4, for a total of five integers. As another particular example, allowable reshaping parameter values may include 0, -1, -2, -3, and -4, for a total of five integers. As another particular example, the allowed reshaping parameter values may include 0, -1, -2, and -3, for a total of four integers. In these examples, the term describing these specified integer ranges is alphabet size. Specifically, the alphabet size for a range of integers is the number of allowed values within this range. In the four examples above, the alphabet size is 4 or 5.

In some embodiments, a single frame can include one or more bands having a reshaping parameter that can be encoded using a first alphabet size, and further includes a first alphabet size and One or more bands can be included having reshaping parameters that can be encoded using different second alphabet sizes. Using different alphabet sizes in this way may allow for more compressed bitstream compression.

An encoder may encode data representing reconstruction parameters for each band into a bitstream. Encoding the reshaping parameters into the bitstream may allow the decoder to reverse the time/frequency reshaping before applying the inverse transform. One simple approach may be to form a reshaping sequence for each frame, with each element of the reshaping sequence being a reshaping parameter for a band within the frame. For a frame with 22 bands, this approach will generate a reshaping sequence made up of 22 reshaping parameters. The reshaping sequence for each frame may describe the reshaping parameters for each band. In some embodiments, the encoder can normalize each entry in each reshaping sequence to a range of possible values for this entry, each range of possible values being a specification of the reshaping parameter for the band. corresponds to the specified range.

As an improvement to this simple approach, the encoder can reduce the size of data required to completely describe these 22 integers. In this improved approach, the encoder calculates the lengths of the four sequences (e.g., the number of bits or integers in each of the four sequences), selects the shortest sequence among the four sequences, and selects this sequence. Data representing the shortest sequence can be embedded in the bitstream. The shortest sequence is the sequence that contains the least number of bits, ie, the sequence that most concisely describes the 22 integers. The four sequences will be explained below.

The encoder may use unary codes to form a first sequence for each frame that describes the reformation parameters for the frame as a sequence representing the reformation parameters for each band. The encoder may use a quasi-uniform code to form a second sequence for each frame that describes the reshaping parameters for the frame as a sequence representing the reshaping parameters for each band. The encoder may use unary codes to form a third sequence for each frame that describes the reformation parameters for the frame as a sequence representing the difference in reformation parameters between adjacent bands. The encoder may use a quasi-uniform code to form a fourth sequence for each frame that describes the reformation parameters for the frame as a sequence representing the difference in reformation parameters between adjacent bands.

The encoder can select the shortest sequence among the first sequence, the second sequence, the third sequence, and the fourth sequence. The encoder may embed the selected shortest sequence into the bitstream for each frame. The encoder may further embed, for each frame, data in the bitstream representing an indicator indicating which of the four sequences is included in the bitstream.

The appendix below provides strict mathematical definitions of the quantities mentioned above.

FIG. 6 depicts a flowchart of an embodiment of a method 600 for encoding an audio signal, according to some embodiments. Method 600 may be performed by encoding system 100 or 200 of FIG. 1 or 2, or by any other suitable encoding system. Method 600 is only one example of a method for encoding an audio signal; other suitable encoding methods can be used as well.

At act 602, an encoding system can receive a digital audio signal.

At operation 604, the encoding system may parse the digital audio signal into multiple frames, each frame including a specified number of audio samples.

At act 606, the encoding system may perform a transform on the audio samples of each frame to generate a plurality of frequency domain coefficients for each frame.

At act 608, the encoding system may divide the plurality of frequency domain coefficients for each frame into a plurality of bands for each frame, each band having reformation parameters representing time resolution and frequency resolution.

At act 610, the encoding system may encode the digital audio signal into a bitstream that includes reshaping parameters. Reshaping parameters for the first band may be encoded using a first alphabet size. Reshaping parameters for a second band different from the first band may be encoded using a second alphabet size different from the first alphabet size.

At act 612, the encoding system can output a bitstream.

FIG. 7 shows a flowchart of an embodiment of a method 700 for decoding an encoded audio signal, according to some embodiments. Method 700 may be performed by decoding system 300 or 400 of FIG. 3 or 4 or by any other suitable encoding system. Method 700 is just one method for decoding an encoded audio signal; other suitable encoding methods can be used as well.

At operation 702, a decoding system can receive a bitstream that includes multiple frames, each divided into multiple bands.

At operation 704, the decoding system may extract reshaping parameters from the bitstream for each band of each frame, where the reshaping parameters represent temporal and frequency resolution for the band. Reshaping parameters for the first band may be embedded in the bitstream using a first alphabet size. Reshaping parameters for a second band different from the first band may be embedded in the bitstream using a second alphabet size different from the first alphabet size.

At operation 706, the decoding system may decode the bitstream using the reshaping parameters to generate a decoded digital audio signal.

FIG. 12 depicts a block diagram of one embodiment of an encoding system 1200, according to some embodiments.

Receiver circuit 1202 can receive digital audio signals.

Framer circuit 1304 may parse the digital audio signal into multiple frames, each frame including a specified number of audio samples.

Transformer circuit 1206 may perform a transform of the audio samples of each frame to generate a plurality of frequency domain coefficients for each frame.

Frequency band divider circuit 1208 can divide the frequency domain coefficients for each frame into multiple bands for each frame, each band having reformation parameters representing time resolution and frequency resolution.

Encoder circuit 120 may encode the digital audio signal into a bitstream that includes reformation parameters for each band. Reshaping parameters for the first band may be encoded using a first alphabet size. Reshaping parameters for a second band different from the first band may be encoded using a second alphabet size different from the first alphabet size.

The output circuit 1212 can output a bitstream.

Many other variations in addition to those described herein will be apparent from the specification. For example, in some embodiments, certain acts, events, or functions of some of the methods and algorithms described herein may be performed in a different order, added, combined, or omitted entirely ( Therefore, not all acts or events described herein may be required to implement the present methods and algorithms). Furthermore, in certain embodiments, operations or events may be performed concurrently rather than sequentially, for example, by multi-threaded processing, interrupt processing, or by multiple processors or processor cores, or on other parallel architectures. Additionally, various tasks or processes can be performed by different machines and computing systems that can function together.

The various example logical blocks, modules, methods, and algorithm processes and sequences described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. can. To clearly illustrate this compatibility between hardware and software, the operations of various example components, blocks, modules, and processes are described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. The functionality described may be implemented in different ways for each particular application, and such implementation decisions should not be construed as resulting in a departure from the scope of this specification.

Various exemplary logic blocks and modules described in connection with embodiments disclosed herein include a general purpose processor, a processing device, a computing device having one or more processing devices, a digital signal processor (DSP), Application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic circuits, discrete hardware components, or other devices configured to perform the functions described herein. It can be implemented or executed by any machine such as any combination of these designed. General purpose processors and processing devices can be microprocessors, but in the alternative, the processor can be a controller, microcontroller, state machine, combinations thereof, or the like. A processor may be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. You can also do it.

Embodiments of the systems and methods described herein are operational within many types of general purpose or special purpose computing system environments or configurations. In general, a computing environment includes one or more microprocessors, mainframe computers, digital signal processors, portable computing devices, personal organizers, devices, to name a few examples, but are not limited to. Can include any type of computer system, including computer systems based on appliances with controllers, computational engines within appliances, mobile phones, desktop computers, mobile computers, tablet computers, smartphones, and embedded computers. .

Such computing devices typically include, but are not limited to, personal computers, server computers, handheld computing devices, laptops or mobile computers, communication devices such as cell phones and PDAs, multiprocessor systems, microprocessor systems, etc. can be found in devices with at least some minimal computing power, including base systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, audio or video media players, etc. . In some embodiments, a computing device will include one or more processors. Each processor may be a specialized microprocessor, such as a digital signal processor (DSP), very long instruction word (VLIW), or other microcontroller, or may be based on a specialized graphics processing unit (GPU) within a multi-core CPU. A conventional central processing unit (CPU) may have one or more processing cores, including a core.

The processing operations of the methods, processes, or algorithms described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in any combination of the two. can do. The software can be included on a computer-readable medium that can be accessed by a computing device. Computer-readable media includes both volatile and non-volatile media, either removable or non-removable, and/or any combination thereof. Computer-readable media are used to store information such as computer-readable or computer-executable instructions, data structures, program modules, or other data. Computer-readable media can include, by way of example and not limitation, computer storage media and communication media.

Computer storage media include, but are not limited to, Bluray® discs (BDs), digital versatile discs (DVDs), compact discs (CDs), floppy disks, tape drives, hard drives, optical drives, solid state A state memory device, RAM memory, ROM memory, EPROM memory, EEPROM memory, flash memory, or other memory technology, magnetic cassette, magnetic tape, magnetic disk storage, or other magnetic storage device, or for storing the desired information. computer-readable or machine-readable media or storage devices, such as any other device usable and accessible by one or more computing devices.

The software may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CDROM, or any other type of non-transitory computer readable storage medium, media, or physical computer storage known in the art. Can exist in other forms. An exemplary storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. A processor and a storage medium may reside on an application specific integrated circuit (ASIC). The ASIC may reside within the user terminal. Alternatively, the processor and the storage medium can reside as separate components within a user terminal.

As used herein, the phrase "non-temporary" means "permanent or long-lived." The phrase "non-transitory computer-readable medium" includes any and all computer-readable media, the only implementation exception being a transitory propagated signal. This term includes, by way of example and not limitation, non-transitory computer-readable media such as register memory, processor cache, and random access memory (RAM).

The phrase "audio signal" is a signal that represents physical sound.

Also, carrying information such as computer-readable or computer-executable instructions, data structures, program modules, etc. may encode one or more modulated data signals, electromagnetic waves (such as carrier waves), or other transmission mechanisms or communication protocols. It can also be implemented using a variety of communication media for communication, including some wired or wireless information delivery mechanism. Generally, these communication media refer to a signal that has one or more of its characteristics set or changed in such a manner as to encode information or instructions in the signal. For example, communication media can include one or more wired media, such as a wired network or direct wired connection, that carries one or more modulated data signals, and one or more wireless media, such as acoustic, radio frequency (RF), infrared, laser, etc. transmitting, receiving, or other wireless media for transmitting and receiving modulated data signals or electromagnetic waves. Combinations of any of the above should also be included within the scope of communication media.

Additionally, any one or any combination of software, programs, computer program products, or combinations thereof, embodying some or all of the various embodiments of the encoding and decoding systems and methods described herein. The portions may be stored on, received from, transmitted to, or read from computer-readable or machine-readable media or any desired combination of storage devices and communication media in the form of computer-executable instructions or other data structures.

Embodiments of the systems and methods described herein may also be described in the general context of computer-executable instructions, such as program modules, being executed by a computing device. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Embodiments described herein can also be practiced in distributed computing environments where tasks are performed by one or more remote processing devices or in a cloud of one or more devices. These devices are linked via one or more communication networks. In a distributed computing environment, program modules may be located in both local and remote computer storage media including media storage devices. Furthermore, the instructions described above may be implemented in part or in whole as hardware logic circuitry that may or may not include a processor.

Conditional words as used herein, especially "can", "might", "may", "e.g.", and the like, , unless explicitly stated otherwise or understood otherwise within the context in which it is used, a particular embodiment generally includes certain features, elements, and/or conditions; The embodiments are intended to convey the exclusion of these. Thus, such conditional language generally does not imply that a feature, element, and/or condition is necessarily required for one or more embodiments, and that it may be used without input or direction from the author. In any particular embodiment, one or more embodiments may include logic for determining whether these features, elements, and/or conditions are included or implemented in any particular embodiment. This does not imply that it necessarily includes. The terms “comprising,” “including,” “having,” and the like are synonymous and are used inclusively and in an open-ended manner, and refer to additional elements, features, and operations. , operations etc. are not excluded. Also, the term "or" is used in its inclusive sense (rather than in its exclusive sense), so that, for example, when used to join a list of elements, the term "or" Refers to one, some, or all of the elements in a list.

While the foregoing detailed description illustrates, describes, and points out novel features that apply to various embodiments, various omissions, substitutions, and changes may be made without departing from the spirit of this disclosure. It will be understood that the methods may be implemented in the form and details of the devices or algorithms illustrated. As will be appreciated, some features may be used or implemented separately from other features, so that certain embodiments of the invention described herein may be modified to include the features and features illustrated herein. It may be implemented within a range of forms that do not necessarily provide all of the advantages.

Furthermore, although the subject matter has been described in terms specific to structural features and methodological acts, the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. I hope you understand that. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

(appendix)

Embodiments of time-frequency modified sequence codecs and methods described herein include techniques for efficiently encoding and decoding sequences that describe time-frequency reshaping sequences. Embodiments of the present codec and method address efficient encoding and decoding of sequences on a heterogeneous alphabet.

Some codecs produce sequences that are much more complex than those typically used in existing codecs. This complexity is due to the fact that these sequences describe a richer set of possible time-frequency reshaping transforms. In some embodiments, this complexity is due to the fact that the elements of the sequence are from four different alphabets that are of different sizes or ranges (depending on the coordinates) and based on the context in which the audio frames are processed. This is something that can be obtained. Simple encoding of these sequences is expensive and negates the benefits of a richer set.

Embodiments of the present codec and method describe a highly efficient method that enables uniform processing of heterogeneous alphabets through various alphabet transformations, optimizing encoding parameters to obtain the shortest possible description. do. Some features of embodiments of the present codecs and methods include uniform processing of heterogeneous alphabets, definition of multiple encoding styles, and selection of the style that minimizes encoding length. These features are part of what provides some of the advantages of embodiments of the present codec and method, including enabling the use of a richer set of time-frequency transforms.

Section 1: Sequence definition

The modified discrete cosine transform (MDCT) transform engine currently operates in two modes: long transform (used by default on most frames) and short transform (used on frames that are considered to contain transients). ) works. If the number of MDCT coefficients in a given band is a quantity N, then in long transform mode these coefficients are arranged as one time slot containing N frequency slots (1×N). In short transform mode, the coefficients are organized as eight time slots (8×N/8), each slot containing N/8 frequency slots.

The time-frequency change sequence or vector is a sequence of integers, one for each band, up to the number of effective bands available for the frame. Each integer indicates how the original time/frequency structure defined by the transform is changed for the corresponding band. If the original structure for the band is T×F (T time slots, F frequency slots) and the modification value is c, then by applying the appropriate local transformation this structure becomes 2 ^c Changed to T×2 ^-c F. The range of allowed values for c is determined by integer constraints that depend on whether the original modes are long or short transforms and the size of the band, as well as by limits on the number of time-frequency configurations supported. Ru.

A band is called narrowband if its size is smaller than 16 MDCT bins. Otherwise, the band is called wideband. All band sizes can be multiples of 8, and in the current implementation, at a sampling rate of 48 kHz, bands numbered 0 to 7 are narrow bands and bands numbered 8 to 21 are narrow bands. The bands numbered from 0 to 5 may be narrow bands and the bands numbered from 6 to 21 may be wide bands at a sampling rate of 44 kHz. .

The next paragraph shows a set of possible changes c for all combinations of long versus short transforms and narrowband versus wideband.

For narrowband and long transforms, it is {0, 1, 2, 3}.

For wideband and long transforms, it is {0, 1, 2, 3, 4}.

For narrowband and short transforms, it is {-3, -2, -1, 0}.

For wideband and short transforms, it is {-3, -2, -1, 0, 1}.

Section 2: Sequence encoding

Section 2.1: Basic elements

The input to the encoding process is the sequence or vector c=[c ₀ , c ₁ , . ．． ．． , c _M-1 ], where the quantity M is the number of effective bands and the value c _i is in the appropriate range from the above paragraph.

From sequence c, a first difference sequence or vector d=[d ₀ , d ₁ , . ．． ．． , d _M-1 ], where d ₀ =c ₀ and d _i =c _i −c _il , 0<i<M. A parameter d of the encoding is defined, which determines which sequence is encoded into the bitstream, i.e. sequence c if parameter d is 0, sequence d if parameter d is l. It is a signal that conveys the A description of how the parameter d is determined follows below.

Sequence or vector s=[s ₀ , s ₁ , . ．． ．． , s _M-1 ], to encode which can be either sequence c or sequence d, the following is defined.

The quantity head(s) is the length of the subsequence of the sequence s extending from the first coordinate to the last non-zero coordinate. This subsequence is called the head of s. Note that head(s)=0 if and only if the sequence s is a sequence of all zeros.

The quantity head(s) is encoded as follows. If the quantity head(s) is equal to zero, the encoder writes a zero bit and stops. In this case, the zero bits represent the entire reformed vector that is all zeros, so no further encoding is required. If the quantity head(s) is greater than zero, the encoder encodes the quantity head(s)-1 using a quasi-uniform code over the alphabet of size M.

A quasi-uniform code on the alphabet of size α can be written using either L ₁ = [log ₂ α] bits or L ₂ = [log ₂ α] bits as follows: {0, 1, . ．． ．． , α-1}.

Symbols x with 0<=x<n ₁ are encoded with their binary representation in L ₁ bits.

A symbol x, with n ₁ <=x<n ₁ +n ₂ , is encoded with a binary representation of x+n ₁ in L ₂ bits.

The symbols at the head of s are encoded symbol by symbol. Before encoding, each symbol is mapped using a mapping that depends on the parameter d, the long versus short transform, and the choice of narrowband versus wideband. This mapping is defined in the pseudocode function MapTFSymbol shown in FIG. Assume that the input symbol sequence s, the variable d, and the Boolean quantities is_long and is_narrow are given as parameters.

Figure 8 shows a mapping that in all cases results in non-negative integers (i.e. {0, 1, ..., α-1}) in the range [0, α), where the quantity α is , 4 for narrowband and 5 for wideband. There are two code choices for the mapped symbols, and these symbols are parameterized with a binary flag k.

k=0: It is a simple code on the alphabet of size α. This unary code is {0, 1, . ．． ．． , α-2} is encoded as a sequence consisting of i "0"s followed by "1" indicating the end of encoding. The integer α-1 is encoded as a sequence of α-1 '0's without a terminating '1'.

k=l: A quasi-uniform code on the alphabet of size α.

How the binary flag k is determined is explained below.

Section 2.2: Encoding

Assume that parameters d and k are known. The pair (d,k) is encoded as one symbol resulting as shown in FIG. The resulting symbols are encoded using the Golomb code, and the permutation array map_dk_pair consists of the pair (d, k), with (d=1, k=0) receiving the most likely shortest codeword. Assign indices in descending order of probability of occurrence.

The encoding procedure is summarized in the pseudocode of FIG. The variable seq represents the input sequence c. The number of bands is available in the global variable num_bands.

Section 2.3: Parameter optimization

To determine the parameters d and k, the encoder tries all four combinations of binary values and chooses the one that gives the shortest code length. This is done using a code length function that does not require actual encoding.

Section 3: Sequence decoding

The decoder is simply a reversal of the steps of the encoder, except that the decoder does not have to read the parameters d and k from the bitstream and optimize these parameters. The decoding procedure is summarized in the pseudocode of FIG. 11, where the quantity num_bands is the known number of bands.

100 encoding system 102 digital audio signal 104 bitstream 106 processor 108 memory device 110 instructions

Claims (20)

a processor;
a memory device storing instructions executable by the processor, the instructions being executable by the processor to perform a method for encoding an audio signal;
In a coding system comprising:
The method includes:
receiving a digital audio signal;
parsing the digital audio signal into a plurality of frames each including a specified number of audio samples;
transforming the audio samples of each frame to generate a plurality of frequency domain coefficients for each frame;
dividing a plurality of frequency domain coefficients for each frame into a plurality of bands for each frame, each band having default values of time resolution and frequency resolution after the transform, and each band having adjusted a reshaping parameter representing a time resolution and an adjusted frequency resolution, the reshaping parameter representing a time resolution and a frequency resolution to the adjusted values of the time resolution and frequency resolution; a step, which is a value indicating a change in frequency resolution from the default value;
encoding the parsed, transformed, and segmented digital audio signal into a bitstream comprising reshaping parameters for each of the bands, the reshaping parameters for a first band being a first the reshaping parameters for a second band different from the first band are coded using a second alphabet size different from the first alphabet size. , step and
outputting the bitstream;
including,
A coding system characterized by: The method further includes:
adjusting the time resolution and frequency resolution of each band of each of the frames, a first of the time resolution and a first of the frequency resolution being selected from one of a plurality of specified ranges of integers; complementarily adjusted by the magnitude described by said reshaping parameter having a value that is an integer;
the first alphabet size is equal to the number of integers in a first specified range of integers among the plurality of specified ranges of integers;
2. The encoding system of claim 1, wherein the second alphabet size is equal to the number of integers in a second specified range of integers of the plurality of specified ranges of integers. 3. The encoding system of claim 2, wherein the first alphabet size is four and the second alphabet size is five. 3. The encoding system of claim 2, wherein, before the adjustment, the temporal resolution of the first band is equal to eight audio samples and the temporal resolution of the second band is equal to one audio sample. . each band has a size equal to the product of the time resolution of the band and the frequency resolution of the band;
3. The encoding system of claim 2, wherein the time resolution of the band and the frequency resolution of the band are complementarily adjusted without changing the size of the band. 6. The encoding system of claim 5, wherein the time resolution is adjusted by a multiple of ^2c , the frequency resolution is varied by a multiple of 2 ^-c , and the quantity c is the reshaping parameter. The method further includes:
forming a reshaping sequence for each frame that describes the reshaping parameters for each band;
normalizing each entry in each reshaping sequence to a range of possible values for said entry;
7. A coding system according to any of claims 2 to 6, wherein each range of possible values corresponds to an integer in the specified range for the band. The method further includes:
forming a first sequence for each frame that describes the reshaping parameters for the frame as a sequence representing the reshaping parameters for each band using a unary code;
forming a second sequence for each frame that describes the reshaping parameters for the frame as a sequence representing the reshaping parameters for each band using a quasi-uniform code;
forming a third sequence for each frame that describes the reshaping parameter for the frame as a sequence representing a difference in the reshaping parameter between adjacent bands using a unary code;
forming a fourth sequence for each frame that describes the reshaping parameter for the frame as a sequence representing a difference in the reshaping parameter between adjacent bands using a quasi-uniform code;
selecting the shortest sequence that is the sequence that includes the least number of elements among the first sequence, the second sequence, the third sequence, and the fourth sequence;
embedding the selected shortest sequence into the bitstream for each frame;
embedding in the bitstream, for each frame, data representing an indicator of which of the four sequences is included in the bitstream;
The encoding system according to claim 1, comprising: The encoding system of claim 1, wherein the transform is a modified discrete cosine transform. The encoding system of claim 1, wherein each frame includes exactly 1024 samples. 2. The encoding system of claim 1, wherein the number of frequency domain coefficients in the respective plurality of frequency domain coefficients is equal to the specified number of audio samples in each frame. The encoding system of claim 1, wherein the plurality of frequency domain coefficients for each frame includes exactly 1024 frequency domain coefficients. 2. The encoding system of claim 1, wherein the plurality of bands for each frame includes exactly 22 bands. The encoding system of claim 1, wherein the encoding system is included in a codec. a processor;
a memory device storing instructions executable by the processor, the instructions being executable by the processor to perform a method for decoding an encoded audio signal;
A decoding system comprising:
The method includes:
receiving a bitstream including a plurality of frames each divided into a plurality of bands;
extracting, for each band of each frame, from the bitstream a reshaping parameter representing an adjusted temporal resolution and an adjusted frequency resolution for the band, the reshaping parameter representing the adjusted temporal resolution and adjusted frequency resolution; a value indicative of a change from default values of time resolution and frequency resolution to the adjusted value of resolution, the reshaping parameter for a first band reshaping the bitstream using a first alphabet size embedded, the reshaping parameters for a second band different from the first band are embedded in the bitstream using a second alphabet size different from the first alphabet size;
decoding the bitstream using the reshaping parameters to generate a decoded digital audio signal;
including;
The decoding includes adjusting the adjusted temporal resolution and the adjusted frequency resolution of each band of each frame and subsequently applying an inverse transform, the adjusting temporal resolution and the adjusting the frequency resolution is adjusted using the reshaping parameter to increase one of the first time resolution and the first frequency resolution and decrease the other, or to leave both unchanged. A decoding system characterized by: The method further includes:
For each band of each frame,
whether the reshaping parameters in the bitstream are represented as unary codes or quasi-uniform codes; and whether the reshaping parameters in the bitstream define the reshaping parameters for each of the bands. or as a sequence representing a difference in said reshaping parameter between adjacent said bands;
16. The decoding system according to claim 15, comprising the step of extracting data indicative of . The decoding system according to claim 15 or 16, wherein the decoding system is included in a codec. A coding system,
a receiver circuit for receiving a digital audio signal;
a framer circuit for parsing the digital audio signal into a plurality of frames each including a specified number of audio samples;
a transformer circuit for performing a transform on the audio samples of each frame to generate a plurality of frequency domain coefficients for each frame;
a frequency band divider circuit for dividing the plurality of frequency domain coefficients for each frame into a plurality of bands for each frame, each band representing an adjusted time resolution and an adjusted frequency resolution; a reshaping parameter, the reshaping parameter indicating a change from default values of time resolution and frequency resolution to the adjusted values of time resolution and frequency resolution; a frequency band divider circuit,
an encoder circuit for encoding the parsed, transformed, and segmented digital audio signal into a bitstream comprising reshaping parameters for each band, the reshaping parameters for a first band comprising: the reshaping parameters for a second band different from the first band are encoded using a second alphabet size different from the first alphabet size; an encoder circuit,
an output circuit for outputting the bitstream;
An encoding system comprising: Further comprising a resolution adjustment circuit for adjusting the time resolution and frequency resolution of each band of each of the frames, the first said time resolution and the first said frequency resolution being within a plurality of specified ranges of integers. complementarily adjusted by the magnitude described by the reshaping parameter having a value that is an integer selected from one;
the first alphabet size is equal to the number of integers in a first specified range of integers among the plurality of specified ranges of integers;
19. The encoding system of claim 18, wherein the second alphabet size is equal to the number of integers in a second specified range of integers of the plurality of specified ranges of integers. 20. The encoding system of claim 19, wherein the temporal resolution is adjusted by a multiple of ^2c , the frequency resolution is varied by a multiple of 2 ^-c , and the quantity c is the reshaping parameter.

Images