KR102386738B1 - Signal encoding method and apparatus, and signal decoding method and apparatus - Google Patents

Abstract

The spectrum encoding method includes the steps of selecting an encoding method based on at least bit allocation information of each band, performing zero encoding on the zero band, and encoding information of the selected important frequency component for each non-zero band. may include

Description

Signal encoding method and apparatus, and signal decoding method and apparatus

The present invention relates to audio or speech signal encoding and decoding, and more particularly, to a method and apparatus for encoding or decoding spectral coefficients in a frequency domain.

For efficient encoding of spectral coefficients in the frequency domain, various types of quantizers have been proposed. For example, there are TCQ (Trellis Coded Quantization), USQ (Uniform Scalr Quantization), FPC (Factorial Pulse Coding), AVQ (Algebraic VQ), PVQ (Pyramid VQ), etc. can be implemented.

SUMMARY OF THE INVENTION An object of the present invention is to provide a method and apparatus for encoding or decoding spectral coefficients adaptively to various bit rates or various subband sizes in the frequency domain.

Another object of the present invention is to provide a computer-readable recording medium in which a program for executing a signal encoding method or a signal decoding method in a computer is recorded.

Another object of the present invention is to provide a multimedia device employing a signal encoding device or a decoding device.

A spectrum encoding method according to one aspect for achieving the above object includes at least selecting an encoding method based on bit allocation information of each band; performing zero encoding on the zero band; and encoding information of a selected important frequency component for each non-zero band.

According to one aspect of the present invention, a spectrum decoding method includes: selecting a decoding method based on at least bit allocation information of each band; performing zero decoding on the zero band; and decoding information of important frequency components obtained for each non-zero band.

It is possible to encode and decode spectral coefficients adaptive to various bit rates and sizes of various subbands. In addition, the spectrum can be encoded as TCQ at a fixed bit rate by using a bit rate control module designed in a codec supporting multi-rate. In this case, it is possible to maximize the encoding performance of the codec by encoding the high performance of the TCQ at an accurate target bit rate.

1A and 1B are block diagrams respectively showing configurations of an audio encoding apparatus and a decoding apparatus to which the present invention can be applied according to an example.
2A and 2B are block diagrams each showing the configuration of an audio encoding apparatus and a decoding apparatus according to another example to which the present invention can be applied.
3A and 3B are block diagrams each showing the configuration of an audio encoding apparatus and a decoding apparatus according to another example to which the present invention can be applied.
4A and 4B are block diagrams respectively showing configurations of an audio encoding apparatus and a decoding apparatus to which the present invention can be applied according to another example.
5 is a block diagram showing the configuration of a frequency domain audio encoding apparatus to which the present invention can be applied.
6 is a block diagram showing the configuration of a frequency domain audio decoding apparatus to which the present invention can be applied.
7 is a block diagram showing the configuration of a spectrum encoding apparatus according to an embodiment.
8 is a diagram illustrating an example of subband division.
9 is a block diagram illustrating a configuration of a spectral quantization apparatus according to an embodiment.
10 is a block diagram showing the configuration of a spectrum encoding apparatus according to an embodiment.
11 is a block diagram showing the configuration of an ISC encoding apparatus according to an embodiment.
12 is a block diagram showing the configuration of an ISC information encoding apparatus according to an embodiment.
13 is a block diagram showing the configuration of a spectrum encoding apparatus according to another embodiment.
14 is a block diagram showing the configuration of a spectrum encoding apparatus according to another embodiment.
15 is a diagram illustrating a concept of an ISC collection and encoding process according to an embodiment.
16 is a diagram illustrating the concept of an ISC collection and encoding process according to another embodiment.
17 is a diagram illustrating an example of TCQ used in the present invention.
18 is a block diagram showing the configuration of a frequency domain audio decoding apparatus to which the present invention can be applied.
19 is a block diagram illustrating a configuration of a spectrum decoding apparatus according to an embodiment.
20 is a block diagram illustrating a configuration of a spectral inverse quantization apparatus according to an embodiment.
21 is a block diagram showing the configuration of a spectrum decoding apparatus according to an embodiment.
22 is a block diagram showing the configuration of an ISC decoding apparatus according to an embodiment.
23 is a block diagram showing the configuration of an ISC information decoding apparatus according to an embodiment.
24 is a block diagram showing the configuration of a spectrum decoding apparatus according to another embodiment.
25 is a block diagram showing the configuration of a spectrum decoding apparatus according to another embodiment.
26 is a block diagram showing the configuration of an ISC information encoding apparatus according to another embodiment.
27 is a block diagram showing the configuration of an ISC information decoding apparatus according to another embodiment.
28 is a block diagram illustrating a configuration of a multimedia device according to an embodiment.
29 is a block diagram illustrating a configuration of a multimedia device according to another embodiment.
30 is a block diagram illustrating a configuration of a multimedia device according to another exemplary embodiment.
31 is a flowchart illustrating an operation of a method for encoding a fine structure of a spectrum according to an embodiment.
32 is a flowchart illustrating an operation of a method for decoding a fine structure of a spectrum according to an embodiment.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to a specific embodiment, it can be understood to include all transformations, equivalents or substitutes included in the spirit and scope of the present invention. In describing the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

Terms such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The terms used in the present invention are only used to describe specific embodiments, and are not intended to limit the present invention. The terms used in the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, but these may vary depending on the intention, precedent, or emergence of new technology of those of ordinary skill in the art. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term.

The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present invention, terms such as "comprises" or "have" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1A and 1B are block diagrams respectively showing configurations of an audio encoding apparatus and a decoding apparatus to which the present invention can be applied according to an example.

The audio encoding apparatus 110 shown in FIG. 1A may include a preprocessor 112 , a frequency domain encoder 114 , and a parameter encoder 116 . Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown).

In FIG. 1A , the preprocessor 112 may perform filtering or downsampling on an input signal, but is not limited thereto. The input signal may mean a media signal such as audio, music or speech, or a sound representing a mixed signal thereof, but hereinafter, it will be referred to as an audio signal for convenience of description.

The frequency domain encoding unit 114 performs time-frequency transformation on the audio signal provided from the preprocessor 112 , selects an encoding tool corresponding to the number of channels, encoding band, and bit rate of the audio signal, and selects the selected encoding tool can be used to encode an audio signal. The time-frequency transform uses a Modified Discrete Cosine Transform (MDCT), a Modulated Lapped Transform (MLT), or a Fast Fourier Transform (FFT), but is not limited thereto. Here, when the given number of bits is sufficient, the general transform encoding method is applied to the entire band, and when the given number of bits is insufficient, the band extension method can be applied to some bands. On the other hand, when the audio signal is stereo or multi-channel, if the given number of bits is sufficient, it is encoded for each channel, and if not enough, the downmixing method may be applied. The frequency domain encoder 114 generates encoded spectral coefficients.

The parameter encoder 116 may extract a parameter from the coded spectral coefficient provided from the frequency domain encoder 114 and encode the extracted parameter. The parameter may be extracted for each subband or each band, for example, and will be referred to as a subband hereinafter for the sake of simplicity of description. Each subband is a unit of grouping spectral coefficients, and may have a uniform or non-uniform length by reflecting a critical band. In the case of having a non-uniform length, a subband existing in a low frequency band may have a relatively small length compared to that in a high frequency band. The number and length of subbands included in one frame vary depending on a codec algorithm and may affect encoding performance. Meanwhile, the parameter may include, but is not limited to, a scale factor, power, average energy, or norm of a subband. The spectral coefficients and parameters obtained as a result of encoding form a bitstream, and may be stored in a storage medium or transmitted through a channel, for example, in the form of a packet.

The audio decoding apparatus 130 illustrated in FIG. 1B may include a parameter decoding unit 132 , a frequency domain decoding unit 134 , and a post-processing unit 136 . Here, the frequency domain decoder 134 may include a frame erasure concealment (FEC) algorithm or a packet loss concealment (PLC) algorithm. Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown).

Referring to FIG. 1B , the parameter decoding unit 132 may decode an encoded parameter from a received bitstream and check whether an error such as erasure or loss occurs from the decoded parameter on a frame-by-frame basis. The error check may use various well-known methods, and information on whether the current frame is a normal frame, an erased frame, or a lost frame is provided to the frequency domain decoder 134 . Hereinafter, an erased or lost frame will be referred to as an error frame for the sake of simplification of description.

When the current frame is a normal frame, the frequency domain decoder 134 may perform decoding through a general transform decoding process to generate synthesized spectral coefficients. On the other hand, when the current frame is an error frame, the frequency domain decoder 134 repeatedly uses the spectral coefficients of the previous normal frame for the error frame through the FEC or PLC algorithm, or by scaling and repeating through regression analysis, synthesized spectral coefficients. can create The frequency domain decoder 134 may generate a time domain signal by performing frequency-time conversion on the synthesized spectral coefficients.

The post-processing unit 136 may perform filtering or upsampling to improve sound quality on the time domain signal provided from the frequency domain decoding unit 134 , but is not limited thereto. The post-processing unit 136 provides the restored audio signal as an output signal.

2A and 2B are block diagrams respectively illustrating configurations of an audio encoding apparatus and a decoding apparatus according to another example to which the present invention can be applied, and have a switching structure.

The audio encoding apparatus 210 shown in FIG. 2A may include a preprocessor 212 , a mode determiner 213 , a frequency domain encoder 214 , a time domain encoder 215 , and a parameter encoder 216 . can Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown).

In FIG. 2A , the pre-processing unit 212 is substantially the same as the pre-processing unit 112 of FIG. 1A , and thus a description thereof will be omitted.

The mode determiner 213 may determine the encoding mode with reference to the characteristics of the input signal. Whether the encoding mode suitable for the current frame is the voice mode or the music mode may be determined according to the characteristics of the input signal, and it may be determined whether the encoding mode effective for the current frame is the time domain mode or the frequency domain mode. Here, the characteristics of the input signal may be determined by using the short-term characteristics of the frame or the long-term characteristics of a plurality of frames, but is not limited thereto. For example, if the input signal corresponds to a voice signal, the voice mode or time domain mode is determined. . The mode determining unit 213 transmits the output signal of the preprocessor 212 to the frequency domain encoding unit 214 when the characteristics of the input signal correspond to the music mode or the frequency domain mode, and the characteristics of the input signal are the voice mode or the time domain. If it corresponds to the domain mode, it may be provided to the time domain encoder 215 .

Since the frequency domain encoder 214 is substantially the same as the frequency domain encoder 114 of FIG. 1A , a description thereof will be omitted.

The time domain encoder 215 may perform Code Excited Linear Prediction (CELP) encoding on the audio signal provided from the preprocessor 212 . Specifically, ACELP (Algebraic CELP) may be used, but is not limited thereto.

The parameter encoder 216 extracts parameters from the encoded spectral coefficients provided from the frequency domain encoder 214 or the time domain encoder 215 and encodes the extracted parameters. Since the parameter encoder 216 is substantially the same as the parameter encoder 116 of FIG. 1A , a description thereof will be omitted. Spectral coefficients and parameters obtained as a result of encoding form a bitstream together with encoding mode information, and may be transmitted in a packet form through a channel or stored in a storage medium.

The audio decoding apparatus 230 shown in FIG. 2B may include a parameter decoding unit 232 , a mode determining unit 233 , a frequency domain decoding unit 234 , a time domain decoding unit 235 , and a post-processing unit 236 . can Here, the frequency domain decoding unit 234 and the time domain decoding unit 235 may each include an FEC or PLC algorithm in a corresponding domain. Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown).

Referring to FIG. 2B , the parameter decoding unit 232 may decode a parameter from a bitstream transmitted in the form of a packet, and check whether an error has occurred from the decoded parameter on a frame-by-frame basis. Various known methods may be used for the error check, and information on whether the current frame is a normal frame or an error frame is provided to the frequency domain decoder 234 or the time domain decoder 235 .

The mode determiner 233 checks the encoding mode information included in the bitstream and provides the current frame to the frequency domain decoder 234 or the time domain decoder 235 .

The frequency domain decoder 234 operates when the encoding mode is the music mode or the frequency domain mode, and when the current frame is a normal frame, performs decoding through a general transform decoding process to generate synthesized spectral coefficients. On the other hand, if the current frame is an error frame and the encoding mode of the previous frame is music mode or frequency domain mode, the spectral coefficients of the previous normal frame are repeatedly used in the error frame through FEC or PLC algorithm in the frequency domain, or regression analysis is performed. It is possible to generate synthesized spectral coefficients by repeating by scaling. The frequency domain decoder 234 may generate a time domain signal by performing frequency-time conversion on the synthesized spectral coefficients.

The time domain decoder 235 operates when the encoding mode is the voice mode or the time domain mode, and when the current frame is a normal frame, performs decoding through a general CELP decoding process to generate a time domain signal. Meanwhile, when the current frame is an error frame and the encoding mode of the previous frame is a voice mode or a time domain mode, the FEC or PLC algorithm in the time domain may be performed.

The post-processing unit 236 may perform filtering or upsampling on the time domain signal provided from the frequency domain decoding unit 234 or the time domain decoding unit 235, but is not limited thereto. The post-processing unit 236 provides the restored audio signal as an output signal.

3A and 3B are block diagrams respectively illustrating configurations of an audio encoding apparatus and a decoding apparatus according to another example to which the present invention can be applied, and have a switching structure.

The audio encoding apparatus 310 shown in FIG. 3A includes a preprocessor 312, a linear prediction (LP) analysis unit 313, a mode determination unit 314, a frequency domain excitation encoding unit 315, and a time domain excitation encoding unit. 316 and a parameter encoder 317 may be included. Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown).

In FIG. 3A , the pre-processing unit 312 is substantially the same as the pre-processing unit 112 of FIG. 1A , and thus a description thereof will be omitted.

The LP analysis unit 313 extracts LP coefficients by performing LP analysis on the input signal, and generates an excitation signal from the extracted LP coefficients. The excitation signal may be provided to one of the frequency domain excitation encoder 315 and the time domain excitation encoder 316 according to the encoding mode.

Since the mode determining unit 314 is substantially the same as the mode determining unit 213 of FIG. 2B , a description thereof will be omitted.

The frequency domain excitation encoder 315 operates when the encoding mode is a music mode or a frequency domain mode, and is substantially the same as the frequency domain encoder 114 of FIG. 1A except that the input signal is an excitation signal. to be omitted.

The time domain excitation encoder 316 operates when the encoding mode is the voice mode or the time domain mode, and is substantially the same as the time domain encoder 215 of FIG. 2A except that the input signal is an excitation signal. to be omitted.

The parameter encoder 317 extracts parameters from the coded spectral coefficients provided from the frequency domain excitation encoder 315 or the time domain excitation encoder 316 , and encodes the extracted parameters. Since the parameter encoder 317 is substantially the same as the parameter encoder 116 of FIG. 1A , a description thereof will be omitted. Spectral coefficients and parameters obtained as a result of encoding form a bitstream together with encoding mode information, and may be transmitted in a packet form through a channel or stored in a storage medium.

The audio decoding apparatus 330 shown in FIG. 3B includes a parameter decoding unit 332 , a mode determining unit 333 , a frequency domain excitation decoding unit 334 , a time domain excitation decoding unit 335 , and an LP synthesis unit 336 . and a post-processing unit 337 . Here, the frequency domain excitation decoding unit 334 and the time domain excitation decoding unit 335 may each include an FEC or PLC algorithm in a corresponding domain. Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown).

Referring to FIG. 3B , the parameter decoding unit 332 may decode a parameter from a bitstream transmitted in the form of a packet and check whether an error has occurred from the decoded parameter on a frame-by-frame basis. Various known methods may be used for error checking, and information on whether the current frame is a normal frame or an error frame is provided to the frequency domain excitation decoding unit 334 or the time domain excitation decoding unit 335 .

The mode determining unit 333 checks the encoding mode information included in the bitstream and provides the current frame to the frequency domain excitation decoding unit 334 or the time domain excitation decoding unit 335 .

The frequency domain excitation decoding unit 334 operates when the encoding mode is the music mode or the frequency domain mode, and when the current frame is a normal frame, performs decoding through a general transform decoding process to generate synthesized spectral coefficients. On the other hand, if the current frame is an error frame and the encoding mode of the previous frame is music mode or frequency domain mode, the spectral coefficients of the previous normal frame are repeatedly used in the error frame through FEC or PLC algorithm in the frequency domain, or regression analysis is performed. It is possible to generate synthesized spectral coefficients by repeating by scaling. The frequency domain excitation decoder 334 may generate an excitation signal that is a time domain signal by performing frequency-time conversion on the synthesized spectral coefficients.

The time domain excitation decoding unit 335 operates when the encoding mode is the voice mode or the time domain mode, and when the current frame is a normal frame, performs decoding through a general CELP decoding process to generate an excitation signal that is a time domain signal. Meanwhile, when the current frame is an error frame and the encoding mode of the previous frame is a voice mode or a time domain mode, the FEC or PLC algorithm in the time domain may be performed.

The LP synthesis unit 336 generates a time domain signal by performing LP synthesis on the excitation signal provided from the frequency domain excitation decoding unit 334 or the time domain excitation decoding unit 335 .

The post-processing unit 337 may perform filtering or upsampling on the time domain signal provided from the LP synthesis unit 336, but is not limited thereto. The post-processing unit 337 provides the restored audio signal as an output signal.

4A and 4B are block diagrams respectively illustrating configurations of an audio encoding apparatus and a decoding apparatus according to another example to which the present invention can be applied, and have a switching structure.

The audio encoding apparatus 410 shown in FIG. 4A includes a preprocessor 412 , a mode determiner 413 , a frequency domain encoder 414 , an LP analyzer 415 , a frequency domain excitation encoder 416 , and time It may include a domain excitation encoder 417 and a parameter encoder 418 . Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown). Since the audio encoding apparatus 410 shown in FIG. 4A can be viewed as a combination of the audio encoding apparatus 210 of FIG. 2A and the audio encoding apparatus 310 of FIG. 3A , descriptions of operations of common parts are omitted, while the mode An operation of the determination unit 413 will be described.

The mode determiner 413 may determine the encoding mode of the input signal by referring to the characteristics and bit rate of the input signal. The mode determining unit 413 determines whether the current frame is a voice mode or a music mode according to the characteristics of the input signal, and whether the efficient encoding mode for the current frame is a time domain mode or a frequency domain mode. mode can be determined. If the characteristic of the input signal is the voice mode, the CELP mode may be determined, the FD mode may be determined in the case of a music mode and a high bit rate, and the audio mode may be determined if the music mode is a low bit rate. The mode determination unit 413 transmits the input signal to the frequency domain encoding unit 414 in the FD mode, the frequency domain excitation encoding unit 416 through the LP analysis unit 415 in the audio mode, and the LP in the CELP mode. It may be provided to the time domain excitation encoding unit 417 through the analysis unit 415 .

The frequency domain encoding unit 414 is connected to the frequency domain encoding unit 114 of the audio encoding apparatus 110 of FIG. 1A or the frequency domain encoding unit 214 of the audio encoding apparatus 210 of FIG. 2A, and the frequency domain excitation encoding unit 416 or the time domain excitation encoder 417 may correspond to the frequency domain excitation encoder 315 or the time domain excitation encoder 316 of the audio encoding apparatus 310 of FIG. 3A .

The audio decoding apparatus 430 shown in FIG. 4B includes a parameter decoding unit 432 , a mode determining unit 433 , a frequency domain decoding unit 434 , a frequency domain excitation decoding unit 435 , and a time domain excitation decoding unit 436 . ), an LP synthesis unit 437 and a post-processing unit 438 may be included. Here, the frequency domain decoding unit 434, the frequency domain excitation decoding unit 435, and the time domain excitation decoding unit 436 may each include an FEC or PLC algorithm in a corresponding domain. Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown). Since the audio decoding device 430 shown in FIG. 4B can be viewed as a combination of the audio decoding device 230 of FIG. 2B and the audio decoding device 330 of FIG. 3B , the description of common parts of the operation is omitted, while the mode An operation of the determination unit 433 will be described.

The mode determination unit 433 checks the encoding mode information included in the bitstream and provides the current frame to the frequency domain decoding unit 434 , the frequency domain excitation decoding unit 435 , or the time domain excitation decoding unit 436 .

The frequency domain decoding unit 434 is connected to the frequency domain decoding unit 134 of the audio encoding apparatus 130 of FIG. 1B or the frequency domain decoding unit 234 of the audio decoding apparatus 230 of FIG. 2B, and the frequency domain excitation decoding unit (435) or the time domain excitation decoding unit 436 may correspond to the frequency domain excitation decoding unit 334 or the time domain excitation decoding unit 335 of the audio decoding apparatus 330 of FIG. 3B.

5 is a block diagram showing the configuration of a frequency domain audio encoding apparatus to which the present invention is applied.

The frequency domain audio encoding apparatus 510 shown in FIG. 5 includes a transient detection unit 511 , a transformation unit 512 , a signal classification unit 513 , an energy encoding unit 514 , a spectrum normalization unit 515 , and a bit allocation unit. 516 , a spectrum encoder 517 and a multiplexer 518 may be included. Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown). Here, the frequency domain audio encoding apparatus 510 may perform all functions of the frequency domain encoding unit 214 shown in FIG. 2 and some functions of the parameter encoding unit 216 . On the other hand, the frequency domain audio encoding apparatus 510 may be replaced with the encoder configuration disclosed in the ITU-T G.719 standard except for the signal classifying unit 513, and in this case, the transforming unit 512 has an overlap of 50%. A transformation window with a section can be used. Also, the frequency domain audio encoding apparatus 510 may be replaced with the encoder configuration disclosed in the ITU-T G.719 standard except for the transient detection unit 511 and the signal classification unit 513 . In each case, although not shown, a noise level estimation unit is further provided at the rear end of the spectrum encoding unit 517 as in the ITU-T G.719 standard, so that in the bit allocation process, a zero bit is allocated for the spectral coefficient. The noise level may be estimated and included in the bitstream.

Referring to FIG. 5 , the transient detection unit 511 may analyze an input signal to detect a section indicating a transient characteristic, and generate transient signaling information for each frame in response to the detection result. In this case, various known methods may be used to detect the transient section. According to an embodiment, the transient detector 511 may first determine whether the current frame is a transient frame, and may secondarily perform verification on the current frame determined as the transient frame. The transient signaling information may be included in the bitstream through the multiplexer 518 and provided to the converter 512 .

The transform unit 512 may determine a window size used for transformation according to a detection result of the transient section, and perform time-frequency transformation based on the determined window size. As an example, a short window may be applied to a subband in which a transient period is detected, and a long window may be applied to a subband in which a transient period is not detected. As another example, a short-term window may be applied to a frame including a transient period.

The signal classifying unit 513 may analyze the spectrum provided from the transform unit 512 in units of frames to determine whether each frame corresponds to a harmonic frame. In this case, various known methods may be used to determine the harmonic frame. According to an exemplary embodiment, the signal classifying unit 513 may divide the spectrum provided from the converting unit 512 into a plurality of subbands, and obtain a peak value and an average value of energy for each subband. Next, for each frame, the number of subbands having a peak value of energy greater than the average value by a predetermined ratio or more may be obtained, and a frame having the obtained number of subbands greater than or equal to a predetermined value may be determined as a harmonic frame. Here, the predetermined ratio and the predetermined value may be determined in advance through experiments or simulations. The harmonic signaling information may be included in the bitstream through the multiplexer 518 .

The energy encoder 514 may obtain energy in units of each subband and perform quantization and lossless encoding. According to an embodiment, as the energy, a Norm value corresponding to the average spectral energy of each subband may be used, and a scale factor or power may be used instead, but is not limited thereto. Here, the Norm value of each subband is provided to the spectrum normalizer 515 and the bit allocator 516 , and may be included in the bitstream through the multiplexer 518 .

The spectrum normalizer 515 may normalize the spectrum by using the Norm value obtained for each subband.

The bit allocator 516 may allocate bits in units of integers or decimal points by using the Norm value obtained in units of each subband. In addition, the bit allocator 516 may calculate a masking threshold using the Norm value obtained for each subband, and estimate the number of perceptually necessary bits, that is, the number of allowable bits using the masking threshold. Next, the bit allocator 516 may limit the number of allocated bits for each subband so as not to exceed the allowable number of bits. Meanwhile, the bit allocator 516 sequentially allocates bits from the subband having a large Norm value, and assigns a weight to the Norm value of each subband according to the perceptual importance of each subband, thereby perceptually important subbands. can be adjusted to allocate more bits to At this time, the quantized Norm value provided from the Norm encoder 514 to the bit allocator 516 is pre-adjusted in order to consider psycho-acoustical weighting and masking effects as in ITU-T G.719. It can then be used for bit allocation.

The spectrum encoder 517 may perform quantization with respect to the normalized spectrum by using the number of allocated bits of each subband, and may perform lossless encoding on the quantized result. As an example, TCQ (Trellis Coded Quantizer), USQ (Uniform Scalar Quantizer), FPC (Factorial Pulse Coder), AVQ (Analog Vector Quantizer), PVQ (Predictive Vector Quantizer) or a combination thereof and each quantizer correspond to spectral coding A lossless encoder can be used. In addition, various spectrum encoding techniques can be applied according to the environment in which the corresponding codec is mounted or the needs of the user. Information on the spectrum encoded by the spectrum encoder 517 may be included in the bitstream through the multiplexer 518 .

6 is a block diagram showing the configuration of a frequency domain audio encoding apparatus to which the present invention is applied.

The audio encoding apparatus 600 illustrated in FIG. 6 may include a preprocessor 610 , a frequency domain encoder 630 , a time domain encoder 650 , and a multiplexer 670 . The frequency domain encoder 630 may include a transient detector 631 , a transform unit 633 , and a spectrum encoder 6355 . Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown).

6 , the preprocessor 610 may perform filtering or downsampling on the input signal, but is not limited thereto. The preprocessor 610 may determine an encoding mode based on signal characteristics. Whether the encoding mode suitable for the current frame is the voice mode or the music mode may be determined according to the signal characteristics, and it may also be determined whether the encoding mode efficient for the current frame is the time domain mode or the frequency domain mode. Here, the signal characteristics may be identified using a short-term characteristic of a frame or a long-term characteristic of a plurality of frames, but is not limited thereto. For example, if the input signal corresponds to a voice signal, the voice mode or time domain mode is determined. . The preprocessor 610 converts the input signal to the frequency domain encoder 630 when the signal characteristic corresponds to the music mode or the frequency domain mode, and converts the input signal to the time domain when the signal characteristic corresponds to the voice mode or the time domain mode. It may be provided to the encoder 650 .

The frequency domain encoder 630 may process the audio signal provided from the preprocessor 610 based on transform encoding. Specifically, the transient detector 631 may determine whether the current frame is a transient frame by detecting a transient component from the audio signal. The transform unit 633 may determine the length or shape of the transform window based on the frame type, ie, transient information, provided from the transient detector 631 , and convert the audio signal into the frequency domain based on the determined transform window. As a transformation method, MDCT, FFT, or MLT may be applied. In general, a short-length transform window may be applied to a frame having a transient component. The spectrum encoder 635 may perform encoding on the audio spectrum transformed into the frequency domain. The spectrum encoder 635 will be described in more detail with reference to FIGS. 7 and 9 .

The time domain encoder 650 may perform Code Excited Linear Prediction (CELP) encoding on the audio signal provided from the preprocessor 610 . Specifically, ACELP (Algebraic CELP) may be used, but is not limited thereto.

The multiplexer 670 multiplexes a spectrum component or a signal component generated as a result of encoding by the frequency domain encoder 630 or the time domain encoder 650 with various indexes to generate a bitstream, and the bitstream is a packet through a channel. It may be transmitted in a form or stored in a storage medium.

7 is a block diagram showing the configuration of a spectrum encoding apparatus according to an embodiment. The apparatus illustrated in FIG. 7 may correspond to the spectrum encoding unit 635 of FIG. 6 , may be included in another frequency domain encoding apparatus, or may be implemented independently.

The spectrum encoding apparatus 700 shown in FIG. 7 includes an energy estimator 710 , an energy quantization and encoding unit 720 , a bit allocator 730 , a spectrum normalization unit 740 , and a spectrum quantization and encoding unit 750 . and a noise peeling unit 760 .

Referring to FIG. 7 , the energy estimator 710 may divide the original spectral coefficients into subbands and estimate the energy of each subband, for example, a Norm value. Here, each subband in one frame may have the same size or the number of spectral coefficients included in each subband may increase from a low band to a high band.

The energy quantization and encoding unit 720 may quantize and encode the Norm value estimated for each subband. In this case, the Norm value may be quantized in various ways, such as vector quantization, scalar quantization, TCQ, and lattice vector quantization (LVQ). The energy quantization and encoding unit 720 may additionally perform lossless encoding to further improve encoding efficiency.

The bit allocator 730 may allocate bits necessary for encoding while considering allowable bits per frame by using the quantized Norm value for each subband.

The spectrum normalizer 740 may normalize the spectrum by using the quantized Norm value for each subband.

The spectrum quantization and encoding unit 750 may perform quantization and encoding on the normalized spectrum based on bits allocated to each subband.

The noise peeling unit 760 may add appropriate noise to the portion quantized to 0 due to the limitation of allowable bits in the spectral quantization and encoding unit 750 .

8 is a diagram illustrating an example of subband division.

Referring to FIG. 8 , when an input signal uses a sampling frequency of 48 kHz and has a frame size of 20 ms, the number of samples to be processed per frame is 960. That is, 960 spectral coefficients are obtained when the input signal is transformed by applying an overlap of 50% using MDCT. Here, the overlapping ratio may be variously set according to the encoding method. In the frequency domain, it is theoretically possible to process up to 24 kHz, but it is decided to express a band up to 20 kHz in consideration of the human audible band. In the low band of 0 to 3.2 kHz, 8 spectrum coefficients are grouped into one subband, and in the band of 3.2 to 6.4 kHz, 16 spectrum coefficients are grouped into one subband. In the band of 6.4~13.6kHz, 24 spectral coefficients are grouped into one subband and used, and in the band of 13.6~20kHz, 32 spectral coefficients are grouped and used as one subband. When encoding is performed by obtaining an actual Norm value, it is possible to obtain and encode a Norm up to a band determined by the encoder. In a specific high band after a predetermined band, encoding based on various methods such as band extension is possible.

9 is a block diagram illustrating a configuration of a spectral quantization apparatus according to an embodiment.

The apparatus shown in FIG. 9 may include a quantizer selection unit 910 , a USQ 930 , and a TCQ 950 .

9 , the quantizer selector 910 may select the most efficient quantizer from among various quantizers according to the characteristics of the input signal, that is, the signal to be quantized. As characteristics of the input signal, bit allocation information for each band, band size information, and the like can be used. According to the selection result, the signal to be quantized is provided to one of the USQ 830 and the TCQ 850 to perform corresponding quantization.

10 is a block diagram showing the configuration of a spectrum encoding apparatus according to an embodiment. The apparatus illustrated in FIG. 10 may correspond to the spectral quantization and encoding unit 750 of FIG. 7 , may be included in another frequency domain encoding apparatus, or may be implemented independently.

The apparatus shown in FIG. 10 includes an encoding method selection unit 1010, a zero encoding unit 1020, a scaling unit 1030, an ISC encoding unit 1040, a quantization component restoration unit 1050, and an inverse scaling unit 1060. may include Here, the quantization component restoration unit 1050 and the inverse scaling unit 1060 may be optionally provided.

Referring to FIG. 10 , the encoding method selection unit 1010 may select an encoding method in consideration of input signal characteristics. The input signal characteristics may include bits allocated for each band. The normalized spectrum may be provided to the zero encoder 1020 or the scaling unit 1030 based on the encoding method selected for each band. According to an embodiment, when the average number of bits allocated to each sample of a band is greater than or equal to a predetermined value, for example, 0.75, the corresponding band is determined to be very important and USQ is used, while TCQ can be used for all other bands. Here, the average number of bits may be determined in consideration of a band length or a band size. The selected encoding method may be set using a 1-bit flag.

The zero encoder 1020 may encode all samples to 0 for a band in which the allocated bit is 0.

The scaling unit 1030 may adjust the bit rate by scaling the spectrum based on the bits allocated to the band. In this case, a normalized spectrum may be used. The scaling unit 1030 may perform scaling in consideration of the average number of bits allocated to each sample included in the band, that is, a spectrum coefficient. For example, the larger the average number of bits, the greater the scaling can be done.

According to an embodiment, the scaling unit 1030 may determine an appropriate scaling value according to bit allocation for each band.

Specifically, first, the number of pulses for the current band may be estimated using the band length and bit allocation information. Here, the pulse may mean a unit pulse. First, based on Equation 1 below, it is possible to calculate the bit (b) actually required in the current band.

Here, n denotes the length of the band, m denotes the number of pulses, and i denotes the number of non-zero positions having the Important Spectral Component (ISC).

On the other hand, the number of non-zero positions can be obtained based on probability, for example, as in Equation 2 below.

And, the number of bits required for non-zero positions can be estimated as in Equation 3 below.

Finally, finally, the number of pulses may be selected by a value of b having a value closest to the bit assigned to each band.

Next, the initial scaling factor may be determined using the estimated number of pulses obtained for each band and the absolute value of the input signal. The input signal may be scaled by an initial scaling factor. If the sum of the number of pulses for the scaled original signal, that is, the quantized signal, is not equal to the estimated value of the number of pulses, a pulse redistribution process may be performed using the updated scaling factor. In the pulse redistribution process, if the number of pulses selected for the current band is less than the estimated number of pulses obtained for each band, the scaling factor is decreased to increase the number of pulses. In this case, it is possible to increase or decrease by a predetermined value by selecting a position that minimizes distortion with the original signal.

Since the distortion function for TSQ requires a relative size rather than an exact distance, it can be obtained as the sum of the square distances of each quantized and inverse-quantized value in each band as in Equation 4 below.

Here, p _i is an actual value, and q _i is a quantized value.

Meanwhile, the distortion function for USQ may use the Euclidean distance to determine the best quantized value. In this case, a modified equation including a scaling factor is used to minimize complexity, and the distortion function may be calculated by the following equation (5).

If the number of pulses per band does not match the required value, it is necessary to add or subtract a predetermined number of pulses while maintaining the minimum metric. This can be performed by repeating the process of adding or subtracting one pulse until the number of pulses reaches a required value.

In order to add or subtract one pulse, it is necessary to obtain n distortion values to obtain the most optimal distortion value. For example, the distortion value j may correspond to adding a pulse to the j-th position in the band as shown in Equation 6 below.

In order to avoid performing Equation 6 n times, a deviation as in Equation 7 below may be used.

는 한번만 계산하면 된다. 한편, n은 밴드 길이 즉, 밴드에 있는 계수 갯수를 나타내며, p는 원신호 즉, 양자화기의 입력신호, q는 양자화된 신호, g는 스케일링 팩터를 나타낸다. 최종적으로, 왜곡 d를 최소화하는 위치 j가 선택되고, qj 가 업데이트될 수 있다. In Equation 7 above

is calculated only once. Meanwhile, n represents the band length, that is, the number of coefficients in the band, p represents the original signal, that is, the input signal of the quantizer, q represents the quantized signal, and g represents the scaling factor. Finally, a position j that minimizes distortion d is selected, and qj can be updated.

Meanwhile, in order to control the bit rate, an appropriate ISC may be selected and encoded using scaled spectral coefficients. Specifically, the spectral component to quantize can be selected using the bit allocation of each band. In this case, the spectral component may be selected based on various combinations according to the distribution and dispersion of the spectral components. Then, the actual non-zero positions can be calculated. The non-zero position can be obtained by analyzing the scaling amount and the redistribution operation, and the non-zero position selected in this way can be called ISC in other words. In summary, it is possible to obtain the optimal scaling factor and non-zero position information corresponding to the ISC by analyzing the magnitude of the signal that has undergone the scaling and redistribution process. Here, the non-zero position information refers to the number and positions of non-zero positions. If the number of pulses is not adjusted through the scaling and redistribution process, the selected pulse is quantized through the actual TCQ process, and the excess bits can be adjusted using the result. Examples of this process are as follows.

In the case of a condition that the number of non-zero positions and the estimated number of pulses obtained for each band are not the same, and the number of non-zero positions is greater than a predetermined value, for example, 1 and the obtained quantizer selection information indicates TCQ, excess bits are obtained through actual TCQ quantization. Can be adjusted. Specifically, when the above condition is satisfied, a TCQ quantization process is first performed in order to adjust the redundant bits. If the number of pulses in the current band obtained through the actual TCQ quantization is smaller than the estimated number of pulses obtained for each band in advance, the scaling factor is increased by multiplying the previously determined scaling factor by a value greater than 1, for example, 1.1, and vice versa. In this case, the scaling factor is reduced by multiplying it by a value less than 1, for example, 0.9. By repeating this process, when the estimated number of pulses obtained for each band and the number of pulses of the current band obtained through TCQ quantization are the same, bits used in the actual TCQ quantization process are calculated and the excess bits are updated. The non-zero position obtained in this way may correspond to the ISC.

The ISC encoder 1040 may encode information on the number of finally selected ISCs and non-zero position information. In this process, lossless encoding may be applied to increase encoding efficiency. The ISC encoder 1040 may perform encoding on a non-zero band in which an allocated bit is not 0 using a selected quantizer. Specifically, the ISC encoder 1040 may select an ISC for each band with respect to the normalized spectrum, and encode information of the selected ISC for each band based on the number, location, size, and sign. In this case, the size of the ISC may be coded in a different way from the number, position, and sign. For example, the size of the ISC may be quantized and arithmetic-coded using one of USQ and TCQ, while arithmetic coding may be performed on the number, position, and sign of the ISC. If it is determined that a specific band contains important information, USQ may be used, and if not, TCQ may be used. According to an embodiment, one of TCQ and USQ may be selected based on signal characteristics. Here, the signal characteristics may include bits allocated to each band or the length of the band. If the average number of bits allocated to each sample included in the band is greater than or equal to a threshold, for example, 0.75, the corresponding band may be determined to contain very important information, and thus USQ may be used. Meanwhile, even in the case of a low band having a short band length, USQ may be used if necessary. According to another embodiment, one of the first joint method and the second joint method may be used according to the bandwidth. For example, for NB and WB, the first joint method in which quantizer selection is performed by additionally using not only original bit allocation information for each band, but also secondary bit allocation processing for residual bits from a previously coded band may be used, and a second joint method using TCQ for LSB (Least Significant Bit) for a band determined to use USQ for SWB and FB may be used. In the first joint scheme, the secondary bit allocation process distributes excess bits from a previously coded band, and two bands can be selected. Meanwhile, in the second joint method, the remaining bits may use USQ.

The quantized component restoration unit 1050 may restore the actual quantized component by adding ISC position, size, and sign information to the quantized component. Here, 0 may be assigned to a zero position, that is, a spectral coefficient encoded as zero.

The inverse scaling unit 1060 may perform inverse scaling on the reconstructed quantized component to output a quantized spectrum coefficient having the same level as that of the normalized input spectrum. The same scaling factor may be used in the scaling unit 1030 and the inverse scaling unit 1060 .

11 is a block diagram showing the configuration of an ISC encoding apparatus according to an embodiment.

The apparatus shown in FIG. 11 may include an ISC selection unit 1110 and an ISC information encoding unit 1130 . The device of FIG. 11 may correspond to the ISC encoder 1040 of FIG. 10 or may be implemented as an independent device.

11 , the ISC selection unit 1110 may select an ISC based on a predetermined criterion from a scaled spectrum in order to adjust the bit rate. The ISC selector 1110 may obtain an actual non-zero position by analyzing the scaled degree from the scaled spectrum. Here, ISC may correspond to an actual non-zero spectral coefficient before scaling. The ISC selector 1110 may select a spectral coefficient to be encoded, that is, a non-zero position, in consideration of the distribution and dispersion of the spectral coefficients based on the bits allocated for each band. TCQ can be used for ISC selection.

The ISC information encoder 1130 may decode ISC information, that is, ISC number information, location information, size information, and a code, based on the selected ISC.

12 is a block diagram showing the configuration of an ISC information encoding apparatus according to an embodiment.

The apparatus shown in FIG. 12 may include a location information encoder 1210 , a size information encoder 1230 , and a code encoder 1250 .

12 , the location information encoder 1210 may encode location information of the ISC selected by the ISC selection unit ( 1110 of FIG. 11 ), that is, location information of non-zero spectral coefficients. The location information may include the number and location of the selected ISC. For encoding the location information, arithmetic coding may be used. On the other hand, a new buffer can be configured by collecting the selected ISCs. Zero bands and unselected spectra can be excluded for ISC collection.

The size information encoder 1230 may encode the size information of the newly configured ISC. In this case, quantization may be performed by selecting one of TCQ and USQ, and then arithmetic encoding may be additionally performed. In order to increase the efficiency of arithmetic encoding, non-zero location information and the number of ISCs may be used.

The code information encoding unit 1250 may perform encoding on the selected ISC code information. Arithmetic encoding may be used for encoding the code information.

13 is a block diagram showing the configuration of a spectrum encoding apparatus according to another embodiment. The apparatus illustrated in FIG. 13 may correspond to the spectral quantization and encoding unit 750 of FIG. 7 , may be included in another frequency domain encoding apparatus, or may be implemented independently.

The apparatus shown in FIG. 13 may include a scaling unit 1330 , an ISC encoding unit 1340 , a quantization component restoration unit 1350 , and an inverse scaling unit 1360 . Compared with FIG. 10 , the operation of each component is the same except that the zero encoder 1020 and the encoding method selector 1010 are omitted, and the ISC encoder 1340 can use TCQ.

14 is a block diagram showing the configuration of a spectrum encoding apparatus according to another embodiment. The apparatus illustrated in FIG. 14 may correspond to the spectral quantization and encoding unit 750 of FIG. 7 , may be included in another frequency domain encoding apparatus, or may be implemented independently.

The apparatus shown in FIG. 14 may include an encoding method selection unit 1410 , a scaling unit 1430 , an ISC encoding unit 1440 , a quantization component restoration unit 1450 , and an inverse scaling unit 1460 . Compared with FIG. 10 , the operation of each component is the same except that the zero encoder 1020 is omitted.

15 is a diagram illustrating a concept of an ISC collection and encoding process according to an embodiment. First, a zero band, that is, a band to be quantized to 0 is excluded. Next, a new buffer may be constructed using an ISC selected from among the spectral components existing in the non-zero band. TCQ may be performed on a band-by-band basis on the newly configured ISC, and corresponding lossless encoding may be performed.

16 is a diagram illustrating the concept of an ISC collection and encoding process and an ISC collection process according to another embodiment. First, a zero band, that is, a band to be quantized to 0 is excluded. Next, a new buffer may be constructed using an ISC selected from among the spectral components existing in the non-zero band. USC or TCQ may be performed on a band-by-band basis with respect to the newly configured ISC, and corresponding lossless encoding may be performed.

17 is a diagram illustrating an example of TCQ used in the present invention, and corresponds to a trellis structure of an 8-state 4 coset having two zero levels. A detailed description of the TCQ is disclosed in US 7605727.

18 is a block diagram showing the configuration of a frequency domain audio decoding apparatus to which the present invention can be applied.

The frequency domain audio decoding apparatus 1800 shown in FIG. 18 may include a frame error detection unit 1810 , a frequency domain decoding unit 1830 , a time domain decoding unit 1850 , and a post-processing unit 1870 . The frequency domain decoding unit 1830 may include a spectrum decoding unit 1831 , a memory update unit 1833 , an inverse transform unit 1835 , and an overlap and add (OLA) unit 1837 . Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown).

Referring to FIG. 18 , the frame error detection unit 1810 may detect whether a frame error has occurred from a received bitstream.

The frequency domain decoding unit 1830 operates when the encoding mode is the music mode or the frequency domain mode, and operates the FEC or PLC algorithm when a frame error occurs. generate a signal Specifically, the spectrum decoding unit 1831 may synthesize spectrum coefficients by performing spectrum decoding using the decoded parameters. The spectrum decoder 1831 will be described in more detail with reference to FIGS. 19 and 20 .

The memory update unit 1833 stores spectral coefficients synthesized with respect to the current frame, which is a normal frame, information obtained using decoded parameters, the number of consecutive error frames, and signal characteristics or frame type information of each frame to the next frame. can be updated for Here, the signal characteristic may include a transient characteristic and a stationary characteristic, and the frame type may include a transient frame, a stationary frame, or a harmonic frame.

The inverse transform unit 1835 may generate a time domain signal by performing time-frequency inverse transform on the synthesized spectral coefficients.

The OLA unit 1837 may perform OLA processing using the time domain signal of the previous frame, and as a result, may generate a final time domain signal for the current frame and provide it to the post-processing unit 1870 .

The time domain decoding unit 1850 operates when the encoding mode is the voice mode or the time domain mode, operates the FEC or PLC algorithm when a frame error occurs, and when a frame error does not occur, the time domain through the general CELP decoding process generate a signal

The post-processing unit 1870 may perform filtering or upsampling on the time domain signal provided from the frequency domain decoding unit 1830 or the time domain decoding unit 1850, but is not limited thereto. The post-processing unit 1670 provides the restored audio signal as an output signal.

19 is a block diagram illustrating a configuration of a spectrum decoding apparatus according to an embodiment. The apparatus shown in FIG. 19 may correspond to the spectrum decoding unit 1831 of FIG. 18 , may be included in another frequency domain decoding apparatus, or may be implemented independently.

The spectrum decoding apparatus 1900 shown in FIG. 19 includes an energy decoding and inverse quantization unit 1910, a bit allocator 1930, a spectrum decoding and inverse quantization unit 1950, a noise filling unit 1970, and a spectrum shaping unit ( 1990) may be included. Here, the noise filling unit 1970 may be located at a rear end of the spectrum shaping unit 1990 . Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown).

Referring to FIG. 19 , the energy decoding and inverse quantization unit 1910 performs lossless decoding on a parameter to which lossless encoding has been performed in the encoding process, for example, energy such as a Norm value, and inverse quantizes the decoded Norm value. can be performed. In the encoding process, inverse quantization may be performed using a method corresponding to the quantization method of the Norm value.

The bit allocator 1930 may allocate the number of bits required for each subband based on the quantized Norm value or the inverse quantized Norm value. In this case, the number of bits allocated in units of subbands may be the same as the number of bits allocated in the encoding process.

The spectral decoding and inverse quantization unit 1950 performs lossless decoding on the coded spectral coefficients using the number of bits allocated to each subband, and performs inverse quantization on the decoded spectral coefficients to generate normalized spectral coefficients. can do.

The noise filling unit 1970 may fill noise with respect to a portion requiring noise filling for each subband among the normalized spectral coefficients.

The spectrum shaping unit 1990 may shape the normalized spectrum coefficients using the inverse quantized Norm value. A finally decoded spectral coefficient may be obtained through a spectral shaping process.

20 is a block diagram illustrating a configuration of a spectral inverse quantization apparatus according to an embodiment.

The apparatus shown in FIG. 20 may include an inverse quantizer selector 2010, a USQ 2030, and a TCQ 2050.

Referring to FIG. 20 , the inverse quantizer selector 2010 may select the most efficient inverse quantizer from among various inverse quantizers according to characteristics of an input signal, that is, a signal to be inversely quantized. As characteristics of the input signal, bit allocation information for each band, band size information, and the like can be used. According to the selection result, the signal to be inverse quantized is provided to one of the USQ 2030 and the TCQ 2050 to perform corresponding inverse quantization.

21 is a block diagram showing the configuration of a spectrum decoding apparatus according to an embodiment. The apparatus shown in FIG. 21 may correspond to the spectrum decoding and inverse quantization unit 1950 of FIG. 19 , may be included in another frequency domain decoding apparatus, or may be implemented independently.

The apparatus shown in FIG. 21 may include a decoding method selection unit 2110, a zero decoding unit 2130, an ISC decoding unit 2150, a quantization component restoration unit 2170, and an inverse scaling unit 2190. Here, the quantization component restoration unit 2170 and the inverse scaling unit 2190 may be optionally provided.

In FIG. 21 , the decoding method selector 2110 may select a decoding method based on bits allocated for each band. The normalized spectrum may be provided to the zero decoder 2130 or the ISC decoder 2150 based on a decoding method selected for each band.

The zero decoder 2130 may decode all samples to 0 for the band in which the allocated bit is 0.

The ISC decoder 2150 may perform decoding with respect to a band having a non-zero allocated bit by using a selected inverse quantizer. The ISC decoder 2150 may obtain information on important frequency components for each band of the encoded spectrum, and decode the information on important frequency components obtained for each band based on the number, location, size, and sign. The magnitude of the significant frequency component can be decoded in a different way from the number, position, and sign. For example, the magnitude of the significant frequency component is arithmetic-decoded and inverse quantized using one of USQ and TCQ, while arithmetic decoding can be performed on the number, position, and sign of the significant frequency component. The inverse quantizer selection may be performed using the same result as in the ISC encoder 1040 shown in FIG. 10 . The ISC decoder 2150 may perform inverse quantization on a band in which the allocated bit is not 0 using one of TCQ and USQ.

The quantization component restoration unit 2170 may reconstruct an actual quantization component based on the position, size, and sign information of the restored ISC. Here, 0 may be assigned to a zero position, that is, a non-quantized portion that is a spectral coefficient decoded to zero.

In addition, an inverse scaling unit (not shown) may be included to perform inverse scaling on the reconstructed quantized component to output a quantized spectrum coefficient having the same level as that of the normalized spectrum.

22 is a block diagram showing the configuration of an ISC decoding apparatus according to an embodiment.

The apparatus of FIG. 22 may include a pulse number estimator 2210 and an ISC information decoder 2230 . The device of FIG. 22 may correspond to the ISC decoder 2150 of FIG. 21 or may be implemented as an independent device.

22 , the pulse number estimator 2210 may determine an estimate of the number of pulses required in the current band by using the band size and bit allocation information. That is, since the bit allocation information of the current frame is the same as that of the encoder, the same pulse number estimate is derived using the same bit allocation information and decoding is performed.

The ISC information decoding unit 2230 may decode ISC information, ie, ISC number information, location information, size information, and a code, based on the estimated number of pulses.

23 is a block diagram showing the configuration of an ISC information decoding apparatus according to an embodiment.

The apparatus shown in FIG. 23 may include a location information decoding unit 2310 , a size information decoding unit 2330 , and a code decoding unit 2350 .

23 , the location information decoding unit 2310 may restore the number and location of ISCs by decoding an index related to location information included in a bitstream. Arithmetic decoding may be used for decoding the location information. The size information decoding unit 2330 may perform arithmetic decoding on an index related to size information included in the bitstream, and may perform inverse quantization by selecting one of TCQ and USQ on the decoded index. In order to increase the efficiency of arithmetic decoding, non-zero location information and the number of ISCs may be used. The code decoder 2350 may restore the ISC code by decoding the index related to the code information included in the bitstream. Arithmetic decoding may be used for decoding the code information. According to an embodiment, the number of pulses required for the non-zero band may be estimated and used for decoding position information, magnitude information, or code information.

24 is a block diagram showing the configuration of a spectrum decoding apparatus according to another embodiment. The apparatus shown in FIG. 24 may correspond to the spectrum decoding and inverse quantization unit 1950 of FIG. 19 , may be included in another frequency domain decoding apparatus, or may be implemented independently.

The apparatus shown in FIG. 24 may include an ISC decoding unit 2150 , a quantization component restoration unit 2170 , and an inverse scaling unit 2490 . Compared with FIG. 21 , the operation of each component is the same except that the decoding method selection unit 2110 and the zero decoding unit 2130 are omitted, and the ISC decoding unit 2150 uses TCQ.

25 is a block diagram showing the configuration of a spectrum decoding apparatus according to another embodiment. The apparatus shown in FIG. 25 may correspond to the spectrum decoding and inverse quantization unit 1950 of FIG. 19 , may be included in another frequency domain decoding apparatus, or may be implemented independently.

The apparatus shown in FIG. 25 may include a decoding method selection unit 2510 , an ISC decoding unit 2550 , a quantization component restoration unit 2570 , and an inverse scaling unit 2590 . Compared with FIG. 21 , the operation of each component is the same except that the zero decoding unit 2130 is omitted.

26 is a block diagram showing the configuration of an ISC information encoding apparatus according to another embodiment.

The apparatus of FIG. 26 may include a probability calculator 2610 and a lossless encoder 2630 .

In FIG. 26 , the probability calculator 2610 may calculate a probability value for magnitude encoding according to Equations 8 and 9 below by using the number of ISCs, the number of pulses, and TCQ information.

는 각 밴드에서 전송될 ISC 개수중에서 부호화되고 남은 개수,

은 각 밴드에서 전송될 펄스의 개수중의 부호화되고 남은 개수를 나타내며, Ms는 트렐리스 상태에서 존재하는 크기의 집합을 의미한다. 그리고 j는 크기 중에서 부호화된 펄스 개수를 의미한다. here

is the remaining number of encoded ISCs among the number of ISCs to be transmitted in each band;

denotes the remaining number of encoded pulses among the number of pulses to be transmitted in each band, and Ms denotes a set of sizes existing in the trellis state. And j denotes the number of encoded pulses among the magnitudes.

와

값에 의해 부호화된다. 여기서

값은 이전 크기의 마지막 펄스의 확률을 의미한다. 그리고

는 그 이외의 다른 펄스들에 해당하는 확률을 의미한다. 최종적으로 이와 같이 구해진 확률값에 의해 부호화된 인덱스를 출력한다The lossless encoding unit 2630 may losslessly encode TCQ size information, ie, size and path information, by using the obtained probability value. The number of pulses of each magnitude is

Wow

coded by value. here

The value refers to the probability of the last pulse of the previous magnitude. And

denotes a probability corresponding to other pulses. Finally, an index encoded by the obtained probability value is output.

27 is a block diagram showing the configuration of an ISC information decoding apparatus according to another embodiment.

The apparatus of FIG. 27 may include a probability calculator 2710 and a lossless decoder 2730 .

27, the probability calculator 2710 calculates a probability value for magnitude encoding using ISC information (number i, position), TCQ information, the number of pulses (m), and the size (n) of the band. can To this end, the necessary bit information (b) is obtained using the first obtained number of pulses and the band size. In this case, it can be obtained as in Equation 1 above. Then, a probability value for size encoding is calculated based on Equations (8) and (9) using the obtained bit information (b), the number of ISCs, ISC positions, and TCQ information.

와

값에 의해 복호화된다. 여기서

값은 이전 magnitude의 마지막 펄스의 확률을 의미한다. 그리고

는 그 이외의 다른 펄스들에 해당하는 확률을 의미한다. 최종적으로 이와 같이 구해진 확률값에 의해 복호화된 TCQ 정보 즉 크기 정보와 경로 정보를 출력한다. The lossless decoding unit 2730 may losslessly decode TCQ size information, that is, magnitude information and path information, by using the obtained probability value and transmitted index information in the same manner as in the encoding apparatus. To this end, first, an arithmetic encoding model for number information is created using a probability value, and TCQ size information is decoded by performing arithmetic decoding on the TCQ size information using the obtained model. Specifically, the number of pulses of each magnitude is

Wow

Decrypted by value. here

The value means the probability of the last pulse of the previous magnitude. And

denotes a probability corresponding to other pulses. Finally, TCQ information decoded by the obtained probability value, that is, size information and path information is output.

28 is a block diagram showing the configuration of a multimedia device including an encoding module according to an embodiment of the present invention.

The multimedia device 2800 shown in FIG. 28 may include a communication unit 2810 and an encoding module 2830 . In addition, according to the purpose of the audio bitstream obtained as a result of encoding, the storage unit 2850 for storing the audio bitstream may be further included. Also, the multimedia device 2800 may further include a microphone 2870 . That is, the storage unit 2450 and the microphone 2870 may be provided as options. Meanwhile, the multimedia device 2800 shown in FIG. 28 may further include an arbitrary decryption module (not shown), for example, a decryption module performing a general decryption function or a decryption module according to an embodiment of the present invention. . Here, the encoding module 2830 may be integrated with other components (not shown) included in the multimedia device 2800 and implemented by at least one processor (not shown).

Referring to FIG. 28 , the communication unit 2810 receives at least one of an audio provided from the outside and an encoded bitstream, or transmits at least one of the restored audio and an audio bitstream obtained as a result of encoding by the encoding module 2830. can

The communication unit 2810 is a wireless Internet, wireless intranet, wireless telephone network, wireless LAN (LAN), Wi-Fi (Wi-Fi), Wi-Fi Direct (WFD, Wi-Fi Direct), 3G (Generation), 4G (4 Generation), Bluetooth Wireless networks such as (Bluetooth), Infrared Data Association (IrDA), Radio Frequency Identification (RFID), Ultra WideBand (UWB), Zigbee, Near Field Communication (NFC) or wired telephone networks, such as wired Internet It is configured to transmit/receive data to/from an external multimedia device or server through a wired network.

According to an embodiment, the encoding module 2830 may select an important frequency component for each band with respect to the normalized spectrum, and encode information of the important frequency component selected for each band based on the number, location, size and sign. there is. The size of the significant frequency component can be coded in a different way from the number, position, and sign. For example, the size of the significant frequency component is quantized and arithmetic-coded using one of USQ and TCQ, while the number of important frequency components , arithmetic encoding can be performed on positions and signs. According to an embodiment, scaling may be performed on the normalized spectrum based on bits allocated to each band, and an important frequency component may be selected from the scaled spectrum.

The storage unit 2850 may store various programs necessary for the operation of the multimedia device 2800 .

The microphone 2870 may provide a user or an external audio signal to the encoding module 2830 .

29 is a block diagram showing the configuration of a multimedia device including a decoding module according to an embodiment of the present invention.

The multimedia device 2900 shown in FIG. 29 may include a communication unit 2910 and a decoding module 2930 . In addition, according to the purpose of the restored audio signal obtained as a result of decoding, the storage unit 2950 for storing the restored audio signal may be further included. Also, the multimedia device 2900 may further include a speaker 2970 . That is, the storage unit 2950 and the speaker 2970 may be provided as options. Meanwhile, the multimedia device 2900 shown in FIG. 16 may further include an arbitrary encoding module (not shown), for example, an encoding module performing a general encoding function or an encoding module according to an embodiment of the present invention. . Here, the decoding module 2930 may be integrated with other components (not shown) included in the multimedia device 2900 and implemented by at least one processor (not shown).

Referring to FIG. 29 , the communication unit 2910 receives at least one of an encoded bitstream and an audio signal provided from the outside, or at least one of a reconstructed audio signal obtained as a result of decoding by the decoding module 2930 and an audio bitstream obtained as a result of encoding. You can send one. Meanwhile, the communication unit 2910 may be implemented substantially similarly to the communication unit 2810 of FIG. 28 .

According to an embodiment, the decoding module 2930 receives the bitstream provided through the communication unit 2910, obtains information on important frequency components for each band of the encoded spectrum, and information on important frequency components obtained for each band. can be decoded based on the number, position, size, and sign. The size of the important frequency component can be decoded in a different way from the number, position, and sign. For example, the size of the important frequency component is arithmetic-decoded and inverse quantized using one of USQ and TCQ, while Arithmetic decoding can be performed on the number, position, and sign.

The storage unit 2950 may store the restored audio signal generated by the decoding module 2930 . Meanwhile, the storage unit 2950 may store various programs necessary for the operation of the multimedia device 2900 .

The speaker 2970 may output the restored audio signal generated by the decoding module 2930 to the outside.

30 is a block diagram showing the configuration of a multimedia device including an encoding module and a decoding module according to an embodiment of the present invention.

The multimedia device 3000 shown in FIG. 30 may include a communication unit 3010 , an encoding module 3020 , and a decoding module 3030 . In addition, the storage unit 3040 for storing the audio bitstream or the restored audio signal according to the purpose of the audio bitstream obtained as a result of encoding or the restored audio signal obtained as a result of decoding may be further included. Also, the multimedia device 3000 may further include a microphone 3050 or a speaker 3060 . Here, the encoding module 3020 and the decoding module 3030 may be integrated with other components (not shown) included in the multimedia device 3000 and implemented by at least one processor (not shown).

Since each component shown in FIG. 30 overlaps with the component of the multimedia device 2800 shown in FIG. 28 or the component of the multimedia device 2900 shown in FIG. 29, a detailed description thereof will be omitted.

The multimedia devices 2800, 2900, and 3000 shown in FIGS. 28 to 30 include a voice communication terminal including a telephone, a mobile phone, etc., a broadcasting or music exclusive device including a TV, an MP3 player, or the like, or a dedicated voice communication terminal. It may include, but is not limited to, a terminal and a fusion terminal of a broadcast or music-only device, and a user terminal of a teleconferencing or interaction system. Also, the multimedia devices 2800 , 2900 , and 3000 may be used as a client, a server, or a converter disposed between the client and the server.

On the other hand, when the multimedia devices 2800 , 2900 , and 3000 are, for example, mobile phones, although not shown, a user input unit such as a keypad, a user interface or a display unit for displaying information processed in the mobile phone, and overall functions of the mobile phone are controlled It may further include a processor that does. In addition, the mobile phone may further include a camera unit having an imaging function and at least one or more components performing functions required by the mobile phone.

On the other hand, if the multimedia device (2800, 2900, 3000) is, for example, a TV, although not shown, it may further include a user input unit such as a keypad, a display unit for displaying received broadcast information, and a processor for controlling overall functions of the TV. can In addition, the TV may further include at least one or more components that perform functions required by the TV.

31 is a flowchart illustrating an operation of a method for encoding a fine structure of a spectrum according to an embodiment.

Referring to FIG. 31 , in operation 3110, an encoding method may be selected. For this, information on each band and bit allocation information may be used. Here, the encoding method may include a quantization method.

In step 3130, it is determined whether the current band is a band in which bit allocation is zero, that is, a zero band.

In operation 3150, all samples in the zero band may be zero-coded.

In operation 3170, a band other than the zero band may be encoded based on the selected quantization method. According to an embodiment, the number of pulses per band may be estimated using the band length and bit allocation information, the number of non-zero positions may be determined, and the number of necessary bits of the non-zero positions may be estimated to determine the final number of pulses. Next, the initial scaling factor may be determined based on the number of pulses per band and the absolute value of the input signal, and the scaling factor may be updated through scaling and pulse redistribution using the initial scaling factor. The spectral coefficient may be scaled using the last updated scaling factor, and an appropriate ISC may be selected using the scaled spectral coefficient. A spectral component to be quantized may be selected based on bit allocation information of each band. Next, the size of the collected ISC can be quantized and arithmetic by the USC and TCQ joint schemes. Here, in order to increase the efficiency of arithmetic encoding, non-zero positions and the number of ISCs may be used. The USC and TCQ joint schemes may have a first joint scheme and a second joint scheme according to bandwidth. The first joint method is that quantizer selection is performed using the secondary bit allocation processing for the excess bits from the previous band, and can be used for NB and WB, and the second joint method is used for the LSB for the band determined by USQ. For SWB and FB, TCQ is used and the remaining bits are USQ. Meanwhile, code information of the selected ISC may be arithmetic-decoded with the same probability for negative and positive codes.

After step 3170, the step of additionally reconstructing the quantization component and the step of inverse-scaling the band may be included. In order to reconstruct the actual quantized component of each band, position, sign, and size information may be added to the quantized component. A zero may be assigned to the zero position. Meanwhile, an inverse scaling factor may be extracted using the same scaling factor as used for scaling, and inverse scaling may be performed on the restored actual quantized component. The descaled signal may have a normalized spectrum, that is, the same level as the input signal.

For each step of FIG. 31 , an operation of each component of the above-described encoding apparatus may be further added as needed.

32 is a flowchart illustrating an operation of a method for decoding a fine structure of a spectrum according to an embodiment. According to the method of FIG. 32, in order to inverse quantize the fine structure of the normalized spectrum, information on the ISC and the selected ISC for each band is decoded by position, number, sign, and size. Here, size information is decoded by arithmetic decoding and a USQ and TCQ joint method, and position, number, and sign information are decoded by arithmetic decoding.

Specifically, referring to FIG. 32 , a decoding method may be selected in operation 3210 . For this, information on each band and bit allocation information may be used. Here, the decoding method may include an inverse quantization method. The inverse quantization method may be selected through the same process as the quantization method selection applied in the above-described encoding apparatus.

In step 3230, it is determined whether the current band is a band in which bit allocation is zero, that is, a zero band.

In operation 3250, all samples in the zero band may be decoded to zero.

In operation 3270, a non-zero band may be decoded based on the selected inverse quantization method. According to an embodiment, the number of pulses per band may be estimated or determined using the band length and bit allocation information. This may be performed through the same process as the scaling applied in the above-described encoding apparatus. Next, location information of the ISC, that is, the number and location of the ISC may be restored. This is processed similarly to the above-described encoding apparatus, and the same probability value may be used for proper decoding. Next, the size of the collected ISC may be decoded by arithmetic decoding and dequantized by a USC and TCQ joint scheme. Here, the non-zero positions and the number of ISCs may be used for arithmetic decoding. The USC and TCQ joint schemes may have a first joint scheme and a second joint scheme according to bandwidth. In the first joint method, quantizer selection is performed by additionally using the secondary bit allocation processing for the excess bits from the previous band, and it can be used for NB and WB, and the second joint method is used for the LSB for the band determined by USQ. For SWB and FB, TCQ is used and the remaining bits are USQ. Meanwhile, code information of the selected ISC may be arithmetic-decoded with the same probability for negative and positive codes.

After step 3270, the step of additionally reconstructing the quantization component and the step of inverse-scaling the band may be included. In order to reconstruct the actual quantized component of each band, position, sign, and size information may be added to the quantized component. A band with no data to be transmitted may be filled with zeros. Next, the number of pulses in the non-zero band is estimated, and position information including the number and position of ISCs may be decoded based on the estimated number of pulses. Lossless decoding and decoding using USC and TCQ joint schemes may be performed on the size information. For a non-zero magnitude value, the sign and the quantized component may be finally reconstructed. Meanwhile, inverse scaling may be performed using the transmitted norm information on the reconstructed actual quantization component.

For each step of FIG. 32 , an operation of each component of the above-described decoding apparatus may be further added as needed.

The above embodiments can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. In addition, the data structure, program command, or data file that can be used in the above-described embodiments of the present invention may be recorded in a computer-readable recording medium through various means. The computer-readable recording medium may include any type of storage device in which data readable by a computer system is stored. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and floppy disks. Hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like may be included. In addition, the computer-readable recording medium may be a transmission medium for transmitting a signal designating a program command, a data structure, and the like. Examples of program instructions may include high-level language codes that can be executed by a computer using an interpreter as well as machine language codes such as those generated by a compiler.

As described above, although one embodiment of the present invention has been described with reference to the limited embodiments and drawings, one embodiment of the present invention is not limited to the above-described embodiments, which are common knowledge in the field to which the present invention pertains. Various modifications and variations are possible from such a base material. Accordingly, the scope of the present invention is shown in the claims rather than the above description, and all equivalents or equivalent modifications thereof will fall within the scope of the technical spirit of the present invention.

Claims (15)

if the bit allocated to the band is 0, encoding the samples in the band as 0;
If the bit allocated to the band is not 0, either a quantization method of USQ (Uniform Scalar Quantization) and TCQ (Trellis Coded Quantization) is selected according to the comparison result of the average number of bits allocated to each sample of the band and a predetermined value. selecting;
performing scaling on a spectral component of the band based on the bits allocated to the band;
selecting a major spectral component of the band based on the scaled spectral component of the band; and
and encoding the size of the main spectral component of the band by applying the selected quantization method to the size of the main spectral component of the band. According to claim 1,
The spectrum encoding method comprises:
If the bit allocated to the band is not 0, encoding information on the number, position, and sign of major spectral components of the band. 3. The method of claim 2, wherein the magnitude of the major spectral component is encoded in a manner different from the number, position, and sign of the major spectral component.

delete delete delete delete delete delete delete delete delete delete delete delete

Images