JP6633547B2 - Spectrum coding method - Google Patents

Description

  The present invention relates to audio signal encoding or speech signal encoding and decoding, and more particularly, to a method and apparatus for encoding or decoding spectral coefficients in the frequency domain.

  Various types of quantizers have been proposed for efficient coding of spectral coefficients in the frequency domain. For example, there are TCQ (trellis coded quantization), USQ (uniform scalar quantization), FPC (factorial pulse coding), AVQ (algebraic VQ), PVQ (pyramid VQ), etc., and lossless loss optimized for each quantizer. The encoder is implemented together.

  An object of the present invention is to provide a method and apparatus for encoding or decoding spectral coefficients adaptively to various bit rates or various sub-band sizes in the frequency domain. .

  Another object of the present invention is to provide a computer-readable recording medium storing a program for causing a computer to execute the signal encoding method or the signal decoding method.

  Another object of the present invention is to provide a multimedia device that employs a signal encoding device or its decoding device.

  According to one aspect of the present invention, there is provided a spectrum encoding method including: selecting an encoding method based on at least bit allocation information of each band; performing zero encoding on a zero band; Encoding information of important frequency components selected for the band.

  According to one aspect of the present invention, there is provided a spectrum decoding method including: selecting a decoding method based on at least bit allocation information of each band; performing zero decoding on a zero band; Decoding the information of the important frequency component obtained for the input signal.

  It is possible to encode and decode spectral coefficients adaptive to various bit rates and various sub-band sizes. Also, a spectrum can be encoded with TCQ at a fixed bit rate using a bit rate control module designed with a codec that supports multi-rate. At this time, it is possible to code the high performance of the TCQ at an accurate target bit rate and maximize the coding performance of the codec.

FIG. 1 is a block diagram illustrating a configuration of an example of an audio encoding device to which the present invention is applied. FIG. 3 is a block diagram illustrating a configuration of an example of an audio decoding device to which the present invention is applied. FIG. 11 is a block diagram illustrating a configuration of another example of an audio encoding device to which the present invention is applied. FIG. 21 is a block diagram illustrating a configuration of another example of an audio decoding device to which the present invention is applied. FIG. 10 is a block diagram illustrating a configuration of another example of an audio encoding device to which the present invention is applied. FIG. 21 is a block diagram illustrating a configuration of another example of an audio decoding device to which the present invention is applied. FIG. 11 is a block diagram illustrating a configuration of another example of an audio encoding device to which the present invention is applied. FIG. 21 is a block diagram illustrating a configuration of another example of an audio decoding device to which the present invention is applied. FIG. 1 is a block diagram illustrating a configuration of a frequency domain audio encoding device to which the present invention is applied. FIG. 2 is a block diagram illustrating a configuration of a frequency domain audio decoding device to which the present invention is applied. 1 is a block diagram illustrating a configuration of a spectrum encoding device according to one embodiment. 6 is a diagram illustrating an example of subband division. FIG. 1 is a block diagram illustrating a configuration of a spectrum quantization device according to an embodiment. 1 is a block diagram illustrating a configuration of a spectrum encoding device according to one embodiment. 1 is a block diagram illustrating a configuration of an ISC encoding device according to one embodiment. It is a block diagram showing the composition of the ISC information encoding device by one embodiment. It is a block diagram showing the composition of the spectrum encoding device by other embodiments. It is a block diagram showing the composition of the spectrum encoding device by other embodiments. 5 is a diagram illustrating a concept of an ISC collection process and an encoding process according to an embodiment. 9 is a diagram illustrating a concept of an ISC collecting process and an encoding process according to another embodiment. 5 is a diagram illustrating an example of a TCQ used in the present invention. FIG. 2 is a block diagram illustrating a configuration of a frequency domain audio decoding device to which the present invention is applied. It is a block diagram showing the composition of the spectrum decoding device by one embodiment. It is a block diagram showing composition of a spectrum dequantization device by one embodiment. It is a block diagram showing the composition of the spectrum decoding device by one embodiment. It is a block diagram showing the composition of the ISC decoding device by one embodiment. It is a block diagram showing the composition of the ISC information decoding device by one embodiment. FIG. 18 is a block diagram illustrating a configuration of a spectrum decoding device according to another embodiment. FIG. 18 is a block diagram illustrating a configuration of a spectrum decoding device according to another embodiment. It is a block diagram showing the composition of the ISC information encoding device by other embodiments. It is a block diagram showing the composition of the ISC information decoding device by other embodiments. FIG. 1 is a block diagram illustrating a configuration of a multimedia device according to an embodiment. FIG. 11 is a block diagram illustrating a configuration of a multimedia device according to another embodiment. FIG. 11 is a block diagram illustrating a configuration of a multimedia device according to another embodiment. 5 is a flowchart illustrating an operation of a spectrum fine structure encoding method according to an embodiment. 5 is a flowchart illustrating an operation of a method for decoding a fine structure of a spectrum according to an embodiment.

  Although the present invention is capable of various modifications and having various embodiments, certain embodiments are illustrated in the drawings and are specifically described by way of the detailed description. It should be understood, however, that the intention is not to limit the invention to particular embodiments, but to cover all transformations, equivalents or alternatives falling within the spirit and scope of the invention. In the description of the present invention, when it is determined that the specific description of the related art will obscure the gist of the present invention, the detailed description will be omitted.

  Terms such as the first and second are used to describe various components, but the components are not limited by the terms. The terms are only used to distinguish one element from another.

  The terms used in the present invention are merely used for describing particular embodiments, and are not intended to limit the present invention. The terms used in the present invention have been selected, wherever possible, from general terms currently used, while taking into account the function of the present invention, which is based on the intention, case, or new technology of those skilled in the art. It depends on the appearance of. Further, in a specific case, there is a term arbitrarily selected by the applicant, and in that case, its meaning is described in detail in the description part of the invention. Therefore, the terms used in the present invention must be defined based on the meanings of the terms and the general content of the present invention, not the names of the simple terms.

  The singular forms include the plural unless the context clearly dictates otherwise. In the present invention, terms such as "comprising" or "having" also indicate that a feature, number, step, act, component, part, or combination thereof, described in the specification is present. It should be understood that this does not exclude the possibility of the presence or addition of one or more other features, figures, steps, acts, components, parts, or combinations thereof.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  1A and 1B are block diagrams each showing a configuration of an example of an audio encoding device and an audio decoding device to which the present invention is applied.

  The audio encoding device 110 illustrated in FIG. 1A may include a pre-processing unit 112, a frequency domain encoding unit 114, and a parameter encoding unit 116. Each component is integrated into at least one or more modules and is also embodied as at least one or more processors (not shown).

  In FIG. 1A, the preprocessing unit 112 can perform filtering, downsampling, or the like on an input signal, but is not limited thereto. The input signal means a media signal such as audio, music, speech, or a sound indicating a mixed signal thereof, but is hereinafter referred to as an audio signal for convenience of description.

  The frequency domain encoding unit 114 performs time / frequency conversion on the audio signal provided from the preprocessing unit 112, and selects an encoding tool according to the number of channels of the audio signal, the encoding band, and the bit rate. The encoding of the audio signal can be performed using the selected encoding tool. The time-frequency transform uses a modified discrete cosine transform (MDCT), a modulated lapped transform (MLT), or a fast Fourier transform (FFT), but is not limited thereto. Here, when the given number of bits is sufficient, a general transform coding method is applied to the entire band, and when the given number of bits is not sufficient, the band extension method is used for some bands. Can be applied. On the other hand, when the audio signal is stereo or multi-channel, if a given number of bits is sufficient, coding is performed for each channel, and if not enough, a down-mixing method can be applied. The frequency domain encoding unit 114 generates encoded spectral coefficients.

  The parameter encoding unit 116 can extract a parameter from the encoded spectral coefficient provided from the frequency domain encoding unit 114, and encode the extracted parameter. The parameters are extracted, for example, for each sub-band or each band. Each sub-band is a unit in which spectral coefficients are grouped, reflects a critical band, and can have a uniform length or a non-uniform length. In case of having a non-uniform length, a sub-band existing in a low frequency band may have a relatively short length compared to a high frequency band. The number and length of subbands included in one frame vary depending on the codec algorithm, and affect coding performance. On the other hand, the parameters may include, but are not limited to, the sub-band scale factor, power, average energy, or norm. The spectral coefficients and parameters resulting from the encoding form a bit stream and are stored on a recording medium or transmitted via a channel, for example, in packet form.

  The audio decoding apparatus 130 illustrated in FIG. 1B may include a parameter decoding unit 132, a frequency domain decoding unit 134, and a post-processing unit 136. Here, the frequency domain decoding unit 134 may include a frame erasure concealment (FEC) algorithm or a packet loss concealment (PLC) algorithm. Each component is integrated into at least one or more modules and is also embodied as at least one or more processors (not shown).

  In FIG. 1B, a parameter decoding unit 132 decodes an encoded parameter from the received bit stream, and determines whether an error such as erasure or loss has occurred on a frame basis from the decoded parameter. Can be checked. The error check may use various known methods, and provides information regarding whether the current frame is a normal frame, an erased frame, or a lost frame to the frequency domain decoding unit 134. Hereinafter, an erasure frame or a lost frame is referred to as an error frame for the sake of simplicity.

  If the current frame is a normal frame, the frequency domain decoding unit 134 may perform decoding through a general transform decoding process to generate a combined spectral coefficient. On the other hand, if the current frame is an error frame, the frequency domain decoding unit 134 may repeatedly use the spectral coefficient of the previous normal frame as the error frame through the FEC algorithm or the PLC algorithm, or may perform the regression analysis. By scaling and iterating, synthesized spectral coefficients can be generated. The frequency domain decoding unit 134 can perform frequency-to-time conversion on the synthesized spectral coefficients to generate a time-domain signal.

  The post-processing unit 136 can perform filtering or up-sampling on the time domain signal provided from the frequency domain decoding unit 134 for improving sound quality, but is not limited thereto. The post-processing unit 136 provides a restored audio signal as an output signal.

  2A and 2B are block diagrams each showing a configuration of another example of an audio encoding device and an audio decoding device to which the present invention is applied, and have a switching structure.

  The audio encoding device 210 illustrated in FIG. 2A may include a pre-processing unit 212, a mode determining unit 213, a frequency domain encoding unit 214, a time domain encoding unit 215, and a parameter encoding unit 216. Each component is integrated into at least one or more modules and is also embodied as at least one or more processors (not shown).

  In FIG. 2A, the pre-processing unit 212 is substantially the same as the pre-processing unit 112 in FIG. 1A, and a description thereof will be omitted.

  The mode determining unit 213 can determine an encoding mode with reference to the characteristics of the input signal. Depending on the characteristics of the input signal, it can be determined whether the coding mode suitable for the current frame is the voice mode or the music mode, and the efficient coding mode for the current frame is determined in the time domain. Mode or frequency domain mode can be determined. Here, the characteristics of the input signal can be grasped by using the short-period characteristics of a frame or the long-period characteristics of a plurality of frames, but the present invention is not limited to this. For example, if the input signal corresponds to an audio signal, the audio mode or the time domain mode is determined.If the input signal corresponds to a signal other than the audio signal, that is, a music signal or a mixed signal, the mode is set to the music mode or the frequency domain mode. Can be determined. When the characteristics of the input signal correspond to the music mode or the frequency domain mode, the mode determination unit 213 provides the output signal of the preprocessing unit 212 to the frequency domain encoding unit 214, and the characteristics of the input signal are set to the voice mode or the frequency domain mode. If it corresponds to the time domain mode, it can be provided to the time domain encoding unit 215.

  The frequency domain coding unit 214 is substantially the same as the frequency domain coding unit 114 of FIG. 1A, and a description thereof will be omitted.

  The time domain encoding unit 215 can perform CELP (code excited linear prediction) encoding on the audio signal provided from the preprocessing unit 212. Specifically, ACELP (algebraic CELP) can be used, but is not limited thereto.

  The parameter encoding unit 216 extracts a parameter from the encoded spectral coefficient provided from the frequency domain encoding unit 214 or the time domain encoding unit 215, and encodes the extracted parameter. Parameter encoding section 216 is substantially the same as parameter encoding section 116 of FIG. The spectral coefficients and parameters resulting from the encoding form a bitstream with the encoding mode information and are transmitted in packets over the channel or stored on a recording medium.

  The audio decoding device 230 illustrated in FIG. 2B may include a parameter decoding unit 232, a mode determination unit 233, a frequency domain decoding unit 234, a time domain decoding unit 235, and a post-processing unit 236. Here, the frequency domain decoding unit 234 and the time domain decoding unit 235 may each include an FEC algorithm or a PLC algorithm in the domain. Each component is integrated into at least one or more modules and is also embodied as at least one or more processors (not shown).

  In FIG. 2B, a parameter decoding unit 232 can decode parameters from a bit stream transmitted in a packet form and check whether an error has occurred in frame units based on the decoded parameters. For the error check, various known methods can be used, and information on whether the current frame is a normal frame or an error frame is sent to the frequency domain decoding unit 234 or the time domain decoding unit 235. provide.

  The mode determining unit 233 checks the encoding mode information included in the bit stream, and provides the current frame to the frequency domain decoding unit 234 or the time domain decoding unit 235.

  The frequency domain decoding unit 234 operates when the encoding mode is the music mode or the frequency domain mode. When the current frame is a normal frame, the frequency domain decoding unit 234 performs decoding through a general transform decoding process to obtain a synthesized spectrum. Generate coefficients. On the other hand, if the current frame is an error frame and the encoding mode of the previous frame is the music mode or the frequency domain mode, the spectrum coefficient of the previous normal frame is converted to the error frame through the FEC algorithm or the PLC algorithm in the frequency domain. Iterative use, or scaling and repetition via regression analysis, can produce synthesized spectral coefficients. The frequency domain decoding unit 234 can perform frequency-to-time conversion on the synthesized spectral coefficient to generate a time-domain signal.

  The time domain decoding unit 235 operates when the encoding mode is the voice mode or the time domain mode. When the current frame is a normal frame, the time domain decoding unit 235 performs decoding through a general CELP decoding process, and converts the time domain signal. Generate. On the other hand, if the current frame is an error frame and the coding mode of the previous frame is the voice mode or the time domain mode, the FEC algorithm or the PLC algorithm in the time domain can be performed.

  The post-processing unit 236 can perform filtering or up-sampling on the time domain signal provided from the frequency domain decoding unit 234 or the time domain decoding unit 235, but is not limited thereto. The post-processing unit 236 provides a restored audio signal as an output signal.

  FIGS. 3A and 3B are block diagrams each showing a configuration of another example of an audio encoding device and an audio decoding device to which the present invention is applied, and have a switching structure.

  The audio encoding device 310 illustrated in FIG. 3A includes a preprocessing unit 312, a linear prediction (LP) analysis unit 313, a mode determination unit 314, a frequency domain excitation encoding unit 315, a time domain excitation encoding unit 316, and a parameter code. May be included. Each component is integrated into at least one or more modules and is also embodied as at least one or more processors (not shown).

  In FIG. 3A, the pre-processing unit 312 is substantially the same as the pre-processing unit 112 in FIG. 1A, and a description thereof will be omitted.

  LP analysis section 313 performs LP analysis on the input signal to extract LP coefficients, and generates an excitation signal from the extracted LP coefficients. The excitation signal is provided to one of the frequency domain excitation encoding unit 315 and the time domain excitation encoding unit 316 according to an encoding mode.

  The mode determining unit 314 is substantially the same as the mode determining unit 213 in FIG.

  The frequency domain excitation encoding unit 315 operates when the encoding mode is the music mode or the frequency domain mode, and is substantially the same as the frequency domain encoding unit 114 of FIG. 1A except that the input signal is an excitation signal. Therefore, the description is omitted.

  The time domain excitation encoding unit 316 operates when the encoding mode is the voice mode or the time domain mode, and is substantially the same as the time domain encoding unit 215 of FIG. 2A except that the input signal is an excitation signal. Therefore, the description is omitted.

  The parameter coding unit 317 extracts parameters from the coded spectral coefficients provided from the frequency domain excitation coding unit 315 or the time domain excitation coding unit 316, and codes the extracted parameters. Parameter encoding section 317 is substantially the same as parameter encoding section 116 in FIG. 1A, and thus description thereof is omitted. The spectral coefficients and parameters resulting from the encoding form a bitstream with the encoding mode information and are transmitted in packets over the channel or stored on a recording medium.

  3B may include a parameter decoding unit 332, a mode determination unit 333, a frequency domain excitation decoding unit 334, a time domain excitation decoding unit 335, an LP synthesis unit 336, and a post-processing unit 337. Here, the frequency domain excitation decoding unit 334 and the time domain excitation decoding unit 335 may each include an FEC algorithm or a PLC algorithm in the relevant domain. Each component is integrated into at least one or more modules and is also embodied as at least one or more processors (not shown).

  In FIG. 3B, the parameter decoding unit 332 can decode a parameter from a bit stream transmitted in a packet form, and check whether an error has occurred on a frame basis from the decoded parameter. For the error check, various known methods can be used, and information on whether the current frame is a normal frame or an error frame is transmitted to the frequency domain excitation decoding unit 334 or the time domain excitation decoding unit. 335.

  The mode determining unit 333 checks the encoding mode information included in the bit stream, and provides the current frame to the frequency domain excitation decoding unit 334 or the time domain excitation decoding unit 335.

  The frequency domain excitation decoding unit 334 operates when the encoding mode is the music mode or the frequency domain mode. When the current frame is a normal frame, the frequency domain excitation decoding unit 334 performs decoding through a general transform decoding process and synthesizes. Generate spectral coefficients. On the other hand, if the current frame is an error frame and the encoding mode of the previous frame is the music mode or the frequency domain mode, the spectrum coefficient of the previous normal frame is converted to the error frame through the FEC algorithm or the PLC algorithm in the frequency domain. Iterative use, or scaling and repetition via regression analysis, can produce synthesized spectral coefficients. The frequency-domain excitation decoding unit 334 can perform frequency-to-time conversion on the synthesized spectral coefficients to generate an excitation signal that is a time-domain signal.

  The time domain excitation decoding unit 335 operates when the encoding mode is the voice mode or the time domain mode. When the current frame is a normal frame, the time domain excitation decoding unit 335 performs decoding through a general CELP decoding process, and performs time domain signal decoding. To generate an excitation signal. On the other hand, if the current frame is an error frame and the coding mode of the previous frame is the voice mode or the time domain mode, the FEC algorithm or the PLC algorithm in the time domain can be performed.

  The LP synthesis unit 336 performs LP synthesis on the excitation signal provided from the frequency domain excitation decoding unit 334 or the time domain excitation decoding unit 335, and generates a time domain signal.

  The post-processing unit 337 can perform filtering or up-sampling on the time domain signal provided from the LP synthesis unit 336, but is not limited thereto. The post-processing unit 337 provides a restored audio signal as an output signal.

  4A and 4B are block diagrams each showing a configuration of another example of an audio encoding device and an audio decoding device to which the present invention is applied, and have a switching structure.

  The audio encoding device 410 illustrated in FIG. 4A includes a preprocessing unit 412, a mode determination unit 413, a frequency domain encoding unit 414, an LP analysis unit 415, a frequency domain excitation encoding unit 416, and a time domain excitation encoding unit 417. And a parameter encoding unit 418. Each component is integrated into at least one or more modules and is also embodied as at least one or more processors (not shown). Since the audio encoding device 410 shown in FIG. 4A can be regarded as a combination of the audio encoding device 210 of FIG. 2A and the audio encoding device 310 of FIG. 3A, the description of the operation of the common part is omitted. Meanwhile, the operation of the mode determination unit 413 will be described.

  The mode determination unit 413 can determine the encoding mode of the input signal with reference to the characteristics and the bit rate of the input signal. The mode determining unit 413 determines whether the current frame is in the audio mode or the music mode according to the characteristics of the input signal, and determines whether the efficient encoding mode for the current frame is the time domain mode, or The CELP mode and other modes can be determined depending on whether the mode is the frequency domain mode. If the characteristic of the input signal is the voice mode, the mode is determined to be the CELP mode. If the high bit rate is the music mode, the mode is determined to be the FD mode. If so, the audio mode can be determined. When the mode is the FD mode, the mode determination unit 413 provides the input signal to the frequency domain coding unit 414, and when the mode is the audio mode, the mode determination unit 413 provides the input signal to the frequency domain excitation coding unit 416 via the LP analysis unit 415, In the case of the CELP mode, it can be provided to the time-domain excitation encoding unit 417 via the LP analyzing unit 415.

  The frequency domain encoding unit 414 corresponds to the frequency domain encoding unit 114 of the audio encoding device 110 of FIG. 1A or the frequency domain encoding unit 214 of the audio encoding device 210 of FIG. The 416 or the time domain excitation encoding unit 417 corresponds to the frequency domain excitation encoding unit 315 or the time domain excitation encoding unit 316 of the audio encoding device 310 in FIG. 3A.

  The audio decoding device 430 illustrated in FIG. 4B includes a parameter decoding unit 432, a mode determination unit 433, a frequency domain decoding unit 434, a frequency domain excitation decoding unit 435, a time domain excitation decoding unit 436, an LP synthesis unit 437, and a post-processing unit. 438. Here, the frequency domain decoding section 434, the frequency domain excitation decoding section 435, and the time domain excitation decoding section 436 may each include an FEC algorithm or a PLC algorithm in the domain. Each component is integrated into at least one or more modules and is also embodied as at least one or more processors (not shown). The audio decoding device 430 shown in FIG. 4B can be regarded as a combination of the audio decoding device 230 of FIG. 2B and the audio decoding device 330 of FIG. 3B. The operation of the mode determining unit 433 will be described.

  The mode determining unit 433 checks the coding mode information included in the bitstream, and provides the current frame to the frequency domain decoding unit 434, the frequency domain excitation decoding unit 435, or the time domain excitation decoding unit 436.

  The frequency domain decoding unit 434 corresponds to the frequency domain decoding unit 134 of the audio encoding device 130 in FIG. 1B or the frequency domain decoding unit 234 of the audio decoding device 230 in FIG. The decoding unit 436 corresponds to the frequency domain excitation decoding unit 334 or the time domain excitation decoding unit 335 of the audio decoding device 330 in FIG. 3B.

  FIG. 5 is a block diagram showing a configuration of a frequency domain audio encoding device to which the present invention is applied.

  The frequency domain audio encoding device 510 illustrated in FIG. 5 includes a transient detection unit 511, a conversion unit 512, a signal classification unit 513, an energy encoding unit 514, a spectrum normalization unit 515, a bit allocation unit 516, and a spectrum encoding unit. 517 and a multiplexing unit 518. Each component is integrated into at least one or more modules and is also embodied as at least one or more processors (not shown). Here, the frequency domain audio encoding device 510 may perform all functions of the frequency domain encoding unit 214 and some functions of the parameter encoding unit 216 shown in FIG. On the other hand, except for the signal classification unit 513, the frequency domain audio encoding device 510 includes the ITU-TG. Instead, the conversion unit 512 may use a conversion window having a 50% overlap interval. In addition, except for the transient detection unit 511 and the signal classification unit 513, the frequency domain audio encoding device 510 includes the ITU-T G. The configuration of the encoder disclosed in the 719 standard is also substituted. In each case, although not shown, ITU-TG. As in the H.719 standard, a noise level estimator is further provided at the rear end of the spectrum encoder 517, and in a bit allocation process, a noise level for a spectral coefficient to which zero bits are allocated is estimated and included in a bit stream. be able to.

  Referring to FIG. 5, the transient detection unit 511 may analyze an input signal, detect a section exhibiting a transient characteristic, and generate transient signaling information for each frame according to a detection result. At that time, various known methods can be used for detecting the transient section. According to one embodiment, the transient detection unit 511 first determines whether the current frame is a transient frame or not, and determines a secondary frame for the current frame determined to be a transient frame. Verification is performed. The transient signaling information is provided to the converter 512 while being included in the bit stream via the multiplexer 518.

  The conversion unit 512 determines a window size used for conversion based on the detection result of the transient section, and performs time / frequency conversion based on the determined window size. As an example, a short window may be applied to a subband in which a transient section is detected, and a long window may be applied to a subband in which a transient section is not detected. As another example, a short section window can be applied to a frame including a transient section.

  The signal classification unit 513 may analyze the spectrum provided from the conversion unit 512 on a frame-by-frame basis, and determine whether each frame corresponds to a harmonic frame. At this time, various known methods can be used to determine the harmonic frame. According to one embodiment, the signal classification unit 513 can divide the spectrum provided from the conversion unit 512 into a plurality of subbands, and calculate a peak value and an average value of energy for each subband. Next, for each frame, the number of sub-bands whose energy peak value is larger than the average value by a predetermined ratio or more is determined, and a frame whose calculated number of sub-bands is equal to or more than a predetermined value is determined as a harmonic frame. Can be. Here, the predetermined ratio and the predetermined value can be determined in advance through experiments or simulations. The harmonic signaling information may be included in the bit stream via the multiplexing unit 518.

  The energy encoding unit 514 may obtain energy for each subband, and perform quantization and lossless encoding. According to one embodiment, the energy may be a Norm value corresponding to the average spectral energy of each subband, and a scale factor or power may be used instead, but not limited thereto. Absent. Here, the Norm value of each subband is provided to the spectrum normalization unit 515 and the bit allocation unit 516, and may be included in the bit stream via the multiplexing unit 518.

  The spectrum normalization unit 515 can normalize the spectrum using the Norm value obtained for each subband.

  The bit allocation unit 516 can perform bit allocation in integer units or decimal point units using the Norm value obtained in each subband. Also, the bit allocating unit 516 calculates a masking threshold using the Norm value obtained for each subband, and uses the masking threshold to determine the number of bits perceptually required, that is, the allowable bit. The number can be estimated. Next, the bit allocation section 516 can limit the number of allocated bits to each subband so as not to exceed the allowable number of bits. On the other hand, the bit allocation unit 516 sequentially allocates bits in order from the sub-band having a large Norm value, and assigns a weight to the Norm value of each sub-band according to the perceptual importance of each sub-band, thereby obtaining a perceptual value. It can be adjusted so that more bits are allocated to important subbands. At this time, the quantized Norm value provided from the Norm encoding unit 514 to the bit allocation unit 516 is based on ITU-TG. Like 719, it is pre-adjusted to account for psycho-acoustical weighting and masking effects and then used for bit allocation.

  The spectrum coding unit 517 can perform quantization on the normalized spectrum using the number of bits allocated to each subband, and perform lossless coding on the quantized result. . As an example, for spectral coding, TCQ (trellis coded quantizer), USQ (uniform scalar quantizer), FPC (factorial pulse encoder), AVQ (analog vector quantizer), PVQ (predictive vector quantizer), or a combination thereof, A lossless encoder corresponding to the quantizer can be used. In addition, various spectrum coding techniques can be applied depending on the environment in which the codec is mounted or the needs of the user. Information on the spectrum encoded by the spectrum encoding unit 517 may be included in the bit stream via the multiplexing unit 518.

  FIG. 6 is a block diagram showing a configuration of a frequency domain audio encoding device to which the present invention is applied. The audio encoding device 600 illustrated in FIG. 6 may include a pre-processing unit 610, a frequency domain encoding unit 630, a time domain encoding unit 650, and a multiplexing unit 670. The frequency domain coding unit 630 may include a transient detection unit 631, a conversion unit 633, and a spectrum coding unit 635. Each component is integrated into at least one or more modules and is also embodied as at least one or more processors (not shown).

  In FIG. 6, the pre-processing unit 610 can perform filtering or down-sampling on an input signal, but is not limited thereto. The preprocessing unit 610 can determine an encoding mode based on the signal characteristics. The signal characteristics can determine whether the coding mode suitable for the current frame is the voice mode or the music mode, and the efficient coding mode for the current frame is the time domain mode. Or it is in frequency domain mode. Here, the signal characteristic can be grasped using the short section characteristic of the frame or the long section characteristic for a plurality of frames, but is not limited thereto. For example, if the input signal corresponds to an audio signal, the mode is determined to be the audio mode or the time domain mode. If the input signal corresponds to a signal other than the audio signal, that is, a music signal or a mixed signal, the mode is set to the music mode or the frequency domain mode. Can be determined. The preprocessing unit 610 provides an input signal to the frequency domain encoding unit 630 when the signal characteristic corresponds to the music mode or the frequency domain mode, and provides the input signal when the signal characteristic corresponds to the voice mode or the time domain mode. The signal may be provided to a time domain encoder 650.

  The frequency domain encoding unit 630 may process the audio signal provided from the pre-processing unit 610 based on transform coding. Specifically, the transient detection unit 631 can detect a transient component from the audio signal and determine whether the current frame is a transient frame. The conversion unit 633 determines the length or form of the conversion window based on the frame type provided by the transient detection unit 631, that is, the transient information, and converts the audio signal into the frequency domain based on the determined conversion window. Can be converted. As a conversion technique, MDCT, FFT, or MLT can be applied. In general, for frames with transient components, a short length transform window can be applied. The spectrum encoding unit 635 can encode the audio spectrum converted into the frequency domain. The spectrum encoding unit 635 will be described more specifically with reference to FIGS.

  The time domain coding unit 650 may perform code excited linear prediction (CELP) coding on the audio signal provided from the preprocessing unit 610. Specifically, ACELP (algebraic CELP) can be used, but is not limited thereto.

  The multiplexing unit 670 multiplexes a spectrum component or a signal component generated as a result of the coding with various indexes in the frequency domain coding unit 630 or the time domain coding unit 650 to generate a bit stream, The bit stream is transmitted in a packet form through a channel or stored in a recording medium.

  FIG. 7 is a block diagram illustrating a configuration of the spectrum encoding device according to the embodiment. The apparatus shown in FIG. 7 corresponds to the spectrum coding unit 635 of FIG. 6, is included in another frequency domain coding apparatus, or is embodied independently.

  7 includes an energy estimating unit 710, an energy quantizing and encoding unit 720, a bit allocating unit 730, a spectrum normalizing unit 740, a spectrum quantizing and encoding unit 750, and a noise filling unit. 760 may be included.

  Referring to FIG. 7, the energy estimating unit 710 may separate an original spectral coefficient into subbands and estimate energy for each subband, for example, a Norm value. Here, in one frame, the number of spectral coefficients included in each subband can be increased as each subband has the same size or goes from a low band to a high band.

  The energy quantization and encoding unit 720 may quantize and encode the Norm value estimated for each subband. At this time, the Norm value is quantized by various methods such as vector quantization, scalar quantization, TCQ, and LVQ (lattice vector quantization). The energy quantization and encoding unit 720 may further perform lossless encoding to further improve encoding efficiency.

  The bit allocating unit 730 may allocate bits necessary for encoding using a Norm value quantized for each subband, while considering allowable bits per frame.

  The spectrum normalization unit 740 may perform normalization on the spectrum using the Norm value quantized for each subband.

  The spectrum quantization and coding unit 750 may perform quantization and coding on the normalized spectrum based on bits allocated to each subband.

  The noise filling unit 760 can add an appropriate noise to the portion quantized to 0 by the restriction of the allowable bit in the spectrum quantization and encoding unit 750.

  FIG. 8 is a diagram illustrating an example of subband division. Referring to FIG. 8, when an input signal uses a sampling frequency of 48 kHz and has a frame size of 20 ms, the number of samples to be processed per frame is 960. That is, if the input signal is converted by applying 50% overlapping using MDCT, 960 spectral coefficients can be obtained. Here, the overlapping ratio is variously set according to the coding method. In the frequency domain, it is theoretically possible to process up to 24 kHz, but a band up to 20 kHz is expressed in consideration of the human audible band. From 0 to 3.2 kHz, which is a low frequency band, eight spectral coefficients are collectively used in one subband, and in the band of 3.2 to 6.4 kHz, 16 spectral coefficients are used in one subband. Use together. In the band of 6.4 to 13.6 kHz, 24 spectral coefficients are collectively used in one subband, and in the band of 13.6 to 20 kHz, 32 spectral coefficients are collected in one subband. use. When encoding is performed by obtaining an actual Norm value, it is possible to obtain Norm up to a band determined by an encoder and perform encoding. In a specific high band after the determined band, encoding based on various schemes such as band extension is possible.

  FIG. 9 is a block diagram illustrating a configuration of the spectrum quantization device according to the embodiment. The apparatus illustrated in FIG. 9 may include a quantizer selection unit 910), USQ 930, and TCQ 950.

  Referring to FIG. 9, a quantizer selector 910 may select the most efficient quantizer among various quantizers according to characteristics of an input signal, that is, a signal to be quantized. As the characteristics of the input signal, bit allocation information for each band, band size information, and the like can be used. According to the selection result, a signal to be quantized is provided to one of USQ 930 and TCQ 950, and corresponding quantization can be performed.

  FIG. 10 is a block diagram illustrating a configuration of the spectrum encoding device according to the embodiment. The device illustrated in FIG. 10 corresponds to the spectrum quantization and coding unit 750 of FIG. 7, is included in another frequency domain coding device, or is embodied independently.

  The apparatus illustrated in FIG. 10 may include a coding scheme selection unit 1010, a zero coding unit 1020, a scaling unit 1030, an ISC coding unit 1040, a quantization component restoration unit 1050, and an inverse scaling unit 1060. Here, the quantization component restoration unit 1050 and the inverse scaling unit 1060 are provided as options.

  In FIG. 10, a coding scheme selection unit 1010 can select a coding scheme in consideration of input signal characteristics. The input signal characteristics may include bits allocated for each band. The normalized spectrum is provided to the zero encoding unit 1020 or the scaling unit 1030 based on the encoding scheme selected for each band. According to one embodiment, if the average number of bits allocated to each sample of a band is greater than or equal to a predetermined value, eg, 0.75, the band is determined to be very important and USQ is used. On the other hand, all other bands use TCQ. Here, the average number of bits can be determined in consideration of the band length or the band size. The selected encoding method is set using a 1-bit flag.

  The zero encoding unit 1020 can encode all samples to 0 for a band where the assigned bits are 0.

  The scaling unit 1030 may adjust the bit rate by performing scaling on the spectrum based on the bits allocated to the band. Then, the normalized spectrum is used. The scaling unit 1030 may perform scaling in consideration of each sample included in the band, that is, the average number of bits allocated to the spectrum coefficient. For example, the larger the average number of bits, the larger the scaling.

  According to an exemplary embodiment, the scaling unit 1030 may determine an appropriate scaling value by bit allocation for each band.

  Specifically, first, the number of pulses for the current band can be estimated using the band length and the bit allocation information. Here, the pulse means a unit pulse. First, the bit b actually required in the current band can be calculated based on the following equation (1).

ここで、ｎは、バンド長を示し、ｍは、パルス個数（number of pulses）を意味し、ｉは、ＩＳＣ（the important spectral component）を有するノンゼロ位置の数を意味する。

Here, n indicates the band length, m indicates the number of pulses, and i indicates the number of non-zero positions having ISC (the important spectral component).

  On the other hand, the number of non-zero positions is obtained based on a probability, for example, as in the following equation (2).

そして、ノンゼロ位置のために必要なビット数は、下記数式（３）のように推定される。

Then, the number of bits required for the non-zero position is estimated as in the following equation (3).

最終的に、パルスの個数は、各バンドに割り当てられたビットに最も近い値を有するｂ値によって選択される。

Finally, the number of pulses is selected by the b value having the value closest to the bit assigned to each band.

  Next, an initial scaling factor can be determined using the pulse number estimated value obtained for each band and the absolute value of the input signal. The input signal is scaled by the initial scaling factor. If the sum of the pulse numbers for the scaled original signal, ie, the quantized signal, is not the same as the pulse number estimate, a pulse redistribution process is performed using the updated scaling factor. It can be carried out. The pulse redistribution process increases the number of pulses by decreasing the scaling factor if the number of pulses selected for the current band is less than the estimated number of pulses determined for each band, Reduces the number of pulses by increasing the scaling factor. At this time, a position where distortion with the original signal is minimized is selected, and the position can be increased or decreased by a predetermined value.

  Since the distortion function for TSQ requires a relative size rather than an exact distance, the quantized and dequantized values are respectively calculated for each band as shown in the following equation (4). As the sum of the squared distances of

ここで、ｐｉは、実際値であり、ｑｉは、量子化された値を示す。

Here, pi is an actual value, and qi indicates a quantized value.

  On the other hand, the distortion function for USQ can use the Euclidean distance to determine the best quantized value. At this time, in order to minimize the complexity, a modified formula including a scaling factor is used, and the distortion function is calculated by the following formula (5).

もしバンド当たりパルス個数が要求される値とマッチングしない場合、最小メトリックを維持しながら、所定数のパルスを加減する必要がある。それは、１つのパルスを加減する過程を、パルス個数が要求される値に至るまで反復する方法によって遂行される。

If the number of pulses per band does not match the required value, a certain number of pulses must be adjusted while maintaining the minimum metric. It is performed by a method of repeating the process of adding or subtracting one pulse until the number of pulses reaches a required value.

  In order to add or subtract one pulse, it is necessary to find n distortion values for finding an optimal distortion value. For example, the distortion value j corresponds to adding a pulse to a j-th position in a band as in the following equation (6).

前記数式（６）をｎ回遂行することを避けるために、下記数式（７）のように、同じ偏差（deviation）を使用することができる。

In order to avoid performing Equation (6) n times, the same deviation can be used as in Equation (7) below.

前記数式（７）において、

In the equation (7),

は、１回だけ計算すればよい。一方、ｎは、バンド長、すなわち、バンドにある係数数を示し、ｐは、原信号、すなわち、量子化器の入力信号を示し、ｑは、量子化された信号を示し、ｇは、スケーリングファクタを示す。最終的に、歪曲ｄを最小化する位置ｊが選択され、ｑｊがアップデートされる。

Need only be calculated once. On the other hand, n indicates the band length, that is, the number of coefficients in the band, p indicates the original signal, that is, the input signal of the quantizer, q indicates the quantized signal, and g indicates the scaling. Indicates a factor. Finally, the position j that minimizes the distortion d is selected, and qj is updated.

  On the other hand, to control the bit rate, the appropriate ISC can be selected and encoded using the scaled spectral coefficients. Specifically, the spectral components to quantize are selected using the bit allocation for each band. At this time, the spectral components can be selected based on various combinations based on the distribution and dispersion of the spectral components. Next, the actual non-zero position can be calculated. The non-zero position can be obtained by analyzing the scaling amount and the redistribution operation, and the non-zero position so selected can be ISC in other words. In summary, it is possible to determine the optimal scaling factor and the non-zero position information corresponding to the ISC by analyzing the magnitude of the signal that has undergone the scaling and redistribution processes. Here, the non-zero position information means the number and positions of the non-zero positions. If the number of pulses is not adjusted through the scaling and redistribution processes, the selected pulses can be quantized through the actual TCQ process, and the surplus bits can be adjusted using the result. The following example is possible in the process.

  If the number of non-zero positions is not the same as the estimated number of pulses determined for each band, and the number of non-zero positions is a predetermined value, for example, if the quantizer selection information determined to be greater than 1 is a condition indicating TCQ, The surplus bits can be adjusted via actual TCQ quantization. Specifically, if the above condition is satisfied, a TCQ quantization process is first performed to adjust the surplus bits. If the number of pulses in the current band determined through actual TCQ quantization is smaller than the estimated number of pulses determined in advance for each band, the previously determined scaling factor is set to a value greater than 1, For example, multiply by 1.1 to increase the scaling factor and vice versa to decrease the scaling factor by a value less than 1, for example 0.9. Such a process is repeated, and when the pulse number estimated value obtained for each band is equal to the pulse number of the current band obtained through the TCQ quantization, the pulse number is used in the actual TCQ quantization process. Calculate bits and update surplus bits. The non-zero position thus obtained corresponds to the ISC.

  The ISC encoding unit 1040 can encode the number information and the non-zero position information of the finally selected ISC. In the process, lossless coding may be applied to increase coding efficiency. The ISC encoding unit 1040 can perform encoding using a quantizer selected for a non-zero band in which the allocated bits are not 0. Specifically, the ISC encoding unit 1040 selects an ISC for each band with respect to the normalized spectrum, and transmits information of the ISC selected for each band based on the number, position, size, and code. Can be encoded. At that time, the magnitude of the ISC can be encoded by a method different from the number, position, and code. For example, the size of the ISC may be quantized using one of USQ and TCQ and arithmetically coded, while the number, position and code of the ISC may be arithmetically coded. If it is determined that the specific band contains important information, USQ may be used, otherwise TCQ may be used. According to the embodiment, one of TCQ and USQ can be selected based on the signal characteristics. Here, the signal characteristic may include a bit or a band length assigned to each band. If the average number of bits allocated to each sample included in the band is a threshold value, for example, 0.75 or more, it can be determined that the band includes very important information. USQ is used. On the other hand, in the case of a low band having a short band length, USQ is used as necessary. According to another embodiment, one of the first joint method and the second joint method is used depending on the bandwidth. For example, for NB and WB, quantizer selection is performed by further utilizing not only the original bit allocation information for each band but also a secondary bit allocation process for surplus bits from a previously coded band. The first joint scheme is used. For SWB and FB, the band determined to use USQ is used, and for the LSB (least significant bit), the second joint scheme using TCQ is used. In the first joint scheme, the secondary bit allocation process can select two bands by distributing surplus bits from previously encoded bands. On the other hand, in the second joint method, the remaining bits can use USQ.

  The quantized component restoration unit 1050 can restore the actual quantized component by adding the position, size, and code information of the ISC to the quantized component. Here, zero is assigned to the zero position, that is, the spectral coefficient coded to zero.

  The inverse scaling unit 1060 may perform inverse scaling on the restored quantized component and output quantized spectral coefficients at the same level as the normalized input spectrum. In the scaling unit 1030 and the inverse scaling unit 1060, the same scaling factor can be used.

  FIG. 11 is a block diagram illustrating a configuration of an ISC encoding device according to an embodiment. The apparatus illustrated in FIG. 11 may include an ISC selector 1110 and an ISC information encoder 1130. The device of FIG. 11 corresponds to the ISC encoder 1040 of FIG. 10 or is embodied as an independent device.

  In FIG. 11, an ISC selection unit 1110 can select an ISC from a scaled spectrum based on a predetermined criterion in order to adjust a bit rate. The ISC selection unit 1110 can analyze the scaled extent from the scaled spectrum to determine an actual non-zero position. Here, ISC corresponds to an actual non-zero spectral coefficient before scaling. The ISC selection unit 1110 can select a spectral coefficient to be coded, that is, a non-zero position, based on bits allocated to each band in consideration of distribution and variance of the spectral coefficient. T, CQ can be used for ISC selection.

  The ISC information encoding unit 1130 can decode the ISC information, that is, the ISC number information, the position information, the size information, and the code based on the selected ISC.

  FIG. 12 is a block diagram illustrating a configuration of the ISC information encoding device according to the embodiment. The apparatus illustrated in FIG. 12 may include a position information encoding unit 1210, a size information encoding unit 1230, and a code encoding unit 1250.

  12, the position information encoding unit 1210 can encode the position information of the ISC selected by the ISC selection unit 1110 (FIG. 11), that is, the position information of the non-zero spectral coefficient. The location information may include the number and location of the selected ISC. Arithmetic coding is used for encoding the position information. On the other hand, the selected ISCs can be collected to form a new buffer. For ISC collection, zero bands and unselected spectra are excluded.

  The size information coding unit 1230 can perform coding on the size information of the newly configured ISC. At this time, quantization can be performed by selecting one of TCQ and USQ, and then arithmetic coding can be additionally performed. To increase the efficiency of arithmetic coding, non-zero position information and the number of ISCs are used.

  The code information coding unit 1250 can perform coding on the code information of the selected ISC. Arithmetic encoding is used for encoding the code information.

  FIG. 13 is a block diagram illustrating a configuration of a spectrum encoding device according to another embodiment. The apparatus shown in FIG. 13 corresponds to the spectrum quantization and coding unit 750 of FIG. 7, is included in another frequency domain coding apparatus, or is embodied independently.

  The apparatus illustrated in FIG. 13 may include a scaling unit 1330, an ISC encoding unit 1340, a quantization component restoration unit 1350, and an inverse scaling unit 1360. As compared with FIG. 10, the zero encoding unit 1020 and the encoding scheme selection unit 1010 are omitted, and the ISC encoding unit 1340 operates in the same way as the operation of each component except that TCQ can be used. Are the same.

  FIG. 14 is a block diagram illustrating a configuration of a spectrum encoding device according to another embodiment. The device illustrated in FIG. 14 corresponds to the spectrum quantization and coding unit 750 of FIG. 7, is included in another frequency domain coding device, or is embodied independently.

  The apparatus illustrated in FIG. 14 may include a coding scheme selection unit 1410, a scaling unit 1430, an ISC coding unit 1440, a quantization component restoration unit 1450, and an inverse scaling unit 1460. As compared with FIG. 10, the operation of each component is the same except that the zero encoding unit 1020 is omitted.

  FIG. 15 is a diagram illustrating the concept of an ISC collection process and an encoding process according to an embodiment. First, a zero band, that is, a band quantized to 0 is excluded. Next, a new buffer can be configured using the ISC selected from the spectral components existing in the non-zero band. The newly configured ISC may perform TCQ on a band-by-band basis and perform corresponding lossless encoding.

  FIG. 16 is a diagram illustrating a concept of an ISC collecting process and an encoding process according to another embodiment. First, a zero band, that is, a band quantized to 0 is excluded. Next, a new buffer can be configured using the ISC selected from the spectral components existing in the non-zero band. USC or TCQ can be performed on the newly configured ISC in band units, and corresponding lossless coding can be performed.

  FIG. 17 is a diagram illustrating an example of the TCQ used in the present invention, which corresponds to an 8-state 4-coset trellis structure having two zero levels. A detailed description of the TCQ is disclosed in US Pat. No. 7,605,727.

  FIG. 18 is a block diagram showing a configuration of a frequency domain audio decoding device to which the present invention is applied.

  The frequency domain audio decoding device 1800 illustrated in FIG. 18 may include a frame error detection unit 1810, a frequency domain decoding unit 1830, a time domain decoding unit 1850, and a post-processing unit 1870. The frequency domain decoding unit 1830 may include a spectrum decoding unit 1831, a memory updating unit 1833, an inverse transform unit 1835, and an OLA (overlap and add) unit 1837. Each component is integrated into at least one or more modules and is also embodied as at least one or more processors (not shown).

  Referring to FIG. 18, the frame error detection unit 1810 can detect whether a frame error has occurred from the received bitstream.

  The frequency domain decoding unit 1830 operates when the encoding mode is the music mode or the frequency domain mode. When a frame error occurs, the FEC algorithm or the PLC algorithm operates, and when the frame error does not occur, A time-domain signal is generated through a general transform decoding process. Specifically, the spectrum decoding unit 1831 can perform spectrum decoding using the decoded parameters and combine spectral coefficients. The spectrum decoding unit 1831 will be described more specifically with reference to FIGS.

  The memory updating unit 1833 performs the processing based on the spectrum coefficient synthesized with the current frame that is a normal frame, information obtained by using the decoded parameters, the number of error frames that have been continued up to the present, the signal characteristics of each frame, or the frame. Type information etc. can be updated for the next frame. Here, the signal characteristic may include a transient characteristic and a stationary characteristic, and the frame type may include a transient frame, a stationary frame, or a harmonic frame.

  The inverse transform unit 1835 can perform a time-frequency inverse transform on the synthesized spectral coefficient to generate a time-domain signal.

  The OLA unit 1837 performs an OLA process using the time domain signal of the previous frame, and as a result, generates a final time domain signal for the current frame and provides the generated signal to the post-processing unit 1870.

  The time domain decoding unit 1850 operates when the encoding mode is the audio mode or the time domain mode. When a frame error occurs, the FEC algorithm or the PLC algorithm is operated. A time domain signal is generated through a general CELP decoding process.

  The post-processing unit 1870 can perform filtering or up-sampling on the time-domain signal provided from the frequency-domain decoding unit 1830 or the time-domain decoding unit 1850, but is not limited thereto. The post-processing unit 1670 provides a restored audio signal as an output signal.

  FIG. 19 is a block diagram illustrating a configuration of a spectrum decoding device according to one embodiment. The device illustrated in FIG. 19 corresponds to the spectrum decoding unit 1831 in FIG. 18, is included in another frequency domain decoding device, or is embodied independently.

  The spectrum decoding apparatus 1900 illustrated in FIG. 19 may include an energy decoding and inverse quantization unit 1910, a bit allocation unit 1930, a spectrum decoding and inverse quantization unit 1950, a noise filling unit 1970, and a spectrum shaping unit 1990. Here, the noise filling unit 1970 may be located at the rear end of the spectrum shaping unit 1990. Each component is integrated into at least one or more modules and is also embodied as at least one or more processors (not shown).

  Referring to FIG. 19, the energy decoding and inverse quantization unit 1910 performs lossless decoding on an energy such as a Norm value, which is a parameter for which lossless encoding has been performed in an encoding process, and is decoded. Inverse quantization can be performed on the calculated Norm value. In the encoding process, the inverse quantization may be performed using a method corresponding to the quantized method of the Norm value.

  The bit allocation unit 1930 can allocate the required number of bits for each subband based on the quantized Norm value or the dequantized Norm value. In this case, the number of bits allocated for each subband is the same as the number of bits allocated in the encoding process.

  The spectrum decoding and inverse quantization unit 1950 performs lossless decoding on the encoded spectral coefficients using the number of bits allocated to each subband, and performs an inverse quantization process on the decoded spectral coefficients. To generate normalized spectral coefficients.

  The noise filling unit 1970 can fill a portion of the normalized spectral coefficients that requires noise filling for each subband with noise.

  The spectrum shaping unit 1990 can shape the normalized spectral coefficients using the dequantized Norm value. Through the spectrum shaping process, finally decoded spectrum coefficients are obtained.

  FIG. 20 is a block diagram illustrating a configuration of the spectrum inverse quantization device according to the embodiment. The apparatus illustrated in FIG. 20 may include an inverse quantization period selector 2010, USQ 2030, and TCQ 2050.

  In FIG. 20, the inverse quantization period selection unit 2010 may select the most efficient inverse quantizer from among various inverse quantizers according to the characteristics of an input signal, that is, a signal to be inversely quantized. it can. As the characteristics of the input signal, bit allocation information for each band, band size information, and the like can be used. According to the selection result, a signal to be dequantized can be provided to one of USQ 2030 and TCQ 2050, and a corresponding dequantization can be performed.

  FIG. 21 is a block diagram illustrating a configuration of a spectrum decoding device according to one embodiment. The apparatus shown in FIG. 21 corresponds to the spectrum decoding and inverse quantization unit 1950 of FIG. 19, is included in another frequency domain decoding apparatus, or is implemented independently.

  The apparatus illustrated in FIG. 21 may include a decoding scheme selection unit 2110, a zero decoding unit 2130, an ISC decoding unit 2150, a quantization component restoration unit 2170, and an inverse scaling unit 2190. Here, the quantization component restoration unit 2170 and the inverse scaling unit 2190 are provided as options.

  In FIG. 21, decoding scheme selection section 2110 can select a decoding scheme based on bits allocated for each band. The normalized spectrum is provided to the zero decoding unit 2130 or the ISC decoding unit 2150 based on the decoding scheme selected for each band.

  The zero decoding unit 2130 can decode all samples to 0 for a band where the assigned bits are 0.

  The ISC decoding unit 2150 can perform decoding by using an inverse quantizer selected for a band in which the allocated bits are not 0. The ISC decoding unit 2150 obtains information of important frequency components for each band of the encoded spectrum, and decodes information of important frequency components obtained for each band based on the number, position, size, and code. can do. The magnitude of the important frequency component can be decoded in a manner different from the number, position and sign. In one example, the magnitude of the important frequency component is arithmetically decoded and inversely quantized using one of USQ and TCQ, while the arithmetic decoding is performed on the number, position and code of the important frequency component. It can be carried out. The inverse quantizer selection can be performed using the same result as that of the ISC encoder 1040 shown in FIG. The ISC decoding unit 2150 can perform inverse quantization on a band whose assigned bit is not 0 using one of TCQ and USQ.

  The quantization component restoration unit 2170 can restore an actual quantization component based on the restored position, size, and code information of the ISC. Here, 0 is assigned to the zero position, that is, the unquantized part that is the spectral coefficient decoded to zero.

  Furthermore, inverse scaling is performed on the restored quantized component including the inverse scaling unit (not shown), and quantized spectral coefficients at the same level as the normalized spectrum can be output.

  FIG. 22 is a block diagram illustrating a configuration of the ISC decoding device according to the embodiment. The device in FIG. 22 may include a pulse number estimating unit 2210 and an ISC information decoding unit 2230. The device of FIG. 22 corresponds to the ISC decoding unit 2150 of FIG. 21 or is embodied as an independent device.

  In FIG. 22, a pulse number estimation unit 2210 can determine an estimated pulse number value for a current band using band size and bit allocation information. That is, since the bit allocation information of the current frame is the same as that of the encoder, the same bit allocation information is used to derive the same pulse number estimation value and proceed with decoding.

  The ISC information decoding unit 2230 can decode the ISC information, that is, the ISC number information, the position information, the size information, and the code based on the estimated number of pulses.

  FIG. 23 is a block diagram showing the configuration of the ISC information decoding device according to one embodiment. The apparatus illustrated in FIG. 23 may include a position information decoding unit 2310, a size information decoding unit 2330, and a code decoding unit 2350.

  In FIG. 23, a position information decoding unit 2310 can decode an index related to position information included in a bitstream and restore the number and position of ISCs. Arithmetic decoding is used for decoding the position information. The size information decoding unit 2330 performs arithmetic decoding on the index related to the size information included in the bitstream, selects one of TCQ and USQ on the decoded index, and performs inverse quantization. It can be performed. To increase the efficiency of arithmetic decoding, non-zero position information and the number of ISCs are used. The code decoding unit 2350 decodes an index related to the code information included in the bit stream and can restore the ISC code. Arithmetic decoding is used for decoding the code information. According to one embodiment, the number of pulses required by a non-zero band can be estimated and used for decoding position information, size information or code information.

  FIG. 24 is a block diagram illustrating a configuration of a spectrum decoding device according to another embodiment. The apparatus illustrated in FIG. 24 corresponds to the spectrum decoding and inverse quantization unit 1950 of FIG. 19, is included in another frequency domain decoding apparatus, or is embodied independently.

  The apparatus illustrated in FIG. 24 may include an ISC decoding unit 2450, a quantization component restoration unit 2470, and an inverse scaling unit 2490. As compared with FIG. 21, the operation of each component is the same except that the decoding scheme selection unit 2110 and the zero decoding unit 2130 are omitted and the ISC decoding unit 2450 uses TCQ.

  FIG. 25 is a block diagram illustrating a configuration of a spectrum decoding device according to another embodiment. The apparatus shown in FIG. 25 corresponds to the spectrum decoding and inverse quantization unit 1950 of FIG. 19, is included in another frequency domain decoding apparatus, or is implemented independently.

  The apparatus illustrated in FIG. 25 may include a decoding scheme selection unit 2510, an ISC decoding unit 2550, a quantization component restoration unit 2570, and an inverse scaling unit 2590. When compared with FIG. 21, the operation of each component is the same except that the zero decoding unit 2130 is omitted.

  FIG. 26 is a block diagram illustrating a configuration of an ISC information encoding device according to another embodiment. The device in FIG. 26 may include a probability calculation unit 2610 and a lossless encoding unit 2630.

  In FIG. 26, the probability calculating unit 2610 can calculate a probability value for magnitude coding by using the following equations (8) and (9) using the number of ISCs, the number of pulses, and TCQ information.

ここで、

here,

は、各バンドで伝送されるＩＳＣ個数のうち符号化されて残った個数を意味し、

Means the number of remaining ISCs among the number of ISCs transmitted in each band,

は、各バンドで伝送されるパルスの個数のうち、符号化されて残った個数を示し、Ｍｓは、トレリス状態で存在する大きさの集合を意味する。そして、ｊは、大きさのうち符号化されたパルス個数を意味する。

Indicates the number of pulses remaining in the number of pulses transmitted in each band, and Ms indicates a set of sizes existing in a trellis state. J indicates the number of encoded pulses in the magnitude.

  The lossless coding unit 2630 can losslessly code the TCQ size information, that is, the size and the path information, using the obtained probability value. The number of pulses of each magnitude is

値と

Value and

値とによって符号化される。ここで、

And encoded by the value. here,

値は、以前大きさの最後のパルスの確率を意味する。そして、

The value refers to the probability of the last pulse of the previous magnitude. And

値は、それ以外の他のパルスに該当する確率を意味する。最終的に、そのように求められた確率値によって、符号化されたインデックスを出力する。

The value means a probability corresponding to another pulse. Finally, an encoded index is output according to the probability value thus obtained.

  FIG. 27 is a block diagram showing a configuration of an ISC information decoding device according to another embodiment. The device in FIG. 27 may include a probability calculation unit 2710 and a lossless decoding unit 2730.

  In FIG. 27, the probability calculation unit 2710 calculates a probability value for magnitude coding using ISC information (number i, position), TCQ information, pulse number m, and band size n. Can be calculated. For that purpose, first, necessary bit information b is obtained by using the obtained pulse number and band size. At this time, it can be obtained as in the above equation (1). Then, using the obtained bit information b, the number of ISCs, the ISC position, and the TCQ information, a probability value for magnitude coding is calculated based on the above equations (8) and (9).

  The lossless decoding unit 2730 uses the probability value calculated in the same way as the encoding device and the transmitted index information to transmit the TCQ size information, that is, the magnitude information and the path information. Can be losslessly decoded. To this end, first, an arithmetic coding model related to the number information is created using the probability value, and the TCQ size information is arithmetically decoded using the obtained model to perform the TCQ size information. Is decrypted. Specifically, the number of pulses of each magnitude is

値と

Value and

値とによって復号される。ここで、

And decoded by the value. here,

値は、以前大きさの最後のパルス確率を意味する。そして、

The value means the last pulse probability of the previous magnitude. And

値は、それ以外の他のパルスに該当する確率を意味する。最終的に、そのように求められた確率値によって復号されたＴＣＱ情報、すなわち、大きさ情報と経路情報とを出力する。

The value means a probability corresponding to another pulse. Finally, the TCQ information decoded by the probability value thus obtained, that is, the size information and the path information are output.

  FIG. 28 is a block diagram illustrating a configuration of a multimedia device including an encoding module according to an embodiment of the present invention.

  The multimedia device 2800 illustrated in FIG. 28 may include a communication unit 2810 and an encoding module 2830. In addition, the storage unit may further include a storage unit 2850 for storing the audio bitstream according to the use of the audio bitstream obtained as a result of the encoding. In addition, the multimedia device 2800 may further include a microphone 2870. That is, the storage unit 2450 and the microphone 2870 are provided as options. Meanwhile, the multimedia device 2800 illustrated in FIG. 28 further includes an arbitrary decoding module (not shown), for example, a decoding module performing a general decoding function, or a decoding module according to an exemplary embodiment of the present invention. May be. Here, the encoding module 2830 is integrated with other components (not shown) included in the multimedia device 2800, and is also embodied as at least one or more processors (not shown).

  Referring to FIG. 28, the communication unit 2810 receives at least one of an externally provided audio and an encoded bit stream, or recovers the restored audio, and encodes the encoded data by the encoding module 2830. And at least one of the resulting audio bit streams.

  The communication unit 2810 includes a wireless Internet, a wireless intranet, a wireless telephone network, a wireless LAN (local area network), Wi-Fi (wireless fidelity), WFD (Wi-Fi direct), 3G (3rd generation), and 4G (4th generation). , Bluetooth (registered trademark), infrared communication (IrDA: infrared data association), RFID (radio frequency identification), UWB (ultra wideband), ZigBee (ZigBee (registered trademark)), NFC (near field communication) It is configured to be able to transmit and receive data to and from an external multimedia device or server via a simple wireless network or a wired network such as a wired telephone network or a wired Internet.

  According to an exemplary embodiment, the encoding module 2830 may select an important frequency component for each band with respect to the normalized spectrum, and may transmit information of the important frequency component selected for each band to a number, a position, and a size. It can be encoded based on the magnitude and the sign. The magnitude of the important frequency component can be encoded by a method different from the number, position, and code. For example, the magnitude of the important frequency component is quantized using one of USQ and TCQ. While performing arithmetic coding, the arithmetic coding can be performed on the number, position, and sign of important frequency components. According to an exemplary embodiment, the normalized spectrum may be scaled based on bits allocated to each band, and important frequency components may be selected for the scaled spectrum.

  The storage unit 2850 may store various programs necessary for operating the multimedia device 2800.

  The microphone 2870 can provide a user or an external audio signal to the encoding module 2830.

  FIG. 29 is a block diagram illustrating a configuration of a multimedia device including a decoding module according to an embodiment of the present invention.

  The multimedia device 2900 illustrated in FIG. 29 may include a communication unit 2910 and a decoding module 2920. In addition, a storage unit 2960 for storing the restored audio signal may be further included depending on the use of the restored audio signal obtained as a result of decoding. In addition, the multimedia device 2900 may further include a speaker 2970. That is, the storage unit 2960 and the speaker 2970 are provided as options. Meanwhile, the multimedia device 2900 illustrated in FIG. 29 may include an arbitrary encoding module (not shown), for example, an encoding module performing a general encoding function, or encoding according to an embodiment of the present invention. It may further include a module. Here, the decoding module 2920 is integrated with other components (not shown) included in the multimedia device 2900, and is also embodied as at least one or more processors (not shown).

  Referring to FIG. 29, the communication unit 2910 may receive at least one of an externally provided encoded bit stream and an audio signal, or may obtain a decoded signal obtained as a decoding result of the decoding module 2920. At least one of the audio signal and the audio bit stream obtained as a result of the encoding. Meanwhile, the communication unit 2910 is substantially similar to the communication unit 2810 of FIG.

  According to one embodiment, the decoding module 2920 receives the bit stream provided through the communication unit 2910, obtains information on important frequency components for each band of the encoded spectrum, and obtains information for each band. The obtained important frequency component information can be decoded based on the number, position, size, and code. The magnitude of the important frequency component can be decoded by a method different from the number, position, and code. For example, the magnitude of the important frequency component is arithmetically decoded and uses one of USQ and TCQ. While performing inverse quantization, arithmetic decoding can be performed on the number, position, and code of important frequency components.

  The storage unit 2960 may store the restored audio signal generated by the decoding module 2920. Meanwhile, the storage unit 2960 can store various programs necessary for operating the multimedia device 2900.

  The speaker 2970 may output the restored audio signal generated by the decoding module 2920 to the outside.

  FIG. 30 is a block diagram illustrating a configuration of a multimedia device including an encoding module and a decoding module according to an embodiment of the present invention.

  The multimedia device 3000 illustrated in FIG. 30 may include a communication unit 3010, an encoding module 3020, and a decoding module 3030. In addition, the storage unit 3040 may further include a storage unit 3040 for storing an audio bitstream or a restored audio signal according to an application of an audio bitstream obtained as a result of encoding or a restored audio signal obtained as a result of decoding. Further, the multimedia device 3000 may further include a microphone 3050 or a speaker 3060. Here, the encoding module 3020 and the decoding module 3030 are integrated with other components (not shown) included in the multimedia device 3000, and are also embodied as at least one or more processors (not shown). .

  Each component illustrated in FIG. 30 is the same as the component of the multimedia device 2800 illustrated in FIG. 28 or the component of the multimedia device 2900 illustrated in FIG. 29, and thus a detailed description thereof will be omitted. I do.

  The multimedia devices 2800, 2900, and 3000 shown in FIGS. 28 to 30 include terminals dedicated to voice communication including a telephone, a mobile phone, and the like; devices dedicated to broadcasting or music including a TV (television) and an MP3 player; Alternatively, an integrated terminal device of a voice communication dedicated terminal and a broadcast dedicated device or a music dedicated device; a user terminal of a teleconferencing system or an interaction system may be included, but is not limited thereto. The multimedia devices 2800, 2900, and 3000 are also used as a client, a server, or a converter arranged between the client and the server.

  On the other hand, when the multimedia devices 2800, 2900, and 3000 are, for example, mobile phones, not shown, a user input unit such as a keypad; a user interface; or displays information processed by the mobile phone. Display unit; may further include a processor for controlling general functions of the mobile phone. In addition, the mobile phone may further include a camera unit having an imaging function and at least one or more components that perform a function required by the mobile phone.

  On the other hand, when the multimedia devices 2800, 2900, and 3000 are, for example, TVs, not shown, a user input unit such as a keypad, a display unit for displaying received broadcast information, and a general TV unit. It may further include a processor that controls various functions. In addition, the TV may further include at least one or more components that perform a function required by the TV.

  FIG. 31 is a flowchart illustrating the operation of the spectrum fine structure encoding method according to an embodiment. Referring to FIG. 31, in step 3110, an encoding method is selected. For this purpose, information related to each band and bit allocation information are used. Here, the encoding scheme may include a quantization scheme.

  In step 3130, it is determined whether the current band is a band having zero bit allocation, that is, a zero band. If the current band is a zero band, the process proceeds to step 3250. Proceed to stage.

  In step 3150, all samples in the zero band can be coded to zero.

  In operation 3170, a band other than the zero band may be encoded based on the selected quantization scheme. According to one embodiment, using band length and bit allocation information to estimate the number of pulses per band, determine the number of non-zero positions, estimate the required number of bits for non-zero positions, and determine the final number of pulses. Can be. Next, the initial scaling factor is determined based on the number of pulses per band and the absolute value of the input signal, and the scaling factor can be updated through scaling and pulse redistribution using the initial scaling factor. Utilizing the last updated scaling factor, the spectral coefficients are scaled, and the appropriate ISC is selected using the scaled spectral coefficients. The spectral components to be quantized are selected based on the bit allocation information of each band. Next, the magnitude of the collected ISC is quantized by the USC joint scheme and the TCQ joint scheme and arithmetically coded. Here, the non-zero position and the number of ISCs are used to increase the efficiency of arithmetic coding. The USC joint scheme and the TCQ joint scheme have a first joint scheme and a second joint scheme depending on the bandwidth. The first joint method is a method in which a quantizer is selected by using a secondary bit allocation process for surplus bits from a previous band, is used for NB and WB, and the second joint method is determined to be USQ. As for the set bands, TCQ is used for the LSB, and the remaining bits are of the system using the USQ, and can be used for SWB and FB. On the other hand, the selected ISC code information is arithmetically decoded with the same probability with respect to positive and negative codes.

  After step 3170, the method may further include restoring a quantized component and de-scaling the band. In order to restore the actual quantization component of each band, position, code, and size information may be added to the quantization component. Zero positions are assigned zero. On the other hand, an inverse scaling factor can be extracted using the same scaling factor used at the time of scaling, and inverse scaling can be performed on the restored actual quantized component. The inversely scaled signal can have a normalized spectrum, ie, the same level as the input signal.

  For each stage in FIG. 31, the operation of each component of the above-described encoding device may be further added as necessary.

  FIG. 32 is a flowchart illustrating an operation of a method for decoding a fine structure of a spectrum according to an embodiment. According to the method of FIG. 32, in order to dequantize the fine structure of the normalized spectrum, for each band, the ISC and the information related to the selected ISC are determined by the position, number, sign, and size. Decrypted. Here, the size information is decoded by arithmetic decoding and the USQ joint method and the TCQ joint method, and the position, number, and code information are decoded by arithmetic decoding.

  Specifically, referring to FIG. 32, in step 3210, a decoding method is selected. For this purpose, information related to each band and bit allocation information are used. Here, the decoding scheme may include an inverse quantization scheme. The inverse quantization scheme is selected through the same process as the quantization scheme selection applied in the encoding device described above.

  In step 3230, it is determined whether the current band is a band in which bit allocation is zero, that is, a zero band. If the current band is a zero band, the process proceeds to step 3250. If the current band is a non-zero band, 3270 Proceed to stage.

  In step 3250, all samples in the zero band can be decoded to zero.

  In operation 3270, a band other than the zero band can be decoded based on the selected inverse quantization scheme. According to one embodiment, band length and bit allocation information may be used to estimate or determine the number of pulses per band. It is performed through the same process as the scaling applied in the encoding device described above. Next, the position information of the ISC, that is, the number and position of the ISC can be restored. It is processed analogously to the encoding device described above, and the same probability values are used for proper decoding. Next, the magnitude of the collected ISC is decoded by arithmetic decoding and dequantized by the USC joint scheme and the TCQ joint scheme. Here, the non-zero position and the number of ISCs are used for arithmetic decoding. The USC joint method and the TCQ joint method have a first joint method and a second joint method according to the bandwidth. In the first joint scheme, a quantizer is selected by additionally using a secondary bit allocation process for surplus bits from a previous band. The first joint scheme is used for NB and WB, and the second joint scheme is , USQ, the TCB is used for the LSB, and the remaining bits use the USQ, and can be used for SWB and FB. On the other hand, the selected ISC code information is arithmetically decoded with the same probability for positive and negative codes.

  After the step 3270, the method may further include a step of restoring the quantized component and a step of de-scaling the band. To restore the actual quantized component of each band, position, sign, and size information may be added to the quantized component. Bands with no data to be transmitted are filled with zeros. Next, the number of pulses in the non-zero band is estimated, and position information including the number and position of the ISC is decoded based on the estimated number of pulses. For the size information, lossless decoding and decoding by the USC joint method and the TCQ joint method are performed. For non-zero magnitude values, the sign and quantized components are finally restored. On the other hand, inverse scaling is performed on the restored actual quantized component using the transmitted norm information.

  For each stage in FIG. 32, the operation of each component of the above-described decoding device may be further added as necessary.

  The embodiment may be embodied in a general-purpose digital computer that operates on a computer-readable recording medium that can be created as a computer-executable program. In addition, the data structure, program instructions, or data files used in the embodiments of the present invention are recorded on a computer-readable recording medium via various means. The computer-readable recording medium may include all types of storage devices that store data that can be read by a computer system. Examples of a computer-readable recording medium include a magnetic medium such as a hard disk, a floppy (registered trademark) disk, and a magnetic tape; a compact disc (CD) -read only memory (ROM); and a digital versatile (DVD). optical media such as discs; magnetic-optical media such as floppy disks; and ROM, random access memory, and flash memory A hardware device specially configured to store and execute the program instructions may be included. Further, the computer-readable recording medium is a transmission medium for transmitting a signal designating a program instruction, a data structure, and the like. Examples of the program instructions may include not only machine language codes generated by a compiler but also high-level language codes executed by a computer using an interpreter or the like.

  As described above, even if one embodiment of the present invention is described with reference to the limited embodiment and the drawings, the one embodiment of the present invention is not limited to the above-described embodiment. Various modifications and variations will be possible from such a description if one of ordinary skill in the art to which this belongs. Therefore, the scope of the present invention is described not in the above description but in the appended claims, and any equivalent or equivalent modifications fall within the technical concept of the present invention.

Claims (4)

Selecting an encoding scheme based on the bits allocated to the band;
Encoding the spectral components of the band to zero if the selected encoding scheme is a zero encoding scheme;
If the selected coding scheme is not the zero coding scheme, USQ (uniform scalar quantization) and TCQ (trellis coded quantization) may be used according to the average number of bits allocated to the spectral components of the band with respect to the size of the important spectral components. ), Encoding using one of the spectrum encoding methods.   The method of claim 1, further comprising coding a number, a position, and a code of the important spectral components when the selected coding scheme is not a zero coding scheme.   The spectrum encoding method according to claim 2, wherein the size of the important spectral component is encoded by a scheme different from the scheme used for the number, position, and code of the important spectral component.   Scaling the normalized spectrum based on the bits allocated to the band, if the selected coding scheme is not the zero coding scheme; and, for the scaled spectrum, the significant spectral component The spectrum encoding method according to claim 1, further comprising the step of:

Images