JP4618634B2 - Compressed audio data processing method - Google Patents

Description

  The present invention relates to a compressed audio data processing method, and more particularly, to a compressed audio data processing method capable of cutting out a frame at an arbitrary location from audio data that has been compressed and encoded and has a frame structure, and that can be favorably reproduced.

  There is a distribution service in which audio data such as music stored in a content distribution server can be downloaded to a mobile phone and used as a ringtone. In the mobile phone, the downloaded audio data can be stored in a memory, read and reproduced when an incoming call is received, and ringed as a ringtone.

  In addition, a technique for cutting out and reproducing data at an arbitrary position of audio data stored in a memory is also known. For example, Patent Document 1 discloses a method for reproducing an incoming melody of a mobile phone that can set a reading start point and a turning point of audio data (ringing melody data) stored in a memory and can freely set a ringing sound ringing section. Is described.

  By using such a distribution service, the user can easily reproduce a ring tone by designating an arbitrary portion of the audio data stored in the memory. Further, in terms of content creation, it is only necessary to create one content to be stored in the content distribution server, which leads to cost reduction.

  Audio data compression-coded with the HEAAC (high efficiency advanced audio coding) profile of MPEG-4 audio has a frame structure. Since this frame structure involves a forward reference mode in which encoding is performed with reference to the previous frame, a frame that can be decoded immediately from there and a frame that can be decoded only when there is a previous frame. There is.

Therefore, when cutting out necessary parts from long-time content, considering the above frame structure, cut out according to the periodic sync point (data position that can be immediately decoded) of the original content. Need to be considered to do. Further, by performing fade-in / fade-out processing after the cut-out, natural audio can be heard.
Japanese Patent Laid-Open No. 2004-23298 Japanese Patent Application No. 2004-109728 (prior application)

  In the above prior art, audio data can be reproduced immediately when clipping is performed at a periodic sync point of the original content. However, when clipping is performed at a position other than that, there is no previous frame as a reference, and audio data cannot be reproduced immediately. As a result of the cutout position being limited to the sync point, the user cannot freely cut an arbitrary part from the long-time content, and there is a problem that the degree of freedom in editing is limited.

  The object of the present invention is to solve the above-mentioned problems and to cut out any part from a long-time content freely and reproducibly even in an environment with limited memory and computing power. It is an object of the present invention to provide a compressed audio data processing method that can increase the degree of freedom.

In order to solve the above-described problem, the present invention is a case where each frame of the compression-encoded audio data is expressed in subbands when a frame at an arbitrary location in the compression-encoded audio data having a frame structure is cut out. Thus, the audio data representation that refers to the previous frame is replaced with the audio data representation that sequentially references the subbands in the own frame in the frequency direction only when temporal direction prediction is applied to the cut-out frame. .

In addition, when temporal direction prediction is applied to a cut-out frame , the present invention adds information on a sync point that appears immediately before the frame to the frame, and also uses a base subband in the subband of the frame. The band is reconstructed into an audio data representation that does not refer to the previous frame using a past frame, and the other subbands are replaced with an audio data representation that is sequentially referenced in the frequency direction within the frame. .
In the present invention, when the data structure of compression-encoded audio data having a frame structure is formed by mixing a component that performs temporal direction prediction and a component that does not perform temporal direction prediction, a component that does not perform temporal direction prediction It is characterized in that only the component for which the temporal direction prediction is performed is replaced as it is.

Here, when the amount of increase in the number of bits necessary for the replaced audio data expression is equal to or greater than a certain threshold value, the audio data expression may not be replaced, and a separate fade-in process may be performed.

  According to the present invention, the user can freely cut out audio data from a long-time content so that the audio data can be reproduced, and thus the degree of freedom in editing can be increased. In HE AAC, SBR data can be reproduced immediately as it is, so that the sound quality of the audio output can be kept good.

  Hereinafter, the present invention will be described in detail with reference to the drawings. First, the principle of the present invention will be described. Here, it is assumed that the audio data is an MPEG-4 audio HE AAC Profile. In audio encoding, sampled digital audio data is collected by a predetermined number of samples, and encoding is performed in this unit. A typical sampling frequency is 44.1 kHz, and encoding is performed in units of 2048 samples to form an audio frame (hereinafter simply referred to as a frame). In this case, the time per frame is about 46 msec.

  In HE AAC, the low frequency component of audio is encoded by the AAC method, the high frequency component is encoded by the SBR (spectral band replication) method, and each frame is composed of the AAC data and the SBR data obtained thereby. Appropriate SBR data includes an SBR header.

  FIG. 1 schematically shows a frame (a) by HE AAC and a frequency band (b) that can be reproduced by AAC data and SBR data. AAC data enables playback of audio from 0 to 11.025 kHz (low frequency component), and SBR data enables playback of audio from 11.025 to 22.05 kHz (high frequency component). Therefore, at the time of decoding, if AAC data is used, audio in a low frequency band of 0 to 11.025 kHz can be reproduced, and if SBR data is used, audio up to a high frequency band of 11.025 to 22.05 kHz can be reproduced.

  When reproducing audio data, since all frames of AAC data are sync points, reproduction can be performed regardless of the first frame, and audio of low frequency components can be reproduced immediately.

  On the other hand, SBR data cannot be reproduced immediately from an arbitrary frame. First, an SBR header is required to fully reproduce each SBR data. Further, when inter-frame prediction is performed, the previous frame is necessary. As for SBR data, a frame without inter-frame prediction is regularly set, and if it is from that frame, it can be reproduced immediately. The inter-frame period without inter-frame prediction is typically 1 second. In this case, the period in which the audio data can be cut out while being reproducible is once per second.

  The HE AAC Profile SBR data has a forward reference mode in which encoding is performed with reference to the previous frame. At this time, the inside of the SBR data is divided into a plurality of subbands, and reference is made for each subband. In the data of SBR data, the location where the flag bs_df_env [i] [j] indicating the reference direction is bs_df_env [i] [j] = 1 (time direction) corresponds to that in the syntax.

  FIG. 2 is an explanatory diagram of reference in encoding of SBR data. Assuming that the k-th frame is bs_df_env [i] [j] = 1 (time direction), that is, the reference in the time direction, each subband of the SBR data of the k-th frame is the first (k -1) Encoded with reference to the corresponding subband of the frame.

  If the k-th frame is cut out as it is as a trimming start point (hereinafter referred to as an IN point), the (k-1) -th frame referred to by the k-th frame does not exist, and the reproduction cannot be performed. . On the other hand, since AAC data is not forward-referenced, it can be reproduced without any problem from the k-th frame.

  For the above reasons, the HE AAC Profile decoder having a general configuration reproduces only AAC data until it reaches a reproducible frame from the frame, and upsamples and outputs it.

  In the observation of SBR data so far, since a frame without forward reference appears and is refreshed within about 5 frames at the longest, the period during which SBR data cannot be decoded is theoretically about 230 msec.

  However, there is no guarantee that it will always be refreshed in about 5 frames, and there is also an implementation that does not start SBR data playback until the sbr_header that represents the sync point of SBR data arrives. The method to be used is desired. Patent Document 2 describes that sbr_header is configured on the basis of frame information traced back from the IN point and a measure is given to the frame at the IN point, but bs_df_env [i] [j] = 1 (time direction) measures are not described.

  In the present invention, intentional refresh is realized by taking measures against bs_df_env [i] [j] = 1 (time direction). That is, the audio data representation that refers to the previous frame of the audio data of the IN point frame is replaced with the audio data representation that refers to the inside of the own frame, and the IN point frame is reconstructed. As a result, the frame at the IN point is refreshed in the form of a reference in its own frame and can be immediately reproduced. If the frame at the IN point is reproduced, the subsequent frame can be reproduced with reference to the frame before the own frame. The above is the principle of the present invention.

  Next, an embodiment of the present invention will be described. In this embodiment, when the frame at the IN point is bs_df_env [i] [j] = 1 (time direction), the subband serving as the base of the frame is reconfigured based on the information of the past frame, For subbands other than the base subband, the expression is replaced so as to sequentially refer to the frequency direction. Furthermore, the reference direction flag bs_df_env [i] [j] is changed to bs_df_env [i] [j] = 0 (time direction) in accordance with the change of the reference direction to the frequency direction.

  For example, as shown in FIG. 3, when the kth frame where bs_df_env [i] [j] = 1 (time direction) is the IN point, the SBR header from the past sync point is added to the kth frame. In addition, the base band among the subbands of the kth frame is reconstructed into a data representation that does not refer to the previous frame using the past frame, and the subbands other than the base subband are the same. This is replaced with an expression that is sequentially referenced (referenced within the frame) in the frequency direction within the frame (kth frame). Furthermore, a change is made so that bs_df_env [i] [j] = 0 (time direction). As a result, the k-th frame is recreated as a sync point, so that SBR data can be immediately reproduced from here.

  FIG. 4 is a functional block diagram showing an example of a compressed audio data processing apparatus to which the present invention is applied. The compressed audio data processing apparatus of this example includes an AAC / SBR separation unit 41, an SBR analysis unit 42, a Huffman decoding unit 43, an SBR reconstruction unit 44, and an AAC / SBR synthesis unit 45. The AAC / SBR separation unit 41, the SBR analysis unit 42, and the Huffman decoding unit 43 are equivalent to the functions of AAC / SBR separation, SBR analysis, and Huffman decoding in a normal HE AAC decoder.

  The AAC / SBR separation unit 41 separates SBR data from the HE ACC encoded stream. The SBR analysis unit 42 analyzes the SBR data separated by the AAC / SBR separation unit 41. In the Huffman decoding unit 43, when cutting is performed, the SBR reconstruction unit 44 reproduces the past frame and the frame necessary for the SBR reconstruction. When the frame at the IN point is bs_df_env [i] [j] = 1 (time direction), the SBR reconstructing unit 44 reconstructs the SBR data of the frame using the frame reproduced by the Huffman decoding unit 43. Constitute. That is, as described above, the base subband is reconstructed into a data representation that does not refer to the previous frame using a past frame, and the other subbands are sequentially referred to in the frequency direction within the same frame ( Replace with the expression to be referenced in the frame. In addition, an SBR header at a past sync point is added. No processing is performed for frames other than the frame at the IN point.

  As described above, the frame at the IN point is represented by the intraframe reference, and the SBR stream that sequentially refers to the subsequent frame in the time direction is transmitted. The AAC / SBR combining unit 45 combines the AAC stream and the SBR stream to generate an encoded stream.

  FIG. 5 is an explanatory diagram showing reproducible frames (AAC data and SBR data) according to the present invention. For AAC data, since all frames are sync points, the 0 to 11.025 kHz component (low frequency component) of the audio signal is the frame at which the IN point is frame (1), (2),. But you can play it right away.

  Since SBR data can be decoded after the frame (8), which is the sync point, has arrived, frames (8) and (9) can be played back up to 11.025-22.05 kHz components (high-frequency components). SBR data cannot be played. However, according to the present invention, for example, if the frame at the IN point is the frame (4), the frame (4) is reconstructed into an audio data representation that refers to the inside of the frame, and thereby intentional refresh is performed. Since this is realized, the SBR data of the frames (4) to (7) can be theoretically reproduced.

  As a result of countermeasure against bs_df_env [i] [j] = 1 (time direction), the frame size of the IN point increases. In particular, in the HE AAC Profile of MPEG-4 audio, a Huffman code is used to represent a reference value, so that a longer codeword is used, and the amount of information increases accordingly. However, only the frame at the IN point increases in size, and there is no change in the size of subsequent frames.

  Regarding the amount of change in frame size, the theoretical worst value such as maximum use of escape expression with Huffman code was found, and it was 500 bytes. Actually, when the amount of change in the frame size was experimentally obtained with a large number of audio files, it was an increase of 4.32 bytes on average. Under the conditions of 48 kbps and 44.1 kHz, the size per frame is originally 286 bytes, so there is no particular problem on average.

  When the increase in frame size becomes a problem, in preparation for the above theoretical worst value, when the increase in code size becomes too large, for example, when exceeding the upper limit of 64 bytes, cancellation of forward reference is canceled, Trimming / fade processing can also be performed while leaving the forward reference state.

  As described above, according to the present invention, data at an arbitrary location can be freely extracted from long-time content, and the sound quality of the audio output can be kept good. It can be used for audio distribution services on mobile and portable devices.

It is a figure which shows schematically the frame (a) by HE AAC, and the frequency band (b) of AAC data and SBR data of each frame. It is explanatory drawing which shows the reference of the SBR data before replacement by this invention. It is explanatory drawing which shows the reference of the SBR data after substitution by this invention. It is a functional block diagram which shows an example of the compression audio data processing apparatus with which this invention was applied. It is explanatory drawing which shows the flame | frame which can be decoded by this invention.

Explanation of symbols

41 ... AAC / SBR separation unit, 42 ... SBR analysis unit, 43 ... Huffman decoding unit, 44 ... SBR reconstruction unit, 45 ... AAC / SBR synthesis unit

Claims (5)

When extracting a frame at an arbitrary location in compression-encoded audio data having a frame structure, each frame of the compression-encoded audio data is represented by a subband, and temporal direction prediction is applied to the extracted frame. A compressed audio data processing method characterized by replacing an audio data representation that refers to a previous frame with an audio data representation that sequentially refers to subbands in the own frame in the frequency direction only if When temporal direction prediction is applied to a cut-out frame, information on a sync point appearing immediately before the frame is added to the frame, and a base subband in the subband of the frame is a past frame. 2. The audio data representation that does not refer to the previous frame is reconstructed using the subframe, and the other subbands are replaced with the audio data representation that is sequentially referred to in the frequency direction within the same frame. Compressed audio data processing method. When the data structure of compression-encoded audio data having a frame structure is formed by mixing a component that performs temporal direction prediction and a component that does not perform temporal direction prediction, the component that does not perform temporal direction prediction is left as it is, 3. The compressed audio data processing method according to claim 2, wherein only the component for which direction prediction is performed is replaced. 4. The compression encoding method of compression encoded audio data is HE-ACC, and the sync point information is an SBR header of a sync point existing immediately before the cut-out frame. The compressed audio data processing method described. 5. If the amount of increase in the number of bits required for the replaced audio data representation exceeds a certain threshold value, the audio data representation is not replaced and a separate fade-in process is performed. A compressed audio data processing method according to claim 1.

Images