JP2015505070A - Audio encoder with parallel architecture - Google Patents

Audio encoder with parallel architecture Download PDF

Info

Publication number
JP2015505070A
JP2015505070A JP2014547840A JP2014547840A JP2015505070A JP 2015505070 A JP2015505070 A JP 2015505070A JP 2014547840 A JP2014547840 A JP 2014547840A JP 2014547840 A JP2014547840 A JP 2014547840A JP 2015505070 A JP2015505070 A JP 2015505070A
Authority
JP
Japan
Prior art keywords
frames
frame
parallel
audio encoder
bits
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP2014547840A
Other languages
Japanese (ja)
Other versions
JP5864776B2 (en
Inventor
シュイルドバッハ,ヴァルフガング
Original Assignee
ドルビー・インターナショナル・アーベー
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US201161578376P priority Critical
Priority to US61/578,376 priority
Application filed by ドルビー・インターナショナル・アーベー filed Critical ドルビー・インターナショナル・アーベー
Priority to PCT/EP2012/075056 priority patent/WO2013092292A1/en
Publication of JP2015505070A publication Critical patent/JP2015505070A/en
Application granted granted Critical
Publication of JP5864776B2 publication Critical patent/JP5864776B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16Vocoder architecture
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/022Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/032Quantisation or dequantisation of spectral components

Abstract

This article relates to methods and systems for audio encoding. In particular, this paper relates to a method and system for high speed audio encoding using a parallel system architecture. A frame-based audio encoder (300, 400, 500, 600) having K parallel conversion units (303, 403) is described. Each of the K parallel conversion units (303, 403) converts each frame of the group of K frames (305) of the audio signal (101) into each of K sets of frequency coefficients. K> 1, and each of the K frames (305) includes a plurality of samples of the audio signal (101).

Description

This application claims priority to US Provisional Patent Application No. 61 / 565,037, filed Nov. 30, 2011.

TECHNICAL FIELD OF THE INVENTION This article relates to methods and systems for audio encoding. Specifically, this paper relates to a method and system for high speed audio encoding using a parallel encoder architecture.

  Today's media players are mp3, mp4, WMA (Windows Media Audio), AAC (Advanced Audio Coding), HE-AAC (High Efficiency AAC), etc. Support a variety of different audio formats. On the other hand, media databases (such as Simfy) provide millions of audio files for download.

  Typically, it is not economical to encode and store these millions of audio files in a variety of different audio formats and a variety of different bit rates supported by different media players. Thus, it provides a high-speed audio encoding scheme that allows encoding of audio files "on the fly", so that the media database is individually encoded (in a specific audio format, at a specific bit rate) It would be beneficial to be able to generate an audio file when it is required.

  According to one aspect, a frame-based audio encoder is described. The audio encoder may be configured to divide an audio signal including a plurality of time domain samples into a sequence of frames. Here, each frame typically includes a predetermined number of samples. As an example, a frame may include a fixed number M (eg, M = 1024) of samples. In some embodiments, the audio encoder is configured to perform advanced audio coding (AAC).

  The audio encoder may include K parallel conversion units that process K frames of the audio signal (eg, K consecutive frames of the audio signal) in parallel. K parallel conversion units may be implemented on K different processing units (eg graphic processing units), thereby accelerating the conversion process by a factor of K (compared to sequential processing of K frames). Also good. The transform unit may be configured to transform the frame into a set of frequency coefficients. In other words, the transform unit may perform a time domain to frequency domain transform such as a modified discrete cosine transform (MDCT). Thus, each of the K parallel conversion units is configured to convert each frame of a group of K frames (also referred to as a frame group) of the audio signal into each set of K sets of frequency coefficients. May be. K may be greater than 1,2,3,4,5,10,20,50,100.

  As described above, the K parallel conversion units may be configured to apply MDCT to K frames of the frame group, respectively. Further, the K parallel conversion units may be configured to apply a window function to each of the K frames of the frame group. It should be noted that the type of transformation and / or window type applied to a frame typically depends on the type of frame (ie, the frame type, also referred to herein as the block type). Therefore, the K parallel conversion units may be configured to convert K frames into sets of K frame type-dependent frequency coefficients, respectively.

  The audio encoder may have K parallel signal attack detection units. The signal attack detection unit may be configured to classify the frame of the audio signal as a frame that includes an acoustic attack (eg, a transient frame) or as a frame that does not include an acoustic attack (eg, a tone-like frame). Therefore, the K parallel signal attack detection units may be configured to classify the K frames of the frame group based on the presence or absence of an acoustic attack in each of the K frames. K parallel signal attack detection units may be implemented on at least K different processing units. In particular, the K parallel signal attack detection units may be implemented on the same respective processing unit as the K parallel conversion units.

  The audio encoder may further include a frame type detection unit configured to determine a frame type of each of the K frames based on the classification of the K frames. Examples of frame types are short block types (which are typically used for frames containing transient audio signals), long block types (which are typically used for frames containing tone-like audio signals), start Block type (which is typically used as transition frame from long block type to short block type) and / or stop type (which is typically used as transition frame from short block type to long block type) ). Thus, the frame type of the frame may depend on the frame type of one or more previous frames. As a result, the frame type detection unit is configured to determine the frame type of the frame k (k = 1,..., K) out of the K frames based on the frame type of the preceding frame k−1. May be.

  As an example, a frame type detection unit may have a short frame k (k = 1,..., K) if frame k is classified as containing an attack and its preceding frame k−1 is a short block type or a starting block type. You may be comprised so that it may determine with a block type. The frame type detection unit is configured such that frame k (k = 1,..., K) is a long block type when frame k is classified as not including an attack and its preceding frame k−1 is a long block type or a stop block type. It may be configured to determine that. The frame type detection unit determines that the frame k (k = 1,..., K) is the start block type when the frame k is classified as including an attack and the preceding frame k−1 is the long block type. It may be configured as follows. Furthermore, the frame type detection unit is classified as a stop block type if frame k is classified as not containing an attack and its preceding frame k−1 is a short block type. It may be configured to determine.

  The K parallel conversion units may be operated in parallel with the K parallel signal attack detection units and the frame type detection unit. Thus, the K parallel conversion units may be implemented in a different processing unit than the K parallel signal attack detection units, thereby allowing further parallelization of the encoders on at least 2K processing units. To do. In such cases, the conversion unit may be configured to perform frame type dependent windowing and / or speculative execution of the conversion process. Specifically, the transform unit may be configured to determine a plurality of frame type dependent sets of frequency coefficients for each frame of the frame group. More specifically, the transform unit may be configured to determine a frame type dependent set of frequency coefficients for each possible frame type of the frame. The audio encoder may then include a selection unit configured to select an appropriate set of frequency coefficients (for each of the K frames) from the set of frame type dependent frequency coefficients. . Here, the appropriate set of frequency coefficients corresponds to the frame type of the frame.

  Alternatively, the K parallel signal attack detection units may be operated sequentially with the frame type detection unit and sequentially with the K parallel conversion units. Therefore, K parallel signal attack detection units may be mounted on the same processing unit as the K parallel conversion units. In this case, the K parallel conversion units may know the frame type of each frame, and thus the K parallel conversion units determine the K frames according to the respective frame type corresponding to the frame type of the frame. It may be configured to convert into a set of frequency coefficients.

  The audio encoder may have K parallel quantization and encoding units. The K parallel quantization and encoding units may be implemented on at least K different processing units (eg, each processing unit of K parallel conversion units). The quantization and encoding unit may be configured to quantize and entropy encode (eg, Huffman encoding) each set of frequency coefficients, taking into account the respective number of assigned bits. In other words, the quantization and encoding of K frames of the frame group may be performed independently by K parallel quantization and encoding units. For this purpose, K parallel quantization and encoding units are given K indications of the respective number of assigned bits. The indication of the number of allocated bits may be determined jointly for the frame group in the joint bit allocation process. This will be outlined later.

  The audio encoder may further have K parallel psychoacoustic units. K parallel psychoacoustic units may be implemented on at least K different processing units. Typically, the K parallel psychoacoustic units may be implemented on the same respective processing unit as the K parallel conversion units. This is because K parallel psychoacoustic units typically further process each of the K sets of frequency coefficients provided by the K parallel conversion units. The K parallel psychoacoustic units are each configured to determine one or more frame-dependent (and typically frequency-dependent) masking thresholds based on the K sets of frequency coefficients. It may be. Alternatively or additionally, the K parallel psychoacoustic units may be configured to determine K perceptual entropy values for the corresponding K frames of the group of frames. In general terms, perceptual entropy values provide an indication of the information content of the corresponding frame. Typically, the perceptual entropy value corresponds to an estimate of the number of bits that should be used to encode the corresponding frame. Specifically, the perceptual entropy value for a given frame is determined based on the assumption that the noise assigned to the quantized frame is immediately below the one or more masking thresholds. It may indicate how many bits are needed to quantize and encode a given frame.

  The K parallel quantization and encoding units are each configured to quantize and entropy encode each of the K sets of frequency coefficients, taking into account one or more frame-dependent masking thresholds. It may be. Therefore, it is guaranteed that the quantization of the set of frequency coefficients is performed based on psychoacoustic considerations, thereby reducing the audible quantization noise.

  The audio encoder may have a bit allocation unit configured to allocate a respective number of bits to the K parallel quantization and encoding units. For this purpose, the bit allocation unit may distribute the total number of available bits to the individual frames of the frame group, taking into account the total number of bits available for the frame group. The bit allocation unit may be configured to allocate the respective number of bits in consideration of the frame type of each frame of the frame group. Further, the bit allocation unit may take into account some or all of the frame types of the frames of the frame group to improve the allocation of bits to the frames of the frame group. Alternatively or additionally, the bit allocation unit assigns K for the K frames of the group of frames determined by the K parallel psychoacoustic units to allocate the respective number of bits to the K frames. Individual perceptual entropy values may be taken into account. Specifically, the bit allocation unit scales or modifies the K perceptual entropy values depending on the total number of available bits for the frame group, thereby adjusting the bit allocation to the K group frame number. It may be configured to match the perceptual entropy of the frame.

  The audio encoder may further comprise a bit saving and tracking unit configured to track the number of previously consumed bits used to encode the frame of the audio signal preceding the K frame. . Typically, the audio encoder is given a target bit rate for the encoded audio signal. Thus, the bit savings tracking unit may be configured to track the number of bits previously consumed in relation to the targeted number of bits. In addition, the bit savings tracking unit uses the number of bits used by the K parallel quantization and encoding units to encode the K sets of frequency coefficients, and the number of previously consumed bits. May be configured to give the number of bits currently consumed. The number of bits currently consumed may then be the basis for the bit allocation process for subsequent frames of the subsequent K frames.

  The bit allocation unit allocates to encode each number of bits (ie, K frames of a group of frames, taking into account the number of previously consumed bits (provided by the bit savings tracking unit) Each bit number) may be assigned. Further, the bit allocation unit may be configured to allocate the respective number of bits in consideration of the target bit rate in order to encode the audio signal.

  Thus, the bit allocation unit may be configured to allocate each bit to a frame of a frame group for each group (not for each frame). To further improve the bit allocation, the bit allocation unit takes each bit into K quantization and encoding units in a “analysis by synthesis” manner by taking into account the number of bits currently consumed. It may be configured to assign a number. In other words, a number of sequential iterations of bit allocation and quantization / encoding may be performed on a group of frames, and in subsequent iterations the bit allocation unit is used by K quantization and encoding units. The number of currently consumed bits that have been consumed may be taken into account.

  Thus, the bit allocation unit allocates the respective number of bits, taking into account the number of currently consumed bits, and thereby the updated respective bits allocated for each of the K parallel quantization and encoding units. It may be configured to give a number. The K parallel quantization and encoding units are configured to quantize and entropy encode each K set of frequency coefficients, taking into account each updated allocated number of bits. It may be. This iterative bit allocation process may be repeated a predetermined number of iterations to improve bit allocation between frames of the group of frames.

  The K parallel quantization and encoding units and the K parallel conversion units may be configured to operate in a pipeline architecture. This is because the K parallel transform units have a subsequent frame group containing K subsequent frames while the K parallel quantization and encoding units are encoding the set of frequency coefficients of the current frame group. May be configured to process. In other words, the K parallel quantization and encoding units are connected to the K frames of the K frame group while the K parallel conversion units are transforming the frames of the K frame group. The K preceding sets of frequency coefficients corresponding to the preceding frame may be quantized and encoded.

  According to certain further aspects, a frame-based audio encoder is described that is configured to encode in parallel K frames (ie, frames) of an audio signal on at least K different processing units. Any of the features related to audio encoders described in this article are applicable. The audio encoder has K parallel conversion units configured to convert K frames to K sets of frequency coefficients respectively; each based on the presence or absence of an acoustic attack in each of the K frames. K parallel signal attack detection units configured to classify K frames; and / or K parallels configured to quantize and entropy encode each of the K sets of frequency coefficients. And / or at least one of the quantization and encoding units.

  According to certain further aspects, a frame-based audio encoder is described that is configured to encode in parallel K frames (ie, frames) of an audio signal on at least K different processing units. Any of the features related to audio encoders described in this article are applicable. The audio encoder has a transform unit configured to transform each of the K frames into K sets of frequency coefficients. In addition, the audio encoder has K parallel quantization and encoding units, and each of the K parallel quantization and encoding units takes into account each assigned number of bits. It is configured to quantize and entropy code K sets of frequency coefficients. In addition, the audio encoder can determine each of the K parallel quantization and encoding values based on the previously consumed number of bits used to encode the frame of the audio signal preceding the K frames. A bit allocation unit configured to allocate a respective number of bits to the unit;

  According to another aspect, a frame-based audio encoder is described that is configured to encode in parallel K frames of an audio signal on at least K different processing units. Any of the features related to audio encoders described in this article are applicable. The audio encoder has K parallel signal attack detection units. These signal attack detection units are configured to classify the K frames based on the presence or absence of an acoustic attack in each frame. Furthermore, the audio encoder is configured to determine the frame type of the frame group frame k (k = 1,..., K) based on the classification of the frame k and based on the frame type of the immediately preceding frame k−1. It has a frame type detection unit. Furthermore, the audio encoder has K parallel conversion units. Here, the K parallel conversion units are configured to convert K frames into K sets of frequency coefficients, respectively. Typically, the set of frequency coefficients corresponding to a frame depends on the frame type of that frame. In other words, the conversion unit is configured to perform frame type dependent conversion.

  According to certain further aspects, a method for encoding an audio signal including a sequence of frames is described. The method includes: converting K frames of an audio signal in parallel to K sets corresponding to frequency coefficients; each of the K frames based on the presence or absence of an acoustic attack in each frame of the K frames. Any one of the following: classifying the frames in parallel; and quantizing and entropy encoding each set of K sets of frequency coefficients in parallel, taking into account the respective number of bits allocated One or more may be included.

  According to another aspect, a method for encoding an audio signal including a sequence of frames is described. The method converts the K frames of the audio signal into K corresponding sets of frequency coefficients; each of the K sets of frequency coefficients, each taking into account the number of allocated bits. Quantizing and entropy-encoding in parallel; said respective number of bits based on a previously consumed number of bits used to encode a frame of the audio signal preceding the K frames May be included.

  According to certain further aspects, a method for encoding an audio signal including a sequence of frames is described. The method classifies each of the K frames of the audio signal based on the presence or absence of an acoustic attack in each frame of the K frames; and based on the classification of the frame k and the frame k−1 Determining the frame type of each frame k (k = 1,..., K) of the K frames based on the type; and paralleling each of the K frames to each of the K sets of frequency coefficients And a set k of frequency coefficients corresponding to the frame k depends on the frame type of the frame k.

  According to a further aspect, a software program is described. The software program may be adapted for execution on a processor and for performing the method steps outlined herein when executed on a computing device.

  According to another aspect, a storage medium is described. The storage medium may have a software program adapted for execution on a processor and for performing the method steps outlined herein when executed on a computing device.

  According to a further aspect, a computer program product is described. A computer program may include executable instructions for executing the method steps outlined herein when executed on a computer.

  It should be noted that the methods and systems outlined in this article, including preferred embodiments, can be used alone or in combination with other methods and systems disclosed herein. Further, all aspects of the methods and systems outlined in this paper may be combined arbitrarily. In particular, the features of the claims may be combined with one another in any way.

The invention will now be described by way of example with reference to the accompanying drawings.
1 is a block diagram of an exemplary audio encoder. FIG. FIG. 2 illustrates an exemplary frame-based time-frequency transform applied by an audio encoder. FIG. 2 is a block diagram of an exemplary audio encoder summary. FIG. 3 is a block diagram of an exemplary parallel architecture for the encoder summary shown in FIG. FIG. 3 is a block diagram of another exemplary parallel architecture for the encoder summary shown in FIG. 1 is a block diagram of an example audio encoder that includes various parallel encoder processes. FIG. 1 is a block diagram of an exemplary pipelined architecture of an audio encoder. FIG. 3 is an exemplary flowchart of a sequential iterative bit allocation process.

  FIG. 1 a shows an exemplary audio encoder 100. In particular, FIG. 1 a shows an exemplary advanced audio coding (AAC) encoder 100. Audio encoder 100 may be used as a core encoder in the context of a spectral band replication (SBR) based encoding scheme such as High Efficiency (HE) AAC. Alternatively, the audio encoder 100 may be used alone. AAC encoder 100 typically breaks audio signal 101 into a sequence of segments called frames. A time-domain process called a window provides a smooth transition from frame to frame by modifying the data in these frames. The AAC encoder 100 may adapt the encoding of the frame of the audio signal to the characteristics of the time domain signal (eg, the tone-like or transient section of the audio signal) contained within the frame. For this purpose, the AAC encoder 100 is dynamically switched between encoding the entire frame as one long block of M = 1028 samples and encoding the frame as a sequence of short blocks of M = 128 samples. Have been adapted. Thus, the AAC encoder 100 may switch between encoding at a relatively high frequency resolution (using long blocks) and encoding at a relatively high time resolution (using short block sequences). Thus, the AAC encoder 100 is tone-like (steady state, harmonically rich complex spectrum signal) (using long blocks) and impulse-like (transient signal) (using a sequence of 8 short blocks). Adapted to encode audio signals that swing between.

  Each block of samples is transformed to the frequency domain using a modified discrete cosine transform (MDCT). To avoid spectral leakage problems that typically occur in the context of block-based (also referred to as frame-based) time-frequency transformations, MDCT utilizes overlapping windows. That is, MDCT is an example of so-called duplicate conversion. This is illustrated in FIG. 1b. FIG. 1 b shows an audio signal 101 comprising a sequence of frames 171. In the illustrated example, each frame 171 includes M samples of the audio signal 101. Rather than applying the above transformation to only a single block, MDCT, which is a duplicate transformation, duplicates two adjacent frames, as shown by sequence 172. To further smooth the transitions between successive frames, a 2M length window function w [k] is further applied. As a result, a sequence of sets of frequency coefficients of size M is obtained. In the corresponding AAC decoder, inverse MDCT is applied to the sequence of frequency coefficient sets, thereby giving a sequence of time domain sample sets having a length of 2M. Using the overlap addition process 173 shown in FIG. 1b, a frame of length M of the decoded sample 174 is obtained.

  FIG. 1 a shows further details of an exemplary AAC encoder 100. The encoder 100 includes a filter bank 151 that applies MDCT transformation to a frame made up of samples of the audio signal 101. As outlined above, the MDCT transform is a duplicate transform, typically processing two frame samples of the audio signal 101 to provide a set of frequency coefficients. The set of frequency coefficients is subjected to quantization and entropy coding in unit 152. The quantization and encoding unit 152 ensures that an optimized tradeoff between the target bit rate and the quantization noise is achieved. A further component of the AAC encoder 100 is a perceptual model 153 that is used to determine (among other things) signal-dependent masking thresholds applied during quantization and encoding. Further, the AAC encoder 100 may include a gain control unit 154 that applies a global adjustment gain to each frame of the audio signal 101. By doing this, the dynamic range of the AAC encoder 100 can be increased. Further, temperature noise shaping (TNS) 155, backward prediction 156, and joint stereo coding 157 (eg, center / side signal encoding) may be applied.

  This article describes various measures to accelerate the audio encoding method shown in FIG. Although these measures are described in the context of AAC encoding, it should be noted that these measures are applicable to audio encoders in general. In particular, these measures are applicable to block-based (or frame-based) audio encoders in general.

  FIG. 2 shows an exemplary block diagram of the summary 200 of the AAC encoder 100. The diagram 200 relates to the filter bank block 151 shown in FIG. As outlined above, in order to adapt the encoding to the individual characteristics (tone-like or transient) of the audio signal 101, the AAC encoder 100 classifies the frames of the audio signal 101 as so-called long blocks and short blocks. . For this purpose, the AAC encoder 100 analyzes each frame of the audio signal 101 (including M = 1024 samples) and makes a decision regarding the appropriate block type for the frame. This is executed in the block type determination unit 201. In addition to a sequence of long blocks and (N = 8) short blocks, AAC is a "start block" (as a transition block between a long block and a short block sequence) and a "stop block" (a sequence of short blocks). Note that it provides an additional block type (as a transition block between and long blocks).

  After determining the block type, an appropriate window is applied to the frame of the audio signal 101 (windowing unit 202). As outlined above, the MDCT transform is a duplicate transform. That is, the window is applied to the current frame k and the previous frame k−1 of the audio signal (ie, a total of 2M = 2048 samples). The windowing unit 202 typically applies a type of window adapted to the block type determined in the block type determination unit 201. That is, the shape of the window depends on the actual type of frame k. After applying a window to a group of adjacent frames, an appropriate MDCT transform is applied to the windowed group of adjacent frames to provide a set of frequency coefficients corresponding to the frame of the audio signal 101. As an example, if the block type of the current frame k is “short block”, a sequence of 8 short blocks of the windowed samples of the current frame k is frequency coefficients using 8 consecutive MDCT transforms 203. Are converted into 8 sets. On the other hand, if the block type of the current frame k is “long block”, the windowed samples of the current frame k are converted to a single set of frequency coefficients using a single MDCT transform.

  The above process is repeated for all frames of the audio signal 101, thereby giving a sequence of sets of frequency coefficients that are sequentially quantized and encoded. Due to the sequential encoding scheme, the overall encoding speed is limited by the processing power of the processing unit used to encode the audio signal 101.

  In this paper, it is proposed to decompose the dependency chain of the conventional audio encoder 100, 200 described in the context of FIGS. 1a and 2 in order to accelerate the overall encoding speed. In particular, it is proposed to parallelize at least the transformation-related encoding tasks described in the context of FIG. An example of a parallelized architecture corresponding to the sequential architecture 200 is shown in FIG. In the parallel architecture 300, multiple frames 305 of the audio signal 101 are collected. As an example, K = 10 frames of the audio signal 101 are collected. For each of a plurality of K frames 305, signal attack detection (signal attack detection unit 301) is performed to determine whether frame k (k = 1,..., K) includes tone-like content or transient content. Executed). Based on this classification of each of the multiple K frames 305, the “attack to block type” unit 304 may determine an individual block type for each of the multiple K frames 305. In particular, the attack-to-block unit 304 determines whether a particular frame k from multiple K frames 305 should be encoded as a sequence of short blocks, as a long block, as a start block or as a stop block. Also good.

  Now that the respective block types have been determined, the windowing and transform unit 303 may apply an appropriate window and an appropriate MDCT transform to each of the multiple K frames 305. This may be done in parallel for K frames 305. In view of the overlap between adjacent frames, a group of adjacent frames may be input to the K parallel windowing and conversion processes. As an example, K parallel windowing and transformation processes may be identified by indices k = 1,. The kth process handles the kth frame among a plurality of K frames. Since the windowing and transformation typically overlap, the kth process may be further given one or more preceding frames (eg, the (k−1) th frame) of the kth frame. Thus, the K processes may be performed in parallel, thereby providing K sets of frequency coefficients for the K frames 305 of the audio signal 101. In contrast to the sequential architecture 200 shown in FIG. 2, the parallel architecture 300 may be run on K parallel processing units, thereby increasing the overall processing speed as described in FIG. May be accelerated by K times compared to simple processing.

  Alternatively or additionally, the architecture 200 of FIG. 2 can be parallelized by decomposing the dependency chain between block type determination of the frame of the audio signal 101 and windowing / transformation. This dependency chain may be broken down by temporarily performing calculations that may be discarded later. The benefit of such speculative execution is that as a result of the speculative execution, a number of uniform processing tasks that can be parallelized are executed. The increased speed typically brought about by parallel computing is more important than the inefficiency caused by discarding some of the computational results.

  As outlined in the context of FIGS. 2 and 3, the AAC encoder 100 first determines the block type and then performs the windowing and conversion process for the first time. This leads to the dependency that windowing and conversion can only be performed after block type determination has been performed. However, when allowing speculative execution as illustrated by the encoding scheme 400 of FIG. 4, four different transforms using the four different window types available in AAC will result in each (overlapping) frame l of the audio signal 101 Can be executed in parallel. Four sets of frequency coefficients for each frame l are determined in parallel in the windowing and transform unit 403. As a result, four sets of frequency coefficients are obtained for each frame l of the audio signal 101 (a set for the long block type, a set for the short block type, a set for the start block type and a set for the stop block type). Block type determination 301 may be performed independently (eg, in parallel) with windowing and transformation of frame k. Depending on the block type of frame l determined in parallel block type determination 301, an appropriate set of frequency coefficients may be selected for frame l using selection unit 406. The other three sets of frequency coefficients provided by the windowing and transform unit 403 may be discarded.

  As a result of such speculative execution, L frames of the audio signal may be subjected to windowing and transforming process 403 in parallel using different processing units. Each of the processing units (eg, l-th processing unit where l = 1,..., L) determines four sets of frequency coefficients for the l-th frame handled by that processing unit. That is, each processing unit performs approximately four times as many processing steps as the windowing and transformation 301 performed when the block type is already known. Nevertheless, the overall encoding speed is increased by a factor of L / 4 by the parallel architecture 400 shown in FIG. L can be selected in the hundreds of ranges. This makes the proposed method suitable for use in a processor farm with a large number of parallel processors.

  Parallel architecture 400 may be used as an alternative to or in combination with parallel architecture 300. However, it should be noted that the encoding delay typically increases as a result of parallelization. On the other hand, the encoding speed can be significantly increased, which makes the parallel architecture interesting in the context of audio download applications where high speed ("on-the-fly") download can be achieved by massive parallelization of the encoding process.

  FIG. 5 shows a further exemplary parallel encoder architecture 500. Architecture 500 is an extension of architecture 300 and includes the additional aspect of applying psychoacoustic model 153 and performing quantization and encoding 152. Similar to FIG. 3, the architecture 500 includes a signal attack detection unit 301 that processes K frames 305 of the audio signal 101 in parallel. Based on the classified frames, an “attack to block type” unit 304 determines the block type of each of the K frames 305. Thereafter, K sets of frequency coefficients corresponding to the K frames 305 are determined in the K parallel processes within the windowing and transform unit 303. These K sets of frequency coefficients may be used in psychoacoustic processing unit 506 to determine a frequency dependent masking threshold for the K sets of frequency coefficients. The masking threshold is used in the quantization and encoding unit 508 to quantize and encode the K sets of frequency coefficients in a frequency dependent manner under psychoacoustic considerations. In other words, for the k th set of frequency coefficients (ie, for the k th frame), psychoacoustic processing unit 506 determines one or more frequency dependent masking thresholds. The determination of the one or more masking thresholds may be performed in parallel for a set of frequency coefficients k = 1,. The one or more masking thresholds of the kth frame are provided to quantization and encoding units 152, 508 (serialized or parallelized) for quantization and encoding of the kth set of frequency coefficients. It is done. Thus, the determination of the frequency dependent masking threshold may be parallelized. That is, the determination of the masking threshold may be performed in parallel on K independent processing units, thereby accelerating the overall encoding speed.

  Further, FIG. 5 illustrates an exemplary parallelization of the quantization and encoding process 152. Quantization is typically performed by power law quantization. By doing this, larger frequency coefficient values are automatically encoded with lower accuracy and some noise shaping is already incorporated into the quantization process. The quantized value is then encoded by Huffman coding. In order to adapt the encoding process to various local statistics of the audio signal 101, a specific (optimal) Huffman table may be selected from several Huffman tables stored in the database. Different Huffman tables may be selected for different parts of the spectrum of the audio signal. As an example, the Huffman table used to encode the k th set of frequency coefficients may depend on the block type of the k th frame.

  It should be noted that the search for a specific (optimal) Huffman table may be further parallelized. Let P be the total number of possible Huffman tables. For the kth frame (k = 1, ..., K), the kth set of frequency coefficients is different for P Huffman tables in P parallel processes (running on P parallel processing units) May be encoded using. This leads to P encoded sets of frequency coefficients. Here, each of the P encoded sets of frequency coefficients has a corresponding bit length. The Huffman table that leads to the encoded set of frequency coefficients with the shortest bit length may be selected as the specific (optimal) Huffman table for the kth frame. Instead of a full parallelization scheme, an intermediate parallelization scheme such as a branching strategy with branch alpha / beta pruning (where each branch is executed in a separate parallel processing unit) It may be used to determine a specific (optimal) Huffman table for the frame.

  Since Huffman coding is a variable code length method and noise shaping should be performed to keep the quantization noise below the frequency-dependent masking threshold, a global gain value (which determines the size of the quantization step) and a scale factor (Determining the noise shaping factor for each scale factor (ie frequency) band) is typically applied before actual quantization. The process of determining the optimal trade-off between global gain value and scale factor (under the target bit rate and / or target perceptual distortion constraints) for a given frame of audio signal 101 is typically It is performed in a synthetic analysis manner by two nested sequential iteration loops. In other words, the quantization and encoding process 152 typically includes two nested sequential iteration loops, a so-called inner sequential iteration loop (or rate loop) and an outer sequential iteration loop (or noise control loop).

  In the context of an inner sequential iterative loop (rate loop), a global gain so that the quantized and encoded set of frequency coefficients meets the target bit rate (or meets the number of allocated bits for a particular frame k) The value is determined. In general, the Huffman code table assigns shorter codewords to (more frequent) smaller quantized values. If the number of bits resulting from the encoding process exceeds the number of bits available to encode frame k, it will give a larger quantization step size, thus leading to a smaller quantized value. It can be corrected by adjusting the global gain. This process is repeated using different quantization step sizes until the number of bits required for Huffman coding is less than or equal to the bits assigned to the frame. This loop is called a rate loop because it modifies the overall encoder bit rate until the bit rate meets the target bit rate.

  In the context of the outer iterative loop (noise control loop), a frequency dependent scale factor is applied to the frequency dependent masking threshold to control the overall perceptual distortion. A scale factor is applied to each scale factor band to shape the quantization noise according to a frequency dependent masking threshold. A scale factor band corresponds to a frequency interval in the audio signal, and each scale factor band includes a different subset of the set of frequency coefficients. Typically, the scale factor band corresponds to a perceptually motivated subdivision into critical subbands of the overall frequency range of the audio signal. The encoder typically starts with a default scale factor of 1 for each scale factor band. If the quantization noise in a given band is found to exceed the frequency dependent masking threshold (ie, acceptable noise in this band), the scale factor for this band is adjusted to reduce the quantization noise Is done. Thus, the scale factor corresponds to a frequency dependent gain value (as opposed to the overall gain value adjusted in the rate adjustment loop), which is used to individually control the quantization step in each scale factor band. Can be used.

  Achieving smaller quantization noise requires a larger number of quantization steps and thus a higher bit rate, so the rate adjustment loop may need to be repeated each time a new scale factor is used . In other words, the rate loop is nested within the noise control loop. The outer (noise control) loop is where the actual noise (calculated from the difference between the original spectral value minus the quantized spectral value) is below the masking threshold for all scale factor bands (ie critical bands) It is executed until.

  Although the inner sequential iteration loop always converges, this is not true for the combination of both sequential iteration loops. As an example, if the perceptual model requires a too small quantization step size and the rate loop must always increase the quantization step size to allow encoding at the target bit rate, both The loop does not converge. Conditions may be set to stop successive iterations if convergence is not achieved. Alternatively or additionally, the determination of the masking threshold may be based on a target bit rate. In other words, the masking threshold determined, for example, in the perceptual processing unit 506 may depend on the target bit rate. This typically allows convergence of the quantization and encoding scheme to the target bit rate.

  Note that the iterative iterative quantization and encoding process described above (also referred to as the noise assignment process) is only one possible process for determining the set of quantized and encoded frequency coefficients. Should be kept. The parallelization scheme described herein applies equally to other implementations of the parallel noise allocation process within quantization and encoding unit 508.

  As a result of the quantization and encoding process, a set of quantized and encoded frequency coefficients is obtained for the corresponding frame of the audio signal 101. This set of quantized and encoded frequency coefficients is typically represented as a number of bits, depending on the number of bits assigned to the frame. The acoustic content of the audio signal 101 can vary significantly from frame to frame, for example, frames containing tone-like content and frames containing transient content. Thus, the number of bits required to encode a frame (when given some perceptible perceptual distortion) can vary from frame to frame. As an example, a frame containing tone-like content may require a reduced number of bits compared to a frame containing transient content. At the same time, the overall encoded audio signal should meet some target bit rate. For example, the average number of bits per frame should meet a predetermined target value.

  In order to guarantee a predetermined target bit rate and take into account the varying bit requirements of the frame, the AAC encoder 100 typically utilizes a bit allocation process that works in conjunction with the overall bit reserve. . The overall bit savings are filled with a certain number of bits per frame according to the target bit rate. At the same time, the overall bit savings are updated with the number of bits used to encode past frames. Thus, the overall bit savings tracks the amount of bits already used to encode the audio signal 101, thereby providing an indication of the number of bits available to encode the current frame of the audio signal 101. To do. This information is used by the bit allocation process to allocate the number of bits for encoding the current frame. For this allocation process, the block type of the current frame may be taken into account. As a result, the bit allocation process may provide quantization and encoding unit 152 with an indication of the number of bits available for encoding the current frame. This indication may include a minimum number of assigned bits, a maximum number of assigned bits, and / or an average number of assigned bits.

  The quantization and encoding unit 152 uses the indication of the number of allocated bits to quantize and encode the set of frequency coefficients corresponding to the current frame, thereby quantizing and encoding the set of frequency coefficients To decide. This consumes some actual number of bits. This actual number of bits is typically known only after performing the quantization and encoding described above (including nested loops) and is within the limits provided by the allocated number of bits indication. Can vary. The overall bit savings are updated using this actual number of bits and the bit allocation process is repeated for subsequent frames.

  FIG. 5 shows a parallelized quantization and encoding scheme 508 that performs quantization and encoding of K sets of frequency coefficients corresponding to K frames 305 in parallel. As outlined above, the actual quantization and encoding of the k th set of frequency coefficients is independent of the quantization and encoding of the other set of frequency coefficients. As a result, the quantization and encoding of the K sets of frequency coefficients can be performed in parallel. However, an indication of the assigned bits (eg, the maximum number, minimum number and / or average number of assigned bits) for quantization and encoding of the k th set of frequencies is typically (k−1 ) Depends on the state of the overall bit savings after quantization and encoding of the second set. Accordingly, a modified bit allocation process 507 and a modified bit savings update process 509 are described herein that allow for the implementation of a parallelized quantization and encoding process 508.

  One example of the bit allocation process 507 may include updating the bit savings after the actual quantization and encoding 508 of the K sets of frequency coefficients. The updated bit savings can then be the basis for the bit allocation process 507 that provides the allocation of bits to subsequent K sets of frequency coefficients in parallel. In other words, the bit savings update process 509 and the bit allocation process 507 may be performed for each group of K frames (rather than performing the process for each frame).

  More specifically, the bit allocation process 507 obtains the total number of available bits T for a group of K frames from the bit savings (rather than obtaining the number of available bits per frame). May be included. Thereafter, the bit allocation process 507 may distribute the total number of available bits T to the individual frames of the group of K frames, so that for each kth frame of the group of K frames. Each number of allocated bits Tk (k = 1,..., K) may be given. The bit allocation process 507 may take into account the block type of the frames of the K frames. Specifically, the bit allocation process 507 considers the block types of all frames of K frames together, as opposed to the sequential bit allocation process 507, where only the block types of individual frames are taken into account. May be included. This additional information regarding the block type of adjacent frames in the group of K frames may be taken into account to provide improved bit allocation.

  To further improve the allocation of bits to frames of a group of K frames, the bit allocation / bit savings update process may be performed in a synthetic analysis manner, thereby optimizing the overall bit allocation May be used. An exemplary sequential iterative bit allocation process 700 that utilizes a combined analysis scheme is illustrated in FIG. In step 701, a total number T of bits for encoding a group of K frames 305 is received from the bit saving unit. This total number T of bits is then distributed to the frames of the group of K frames, so that the assigned number of bits Tk for each of the frames k (k = 1,..., K) of the group of K frames. (Step 702). In the first iteration of bit allocation process 700, distribution step 702 may be based primarily on the block type of K frames 305 in the group. The number Tk is passed to each quantization and encoding unit 508, where the K frames are quantized and encoded, thereby giving K encoded frames. Each of the K encoded frames uses Uk bits (k = 1, ..., K). The number of used bits Uk is received in step 703.

  Thereafter, it is verified whether the stopping criteria for the iterative bit allocation process 700 are met (step 704). Exemplary stop criteria may include an AND or OR combination of one or more of the following criteria: The sequential iterative bit allocation process has performed a predetermined maximum number of iteration steps; The sum, ie, ΣUk satisfies a predetermined relationship with the number of available bits T; the numbers Uk and Tk satisfy a predetermined relationship for some or all of k = 1,. As an example, if Ul <Tl for frame l, it may be beneficial to perform another iteration of bit allocation process 700. Here, Tl is reduced by the difference between Tl and Ul, and the available bit (Tl-Ul) is assigned to another frame.

  If the stop criterion is not met (reference number 705), a further iteration of the bit allocation process 700 is performed. Here, the distribution of T bits (step 702) is performed in consideration of the used bits Uk (k = 1,..., K) of the immediately preceding iteration. On the other hand, if the stop criterion is met (reference number 706), the sequential iteration process is aborted and the bit savings are updated with the number of bits Uk actually used (ie, the used bits of the last iteration). .

  In other words, for a group of K frames, preliminary bits may be initially allocated for each of the K parallel quantization and encoding processes 508. As a result, K sets of quantized and encoded frequency coefficients and the actual number of bits used K are determined. These actual distributions of K bits may then be analyzed and the bit assignments to K parallel quantization and encoding processes 508 may be modified. As an example, assigned bits that were not used by a particular frame may be assigned to another frame (eg, a frame that has used up all of the assigned bits). K parallel quantization and encoding processes 508 may be repeated using a modified bit allocation process, and so on. In order to optimize the per-group bit allocation process 507, several iterations of this process (eg, two or three iterations) may be performed.

  FIG. 6 shows a pipeline scheme 600 that can be used in place of or in addition to the parallelization scheme outlined in FIGS. 3, 4 and 5. In the pipeline method 600, the set of frequency coefficients of the current frame k (reference numerals 301, 304, 303, 506) is quantized and encoded (reference numerals 608, 609) of the set of frequency coefficients of the preceding frame (k−1). ) And in parallel. These parallel processes are combined in the bit allocation stage 607 for the current frame k. As outlined above, the bit allocation stage 607 receives as input the bit storage updated and / or current with the actual number of bits used to encode the set of frequency coefficients of the previous frame (k−1). Use the block type of frame k as input. When the pipeline scheme 600 of FIG. 6 is used, the determination of the set of frequency coefficients of the current frame k (reference numerals 301, 304, 303, 506) and the quantization of the set of frequency coefficients of the preceding frame (k−1) are performed. And different processing units may be used for encoding (reference numerals 608, 609). As a result, it leads to about twice the speed of the encoding method.

  As shown in FIG. 6, the pipeline method 600 may be used in combination with the parallelization methods 300, 400, and 500. This is the frequency of the previous group of K frames while the current group of K frames is transformed to give K sets of frequency coefficients (reference numbers 301, 304, 303, 506). This means that the K sets before the coefficients can be quantized (reference numerals 608, 609). As outlined above, the parallelization of the determination of the K sets of frequency coefficients for the K frames allows the implementation of these parallel processes on K different processing units. Similarly, K parallel quantization and encoding processes 608 may be implemented on K different processing units. Overall, in pipeline scheme 600, 2K parallel processing units may be used, giving an overall acceleration of 2K times (eg, 20 times if K = 10).

  In FIGS. 3, 4 and 5, several structures are shown that may be used to provide an implementation of a high speed audio encoder. Alternatively or additionally, measures can be taken to accelerate the actual implementation of the encoder on one or more processing units. In particular, predicate logic may be used to provide an accelerated implementation of an audio encoder. Processing units with long processing pipelines typically have the problem of conditional jumps. This is because such conditional jumps prevent (delay) pipeline execution. Pipeline conditional execution is a feature of several processing units that may be used to provide an accelerated implementation. Alternatively, conditional execution may be emulated using a bit mask (instead of an explicit condition). This article describes various methods and systems for high-speed audio encoding. Several parallel encoder architectures are presented that allow implementation of various components of an audio encoder on a parallel processing unit, thereby reducing overall encoding time. The method and system for high speed audio encoding may be used for faster than real time audio encoding, for example, in the context of an audio download application.

  It should be noted that the above description and drawings merely illustrate the principles of the proposed method and system. Thus, those skilled in the art will appreciate that the principles of the invention may be embodied and modified in various ways within the spirit and scope thereof, even if not explicitly described or illustrated herein. It will be. Furthermore, all examples described in this article are primarily for educational purposes only to assist readers in understanding the principles of the proposed method and system and the concepts contributed by the inventors to advance the technology. It is expressly intended and is not to be construed as limiting to such individually described examples and conditions. Furthermore, any statement in this article that describes the principles, aspects and embodiments of the invention and specific examples thereof is intended to encompass equivalents thereof.

  The methods and apparatus described herein may be implemented as software, firmware and / or hardware. Certain components may be implemented as software running on a digital signal processor or microprocessor, for example. Other components may be implemented, for example, as hardware and / or application specific integrated circuits. The signals encountered in the described methods and systems may be stored on a medium such as a random access memory or an optical storage medium. The signal may be transferred via a radio network, a satellite network, a wireless network or a wired network, for example a network such as the Internet. Typical devices that utilize the methods and systems described herein are portable electronic devices or other consumer equipment that are used to store and / or represent audio signals.

Typically, it is not economical to encode and store these millions of audio files in a variety of different audio formats and a variety of different bit rates supported by different media players. Thus, it provides a high-speed audio encoding scheme that allows encoding of audio files "on the fly", so that the media database is individually encoded (in a specific audio format, at a specific bit rate) It would be beneficial to be able to generate an audio file when it is required. US2004 / 0024592A1 describes a system having a plurality of mp3 encoding units for encoding a plurality of divided data segments.

The methods and apparatus described herein may be implemented as software, firmware and / or hardware. Certain components may be implemented as software running on a digital signal processor or microprocessor, for example. Other components may be implemented, for example, as hardware and / or application specific integrated circuits. The signals encountered in the described methods and systems may be stored on a medium such as a random access memory or an optical storage medium. The signal may be transferred via a radio network, a satellite network, a wireless network or a wired network, for example a network such as the Internet. Typical devices that utilize the methods and systems described herein are portable electronic devices or other consumer equipment that are used to store and / or represent audio signals.
Several aspects are described.
[Aspect 1]
A frame-based audio encoder,
-K parallel conversion units, each of the K parallel conversion units, each frame of the group of K frames of the audio signal to each of the K sets of frequency coefficients Configured to convert, K> 1, and each of the K frames includes a plurality of samples of the audio signal;
Audio encoder.
[Aspect 2]
The audio encoder of aspect 1, wherein each of the K parallel transform units is configured to apply a modified discrete cosine transform to each of the K frames.
[Aspect 3]
The audio encoder according to aspect 1 or 2, wherein each of the K parallel conversion units is configured to apply a window function to each of the K frames.
[Aspect 4]
4. The aspect according to claim 1, wherein each of the K parallel conversion units is configured to convert each of the K frames into a set of frequency coefficients depending on a frame type. 5. Audio encoder.
[Aspect 5]
-Further comprising K parallel signal attack detection units, wherein each signal attack detection unit determines each of the K frames based on the presence or absence of an acoustic attack within each of the K frames. Configured to classify,
The audio encoder according to aspect 4.
[Aspect 6]
-Further comprising a frame type detection unit configured to determine a frame type of each of the K frames based on the classification of the K frames;
The audio encoder according to aspect 5.
[Aspect 7]
The audio encoder according to aspect 6, wherein the frame type is one of: short block type, long block type, start block type, and stop type.
[Aspect 8]
The frame type detection unit is configured to determine the frame type of each frame k out of the K frames based on the frame type of the frame k−1, where k = 1,. 8. An audio encoder according to aspect 6 or 7.
[Aspect 9]
Any one of aspects 6 to 8, wherein the K parallel conversion units (403) are operated in parallel with the K parallel signal attack detection units (301) and the frame type detection units (304). The described audio encoder (400).
[Aspect 10]
Each of the K parallel conversion units is configured to convert each of the K frames into a plurality of frame type dependent sets of frequency coefficients;
The encoder further comprises a selection unit configured to select the set of frequency coefficients from the plurality of frame type dependent sets of frequency coefficients, the selected set corresponding to a frame type of each frame To do,
The audio encoder according to any one of aspects 6 to 9.
[Aspect 11]
Aspect 6 wherein the K parallel signal attack detection units (301) are sequentially operated with the frame type detection units (304) that are sequentially operated with the K parallel conversion units (403). An audio encoder (400) according to any one of claims 1 to 8.
[Aspect 12]
The K parallel conversion units (303) convert each of the K frames into a set of frequency coefficients corresponding to the frame type of each frame determined by the frame type detection unit (304). 12. An audio encoder (300, 500, 600) according to any one of aspects 6 to 8 and 11, which is configured.
[Aspect 13]
-Further comprising K parallel quantization and encoding units, each of the K parallel quantization and encoding units assigned to each of the K sets of frequency coefficients, respectively. It is configured to quantize and entropy code based on the number of bits.
The audio encoder according to any one of aspects 1 to 12.
[Aspect 14]
-Further comprising K parallel psychoacoustic units, wherein each of the K parallel psychoacoustic units is based on each of the K sets of frequency coefficients, or Configured to determine a plurality of frame dependent masking thresholds;
The audio encoder according to aspect 13.
[Aspect 15]
The audio encoder of aspect 14, wherein each of the K parallel psychoacoustic units is configured to determine a perceptual entropy value indicative of information content of each of the K frames.
[Aspect 16]
Each of the K parallel quantization and encoding units quantizes and entropy each of the K sets of frequency coefficients, taking into account the respective one or more frame-dependent masking thresholds. 16. An audio encoder according to aspect 14 or 15, wherein the audio encoder is configured to encode.
[Aspect 17]
A bit allocation unit configured to allocate a respective number of bits to each of the K parallel quantization and encoding units;
The audio encoder according to any one of aspects 13 to 16.
[Aspect 18]
18. An audio encoder according to aspect 17, when citing aspect 6, wherein the bit allocation unit is configured to allocate the respective number of bits in consideration of a frame type of the K frames.
[Aspect 19]
The audio encoder of aspect 15, wherein the bit allocation unit is configured to allocate a respective number of bits taking into account perceptual entropy values of the K frames.
[Aspect 20]
Further comprising a bit saving and tracking unit configured to track the number of previously consumed bits used to encode the frame of the audio signal preceding the K frames;
The audio encoder according to any one of aspects 17 to 19.
[Aspect 21]
The bit savings tracking unit updates the number of previously consumed bits with the number of bits used by the K parallel quantization and encoding units to encode a set of frequency coefficients The audio encoder of aspect 20, wherein the audio encoder is configured to provide a number of bits currently consumed thereby.
[Aspect 22]
An audio encoder according to aspect 20 or 21, wherein the bit allocation unit is configured to allocate a respective number of bits, taking into account the number of previously consumed bits.
[Aspect 23]
The audio bit according to any one of aspects 20 to 22, wherein the bit allocation unit is configured to allocate a respective number of bits in consideration of a target bit rate for encoding the audio signal. Encoder.
[Aspect 24]
24. Audio according to any one of aspects 21 to 23, wherein the bit allocation unit is arranged to allocate the respective number of bits in an analysis by synthesis taking into account the number of bits currently consumed.・ Encoder.
[Aspect 25]
The bit allocation unit allocates a respective number of bits, taking into account the number of bits currently consumed, so that for each of the K parallel quantization and encoding units, the allocated bits; Is configured to give each updated number of
Each of the K parallel quantization and encoding units quantizes each of the K sets of frequency coefficients, entropy, taking into account each updated number of assigned bits; Configured to encode,
25. The audio encoder according to any one of aspects 21 to 24.
[Aspect 26]
The K parallel quantization and encoding units and the K parallel conversion units are configured to operate in a pipeline architecture;
The K parallel quantizing and encoding units are configured so that the K predecessors of the group of K frames while the K parallel transform units are transforming the frames of the group of K frames. Quantize and encode K preceding sets of frequency coefficients corresponding to a frame;
26. The audio encoder according to any one of aspects 20 to 25.
[Aspect 27]
27. An audio encoder according to any one of aspects 1 to 26, wherein the K parallel conversion units are implemented on at least K different processing units.
[Aspect 28]
28. An audio encoder according to any one of aspects 1 to 27, wherein the audio encoder is configured to perform advanced audio encoding referred to as AAC.
[Aspect 29]
A frame-based audio encoder configured to encode in parallel K frames of an audio signal on at least K different processing units, where K> 1, the audio encoder is:
• K parallel conversion units each configured to convert each of the K frames to each of K sets of frequency coefficients;
-K parallel signal attack detection units each configured to classify each of the K frames based on the presence or absence of an acoustic attack in the respective frames of the K frames;
K parallel quantization and encoding units each configured to quantize and entropy encode the K sets of frequency coefficients, taking into account the respective number of assigned bits;
An audio encoder comprising at least one of the following:
[Aspect 30]
A frame-based audio encoder configured to encode in parallel K frames of an audio signal on at least K different processing units, where K> 1, the audio encoder is:
A transform unit configured to transform the K frames into K corresponding sets of frequency coefficients;
K parallel quantizing and encoding units, each of the K parallel quantizing and encoding units taking into account the assigned number of bits and said frequency coefficient A quantization and encoding unit configured to quantize and entropy encode each of the K sets;
Each of the K parallel quantization and encoding units, respectively, based on the previously consumed number of bits used to encode the frame of the audio signal preceding the K frames; A bit allocation unit configured to allocate a number of bits;
Audio encoder.
[Aspect 31]
A frame-based audio encoder configured to encode in parallel K frames of an audio signal on at least K different processing units, where K> 1, wherein the audio encoder is
• K parallel signal attack detection units, wherein each signal attack detection unit classifies each of the K frames based on the presence or absence of an acoustic attack in each of the K frames. A configured signal attack detection unit;
K = 1,..., K is configured to determine the frame type of each frame k out of the K frames based on the classification of the frame k and the frame type of the frame k−1. A frame type detection unit;
K parallel conversion units, each of the K parallel conversion units configured to convert each of the K frames to each of K sets of frequency coefficients, and frame k A set k of frequency coefficients corresponding to a parallel conversion unit, which depends on the frame type of frame k,
Audio encoder.
[Aspect 32]
A method for encoding an audio signal including a sequence of frames, comprising:
Converting K frames of the audio signal into K sets corresponding to frequency coefficients in parallel, where K>1;
Classifying each frame of the K frames in parallel based on the presence or absence of an acoustic attack in each of the K frames; and
-Quantizing and entropy encoding each set of the K sets of frequency coefficients in parallel, taking into account the number of assigned bits,
Including any one or more of
Method.
[Aspect 33]
A method for encoding an audio signal comprising a sequence of frames, comprising:
Transforming the K frames of the audio signal into K corresponding sets of frequency coefficients, where K>1;
Quantizing and entropy encoding each of the K sets of frequency coefficients in parallel, taking into account the number of assigned bits;
Assigning the respective number of bits based on a previously consumed number of bits used to encode the frame of the audio signal preceding the K frames;
Method.
[Aspect 34]
A method for encoding an audio signal comprising a sequence of frames, comprising:
Classifying each of the K frames of the audio signal as K> 1 based on the presence or absence of an acoustic attack in each of the K frames;
Determining the frame type of each frame k out of the K frames as k = 1,..., K based on the classification of the frame k and the frame type of frame k−1;
Converting each of the K frames in parallel to each of K sets of frequency coefficients, wherein the set of frequency coefficients corresponding to frame k depends on the frame type of frame k; and including,
Method.
[Aspect 35]
35. A software program for performing the steps of the method of any one of aspects 32-34 when executed on a processor, adapted for execution on a processor.
[Aspect 36]
35. A storage medium having a software program for performing the steps of the method of any one of aspects 32-34 when executed on a computing device, adapted for execution on a processor.
[Aspect 37]
35. A computer program product comprising executable instructions for performing the steps of the method of any one of aspects 32-34 when executed on a computer.

Claims (37)

  1. A frame-based audio encoder,
    -K parallel conversion units, each of the K parallel conversion units, each frame of the group of K frames of the audio signal to each of the K sets of frequency coefficients Configured to convert, K> 1, and each of the K frames includes a plurality of samples of the audio signal;
    Audio encoder.
  2.   The audio encoder of claim 1, wherein each of the K parallel transform units is configured to apply a modified discrete cosine transform to each of the K frames.
  3.   The audio encoder according to claim 1 or 2, wherein each of the K parallel conversion units is configured to apply a window function to each of the K frames.
  4.   4. Each of the K parallel conversion units is configured to convert each of the K frames into a set of frequency coefficients depending on a frame type. 5. Audio encoder.
  5. -Further comprising K parallel signal attack detection units, wherein each signal attack detection unit determines each of the K frames based on the presence or absence of an acoustic attack within each of the K frames. Configured to classify,
    The audio encoder according to claim 4.
  6. -Further comprising a frame type detection unit configured to determine a frame type of each of the K frames based on the classification of the K frames;
    The audio encoder according to claim 5.
  7.   The audio encoder according to claim 6, wherein the frame type is one of: a short block type, a long block type, a start block type, and a stop type.
  8.   The frame type detection unit is configured to determine the frame type of each frame k out of the K frames based on the frame type of the frame k−1, where k = 1,. The audio encoder according to claim 6 or 7.
  9.   The K parallel conversion units (403) are operated in parallel with the K parallel signal attack detection units (301) and the frame type detection units (304), respectively. The audio encoder (400) according to claim.
  10. Each of the K parallel conversion units is configured to convert each of the K frames into a plurality of frame type dependent sets of frequency coefficients;
    The encoder further comprises a selection unit configured to select the set of frequency coefficients from the plurality of frame type dependent sets of frequency coefficients, the selected set corresponding to a frame type of each frame To do,
    The audio encoder according to any one of claims 6 to 9.
  11.   The K parallel signal attack detection units (301) are sequentially operated with the frame type detection units (304) sequentially operated with the K parallel conversion units (403). Audio encoder (400) according to any one of claims 6 to 8.
  12.   The K parallel conversion units (303) convert each of the K frames into a set of frequency coefficients corresponding to the frame type of each frame determined by the frame type detection unit (304). Audio encoder (300, 500, 600) according to any one of claims 6 to 8 and 11, wherein the audio encoder is configured.
  13. -Further comprising K parallel quantization and encoding units, each of the K parallel quantization and encoding units assigned to each of the K sets of frequency coefficients, respectively. It is configured to quantize and entropy code based on the number of bits.
    The audio encoder according to any one of claims 1 to 12.
  14. -Further comprising K parallel psychoacoustic units, wherein each of the K parallel psychoacoustic units is based on each of the K sets of frequency coefficients, or Configured to determine a plurality of frame dependent masking thresholds;
    The audio encoder according to claim 13.
  15.   The audio encoder of claim 14, wherein each of the K parallel psychoacoustic units is configured to determine a perceptual entropy value indicative of information content of each of the K frames. .
  16.   Each of the K parallel quantization and encoding units quantizes and entropy each of the K sets of frequency coefficients, taking into account the respective one or more frame-dependent masking thresholds. 16. An audio encoder according to claim 14 or 15, wherein the audio encoder is configured to encode.
  17. A bit allocation unit configured to allocate a respective number of bits to each of the K parallel quantization and encoding units;
    The audio encoder according to any one of claims 13 to 16.
  18.   18. An audio encoder as recited in claim 17, wherein the bit allocation unit is configured to allocate a respective number of bits in consideration of a frame type of the K frames.
  19.   The audio encoder of claim 15, wherein the bit allocation unit is configured to allocate a respective number of bits taking into account perceptual entropy values of the K frames.
  20. Further comprising a bit saving and tracking unit configured to track the number of previously consumed bits used to encode the frame of the audio signal preceding the K frames;
    The audio encoder according to any one of claims 17 to 19.
  21.   The bit savings tracking unit updates the number of previously consumed bits with the number of bits used by the K parallel quantization and encoding units to encode a set of frequency coefficients The audio encoder of claim 20, wherein the audio encoder is configured to provide a number of bits currently consumed.
  22.   22. Audio encoder according to claim 20 or 21, wherein the bit allocation unit is arranged to allocate a respective number of bits, taking into account the number of previously consumed bits.
  23.   23. Audio according to any one of claims 20 to 22, wherein the bit allocation unit is configured to allocate a respective number of bits in consideration of a target bit rate for encoding the audio signal.・ Encoder.
  24.   24. The bit allocation unit according to any one of claims 21 to 23, wherein the bit allocation unit is arranged to allocate the respective number of bits in an analysis by synthesis taking into account the number of bits currently consumed. Audio encoder.
  25. The bit allocation unit allocates a respective number of bits, taking into account the number of bits currently consumed, so that for each of the K parallel quantization and encoding units, the allocated bits Is configured to give each updated number of
    Each of the K parallel quantization and encoding units quantizes each of the K sets of frequency coefficients, entropy, taking into account each updated number of assigned bits; Configured to encode,
    25. An audio encoder according to any one of claims 21 to 24.
  26. The K parallel quantization and encoding units and the K parallel conversion units are configured to operate in a pipeline architecture;
    The K parallel quantizing and encoding units are configured so that the K predecessors of the group of K frames while the K parallel transform units are transforming the frames of the group of K frames. Quantize and encode K preceding sets of frequency coefficients corresponding to a frame;
    26. An audio encoder according to any one of claims 20 to 25.
  27.   27. An audio encoder according to any one of the preceding claims, wherein the K parallel conversion units are implemented on at least K different processing units.
  28.   28. An audio encoder according to any one of the preceding claims, wherein the audio encoder is configured to perform advanced audio coding, referred to as AAC.
  29. A frame-based audio encoder configured to encode in parallel K frames of an audio signal on at least K different processing units, where K> 1, the audio encoder is:
    • K parallel conversion units each configured to convert each of the K frames to each of K sets of frequency coefficients;
    -K parallel signal attack detection units each configured to classify each of the K frames based on the presence or absence of an acoustic attack in the respective frames of the K frames;
    K parallel quantization and encoding units each configured to quantize and entropy encode the K sets of frequency coefficients, taking into account the respective number of assigned bits;
    An audio encoder comprising at least one of the following:
  30. A frame-based audio encoder configured to encode in parallel K frames of an audio signal on at least K different processing units, where K> 1, the audio encoder is:
    A transform unit configured to transform the K frames into K corresponding sets of frequency coefficients;
    K parallel quantizing and encoding units, each of the K parallel quantizing and encoding units taking into account the assigned number of bits and said frequency coefficient A quantization and encoding unit configured to quantize and entropy encode each of the K sets;
    Each of the K parallel quantization and encoding units, respectively, based on the previously consumed number of bits used to encode the frame of the audio signal preceding the K frames; A bit allocation unit configured to allocate a number of bits;
    Audio encoder.
  31. A frame-based audio encoder configured to encode in parallel K frames of an audio signal on at least K different processing units, where K> 1, wherein the audio encoder is
    • K parallel signal attack detection units, wherein each signal attack detection unit classifies each of the K frames based on the presence or absence of an acoustic attack in each of the K frames. A configured signal attack detection unit;
    K = 1,..., K is configured to determine the frame type of each frame k out of the K frames based on the classification of the frame k and the frame type of the frame k−1. A frame type detection unit;
    K parallel conversion units, each of the K parallel conversion units configured to convert each of the K frames to each of K sets of frequency coefficients, and frame k A set k of frequency coefficients corresponding to a parallel conversion unit, which depends on the frame type of frame k,
    Audio encoder.
  32. A method for encoding an audio signal including a sequence of frames, comprising:
    Converting K frames of the audio signal into K sets corresponding to frequency coefficients in parallel, where K>1;
    Classifying the frames of the K frames in parallel based on the presence or absence of an acoustic attack in each of the K frames; and, taking into account the number of allocated bits, Quantizing and entropy encoding each set of the K sets of frequency coefficients in parallel;
    Including any one or more of
    Method.
  33. A method for encoding an audio signal comprising a sequence of frames, comprising:
    Transforming the K frames of the audio signal into K corresponding sets of frequency coefficients, where K>1;
    Quantizing and entropy encoding each of the K sets of frequency coefficients in parallel, taking into account the number of assigned bits;
    Assigning the respective number of bits based on a previously consumed number of bits used to encode the frame of the audio signal preceding the K frames;
    Method.
  34. A method for encoding an audio signal comprising a sequence of frames, comprising:
    Classifying each of the K frames of the audio signal as K> 1 based on the presence or absence of an acoustic attack in each of the K frames;
    Determining the frame type of each frame k out of the K frames as k = 1,..., K based on the classification of the frame k and the frame type of frame k−1;
    Converting each of the K frames in parallel to each of K sets of frequency coefficients, wherein the set of frequency coefficients corresponding to frame k depends on the frame type of frame k; and including,
    Method.
  35.   35. A software program for performing the steps of the method according to any one of claims 32 to 34 when executed on a processor, adapted for execution on a processor.
  36.   35. A storage medium having a software program for performing the steps of the method according to any one of claims 32 to 34 when executed on a computing device, adapted for execution on a processor .
  37.   35. A computer program product comprising executable instructions for performing the steps of the method according to any one of claims 32 to 34 when executed on a computer.
JP2014547840A 2011-12-21 2012-12-11 Audio encoder with parallel architecture Active JP5864776B2 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US201161578376P true 2011-12-21 2011-12-21
US61/578,376 2011-12-21
PCT/EP2012/075056 WO2013092292A1 (en) 2011-12-21 2012-12-11 Audio encoder with parallel architecture

Publications (2)

Publication Number Publication Date
JP2015505070A true JP2015505070A (en) 2015-02-16
JP5864776B2 JP5864776B2 (en) 2016-02-17

Family

ID=47469935

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2014547840A Active JP5864776B2 (en) 2011-12-21 2012-12-11 Audio encoder with parallel architecture

Country Status (5)

Country Link
US (1) US9548061B2 (en)
EP (1) EP2795617B1 (en)
JP (1) JP5864776B2 (en)
CN (1) CN104011794B (en)
WO (1) WO2013092292A1 (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3095117B1 (en) * 2014-01-13 2018-08-22 Nokia Technologies Oy Multi-channel audio signal classifier
US10573324B2 (en) * 2016-02-24 2020-02-25 Dolby International Ab Method and system for bit reservoir control in case of varying metadata
CN109688990A (en) * 2016-09-06 2019-04-26 新感知公司 For providing a user the method and system of attached sensory information
US10181331B2 (en) 2017-02-16 2019-01-15 Neosensory, Inc. Method and system for transforming language inputs into haptic outputs
US10744058B2 (en) 2017-04-20 2020-08-18 Neosensory, Inc. Method and system for providing information to a user

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000500247A (en) * 1996-07-11 2000-01-11 ドルビー ラボラトリィーズ インコーポレーテド Audible signal coding and decoding method
JP2001242894A (en) * 1999-12-24 2001-09-07 Matsushita Electric Ind Co Ltd Signal processing apparatus, signal processing method and portable equipment
JP2002014696A (en) * 2000-06-29 2002-01-18 Matsushita Electric Ind Co Ltd Device and method for control
JP2004069773A (en) * 2002-08-01 2004-03-04 Yamaha Corp Device and method for encoding audio data
JP2007212895A (en) * 2006-02-10 2007-08-23 Matsushita Electric Ind Co Ltd Apparatus and method for coding audio signal, and program
JP2008539462A (en) * 2005-04-28 2008-11-13 ドルビー・ラボラトリーズ・ライセンシング・コーポレーションDolby Laboratories Licensing Corporation Method and system for operating audio encoders in parallel

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5848391A (en) 1996-07-11 1998-12-08 Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V. Method subband of coding and decoding audio signals using variable length windows
IL129345A (en) 1999-04-06 2004-05-12 Broadcom Corp Video encoding and video/audio/data multiplexing device
US6567781B1 (en) 1999-12-30 2003-05-20 Quikcat.Com, Inc. Method and apparatus for compressing audio data using a dynamical system having a multi-state dynamical rule set and associated transform basis function
AU3840201A (en) 2000-02-18 2001-08-27 Intelligent Pixels Inc Very low-power parallel video processor pixel circuit
JP3826807B2 (en) 2002-02-13 2006-09-27 日本電気株式会社 Positioning system in mobile communication network
US7363230B2 (en) * 2002-08-01 2008-04-22 Yamaha Corporation Audio data processing apparatus and audio data distributing apparatus
JP2004309921A (en) * 2003-04-09 2004-11-04 Sony Corp Device, method, and program for encoding
AT537536T (en) * 2004-10-26 2011-12-15 Panasonic Corp Language coding device and language coding method
US8532984B2 (en) * 2006-07-31 2013-09-10 Qualcomm Incorporated Systems, methods, and apparatus for wideband encoding and decoding of active frames
US8374857B2 (en) * 2006-08-08 2013-02-12 Stmicroelectronics Asia Pacific Pte, Ltd. Estimating rate controlling parameters in perceptual audio encoders
US7676647B2 (en) 2006-08-18 2010-03-09 Qualcomm Incorporated System and method of processing data using scalar/vector instructions
US8515052B2 (en) 2007-12-17 2013-08-20 Wai Wu Parallel signal processing system and method
US9678775B1 (en) 2008-04-09 2017-06-13 Nvidia Corporation Allocating memory for local variables of a multi-threaded program for execution in a single-threaded environment
KR101400535B1 (en) * 2008-07-11 2014-05-28 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베. Providing a Time Warp Activation Signal and Encoding an Audio Signal Therewith
CN101350199A (en) 2008-07-29 2009-01-21 北京中星微电子有限公司 Audio encoder and audio encoding method
US9342486B2 (en) 2008-10-03 2016-05-17 Microsoft Technology Licensing, Llc Fast computation of general fourier transforms on graphics processing units
KR101797033B1 (en) * 2008-12-05 2017-11-14 삼성전자주식회사 Method and apparatus for encoding/decoding speech signal using coding mode
US9165394B2 (en) 2009-10-13 2015-10-20 Nvidia Corporation Method and system for supporting GPU audio output on graphics processing unit

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000500247A (en) * 1996-07-11 2000-01-11 ドルビー ラボラトリィーズ インコーポレーテド Audible signal coding and decoding method
JP2001242894A (en) * 1999-12-24 2001-09-07 Matsushita Electric Ind Co Ltd Signal processing apparatus, signal processing method and portable equipment
JP2002014696A (en) * 2000-06-29 2002-01-18 Matsushita Electric Ind Co Ltd Device and method for control
JP2004069773A (en) * 2002-08-01 2004-03-04 Yamaha Corp Device and method for encoding audio data
JP2008539462A (en) * 2005-04-28 2008-11-13 ドルビー・ラボラトリーズ・ライセンシング・コーポレーションDolby Laboratories Licensing Corporation Method and system for operating audio encoders in parallel
JP2007212895A (en) * 2006-02-10 2007-08-23 Matsushita Electric Ind Co Ltd Apparatus and method for coding audio signal, and program

Also Published As

Publication number Publication date
US20150025895A1 (en) 2015-01-22
CN104011794B (en) 2016-06-08
EP2795617B1 (en) 2016-08-10
JP5864776B2 (en) 2016-02-17
WO2013092292A1 (en) 2013-06-27
US9548061B2 (en) 2017-01-17
CN104011794A (en) 2014-08-27
EP2795617A1 (en) 2014-10-29

Similar Documents

Publication Publication Date Title
US9959879B2 (en) Context-based arithmetic encoding apparatus and method and context-based arithmetic decoding apparatus and method
JP6673957B2 (en) High frequency encoding / decoding method and apparatus for bandwidth extension
US8898068B2 (en) Audio encoder, audio decoder, method for encoding and audio information, method for decoding an audio information and computer program using a modification of a number representation of a numeric previous context value
JP5658307B2 (en) Frequency segmentation to obtain bands for efficient coding of digital media.
KR101975066B1 (en) Signal processing device and method, and computer readable recording medium
JP2019080347A (en) Method for parametric multi-channel encoding
RU2455709C2 (en) Audio signal processing method and device
KR101327194B1 (en) Audio decoder and decoding method using efficient downmixing
KR101953648B1 (en) Time domain level adjustment for audio signal decoding or encoding
KR101330362B1 (en) Modification of codewords in dictionary used for efficient coding of digital media spectral data
EP2077550B1 (en) Audio encoder and decoder
KR101019678B1 (en) Low bit-rate audio coding
US8645127B2 (en) Efficient coding of digital media spectral data using wide-sense perceptual similarity
KR100904542B1 (en) Apparatus and method for generating multi-channel synthesizer control signal and apparatus and method for multi-channel synthesizing
US7062445B2 (en) Quantization loop with heuristic approach
CA2612537C (en) Selectively using multiple entropy models in adaptive coding and decoding
ES2312852T3 (en) Reduction of the transmission cost of the scale factor for mpeg-2 aac through a reticle.
JP4918841B2 (en) Encoding system
JP5298245B2 (en) SBR bitstream parameter downmix
US7693709B2 (en) Reordering coefficients for waveform coding or decoding
US7684981B2 (en) Prediction of spectral coefficients in waveform coding and decoding
KR100547113B1 (en) Audio data encoding apparatus and method
ES2718400T3 (en) Audio signal encoding and decoding method and device
CN1957398B (en) Methods and devices for low-frequency emphasis during audio compression based on acelp/tcx
AU2011226140B2 (en) Audio signal decoder, audio signal encoder, methods and computer program using a sampling rate dependent time-warp contour encoding

Legal Events

Date Code Title Description
A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20150724

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20150804

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20151028

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20151124

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20151224

R150 Certificate of patent or registration of utility model

Ref document number: 5864776

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250