Classifications

• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
  • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
  • G10L19/032—Quantisation or dequantisation of spectral components
  • G10L19/038—Vector quantisation, e.g. TwinVQ audio
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
  • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
  • G10L19/16—Vocoder architecture
  • G10L19/18—Vocoders using multiple modes
• • H—ELECTRICITY
  • H03—ELECTRONIC CIRCUITRY
  • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
  • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
  • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
  • H03M7/3082—Vector coding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
  • H04N19/94—Vector quantisation
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
  • G10L2019/0001—Codebooks
  • G10L2019/0013—Codebook search algorithms

Definitions

• the suggested technology relates generally to vector quantization (VQ), and especially to the accuracy and computational complexity of the same.
• SQs scalar quantizers
• VQs vector quantizers
• VQs are superior to the SQs, but at a cost of increased computational complexity and memory storage.
• the target vector to quantize be M dimensional:
• the optimal index k opt is transmitted to the decoder, and the corresponding codevector is extracted from the CB (identical CBs are available both at the encoder and the decoder) and is used to reconstruct the target vector.
• the CB is typically trained offline and captures the statistical properties of the data.
• the simple squared error (cf. equation (2)) is modified with weights, such that: d c k ) w (m) ⁇ (s(m) - c k (m)f (3)
• the accuracy or quality of the reconstructed target signal is dependent on the size K of the codebook; where a large CB leads to higher accuracy, and thus better quality, than a smaller CB.
• the main computational complexity also is related to the size of the CB, assuming that the vector dimensionality is fixed by the application.
• audio transmission systems are built under the constraints of limited computational complexity. That is, the worst case complexity should not exceed a certain pre-defined L MAX .
• the computational complexity of an audio codec is typically measured by means of Weighted Millions of Operations per Second (WMOPS), but as we consider a VQ module, the complexity is directly related to the size of the search space (size of the CB).
• WOPS Weighted Millions of Operations per Second
• a VQ is typically the most complex module in a codec, and further, the CB search (number of comparisons with CB vectors) is what makes the VQ so complex.
• the technology described herein is applicable e.g. for audio and video compression/transmission systems that perform lossy compression on the input stream, and could be described in a number of different aspects.
• the herein described technology involves a codebook, which is divided into classes and sorted, and the classification of an input target vector s to be quantized into one of said classes.
• the herein described technology enables that the class of codevectors, in the codebook, which comprises the most probable set of candidate codevectors in regard of the input vector s is searched first of the classes in the codebook. Thus, the best match codevector for the input vector s may be found early in a search, and the computational complexity may be reduced.
• a method in a Vector Quantizer comprises comparing an input target vector s with a plurality of centroids, i.e. reference vectors, where each centroid represents a respective class of codevectors in a codebook.
• the method further comprises determining a starting point, in a codebook, for a search related to the input target vector in the codebook, where the starting point is determined based on the result of the comparison.
• a Vector Quantizer comprising functional units adapted to execute the method according to the first aspect.
• the Vector Quantizer comprises a comparing unit adapted to compare an input target vector s with a plurality of centroids, each centroid representing a respective class of codevectors in a codebook.
• the Vector Quantizer further comprises a determining unit adapted to determine a starting point for a search in the codebook, based on the result of the comparison.
• the codevectors in the codebook are sorted according to a distortion measure reflecting the distance between each codevector and the centroids.
• the Vector Quantizer enables that the class of codevectors comprising the most probable candidate codevectors in regard of the input vector s are searched first of the classes in the codebook.
• a codec which comprises a Vector Quantizer according to the second aspect.
• a mobile terminal which comprises a Vector Quantizer according the second aspect above.
• a codebook for vector quantization is provided, which codebook is arranged such that the codevectors of the codebook are divided into a plurality of classes, each represented by a centroid, and where the codevectors further are sorted according to a distortion measure reflecting their distance to the centroids of the plurality of classes.
• the codevectors may be sorted according to e.g. descending or increasing distortion value.
• a use of a codebook for vector quantization is provided, which codebook is arranged such that the codevectors of the codebook are divided into a plurality of classes, each represented by a centroid, and where the codevectors further are sorted according to a distortion measure reflecting their distance to the centroids of the plurality of classes.
• the codevectors may be sorted according to e.g. descending or increasing distortion value.
• a computer program comprising computer readable code, which when run in a processing unit, causes a Vector Quantizer to perform a method according to the first aspect.
• a computer program product comprising a computer readable medium and a computer program according to the sixth aspect, stored on the computer readable medium.
• a search could be performed in the codebook in the determined search space, starting at the determined starting point, where the search delivers a best match to the input target vector s.
• “best match” is here meant the closest match, shortest distance, in regard of a distance measure between the input target vector and a candidate vector in the codebook, i.e. a best match is a code vector which has the shortest distance to the input target vector, according to the distance measure.
• a fifth aspect may be a codec comprising a vector quantizer according to the second aspect.
• a sixth aspect may be a mobile terminal comprising a vector quantizer according to the second aspect.
• Figure 1 a and 1 b shows the structure of an ordered CB according to a solution described herein.
• the search starts from point 0 or point 1 towards the other end of the CB.
• Figure 2 shows an exemplifying CB structure exploiting symmetry. Only
• codevectors of C 0 and C l are stored in a ROM memory.
• Figure 3 illustrates an exemplifying functional unit for determining an optimal class for an input vector s by comparing the input vector s to each of a number of centroids ] I C n 0 C 1 , C l , rule h .P C 0,flip ) ⁇ ,' each associated with a class in a codebook.
• Figure 4 illustrates a functional unit for determining a size of a search region in a codebook based on a number of spectral peaks detected in a current frame, and possibly on the codec's bitrate.
• Figure 5 is a table illustrating that a search region increases as the number of peaks per-frame decrease.
• the search is performed only in 7-bit CB (defined to be the minimum search space in this example), but with 8 peaks or less, the search is performed in 8-bit CB (maximum search space), since this can be "afforded" under the maximum complexity constraint.
• Figures 6a-d show examples of different search regions.
• Figure 7 illustrates that the allowed complexity may be signaled to the system from an external entity.
• the parameter could be based on e.g. CPU load, or battery status.
• Figure 8 is a flow chart illustrating the actions in a procedure for creating a codebook, CB, to be used in the suggested technology.
• Figure 10 is a block diagram illustrating a vector quantizer, according to an example of the herein suggested technology.
• Figure 1 1 is a block diagram illustrating a codec comprising a vector quantizer, according to an example of the herein suggested technology.
• the solution described herein relates to dynamically adapting the search space of a VQ, such that, for any number of target (input) vectors (per block or time interval), a high accuracy, and thus quality, quantization is achieved within a given complexity constraint. That is, the requirements of computational complexity (cf. L max ) are not to be violated.
• This is achieved by that the search is performed in a special classified and ordered CB.
• the starting point in the search space for each target vector is based on a classification procedure, and the size of the search space is increased or reduced, based on the number of target vectors.
• the VQ algorithm described herein may be regarded as a "tool" for data compression, independent of what the data is, i.e. the data could be e.g. video and/or audio.
• the VQ is described in the context of an audio codec, but the concept described herein is not limited to audio codecs. For example, it could also be implemented in video codecs.
• the algorithm described herein is based on a specially designed CB. Some variants of such a codebook will be described in more detail below. First a basic case will be described, and further below a more advanced scheme will be discussed.
• the codevectors of the CB may be arranged according to the solution described herein in an offline mode.
• centroid vectors are vectors of the same dimension as the vectors from the data set, but they do not belong to the data set. That is, the centroid vectors are outside the CB and do not need to coincide with some existing codevectors.
• centroid is herein generally meant a reference vector representing a class of vectors.
• the codevectors which are equally distanced from the centroids (Co and C-i) of the two classes produce a distortion measure d which is close to zero.
• the codevectors are ordered e.g. by increasing distortion measure, as illustrated in figure 1 a and b.
• Each input target vector is compared with the two centroids (the respective centroid of the two classes) and is, depending on the result, assigned to, i.e.
• the starting point of the search is either selected to be the most upper point (figure 1 a) or most left (figure 1 b) point 0 (when the target vector belongs to class Co) or the lowest point (figure 1 a) or most right (figure 1 b) point 1 (when the target vector belongs to class C-i).
• the size of the search space should be made dependent on the number of input target vectors N per block or time segment/interval. If we re-define the search space K not to be the size of the entire CB, but to be variable, the concept behind the adaption described herein may be defined in equation (5)
• an input target vector may reflect a spectral peak (region) of a segment of the audio signal being processed.
• the number of spectral peaks in the signal spectrum of a time segment, e.g. 30 ms, of an audio signal depends on the spectral properties of the audio signal in that time segment.
• the number of spectral peaks may vary between different time segments and between different audio signals.
• the number of input vectors, per block or time segment, to the VQ will vary.
• the maximum number of input vectors, corresponding to a number of spectral peaks in a time segment of an audio signal is 17.
• this number is only an example, and should not be interpreted as limiting the solution in general.
• the search space K may be increased, at most up to the size of the entire CB, which leads to higher accuracy and thus quality of the reconstructed vector.
• the accuracy of a vector quantizer may be measured as a squared error between an original signal and corresponding reconstructed data.
• the codebook of the VQ need not be designed for the worst case scenario (i.e. maximum number of input target vectors). Instead, it could be designed e.g. for a best case scenario, thus comprising more codevectors than could possibly be searched for the maximum number of input target vectors within the maximum complexity constraint LMAX-
• the maximum complexity requirement will be fulfilled by that the extent of the search, i.e. the size of search space, in the CB depends on the number of input target vectors. However, if this would be done "blindly", e.g.
• a set of target vectors s represent spectral peak regions in a transform- domain audio coding, e.g. , transform coefficients in the neighborhood of MDCT peaks.
• the number of target vectors varies over time, since the number of spectral peaks varies from one time-block to another.
• the codevectors of these classes are flipped versions of the codevectors of the right side of the CB.
• the search is performed on the codevectors of the left side, which are stored in memory, but with the elements of the codevectors flipped around the center, such that the codevectors c k jp are given by equation (6) c k,flip (6) where c k (m) are the vector elements of the corresponding class Cj in the stored
• CB i.e. Co or Ci>. That is, if the elements of a certain codevector in Co are ⁇ C01 C02 Co3 C04 ⁇ , the elements of a corresponding codevector in Co , ni P are ⁇ Co 4 C03 C02 C01 ⁇
• an input target vector is compared with four centroids and assigned to a class in order to determine a starting point for the search, i.e. the optimal class for the input target vector is determined by comparing the input vector S to each of the centroids
• FIG. 3 ( C 0 C l C l flip C Q flip ).
• a target vector S is input to a class assigning unit 302, which delivers a class indicator, Cj, as output.
• the centroids C l flip and C 0 flip are not stored in a table, but "created” by flipping the elements of the centroids C 0 and C l .
• the elements does not need to be literally flipped, instead a modified search operation may read the elements of CO and C1 in the reverse order when reading CO, flip and C1 , flip, i.e. both centroids and codebook vectors .
• the CB may be extended to comprising twice the number of codevectors, as compared to what is physically stored in the CB, which means saving of memory resources.
• This is possible due to that the symmetry of the peak regions is exploited, as described above. More specifically, the solution is based on the observation that the symmetry may be exploited by that a flipped valid codevector also is a valid codevector.
• bitrate information reflects changes in quality and complexity requirements, which may be taken into consideration in the vector quantization.
• the table in figure 5 illustrates how the search region is adapted to the number of peaks.
• the search region is indicated as the number of coefficients (codevectors) in the search per input vector.
• the numbers in the table in figure 5 are derived under the assumption that the CB from figure 2 comprises four 7-bit segments (four segments with 128 codevectors each) of which two are "normal” or “physical” and two are "flipped" or “virtual”).
• the input vector S belongs to class C l (position indicated with down arrow).
• the search space is then limited to sizes between the class C l only, and the joint space of the classes C 0 and C l .
• this is illustrated with three broken arrows indicating search spaces of different sizes.
• Figure 6b illustrates the case where an input vector belongs to class Co (position indicated with down arrow), in which case the search has a different starting point.
• the search can be performed in classes C l flip and/or C 0 flip if the input vector belongs to one of these classes. No searches are performed in the joint space of C l and C l flip .
• Exemplifying embodiment 2 Communication system with external control of maximum allowed complexity
• VQ having a complexity which is dynamically adjusted to the number of target vectors N
• the complexity limit is not pre-determined, but may vary e.g. based on some criterion, and be signaled to the VQ and/or to the entity in which the VQ is applied.
• FIG 7 shows a functional unit 702 taking an indicator or the number of peaks/vectors as input, and further taking a complexity constraint LMAX as input.
• the functional unit 702 delivers an indicator of a search space/a region of the CB, cf. the broken arrows in figures 6a-d.
• VQ algorithm with adjustable complexity gives the optimal balance between accuracy of quantization (i.e. quality) and maintaining computational complexity below a pre-defined threshold.
• the CB is divided into classes in an action 802, e.g. by use of a so-called K- means algorithm, as previously described.
• the codevectors of the CB are then sorted in the CB based on a distortion measure, e.g. as the one described in equation (4).
• the distortion measure for each codevector depends on a relation between the codevector and centroids representing each class of the CB, as previously described.
• This organization of the CB enables adaptation of the search space, and thus of the search complexity in VQ, at a highly preserved VQ quality (e.g. quality of the reconstructed target vectors).
• VQ vector quantizer
• a number N of input target vectors are received by the VQ, as previously described. Below, the actions associated with one of the input target vectors will be described, for reasons of simplicity.
• An input target vector s is compared with a number of codevectors each representing a CB class (cf. classes Co and C-i, etc. described earlier), preferably the centroid of each class.
• the comparison is illustrated as action 902 in figure 9a- c.
• Action 902 could alternatively be regarded as integrated with an action 904, illustrated in figure 9c.
• the input target vector s is assigned one of the classes, or sections, of the CB, in an action 904.
• the input target vector s is assigned, or concluded to belong to, the class to which it has the shortest distance, i.e. to which it is most similar, according to some distance measure (error measure).
• the starting point of the search in the CB is determined in an action 906, based on the class assignment or distance measure.
• a search may be performed in the codebook in an action 910.
• the search is initiated in the selected starting point, and is performed over a search space, which may be of a determined size, comprising one or more classes, or parts thereof. Due to the advantageously designed and organized CB, the probability of that the best match, of all candidate codevectors within the whole CB, for the input target vector s will be found within the search space is very high, even when the search space is limited to e.g. half the CB. In a case where the search space would comprise the entire codebook, the best match codevector would be found early in the search when starting the search at the determined starting point.
• the index of the best match codevector is provided, as a result from the VQ, in an action 912, e.g. for use in an audio decoder.
• VQ an exemplifying VQ arrangement suitable for use in a transform encoder/codec will be described with reference to figure 10.
• the transform codec could be e.g. an MDCT codec.
• the VQ is adapted to perform the actions of the procedure described above.
• the VQ 1001 is illustrated as to communicate with other entities (e.g. audio codec) via a communication unit 1002.
• the VQ may further comprise other functional units 1016, such as e.g. functional units providing regular functions, and may further comprise one or more storage units 1014.
• the VQ 1001 could be implemented e.g. by one or more of: a processor or a micro processor and adequate software with suitable storage therefore, a Programmable Logic Device (PLD) or other electronic component(s) and/or circuits.
• PLD Programmable Logic Device
• the VQ may comprise a comparison unit 1004, which is adapted to compare an input target vector s with vectors representing each class of the CB, e.g. the centroid vector of each class. Further, the VQ may comprise an assigning unit 1006, which is adapted to assign a class to the input target vector s (or assign the vector s to a class), i.e. conclude to which class the vector belongs, based on the comparison. Further, the VQ may comprise a determining unit 1008, adapted to determine an adequate starting point for a search in the CB, based on the class assigned to the vector s. The determining unit may further be adapted to determine the size of a search space in the CB, based e.g. on a number of received input target vectors and a computational complexity constraint.
• the VQ may comprise a search unit 1010, which is adapted to perform a search in the CB, starting at the determined starting point and searching the determined search space. The search should result in one or more CB indices pointing to the codevector which best matches the input target vector s.
• the VQ may further comprise a providing unit 1012, which is adapted to provide said index or indices to another entity, e.g. to (or for use by) a transform codec.
• FIG. 12 schematically shows an embodiment of an arrangement 1200 suitable for use in e.g. a transform audio decoder, which also can be an alternative way of disclosing an embodiment of the VQ illustrated in figure 5.
• a processing unit 1206 e.g. with a DSP (Digital Signal Processor).
• the processing unit 1206 can be a single unit or a plurality of units to perform different steps of procedures described herein.
• the arrangement 1200 may also comprise the input unit 1202 for receiving signals, such as input target vectors and indicators of e.g. bitrate and/or complexity constraint; and further the output unit 1204 for output signal(s), such as the CB indices for the best match codevectors.
• the input unit 1202 and the output unit 1204 may be arranged as one in the hardware of the arrangement.
• the arrangement 1200 comprises at least one computer program product 1208 in the form of a non-volatile memory, e.g. an EEPROM, a flash memory and a hard drive.
• the computer program product 1208 comprises a computer program 1210, which comprises code means, which when run in the processing unit 1206 in the arrangement 1200 causes the arrangement to perform the actions of a procedure described earlier in conjunction with figures 9a-c.
• the code means in the computer program 1210 of the arrangement 1200 may comprise a comparison module 1210a for comparing an input target vector with class centroids of a CB.
• the computer program may comprise an assigning module 1210b for assigning a class to the input target vector.
• the computer program 1210 may further comprise a determining unit 1210c for determining a starting point for a search in the CB; and further for determining a search space or region based on input parameters.
• the computer program 1210 may further comprise a search unit 121 Od for searching the CB according to the above.
• the computer program 1210 may comprise a providing module 1210e, for providing indices, which are output from the search to other entities.
• the computer program 1210 is in the form of computer program code structured in computer program modules.
• the modules 1210a-e may essentially perform the actions of the flow illustrated in any of figures 9a-c to emulate at least part of the VQ 1001 illustrated in figure 10. In other words, when the different modules 1210a-d are run on the processing unit 1206, they correspond at least to the units 1004-1012 of figure 10.
• code means in the embodiment disclosed above in conjunction with figure 12 are implemented as computer program modules which when run on the processing unit causes the arrangement and/or transform audio encoder to perform steps described above in the conjunction with figures mentioned above, at least one of the code means may in alternative embodiments be implemented at least partly as hardware circuits.
• the functional blocks may include or encompass, without limitation, digital signal processor (DSP) hardware, reduced instruction set processor, hardware (e.g., digital or analog) circuitry including but not limited to application specific integrated circuit(s) (ASIC), and (where appropriate) state machines capable of performing such functions.
• DSP digital signal processor
• ASIC application specific integrated circuit

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Spectroscopy & Molecular Physics (AREA)
• Computational Linguistics (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Human Computer Interaction (AREA)
• Acoustics & Sound (AREA)
• Health & Medical Sciences (AREA)
• Theoretical Computer Science (AREA)
• Compression, Expansion, Code Conversion, And Decoders (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)

Priority Applications (16)

Applications Claiming Priority (2)

Related Child Applications (2)

Publications (1)

Family

ID=47631684

Family Applications (1)

Country Status (12)

Cited By (1)

Families Citing this family (4)

Family Cites Families (39)

• 2012
  • 2012-12-12 IN IN7726DEN2014 patent/IN2014DN07726A/en unknown
  • 2012-12-12 EP EP12821073.9A patent/EP2831757B1/en active Active
  • 2012-12-12 EP EP24213706.5A patent/EP4521350A1/en active Pending
  • 2012-12-12 CN CN201280072059.0A patent/CN104221287B/zh active Active
  • 2012-12-12 RU RU2017121373A patent/RU2726158C2/ru active
  • 2012-12-12 US US14/387,716 patent/US9401155B2/en active Active
  • 2012-12-12 RU RU2014143442A patent/RU2624586C2/ru active
  • 2012-12-12 ES ES19167463T patent/ES2960582T3/es active Active
  • 2012-12-12 WO PCT/SE2012/051381 patent/WO2013147667A1/en not_active Ceased
  • 2012-12-12 BR BR112014022848-5A patent/BR112014022848B1/pt active IP Right Grant
  • 2012-12-12 FI FIEP19167463.9T patent/FI3547261T3/fi active
  • 2012-12-12 PL PL12821073T patent/PL2831757T3/pl unknown
  • 2012-12-12 TR TR2019/11121T patent/TR201911121T4/tr unknown
  • 2012-12-12 EP EP23186309.3A patent/EP4274235B1/en active Active
  • 2012-12-12 CN CN201710451005.XA patent/CN107170459B/zh active Active
  • 2012-12-12 ES ES23186309T patent/ES3000058T3/es active Active
  • 2012-12-12 DK DK12821073.9T patent/DK2831757T3/da active
  • 2012-12-12 EP EP19167463.9A patent/EP3547261B1/en active Active
  • 2012-12-12 ES ES12821073T patent/ES2745143T3/es active Active
• 2016
  • 2016-06-21 US US15/187,943 patent/US9842601B2/en active Active
• 2017
  • 2017-11-07 US US15/805,768 patent/US10468044B2/en active Active
• 2019
  • 2019-08-23 US US16/549,270 patent/US11017786B2/en active Active
• 2020
  • 2020-05-12 RU RU2020115683A patent/RU2020115683A/ru unknown
• 2021
  • 2021-04-21 US US17/236,563 patent/US11741977B2/en active Active

Non-Patent Citations (5)

Also Published As

Similar Documents

Legal Events