WO2021209470A1 - Decoder for decoding weight parameters of a neural network, encoder, methods and encoded representation using probability estimation parameters - Google Patents

Decoder for decoding weight parameters of a neural network, encoder, methods and encoded representation using probability estimation parameters Download PDF

Info

Publication number
WO2021209470A1
WO2021209470A1 PCT/EP2021/059594 EP2021059594W WO2021209470A1 WO 2021209470 A1 WO2021209470 A1 WO 2021209470A1 EP 2021059594 W EP2021059594 W EP 2021059594W WO 2021209470 A1 WO2021209470 A1 WO 2021209470A1
Authority
WO
WIPO (PCT)
Prior art keywords
parameters
neural network
probability estimation
parameter values
useable
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/EP2021/059594
Other languages
English (en)
French (fr)
Inventor
Paul Haase
Karsten Müller
Heiner Kirchhoffer
Heiko Schwarz
Detlev Marpe
Thomas Wiegand
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fraunhofer Gesellschaft zur Foerderung der Angewandten Forschung eV
Original Assignee
Fraunhofer Gesellschaft zur Foerderung der Angewandten Forschung eV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fraunhofer Gesellschaft zur Foerderung der Angewandten Forschung eV filed Critical Fraunhofer Gesellschaft zur Foerderung der Angewandten Forschung eV
Priority to KR1020227039550A priority Critical patent/KR20230005868A/ko
Priority to EP21717116.4A priority patent/EP4136756A1/en
Priority to CN202180042614.4A priority patent/CN115699585A/zh
Priority to JP2022562944A priority patent/JP7695954B2/ja
Publication of WO2021209470A1 publication Critical patent/WO2021209470A1/en
Priority to US18/046,227 priority patent/US20230141029A1/en
Anticipated expiration legal-status Critical
Priority to JP2025030998A priority patent/JP7817477B2/ja
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/70—Type of the data to be coded, other than image and sound
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G06F18/2193—Validation; Performance evaluation; Active pattern learning techniques based on specific statistical tests
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/40—Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code
    • H03M7/4006—Conversion to or from arithmetic code
    • H03M7/4012—Binary arithmetic codes
    • H03M7/4018—Context adapative binary arithmetic codes [CABAC]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Definitions

  • Decoder for decoding weight parameters of a neural network, Encoder, Methods and encoded representation using probability estimation parameters
  • Embodiments according to the invention comprise an encoder for encoding weight parameters of a neural network, a decoder, methods and an encoded representation using probability estimation parameters.
  • neural networks constitute, for example, a chain of affine transformations followed by an element-wise non-linear function. They may, for example, be represented as a directed acyclic graph, as depicted in Figure 1.
  • Figure 1 shows an example for a graph representation of a feed forward neural network.
  • this 2-layered neural network is a non-linear function which maps a 4-dimensional input vector into the real line.
  • each node entails a particular value, which is forward propagated into the next node, for example, by multiplication with the respective weight value of the edge. All incoming values are then, for example, simply aggregated.
  • L i (X) N i (B i (X)) and where, for example, B i is the affine transformation of layer i and where, for example, N i is some non-linear function of layer i.
  • B i is the affine transformation of layer i and where, for example, N i is some non-linear function of layer i.
  • a simple example for B i is a matrix multiplication of weight parameters (edge weights) w i associated with layer i with the input X:
  • the operator * shall denote matrix multiplication.
  • convolutional layers may also be used by casting them as matrix-matrix products as described, for example, in “cuDNN: Efficient Primitives for Deep Learning” (Sharan Chetlur, et al.; arXiv: 1410.0759, 2014).
  • inference the procedure of calculating the output from a given input.
  • intermediate results as hidden layers or hidden activation values, which constitute a linear transformation + element-wise nonlinearity, e.g. such as the calculation of the first dot product + non-linearity above.
  • bias- and batch- norm operation for example, as follows:
  • Equation 1 where b is denoted bias and where ⁇ , ⁇ 2 , ⁇ , and ⁇ are denoted batch norm parameters.
  • W is a weight matrix, for example, with dimensions n x k and X is the input matrix, for example, with dimensions k x m.
  • Bias b and batch norm parameters ⁇ , ⁇ 2 , ⁇ , and ⁇ are, for example, transposed vectors, for example, of length n.
  • Operator * denotes a matrix multiplication. Note that, for example, all other operations (summation, multiplication, division) on a matrix with a vector are element-wise operations on the columns of the matrix. For example, X ⁇ ⁇ means that each column of X is multiplied element-wise with ⁇ .
  • ⁇ is, for example, a small scalar number (like e.g. 0.001 ) required (or useful) to avoid divisions by 0. However, it may also be 0.
  • Equation 1 refers to a batch-norm layer.
  • ⁇ and all vector elements of ⁇ and ⁇ are set to zero and all elements of ⁇ and ⁇ 2 are set to 1, a layer without batch norm (bias only) is addressd.
  • neural networks contain, for example, millions of parameters, and may thus, for example, require hundreds of MByte for their representation. Consequently, they require high computational resources in order to be executed since their inference procedure involves computations of many dot product operations, for example, between large matrices. Hence, it is of high importance to reduce the complexity of performing these dot products.
  • Embodiments according to the invention comprise a decoder for decoding weight parameters of a neural network, wherein the decoder is configured to obtain a plurality of neural network parameters, e.g., at least one of entries w i of matrix W, b, ⁇ , ⁇ 2 , ⁇ , ⁇ , and/or ⁇ , of the neural network on the basis of an encoded bitstream.
  • a plurality of neural network parameters e.g., at least one of entries w i of matrix W, b, ⁇ , ⁇ 2 , ⁇ , ⁇ , and/or ⁇
  • the decoder is configured to decode the neural network parameters, e.g., entries w i of matrix W, or b, or ⁇ , or ⁇ 2 , or ⁇ , or ⁇ ,or ⁇ ,of the neural network, e.g., a quantized version of the neural network parameters, using a context-dependent arithmetic decoding, e.g., using a context-adaptive binary arithmetic decoding (CABAC).
  • CABAC context-adaptive binary arithmetic decoding
  • probabilities of bin values may be determined for different contexts, wherein, for example, each bin is associated with a context.
  • the decoder is configured to obtain a probability estimate, e.g.
  • P(t) or p k lake which may, for example, be associated with a context, for a, e.g. arithmetic, decoding of a bin of a number representation of a neural network parameter, e.g. on the basis of one or more previously decoded neural network parameters or bins thereof, using one or more probability estimation parameters, e.g., probability estimator parameters, e.g. N, .
  • the decoder is configured to use different probability estimation parameter values for a decoding of different neural network parameters and/or to use different probability estimation parameter values for a decoding of bins associated with different context models, e.g. C k .
  • FIG. 1 For embodiments according to the invention, comprise a decoder for decoding weight parameters of a neural network, wherein the decoder is configured to obtain a plurality of neural network parameters, e.g., at least one of entries w i of matrix W, b, ⁇ , ⁇ 2 , ⁇ , ⁇ , and/or ⁇ , of the neural network on the basis of an encoded bitstream.
  • a decoder for decoding weight parameters of a neural network
  • the decoder is configured to obtain a plurality of neural network parameters, e.g., at least one of entries w i of matrix W, b, ⁇ , ⁇ 2 , ⁇ , ⁇ , and/or ⁇ , of the neural network on the basis of an encoded bitstream.
  • the decoder is configured to decode the neural network parameters, e.g., entries W i of matrix W, or b, or ⁇ , or ⁇ 2 , or ⁇ , or Y, or ⁇ of the neural network, e.g., a quantized version of the neural network parameters, using a context-dependent arithmetic decoding, e.g., using a context-adaptive binary arithmetic decoding (CABAC).
  • CABAC context-adaptive binary arithmetic decoding
  • probabilities of bin values may be determined for different contexts, wherein, for example, each bin is associated with a context.
  • the decoder is configured to obtain a probability estimate, e.g.
  • P(t) or p k which may, for example, be associated with a context, for a, e.g. arithmetic, decoding of a bin of a number representation of a neural network parameter, e.g. on the basis of one or more previously decoded neural network parameters or bins thereof, using one or more probability estimation parameters, e.g., probability estimator parameters, e.g. N, .
  • the decoder is configured to use different probability estimation parameter values for a decoding of neural network parameters associated with different layers of the neural network.
  • Embodiments according to the invention comprise an encoder for encoding weight parameters of a neural network, wherein the encoder is configured to obtain a plurality of neural network parameters, e.g., at least one of entries w i of matrix W, b, ⁇ , ⁇ 2 , ⁇ , ⁇ , and/or ⁇ , of the neural network.
  • a plurality of neural network parameters e.g., at least one of entries w i of matrix W, b, ⁇ , ⁇ 2 , ⁇ , ⁇ , and/or ⁇ , of the neural network.
  • the encoder is configured to encode the neural network parameters, e.g., entries w i of matrix W, or b, or ⁇ , or ⁇ 2 , or ⁇ , or ⁇ , or ⁇ , of the neural network, e.g., a quantized version thereof, using a context-dependent arithmetic coding, e.g., using a context- adaptive binary arithmetic coding (CABAC).
  • a context-dependent arithmetic coding e.g., using a context- adaptive binary arithmetic coding (CABAC).
  • CABAC context- adaptive binary arithmetic coding
  • probabilities of bin values may be determined for different contexts, wherein, for example, each bin is associated with a context.
  • the encoder is configured to obtain a probability estimate, e.g.
  • P(t) or p k which may, for example, be associated with a context, for an, optionally arithmetic, encoding of a bin of a number representation of a neural network parameter, e.g. on the basis of one or more previously encoded neural network parameters or bins thereof, using one or more probability estimation parameters, e.g., probability estimator parameters, e.g. N , A , .
  • the encoder is configured to use different probability estimation parameter values for an encoding of different neural network parameters and/or to use different probability estimation parameter values for an encoding of bins associated with different context models, e.g. C k .
  • the encoder is configured to encode the neural network parameters, e.g., entries w i of matrix W, or b, or ⁇ , or ⁇ 2 , or ⁇ , or ⁇ , or ⁇ ,of the neural network, e.g., a quantized version thereof, using a context-dependent arithmetic coding, e.g., using a context-adaptive binary arithmetic coding (CABAC).
  • a context-dependent arithmetic coding e.g., using a context-adaptive binary arithmetic coding (CABAC).
  • CABAC context-adaptive binary arithmetic coding
  • probabilities of bin values may be determined for different contexts, wherein, for example, each bin is associated with a context.
  • the encoder is configured to obtain a probability estimate, e.g.
  • P(t) or p kango which may, for example, be associated with a context, for an, optionally arithmetic, encoding of a bin of a number representation of a neural network parameter, e.g. on the basis of one or more previously encoded neural network parameters or bins thereof, using one or more probability estimation parameters, e.g., probability estimator parameters, e.g. N ,
  • the encoder is configured to use different probability estimation parameter values for an encoding of neural network parameters associated with different layers of the neural network.
  • Embodiments according to the invention comprise a method for decoding weight parameters of a neural network, wherein the method comprises obtaining a plurality of neural network parameters, e.g., at least one of entries w i of matrix W, b, ⁇ , ⁇ 2 , ⁇ , ⁇ , and/or ⁇ , of the neural network on the basis of an encoded bitstream, wherein the method comprises decoding the neural network parameters, e.g., entries w i of matrix W, or b, or ⁇ , or ⁇ 2 , or ⁇ , or ⁇ , or ⁇ , of the neural network, e.g., a quantized version of the neural network parameters, using a context- dependent arithmetic decoding, e.g., using a context-adaptive binary arithmetic decoding (CABAC).
  • CABAC context-adaptive binary arithmetic decoding
  • probabilities of bin values are determined for different contexts, wherein, for example, each bin may be associated with a context.
  • the method comprises obtaining a probability estimate, e.g. P(t) or p k , which may, for example, be associated with a context, for an, e.g. arithmetic, decoding of a bin of a number representation of a neural network parameter, e.g. on the basis of one or more previously decoded neural network parameters or bins thereof, using one or more probability estimation parameters, e.g., probability estimator parameters, e.g. N, .
  • the method comprises using different probability estimation parameter values for a decoding of different neural network parameters and/or using different probability estimation parameter values for a decoding of bins associated with different context models, e.g. C k .
  • Further embodiments according to the invention comprise a method for decoding weight parameters of a neural network, wherein the method comprises obtaining a plurality of neural network parameters, e.g., at least one of entries w i of matrix W, b, ⁇ , ⁇ 2 , ⁇ , ⁇ , and/or ⁇ , of the neural network on the basis of an encoded bitstream.
  • the method comprises decoding the neural network parameters, e.g., entries w i of matrix W, or b, or ⁇ , or ⁇ 2 , or ⁇ , or ⁇ , or ⁇ , of the neural network, e.g., a quantized version of the neural network parameters, using a context-dependent arithmetic decoding, e.g., using a context-adaptive binary arithmetic decoding (CABAC).
  • CABAC context-adaptive binary arithmetic decoding
  • probabilities of bin values may be determined for different contexts, wherein, for example, each bin is associated with a context.
  • the method comprises obtaining a probability estimate, e.g.
  • P(t) or p k which may, for example, be associated with a context, for an, e.g. arithmetic, decoding of a bin of a number representation of a neural network parameter, e.g. on the basis of one or more previously decoded neural network parameters or bins thereof, using one or more probability estimation parameters, e.g., probability estimator parameters, e.g. N, .
  • the method comprises using different probability estimation parameter values for a decoding of neural network parameters associated with different layers of the neural network.
  • Embodiments according to the invention comprise a method for encoding weight parameters of a neural network, wherein the method comprises obtaining a plurality of neural network parameters, e.g., at least one of entries w i of matrix W, b, ⁇ , ⁇ 2 , ⁇ , ⁇ , and/or ⁇ , of the neural network.
  • a plurality of neural network parameters e.g., at least one of entries w i of matrix W, b, ⁇ , ⁇ 2 , ⁇ , ⁇ , and/or ⁇ , of the neural network.
  • the method comprises encoding the neural network parameters, e.g., entries w i of matrix W, or b, or ⁇ , or ⁇ 2 , or ⁇ , or ⁇ , or ⁇ , of the neural network, e.g., a quantized version of the neural network parameters, using a context-dependent arithmetic coding, e.g., using a context-adaptive binary arithmetic coding (CABAC).
  • CABAC context-adaptive binary arithmetic coding
  • probabilities of bin values may be determined for different contexts, wherein, for example, each bin may be associated with a context.
  • the method comprises obtaining a probability estimate, e.g.
  • P(t) or p k which may, for example, be associated with a context, for a, e.g. arithmetic, encoding of a bin of a number representation of a neural network parameter, e.g. on the basis of one or more previously encoded neural network parameters or bins thereof, using one or more probability estimation parameters, e.g., probability estimator parameters, e.g. N , .
  • the method comprises using different probability estimation parameter values for an encoding of different neural network parameters and/or using different probability estimation parameter values for an encoding of bins associated with different context models, e.g. C k .
  • Further embodiments according to the invention comprise a method for encoding weight parameters of a neural network, wherein the method comprises obtaining a plurality of neural network parameters, e.g., at least one of entries w i of matrix W, b, ⁇ , ⁇ 2 , ⁇ , ⁇ , and/or ⁇ , of the neural network.
  • a plurality of neural network parameters e.g., at least one of entries w i of matrix W, b, ⁇ , ⁇ 2 , ⁇ , ⁇ , and/or ⁇ , of the neural network.
  • the method comprises encoding the neural network parameters, e.g., entries w, of matrix W, or b, or ⁇ , or ⁇ 2 , or ⁇ , or ⁇ , or ⁇ , of the neural network, e.g., a quantized version of the neural network parameters, using a context-dependent arithmetic coding, e.g., using a context-adaptive binary arithmetic coding (CABAC).
  • CABAC context-adaptive binary arithmetic coding
  • probabilities of bin values may be determined for different contexts, wherein, for example, each bin may be associated with a context.
  • the method comprises obtaining a probability estimate, e.g.
  • P(t) or p k which may, for example, be associated with a context, for an, e.g. arithmetic, encoding of a bin of a number representation of a neural network parameter, e.g. on the basis of one or more previously encoded neural network parameters or bins thereof, using one or more probability estimation parameters, e.g., probability estimator parameters, e.g. N, .
  • the method comprises using different probability estimation parameter values for an encoding of neural network parameters associated with different layers of the neural network.
  • Embodiments according to the invention comprise a computer program for performing a method according to the invention when the computer program runs on a computer.
  • Embodiments according to the invention comprise an encoded representation of weight parameters of a neural network, comprising a plurality of encoded weight parameters of the neural network and an encoded representation of one or more probability estimation parameters determining characteristics of a probability estimation for an adaptation of a context of an arithmetic decoding of the encoded weight parameters.
  • the parameters W, b, ⁇ , ⁇ 2 , ⁇ , and ⁇ may be collectively denoted parameters of a layer or layer parameters.
  • One or more of these parameters may be examples for the neural network parameters, as explained before. They usually need to be signaled in a bitstream (e.g. in an encoded video representation, for example, if the neural network is used in a video decoder). For example, they could be represented as 32 bit floating point numbers or they could, for example, be quantized to an integer representation, also denoted as quantization indices. Note that e is usually not signaled in the bitstream.
  • a particularly efficient approach for encoding such parameters employs a uniform reconstruction quantizer (URQ) where, for example, each value is represented as integer multiple of a so-called quantization step size value.
  • the corresponding floating point number can, for example, be reconstructed by multiplying the integer with the quantization step size, which is usually (but not necessarily) a single floating point number.
  • efficient implementations for neural network inference that is, calculating the output of the neural network for an input
  • a set of quantizers is applied where each value is, for example, represented as integer multiple of a quantization step size value.
  • each quantizer in the set employs a disjoint set of integer multiples of the quantization step size parameter as applicable reconstruction values, but two or more quantizers may share one or more reconstruction values.
  • the applied quantizer depends, for example, on the values of previous quantization indices in coding order.
  • the corresponding floating point number can, for example, be reconstructed by multiplying the integer with the quantization step size, which is usually, for example, a floating point number which depends on the chosen quantizer.
  • An example for such a quantizer design is trellis coded quantization (TCQ), also denoted as dependent quantization (DQ).
  • a set of two quantizers is used.
  • the first quantizer employs, for example, all even multiples of the quantization step size including zero
  • the second quantizer employs all the even multiples of the quantization step size including zero.
  • the quantization indices that are output, for example, by the quantization method, e.g. as explained above, may then be entropy coded using a suitable entropy coding method.
  • a particularly suitable entropy coding method for encoding such quantization indices is Context-based Adaptive Binary Arithmetic Coding, also denoted as CABAC.
  • CABAC Context-based Adaptive Binary Arithmetic Coding
  • each quantization index is, for example, decomposed into a sequence of binary decisions, so-called bins.
  • bins are usually, for example, each bin is associated with a probability model, also denoted as context model, which models the statistics of the associated bins, for example, using a probability estimation method.
  • a probability estimator is an apparatus, that models the probability P(t) for a bin being equal to x, where x ⁇ ⁇ 0,1 ⁇ , for example, based on already coded bins associated with the probability estimator.
  • P(t) may be an example for the probability estimate.
  • estimator design is explained. Details of the estimator design are optional features of embodiments according to the invention, and are especially combinable with embodiments comprising the above explained features.
  • each context model c k one or more state variables are maintained with N ⁇ 1.
  • Each state variable is implemented, for example, as signed integer value and represents, for example, a probability value .
  • the probability estimate p k of a context model c k shall be defined, for example, as weighted sum of the probability values of all state variables of the context model.
  • negative state variables may, for example, correspond to ⁇ 0.5.
  • P( ⁇ ) for each state variable of each context model.
  • a state representation that is used in neural network compression can be achieved with the following equation: is a weighting factor, ⁇ is a parameter with 0 ⁇ ⁇ ⁇ 1.
  • all state variables are optionally initialized with same values, denoted as , that may, for example, be optimized to the compression application.
  • a probability estimate is derived from the state variables of a context model.
  • Three alternative approaches are presented in the following as examples. Approach 1 yields more accurate results than approach 2 and approach 3, but also has a higher computational complexity.
  • Step 1
  • a lookup table LUT 1 is employed for converting a state variable into the corresponding probability value , for example according to Eq. (1).
  • LUT1 is a lookup table containing probability values, is a weighting factor that adapts to the size of LUT1.
  • the probability estimate p k is derived from the probability values , for example according to; is a weighting factor that controls the influence of the individual state variables.
  • (3) is a weighting factor that controls the influence of each state variable.
  • the probability estimate p k is derived from the weighted sum of state variables s k , for example according to: LUT2 is a lookup table containing probability estimates, ⁇ k is a weighting factor that adapts s k to the size of LUT2.
  • the weighted sum s k of the state variables is derived, for example, as in approach 2.
  • the probability estimate p k is derived from the weighted sum of state variables s k , for example according to:
  • LUT2 is a lookup table containing probability estimates.
  • a further approach uses a linear relation between the state values and the probability P(x, i, k).
  • the derivation of the probability estimate is, for example, using the approach of equation (2).
  • An example of approach 4 is the probability estimation scheme used in the current draft of Versatile Video Coding (VVC).
  • the look-up table containing the probability estimates is, for example:
  • LUT2 ⁇ 0.5000, 0.4087, 0.3568, 0.3116, 0.2721, 0.2375, 0.2074, 0.1811,
  • one or more state variables of a context model may be updated in order to track the statistical behavior of the symbol sequence.
  • the update is, for example, carried out as follows:
  • A is a lookup table storing, for example, integer values, and are weighting factors that control, for example, the update ‘agility’.
  • the factors can be written, for example, according to , where also denoted as adaptation parameter, z is an offset that ensures, for example, that look table A is accessed only with non-negative values.
  • lookup table A can, for example, be chosen so that stays in a particular given interval.
  • the values of look-up A approximate, for example, an update function.
  • the estimation method of WC following approach 4, applies update functions for the state update and uses bit shifts, which, for example, determine the ‘agility’ of the update.
  • update functions for the state update and uses bit shifts, which, for example, determine the ‘agility’ of the update.
  • bit shifts which, for example, determine the ‘agility’ of the update.
  • the look-up table A is, for example:
  • A ⁇ 157, 143, 129, 115, 101 , 87, 73, 59, 45, 35, 29, 23, 17, 13, 9, 5, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 0 ⁇ .
  • probability estimators Before encoding a symbol, shall, for example, be initialized with a value from the interval [-127, 127] and shall be initialized, for example, with a value from the interval [-2047, 2047] , Consequently, can, for example, be implemented with an 8 bit signed integer value and can, for example, be implemented with a 12 bit signed integer.
  • probability estimators have several parameters, denoted as probability estimator parameters or estimator parameters (or also as probability estimation parameters), that affect the probability estimates, e.g. the adaptation rate. Usually, those estimator parameters are, for example, chosen globally, depending on the application scenario, e.g. encoding of neural network parameters.
  • each neural network parameter applies the same set of estimator parameters. But, it has been found by the inventors that the compression efficiency can be improved by selecting optimized estimator parameters for a current neural network parameter. So, according to an aspect, the basic idea is to select suitable estimator parameters out of a set of parameters, which are then signaled to the decoder.
  • embodiments according to the invention are based on the idea to use different probability estimation parameters values for decoding and respectively encoding of neural network parameters or entities, e.g. bins associated with different contexts, associated with neural network parameters.
  • an adapted or for example even optimal, individual choice of probability estimation parameters values may be performed. Selection of probability estimation values may be performed based on any suitable criterion, e.g. a priori known, characteristics of certain neural network parameters, stochastic features of context models of certain neural network parameters, or of bins of neural network parameters respectively.
  • different neural network parameters may comprise different correlation characteristics, and that a adaptation of the probability estimation parameters to these different correlation characteristics may result in an improved coding efficiency.
  • a correlation strength between adjacent branches or edges of the neural network may differ.
  • an extension of the correlation between the edges (or branches) of the neural network may vary.
  • the probability estimation parameter values may by adapted to the correlation characteristics (correlation strength/extension of the correlation) of the neural network (e.g. in a dynamic manner, e.g. even between an encoding of different neural network parameters of a single neural net), thus resulting in a particularly efficient encoding.
  • the correlation characteristics of neural network parameters associated with different layers of the neural network differ substantially in some cases. For example, if different layers of the neural network reflect convolutions of different sizes (or widths), this can be accomplished by using different probability estimation parameter values for the encoding or decoding of neural network parameters of different neural network layers, resulting in an improved coding efficiency.
  • a usage of different probability estimation parameters when encoding or decoding different neural network parameters or when encoding or decoding bins associated with different context models provides for an improved tradeoff between coding efficiency, complexity and resource usage. For example, it has been recognized that an overhead which may be required for a signaling (or even for a dynamic signaling) of the actually used probability estimation parameter values may be over-compensated by an increase in coding efficiency.
  • a second aspect according to embodiments of the invention is that encoding and/or decoding of the probability estimation parameters may comprise similar or even equal steps as the encoding and/or decoding of neural network parameters.
  • the probability estimator parameters may be encoded/decoded using analogous steps as the neural network parameters.
  • the probability estimation parameters, or an integer index q representing the probability parameter may be encoded/decoded using a sequence of bins, e.g. bins greaterThan_0, greaterThan_1, ..., and context models analogously to the neural network parameters.
  • a decoder may obtain an encoded bitstream, the bitstream comprising neural network parameters.
  • the neural network parameters may be encoded in the bitstream as a sequence of bins. These bins may be decoded using context-dependent arithmetic decoding, for example the same context dependent arithmetic coding method used to encode the bins representing neural network parameters to the bitstream. Therefore, the decoder is configured to obtain a probability estimate using a probability estimator comprising one or more probability estimation parameters, for example at least one of the beforementioned probability estimator parameters e.g.
  • the decoder may then use different probability estimation parameter values for a decoding of different neural network parameters and/or different probability estimation parameter values for a decoding of bins associated with different context models, e.g. C k . Furthermore, the decoder may use different probability estimation parameter values for a decoding of neural network parameters associated with different layers of the neural network. By adapting the probability estimation parameter values according to certain characteristics of the neural network parameters, e.g. the layer or the context model of their bins decoding and encoding efficiency may be improved.
  • the parameters i.e. N, initValf and/or any other parameter related to the probability estimator, e.g. context model may be collectively denoted as probability estimator parameters or estimator parameters (or probability estimation parameters).
  • the probability estimation can, for example, be improved, if the parameters are chosen individually for each parameter or a subset of parameters of a layer (i.e. W, b, m, s 2 , g, and b) and/or context model C k .
  • the estimator parameter to be used is, for example, determined among the parameters of a set of parameters, which can, for example, be the base set or any subset of the base set.
  • Each parameter of the set may, for example, be associated with an integer index q.
  • one parameter of the set may be denoted as default parameter.
  • the default parameter is, for example, associated with an integer index equal to zero.
  • the index associated with the chosen estimator parameter is then, for example, signaled to the decoder.
  • the decoder is configured to choose one or more probability estimation parameters from a base set or from a true subset of the base set.
  • the base set may comprise a plurality of useable parameter values associated with one or more probability estimation parameters, or wherein the base set may comprise a plurality of tuples of useable parameter values associated with a plurality of probability estimation parameters, e.g. a list of useable pairs
  • pairs, tuples and/or sets of allowable probability estimation parameters may be provided that are easy to select, e.g. with respect to one or more criteria. For example, according to a certain layer of the neural network a subset may be chosen with low computational effort, e.g. by just comparing a layer ID to an a priori defined list, in order to choose probability estimation parameters providing an improved decoding efficiency. In addition, such a selection of a subset may be performed according to a certain type of quantization. By providing subsets, advantageous probability estimation parameters may be provided, without a large allocation effort. Instead of checking a plurality of conditions, for example one condition for each probability estimation parameter, one check may be performed, e.g.
  • the decoder is configured to choose one or more probability estimation parameters from different sets of useable parameter values or of useable tuples of parameter values in dependence on a quantization mode or wherein the decoder is configured to use different mapping rules, e.g. different mapping tables, mapping an encoded value representing one or more probability estimation parameters, e.g. an encoded index value q, onto one or more probability estimation parameters, e.g. , in dependence on a quantization mode.
  • mapping rules e.g. different mapping tables, mapping an encoded value representing one or more probability estimation parameters, e.g. an encoded index value q, onto one or more probability estimation parameters, e.g. , in dependence on a quantization mode.
  • the decoder may chose the one or more probability estimation parameters from a first set of useable parameter values or of useable tuples of parameter values for the case that a first quantization mode, e.g. uniform quantization URQ, is used, and the decoder may chose the one or more probability estimation parameters from a second set of useable parameter values for the case that a second quantization mode, e.g. dependent quantization DQ, is used.
  • a first quantization mode e.g. uniform quantization URQ
  • DQ dependent quantization
  • the sets of useable parameters may, for example, define different associations between an index value included in an encoded bitstream and an associated probability estimation parameter or an associated tuple of probability estimation parameter, wherein, for example, a first set of useable parameters defines a mapping of a given index value onto a first tuple of probability estimation parameters and wherein, for example, a second set of useable parameters defines a mapping of the given index value onto a second tuple of probability estimation parameters which is different from the first tuple.
  • the decoder is configured to selectively choose one or more probability estimation parameters from a first set of useable parameter values or from a first set of useable tuples of parameter values in case that a uniform quantization, e.g.
  • a uniform reconstruction quantization (URQ), or a time-invariant quantization, of the one or more probability estimation parameters is used, and the decoder is configured to selectively choose one or more probability estimation parameters from a second set of useable parameter values or from a second set of useable tuples of parameter values in case that a variable quantization, e.g. a trellis-coded quantization (TCQ) or a dependent quantization (DQ), of the one or more probability estimation parameters is used, wherein, for example, the variable quantization, like a DQ, may use more context models than the uniform quantization, such that there may be less bins to be decoded per context model in the case of variable quantization.
  • TCQ trellis-coded quantization
  • DQ dependent quantization
  • the decoder is configured to use a first mapping rule, e.g. a first mapping table, mapping an encoded value representing one or more probability estimation parameters, e.g. an encoded index value q, onto one or more probability estimation parameters, e.g. , in case that a uniform quantization, e.g. a uniform reconstruction quantization (URQ) or a time- invariant quantization, of the one or more probability estimation parameters is used.
  • a uniform quantization e.g. a uniform reconstruction quantization (URQ) or a time- invariant quantization
  • the decoder may be configured to use a second mapping rule, e.g. a second mapping table, mapping an encoded value representing one or more probability estimation parameters, e.g. an encoded index value q, onto one or more probability estimation parameters in case that a variable quantization, e.g. a trellis-coded quantization (TCQ) or a dependent quantization (DQ), of the one or more probability estimation parameters is used.
  • a variable quantization e.g. a trellis-coded quantization (TCQ) or a dependent quantization (DQ)
  • the variable quantization like a DQ, may use more context models than the uniform quantization, such that there may be less bins to be decoded per context model in the case of variable quantization.
  • the first set of useable parameter values is different from the second set of useable parameter values
  • the first set of useable tuples of parameter values is different from the second set of useable tuples of parameter values and/or the second mapping rule is different from the first mapping rule.
  • a concept according to the invention may be applied to any form of quantization and is not limited to a certain type of quantization.
  • the type of quantization and the used set and/or mapping of probability estimation parameters may be adapted. Therefore, coding efficiency may be improved, in addition to providing a high amount of flexibility.
  • useable parameter values of the second set of useable parameter values, or for example of the second mapping rule allow for a faster adaptation of a probability estimate, e.g. to a change of bin value frequencies, than useable parameter values of the first set of useable parameter values, or for example of the first mapping rule.
  • useable tuples of parameter values of the second set of useable tuples of parameter values allow for a faster adaptation of a probability estimate, e.g. to a change of bin value frequencies, than useable tuples of parameter values of the first set of useable tuples of parameter values, or for example of the first mapping rule.
  • This embodiment is based on the finding that such a concept results in an improved coding efficiency.
  • the second set of useable parameter values comprises a useable parameter value which allows for a faster adaptation of a probability estimate, e.g. to a change of bin value frequencies, than useable parameter values, or for example even than all useable parameter values, of the first set of useable parameter values, or for example of the first mapping rule.
  • the second set of useable tuples of parameter values comprises a useable tuple of parameter values which allows for a faster adaptation of a probability estimate, e.g. to a change of bin value frequencies, than useable tuples, or for example even than all useable tuples, of parameter values of the first set of useable tuples of parameter values.
  • This embodiment is based on the finding that such a concept results in an improved coding efficiency.
  • the decoder is configured to choose one or more probability estimation parameters from different sets of useable, e.g. allowable, parameter values or of useable, e.g. allowable tuples of parameter values in dependence on a number of parameters of a layer of the neural network, or in dependence on a number of neural network parameters to be decoded using the chosen one or more probability estimation parameters, or in dependence on a number of elements of a layer parameter, e.g. a number of entries of matrix W, or a number of elements of (transposed) vector b, etc..
  • the decoder is configured to use different mapping rules, e.g. different mapping tables, mapping an encoded value representing one or more probability estimation parameters, e.g. an encoded index value q, onto one or more probability estimation parameters, e.g. in dependence on a number of parameters of a layer of the neural network, or in dependence on a number of neural network parameters to be decoded using the chosen one or more probability estimation parameters, or in dependence on a number of elements of a layer parameter, e.g. a number of entries of matrix W, or a number of elements of (transposed) vector b, etc..
  • mapping rules e.g. different mapping tables, mapping an encoded value representing one or more probability estimation parameters, e.g. an encoded index value q, onto one or more probability estimation parameters, e.g. in dependence on a number of parameters of a layer of the neural network, or in dependence on a number of neural network parameters to be decoded using the chosen one or more probability estimation parameters, or in dependence on a number of elements
  • an adaptation parameter of the probability estimator may be chosen depending on the number of neural network parameters to be decoded. For example, the higher the number of bins already decoded, the better a stochastic estimation of a probability of such a bin may be determinable. Consequently, the coding efficiency may be improved.
  • a first mapping rule e.g. a first mapping table, mapping an encoded value representing one or more probability estimation parameters, e.g. an encoded index value q, onto one or more probability estimation parameters, e.g. if the number of parameters of a layer of the neural network is below a threshold value
  • a second mapping rule e.g. a second mapping table, mapping an encoded value representing one or more probability estimation parameters, e.g. an encoded index value q, onto one or more probability estimation parameters, e.g. if the number of parameters of a layer of the neural network is above the threshold value, e.g
  • the second set of useable parameter values comprises more useable parameter values than the first set of useable parameter values
  • the second set of useable tuples of parameter values comprises more useable tuples than the first set of useable tuples of parameter values
  • the second mapping rule is different from the first mapping rule
  • probability estimation parameters may provide a computationally inexpensive possibility to adapt the probability estimation parameters.
  • statistic adaptation e.g. of context models, may be performed, for example better, if a large number of parameters is decoded, and therefore probability estimation parameters may be adjusted according to such a number.
  • a signaling overhead e.g. a number of bits required for an encoding of an index of a probability estimation parameter or of a set of probability estimation parameters
  • the signaling is limited to probability estimation parameters (or sets of probability estimation parameters) which are well suited for the currently considered neural network.
  • the decoder is configured evaluate a signaling, which may, for example, be included in the encoded bit stream, for example, in the form of a dedicate flag, from which set of useable parameter values, for example small set or large set, e.g. out of a plurality of sets of useable parameter values, which may be different and possibly overlapping subsets of a base set, or from which set of useable tuples of parameter values, for example small set or large set, e.g. out of a plurality of sets of useable tuples of parameter values, which may be different and possibly overlapping subsets of a base set, the one or more probability estimation parameters are elected, e.g. using one or more flags which are decoded, e.g. using a flag “useSecondSubset”.
  • a signaling which may, for example, be included in the encoded bit stream, for example, in the form of a dedicate flag, from which set of useable parameter values, for example small set or large set, e.
  • the decoder is configured evaluate a signaling, which may, for example, be included in the encoded bit stream, for example, in the form of a dedicate flag, indication which mapping rule out of a plurality of mapping rules, e.g. mapping tables, should be used to map an encoded value representing one or more probability estimation parameters, e.g. an encoded index value q, onto one or more probability estimation parameters, e.g. sh, k .
  • a signaling which may, for example, be included in the encoded bit stream, for example, in the form of a dedicate flag, indication which mapping rule out of a plurality of mapping rules, e.g. mapping tables, should be used to map an encoded value representing one or more probability estimation parameters, e.g. an encoded index value q, onto one or more probability estimation parameters, e.g. sh, k .
  • choice of probability estimation parameters may be performed with low computational costs.
  • an error probability of using different probability estimation parameters in the decoder than in the encoder may be reduced.
  • a beneficial mapping rule e.g. the mapping rule used in the encoder used to encode the encoded bitstream, may be communicated via such a signaling.
  • the coding may be adapted to the characteristics of the neural network. For example, a first set of useable tuples of parameter values may be selected for a first type of neural network (e.g. because the first set of useable parameter values better fits the statistics of the first type of neural network), and a second set of useable tuples of parameter values may be selected for a second type of neural network. For example, it may be sufficient to signal the selection of the useable set of parameter values only once for a neural network (or at least more rarely than the actual selection of an individual set of parameter values). Consequently, a coding efficiency may be improved.
  • the decoder is configured to decode one or more index values, for example generally integer values, e.g. an index q or a plurality of indices, describing a probability estimation parameter value, or describing a plurality of probability estimation parameter values, or describing a tuple of probability estimation parameter values, e.g. an index q.
  • index values for example generally integer values, e.g. an index q or a plurality of indices, describing a probability estimation parameter value, or describing a plurality of probability estimation parameter values, or describing a tuple of probability estimation parameter values, e.g. an index q.
  • probability estimation parameters may be represented in an easily compressible manner.
  • the probability estimation parameters may be encoded and therefore decoded analogously the neural network parameters, providing an improved encoding/decoding efficiency.
  • the decoder is configured to decode the one or more index values using one or more context models, which may, for example, determine probabilities of bin values of one or more bins used for decoding the index value.
  • the sequence of encoding/decoding of neural network parameters may be applied analogously on the probability estimation parameters. Therefore, probability estimation parameters or bins representing probability estimation parameters, or index values representing probability estimation parameters may be associated with a context model.
  • the benefits of context-depended coding may be exploited twice, with the encoding/decoding of the probability estimation parameters, used to context-dependent encode/decode the neural network parameters.
  • the decoder is configured to decode a first bin, e.g. a useNotDefault bin, which describes whether a currently considered index value takes a default value, and the decoder is configured to selectively decode one or more additional bins representing the currently considered index value, or a value derived therefrom, e.g. q-1 , in a binary representation, if the currently considered index value does not take the default value.
  • a first bin e.g. a useNotDefault bin, which describes whether a currently considered index value takes a default value
  • the decoder is configured to selectively decode one or more additional bins representing the currently considered index value, or a value derived therefrom, e.g. q-1 , in a binary representation, if the currently considered index value does not take the default value.
  • the first bin indicating whether the currently considered index value takes a default value, is decoded using a context, e.g. considering a probability estimate, while the one or more additional bins are decoded with a fixed length of one bit per bin.
  • Decoding such a first bin may improve coding efficiency, since no further decoding may be required if the neural network parameters is identified, for example in a first bin decoding step, as a default value. Since the first bin may always be present, a e.g. more complex context dependent coding may provide an increased coding efficiency. For additional bins that may not always be present an less complex coding, e.g. with fixed probabilities may be implemented.
  • the decoder is configured to decode the one or more index values using a unary code decoding, or using a truncated unary code decoding, or using a variable length code decoding, wherein optionally the code lengths are chosen according to probabilities of occurrence of different index values.
  • Usage of a unary code may allow for a prefix-free and a self-synchronizing code.
  • index values with a high probability of occurrence may have short code lengths, allowing for an improved efficiency, since symbols and time may be saved with short code lengths for indexes that occur often.
  • the decoder is configured to vary a number of bins or a maximum number of bins, e.g. in case of unary code or Huffmann code, used for decoding the one or more probability estimation parameters, e.g. an integer index q designating a selected probability estimation parameter or a selected tuple of probability estimation parameters, in dependence on a quantization mode used for quantizing the one or more probability estimation parameters, e.g. to adapt to a selected set of useable parameter values or to a selected set of useable tuples of parameter values.
  • a quantization mode used for quantizing the one or more probability estimation parameters
  • a certain accuracy or precision of the probability estimation parameters may be achievable. Therefore, a coding efficiency may be improved, if the number of bins or a maximum number of bins may be chosen with respect to the quantization or an expected quantization error.
  • the decoder is configured to vary a number of bins or a maximum number of bins, e.g. in case of unary code or Huffmann code, used for decoding the one or more probability estimation parameters, e.g. an integer index q designating a selected probability estimation parameter or a selected tuple of probability estimation parameters, in dependence on a number of parameters of a layer of the neural network, or in dependence on a number of neural network parameters to be decoded using the one or more probability estimation parameters, or in dependence on a number of elements of a layer parameter, e.g. a number of entries of matrix W, or a number of elements of (transposed) vector b, etc., e.g.
  • Coding efficiency may be improved by varying the number of bins used for decoding probability estimation parameters with respect to a number of neural network parameters to be decoded. A good trade-off between accuracy and computational costs and time effort may be implemented.
  • stochastic characteristics of bins may be determined depending on the number of neural network parameters, for example adapting a context model more accurately with an increased number of neural network parameters being dependent on said context model.
  • the decoder is configured to switch between different sets of usable parameter values associated with the one or more probability estimation parameters, or between different sets of tuples of useable parameter values associated with a plurality of probability estimation parameters, or between different mapping rules for mapping an encoded value representing one or more probability estimation parameters, e.g. an encoded index value q, onto one or more probability estimation parameters, e.g.
  • Such a switching may, for example, be performed after switching to neural network parameters of another layer that are to be decoded. This flexibility may allow for a more sophisticated adaptation of the decoding for an increased coding efficiency.
  • the decoder is configured to vary a number of bins or a maximum number of bins, e.g. in case of unary code or Huffmann code, used for decoding the one or more probability estimation parameters, e.g. an integer index q, designating a selected probability estimation parameter or a selected tuple of probability estimation parameters in accordance with a switching between different sets of usable parameter values associated with the one or more probability estimation parameters, or between different sets of tuples of useable parameter values associated with a plurality of probability estimation parameters or between different mapping rules.
  • a number of bins or a maximum number of bins e.g. in case of unary code or Huffmann code, used for decoding the one or more probability estimation parameters, e.g. an integer index q, designating a selected probability estimation parameter or a selected tuple of probability estimation parameters in accordance with a switching between different sets of usable parameter values associated with the one or more probability estimation parameters, or between different sets of tuples of useable parameter values associated
  • An index to be encoded is, for example, decomposed into a sequence of bins, which are then encoded.
  • Each bin may, for example, be coded using a context model or using a fixed probability.
  • the encoding procedure may, for example, be according to one of the following schemes: 1.
  • a series of bins e.g. additional bins, is encoded, which denote, for example, the index of the chosen parameter minus one (q - 1), indexMinusOne.
  • the number of bins encoded for index is, for example, equal to 2 (setLength - 1)1, where setLength, denotes the number of elements of the set.
  • This procedure applies a truncated unary code, which is, for example, identical to the unary code used in encoding method 2., except for the case where the index to encode q is equal to q MAX .
  • the index to encode q is equal to q MAX .
  • the bin greaterThan_(q MAX - 1) no further bins are encoded.
  • the value of q is inferred to be q MAX , if greaterThan_(q MAX - 1) is equal to one.
  • This procedure uses a variable length code, where the code lengths are chosen according to the probability of occurrence of a symbol, for example a Huffman code.
  • the decoder is configured to determine one or more state variables, e.g. or S R , and to derive the probability estimate, e.g. p k , using the one or more state variables, e.g. using equations (1) and (2) or using equations (3) and (4) or using equations (3) and (5) or using a linear relation between the one or more state variables and the probability estimate, e.g. P(x,i,k).
  • State variables provide an efficient mean for an evaluation of a probability model, e.g. a context model, describing a stochastic of a neural network parameter or a bin thereof.
  • the state variables may also be used for the coding of the probability estimation parameters.
  • the probability estimation parameters may be decoded using a context model, being evaluated using state variables, that may, for example, be updated based on probability estimation parameters already decoded or decoded, e.g. recently.
  • Lookup tables as shown, for example, in eqn. (1) may, in addition, reduce computational costs.
  • the decoder is configured to update the state variables according to If decoded symbol is 1. If decoded symbol is 0.’ wherein and are weighting factors, and optionally constitute probability estimation parameters, wherein, for example, may be equal to 2 '3 and wherein may be equal to 2 7 .
  • A is a lookup table, e.g. storing integer values, which may, for example be defined as described herein.
  • z is an offset value, e.g. a predetermined value, which may, for example, be equal to 16.
  • the decoder is configured to vary the weighting factors so as to use different probability estimation parameter values, e.g. different values for , for a decoding of different neural network parameters and/or to use different probability estimation parameter values for a decoding of bins associated with different context models and/or to use different probability estimation parameter values for a decoding of neural network parameters associated with different layers of the neural network.
  • different probability estimation parameter values e.g. different values for
  • the decoder is configured to decode an information describing the adaptation parameters, e.g. an index q describing a tuple of adaptation parameters, wherein optionally a meaning of the decoded index value q may, for example, be defined as provided in the following table 2 or in table 3, or in table 4 or in table 5 or in table 6 or in table 7.
  • an information describing the adaptation parameters e.g. an index q describing a tuple of adaptation parameters
  • a meaning of the decoded index value q may, for example, be defined as provided in the following table 2 or in table 3, or in table 4 or in table 5 or in table 6 or in table 7.
  • an estimator applies, for example, a base set of adaptation parameters, which are N-tuple of adaptation parameters Then a subset of the base set is chosen. One parameter out of the subset may be signaled.
  • the configuration is, for example, equal to the previous preferred embodiment, but an estimator is used, which is configured such is identical to the estimator used the current draft of MPEG-7 part 17 standard for compression of neural networks for multimedia content description and analysis and the base set contains, for example, the following 28 pairs for
  • the subset of size 3 is defined and ordered, for example, such that the indexes q according to Table 2 are assigned, for example, in the case all parameters of a layer are quantized with DQ.
  • Table 2 Preferred subset of adaptation parameters for set size 3
  • one parameter out of the subset is signaled, for example, by encoding q according to encoding scheme 1.
  • the bin useNotDefault is encoded using a context model and all other bins are encoded with a fixed length of one bit per bin.
  • an arbitrary mix of context dependent coding and/or other coding e.g. variable length or fixed length coding may be applied to any bin of probability estimation parameters and/or neural network parameters.
  • the configuration is identical to the previous preferred embodiment, except for the assigned adaptation parameter pairs and the size of the chosen subset (Table 3), which is equal to 5.
  • the configuration is identical to the previous preferred embodiment, except for assigned adaptation parameter pairs (Table 4): Table 4: Second preferred subset of adaptation parameters for set size 5
  • the configuration is identical to the previous preferred embodiment, except for the assigned adaptation parameter pairs and the size of the chosen subset (Table 5), which is equal to 9.
  • Table 5 Preferred subset of adaptation parameters for set size 9
  • the configuration is identical to the previous preferred embodiment, except for the assigned adaptation parameter pairs (Table 6).
  • the configuration is identical to the previous preferred embodiment, except for assigned adaptation parameter pairs (Table 7), the size of the chosen subset (5), and the used quantization method, which uses URQ:
  • an estimator is used, which is configured such that it is identical to the estimator used the current draft of MPEG-7 part 17 standard for compression of neural networks for multimedia content description and analysis and the base set of Table 1 is used. This is denoted as base configuration.
  • the subset (of size 9) of parameter pairs in Table 5 may be applied. If a layer parameter is quantized with URQ the subset in Table 8 may be used.
  • Table 8 Preferred subset of adaptation parameters for set size 9
  • the base configuration of the previous preferred embodiment may be applied.
  • the subset with size 3 of parameter pairs for example, in Table 2, denoted as first subset
  • the subset with size 9 for example, in Table 5, denoted as second subset
  • the configuration is identical to the previous preferred embodiment, but instead of using a threshold, a flag (for example, useSecondSubset) is encoded, which determines, for example, the subset to be used. For example, if the flag is equal to zero, the first subset may be used. If the flag is equal to one, the second subset may be used.
  • a flag for example, useSecondSubset
  • the encoder is configured to choose one or more probability estimation parameters from a base set or from a true subset of the base set.
  • the base set may comprise a plurality of useable parameter values associated with one or more probability estimation parameters, or the base set may comprise a plurality of tuples of useable parameter values associated with a plurality of probability estimation parameters, e.g. a list of useable pairs
  • the encoder is configured to choose one or more probability estimation parameters from different sets of useable parameter values or of useable tuples of parameter values in dependence on a quantization mode, for example, such that the encoder chooses the one or more probability estimation parameters from a first set of useable parameter values or of useable tuples of parameter values for the case that a first quantization mode, e.g. uniform quantization URQ, is used, and for example such that the encoder chooses the one or more probability estimation parameters from a second set of useable parameter values for the case that a second quantization mode, e.g. dependent quantization DQ, is used.
  • a first quantization mode e.g. uniform quantization URQ
  • the encoder is configured to selectively choose one or more probability estimation parameters from a first set of useable parameter values or from a first set of useable tuples of parameter values in case that a uniform quantization, e.g. a uniform reconstruction quantization (URQ) or a time-invariant quantization, of the one or more probability estimation parameters is used, and the encoder is configured to selectively choose one or more probability estimation parameters from a second set of useable parameter values or from a second set of useable tuples of parameter values in case that a variable quantization, e.g. a trellis-coded quantization (TCQ) or a dependent quantization (DQ), of the one or more probability estimation parameters is used.
  • the variable quantization like a DQ, may use more context models than the uniform quantization, such that there may be less bins to be encoded per context model in the case of variable quantization.
  • first set of useable parameter values is different from the second set of useable parameter values
  • first set of useable tuples of parameter values is different from the second set of useable tuples of parameter values.
  • useable parameter values of the second set of useable parameter values allow for a faster adaptation of a probability estimate, e.g. to a change of bin value frequencies, than useable parameter values of the first set of useable parameter values.
  • useable tuples of parameter values of the second set of useable tuples of parameter values allow for a faster adaptation of a probability estimate, e.g. to a change of bin value frequencies, than useable tuples of parameter values of the first set of useable tuples of parameter values.
  • the second set of useable parameter values comprises a useable parameter value which allows for a faster adaptation of a probability estimate, e.g. to a change of bin value frequencies, than useable parameter values or for example even than all useable parameter values, of the first set of useable parameter values.
  • the second set of useable tuples of parameter values comprises a useable tuple of parameter values which allows for a faster adaptation of a probability estimate, e.g. to a change of bin value frequencies, than useable tuples, or for example even than all useable tuples, of parameter values of the first set of useable tuples of parameter values.
  • the encoder is configured to choose one or more probability estimation parameters from different sets of useable, e.g. allowable, parameter values or of useable, e.g. allowable, tuples of parameter values in dependence on a number of parameters of a layer of the neural network, or in dependence on a number of neural network parameters to be encoded using the chosen one or more probability estimation parameters, or in dependence on a number of elements of a layer parameter, e.g. a number of entries of matrix W or a number of elements of (transposed) vector b, etc..
  • the second set of useable parameter values comprises more useable parameter values than the first set of useable parameter values
  • the second set of useable tuples of parameter values comprises more useable tuples than the first set of useable tuples of parameter values
  • the encoder is configured to signal from which set of useable parameter values, for example small set or large set, e.g. out of a plurality of sets of useable parameter values which may be different and possibly overlapping subsets of a base set, or from which set of useable tuples of parameter values, for example small set or large set, e.g. out of a plurality of sets of useable tuples of parameter values which may be different and possibly overlapping subsets of a base set, the one or more probability estimation parameters are elected, e.g. using one or more flags which are encoded, e.g. using a flag “useSecondSubset”.
  • the encoder is configured to encode one or more index values, for example generally integer values, e.g. an index q, or a plurality of indices, describing a probability estimation parameter value, or describing a plurality of probability estimation parameter values, or describing a tuple of probability estimation parameter values, e.g. an index q.
  • index values for example generally integer values, e.g. an index q, or a plurality of indices, describing a probability estimation parameter value, or describing a plurality of probability estimation parameter values, or describing a tuple of probability estimation parameter values, e.g. an index q.
  • the encoder is configured to encode the one or more index values using one or more context models, which may, for example, determine probabilities of bin values of one or more bins used for encoding the index value.
  • the encoder is configured to encode a currently considered index value using a first bin, which describes that the currently considered index value takes a default value, or for example using the fist bin only, if the currently considered index value takes the default value.
  • the encoder is configured to encode the currently considered index value using a first bin, which describes that the currently considered index value does not take the default value, and using one or more additional bins representing the currently considered index value, or a value derived therefrom, e.g. q-1, in a binary representation, if the currently considered index value does not take the default value.
  • the first bin indicating whether the currently considered index value takes a default value, is encoded using a context, e.g. considering a probability estimate, while the one or more additional bins are encoded with a fixed length of one bit per bin.
  • the encoder is configured to encode the one or more index values using a unary code, or using a truncated unary code, or using a variable length code, wherein optionally the code lengths are chosen according to probabilities of occurrence of different index values.
  • the encoder is configured to vary a number of bins or a maximum number of bins, e.g. in case of unary code or Huffmann code, used for encoding the one or more probability estimation parameters, e.g. an integer index q designating a selected probability estimation parameter or a selected tuple of probability estimation parameters, in dependence on a quantization mode used for quantizing the one or more probability estimation parameters, e.g. to adapt to a selected set of useable parameter values or to a selected set of useable tuples of parameter values.
  • a quantization mode used for quantizing the one or more probability estimation parameters
  • the encoder is configured to vary a number of bins or a maximum number of bins, e.g. in case of unary code or Huffmann code, used for encoding the one or more probability estimation parameters, e.g. an integer index q designating a selected probability estimation parameter or a selected tuple of probability estimation parameters, in dependence on a number of parameters of a layer of the neural network, or in dependence on a number of neural network parameters to be encoded using the one or more probability estimation parameters, or in dependence on a number of elements of a layer parameter, e.g. a number of entries of matrix W, or a number of elements of (transposed) vector b, etc., e.g. to adapt to a selected set of useable parameter values or to a selected set of useable tuples of parameter values.
  • a number of bins or a maximum number of bins e.g. in case of unary code or Huffmann code, used for encoding the one or more probability estimation parameters, e.
  • the encoder is configured to switch between different sets of usable parameter values associated with the one or more probability estimation parameters, or between different sets of tuples of useable parameter values associated with a plurality of probability estimation parameters.
  • the encoder is configured to vary a number of bins or a maximum number of bins, e.g. in case of unary code or Huffmann code, used for encoding the one or more probability estimation parameters, e.g. an integer index q, designating a selected probability estimation parameter or a selected tuple of probability estimation parameters in accordance with a switching between different sets of usable parameter values associated with the one or more probability estimation parameters, or between different sets of tuples of useable parameter values associated with a plurality of probability estimation parameters
  • the encoder is configured to determine one or more state variables, e.g. s, k or s k , and to derive the probability estimate, e.g. p k , using the one or more state variables, e.g. using equations (1) and (2) or using equations (3) and (4) or using equations (3) and (5) or using a linear relation between the one or more state variables and the probability estimate, e.g. P(x,i,k).
  • the encoder is configured to derive the probability estimate p k from two state variables according to and
  • the encoder is configured to update the state variables according to If symbol to be encoded is 1. If symbol to be encoded is 0. wherein and are weighting factors, and for example constitute probability estimation parameters, wherein, for example, may be equal to 2 -3 and wherein mf may be equal to 2
  • A is a lookup table, e.g. storing integer values, which may, for example be defined as described herein and z is an offset value, e.g. a predetermined value, which may, for example, be equal to 16.
  • offset value e.g. a predetermined value, which may, for example, be equal to 16.
  • the encoder is configured to vary the weighting factors , so as to use different probability estimation parameter values, e.g. different values for , for an encoding of different neural network parameters and/or to use different probability estimation parameter values for an encoding of bins associated with different context models and/or to use different probability estimation parameter values for an encoding of neural network parameters associated with different layers of the neural network.
  • different probability estimation parameter values e.g. different values for
  • the encoder is configured to encode an information describing the adaptation parameters, e.g. an index q describing a tuple of adaptation parameters, wherein for example a meaning of the encoded index value q may, for example, be defined as provided in table 2 or in table 3, or in table 4 or in table 5 or in table 6 or in table 7.
  • a decoder may correspond to an encoder
  • decoding neural network may correspond to encoding neural network parameters
  • previously decoded neural network parameters or bins thereof may correspond to previously encoded neural network parameters or bins thereof
  • decoding of different neural network parameters and decoding of bins associated with different context models may correspond to encoding of different neural network parameters and encoding of bins associated with different context models
  • decoding of neural network parameters associated with different layers of the neural network may correspond to encoding of neural network parameters associated with different layers of the neural network.
  • the encoded representation comprises separate encoded representations of separate probability estimation parameters associated with different neural network parameters and/or of separate probability estimation parameters associated with different context models and/or of separate probability estimation parameters associated with different layers of the neural network.
  • the encoded representation comprises a flag indication which mapping rule out of a plurality of mapping rules is to be used for mapping an encoded value representing one or more probability estimation parameters, e.g. an encoded index value q, onto one or more probability estimation parameters, e.g.
  • Fig. 1 shows an example for a graph representation of a feed forward neural network
  • Fig. 2 shows a block schematic diagram of a decoder according to embodiments of the invention
  • Fig. 3 shows a schematic representation of an example of a selection of probability estimation parameters according to embodiments of the invention
  • Fig. 4 shows a schematic representation of an example of a decoder selection entity according to embodiments of the invention
  • Fig. 5 shows a schematic representation of an example of an encoded bitstream and index values describing a probability estimation parameter values according to embodiments of the invention.
  • Fig. 6 shows a schematic block diagram of methods according to embodiments of the invention.
  • Fig. 2 shows a block schematic diagram of a decoder according to embodiments of the invention.
  • Decoder 200 comprises a context dependent arithmetic decoding unit 210, a probability estimator 220 and probability estimation parameter values 230.
  • the decoder 200 comprises a bitstream disassembly unit 240 and a parameter reassembly unit
  • the decoder 200 may be configured to receive an encoded bitstream 202.
  • the encoded bitstream 202 may comprise an information about a plurality of neural network parameters.
  • the optional bitstream disassembly unit 240 may be configured to convert the encoded bitstream 202 in a processible information for the context dependent arithmetic decoding unit 210. This functionality may be provided by the decoding unit 210, therefore disassembly unit 240 is illustrated only for explanatory purposes.
  • the disassembly unit may be configured to disassemble the encoded bitstream 202 in one part comprising an information about encoded neural network parameters and another part, e.g. flags, indicating a start of the bitstream and/or bits for error correction or other overhead.
  • the decoding unit 210 may be configured to decode the encoded neural network parameters in order to provide a plurality of, for example decoded, neural network parameters 204. In order to decode the neural network parameters, the decoding unit 210 is configured to receive a probabiiity estimate from the probability estimator 220.
  • the neural network parameters may be encoded in the bitstream 202 as a sequence of bins. One bin or a plurality of bins may represent a neural network parameter. The bins may, for example, be associated with a context, or in other words a probabiiity model.
  • the probability estimate may indicate a probability of a bin to have a certain value, e.g. 1 or 0. The probability estimate may be determined depending on a context of the bin, or in other words its probability model.
  • the probability estimator 220 comprises probability estimation parameters, in order to determine the probability estimate.
  • the decoder or the context dependent arithmetic decoding unit 210 is configured to use different probability estimation parameters values for a decoding of different neural network parameters. Consequently, stochastic, individual characteristics of neural network parameters may be taken into account by adapting the probability estimation parameters.
  • the decoder or the context dependent arithmetic decoding unit 210 may be configured to use different probability estimation parameter values for a decoding of bins associated with different context models. Individual bins, or sets of bins may be associated with a context model.
  • the context models may be adapted according to, for example recently, decoded neural network parameters or bins thereof. As a result, for individual bins, or individual context models associated with bins, the parametrization of the probability estimator 220 may be adapted.
  • the optional parameter reassembly unit 250 may be configured to reassemble decoded bins to neural network parameters and/or may be configured to interpret the decoded entities that are provided by the decoding unit 210 in order to provide the plurality of neural network parameters 204.
  • the probability estimator 220 may receive feedback information from the output of the decoding unit 210 and/or from the optional parameter reassembly unit 250.
  • decoder 200 may alternatively, or in addition be configured to use different probability estimation parameter values for a decoding of neural network parameters associated with different layers of the neural network.
  • the information of the layer of a certain neural network parameter may be encoded in the bitstream, and may trigger a changing of probability estimation parameters values 230.
  • Fig. 3 shows a schematic representation of an example of a selection of probability estimation parameters according to embodiments of the invention.
  • Fig. 3 shows, as an example, probability estimation parameters of approaches 1 - 4 and probability estimation parameters for updating the context model.
  • the decoder may choose one or more probability estimation parameters, e.g. parameters 310 and/or 320 from a base set 300. As shown with parameters 310 and 320 these parameters may be a true subset of the base set 300.
  • the decoder may be configured to use not only different probability estimation parameter values but also different probability estimation parameters.
  • the decoder may use any of the approaches 1 to 4, and therefore only need a subset of probability estimation parameters of the base set 300.
  • the decoder may not only choose probability estimation parameters but also their values. For example, in a first selection step, the decoder may choose parameters 310. In the next step the decoder may as well choose from different subsets of probability estimation parameter values for these probability estimation parameters. In other words, for each probability estimation parameter, there may be a plurality of sets of admissible probability estimation parameter values and the decoder may choose probability estimation parameters and corresponding values.
  • Fig. 4 shows a schematic representation of an example of a decoder selection entity according to embodiments of the invention.
  • the decoder may be configured to choose one or more probability estimation parameters from different sets 410, 420 of usable parameter values, wherein, as an example, set 410 comprises a subset 412 (parameters according to approach 3) and a subset 414.
  • set 410 comprises a subset 412 (parameters according to approach 3) and a subset 414.
  • the decoder may choose from usable sets of tuples of parameters values 430, 440 and/or from different mapping rules, here shown as an example in the form of the Tables 5, 450, and 7, 460.
  • Choice of probability estimation parameters values, tuples or mappings may be performed based on a quantization mode. It is to be noted, that different sets of usable parameters values may comprise the same probability estimation parameters but with different values.
  • an adaption may be performed with regard to the parameters used for encoding/decoding as well as with regard to the respective values of the parameters.
  • a selection of the values may be performed, e.g. a decoder decision between set 412 and 414, wherein the selection of parameters is not limited to the shown parameters of approach 3.
  • the decoder may selectively choose one or more probability estimation parameters from a first set 410 of useable parameter values or from a first set of useable tuples 430 of parameter values in case that a uniform quantization of the one or more probability estimation parameters is used.
  • the decoder may choose between sets 412 and 414 in case of the uniform quantization.
  • the decoder may selectively choose one or more probability estimation parameters from a second set 420 of useable parameter values or from a second set of useable tuples 440 of parameter values in case that a variable quantization of the one or more probability estimation parameters is used.
  • the decoder may use a first mapping rule 460 mapping an encoded value representing one or more probability estimation parameters onto one or more probability estimation parameters in case that a uniform quantization of the one or more probability estimation parameters is used, and may use a second mapping rule 450, mapping an encoded value representing one or more probability estimation parameters onto one or more probability estimation parameters in case that a variable quantization of the one or more probability estimation parameters is used.
  • First and second sets, tuples and mapping rules may be different from each other. It is to be noted that the different sets, tuples and mapping rules may comprise different probability estimation parameters, e.g. to perform calculations according to the different approaches 1 -4 or may comprise the same probability estimation parameters but with different values.
  • the decoder may optionally choose a calculation routine first and then its parametrization, or in other words its values.
  • the decoder may only choose the probability estimation parameters values according to the quantization mode, e.g. choosing between sets 412 and 414 or, for example between tuples 430, 440, describing the same probability estimation parameters but with different values.
  • useable parameter values of the second set of useable parameter values or of the second mapping rule may allow for a faster adaptation of a probability estimate, e.g. to a change of bin value frequencies, than useable parameter values of the first set of useable parameter values, or of the first mapping rule.
  • useable tuples of parameter values of the second set of useable tuples of parameter values, or of the second mapping rule may allow for a faster adaptation of a probability estimate, e.g. to a change of bin value frequencies, than useable tuples of parameter values of the first set of useable tuples of parameter values, or of the first mapping rule.
  • the second set of useable parameter values, or the second mapping rule may comprise a useable parameter value which allows for a faster adaptation of a probability estimate, e.g. to a change of bin value frequencies, than useable parameter values, or even than all useable parameter values, of the first set of useable parameter values, or of the first mapping rule.
  • the second set of useable tuples of parameter values comprises a useable tuple of parameter values which allows for a faster adaptation of a probability estimate, e.g. to a change of bin value frequencies, than useable tuples, or even than all useable tuples, of parameter values of the first set of useable tuples of parameter values.
  • sets 410 and 420 may be useable, or for example allowable, parameter values
  • tuples 430, 440 may be useable, or for example allowable, tuples.
  • Decoder choice of such sets or tuples may be performed based on, or in dependence on a number of parameters of a layer of the neural network or in dependence on a number of neural network parameters to be decoded using the chosen one or more probability estimation parameters, or in dependence on a number of elements of a layer parameter.
  • usage of different mapping rules 450 ,460 may be performed by the decoder in dependence on a number of neural network parameters to be decoded using the chosen one or more probability estimation parameters, or in dependence on a number of elements of a layer parameter.
  • the decoder may be configured to selectively choose one or more probability estimation parameters from a second set of useable parameter values or from a second set of useable tuples of parameter values if the number of parameters of a layer of the neural network is above the threshold value or if the number of neural network parameters to be decoded using the chosen one or more probability estimation parameters is above the threshold value, or if the number of elements of the layer parameter is above the threshold value.
  • the decoder is configured to selectively use a first mapping rule mapping an encoded value representing one or more probability estimation parameters onto one or more probability estimation parameters if the number of parameters of a layer of the neural network is below a threshold value or if the number of neural network parameters to be decoded using the chosen one or more probability estimation parameters is below a threshold value, or if the number of elements of the layer parameter is below a threshold value
  • the decoder may be configured to selectively use a second mapping rule mapping an encoded value representing one or more probability estimation parameters onto one or more probability estimation parameters if the number of parameters of a layer of the neural network is above the threshold value or if the number of neural network parameters to be decoded using the chosen one or more probability estimation parameters is above the threshold value, or if the number of elements of the layer parameter is above the threshold value;
  • the second set of useable parameter values may comprise more useable parameter values than the first set of useable parameter values
  • the second set of useable tuples of parameter values may comprise more useable
  • the second mapping rule may be different from the first mapping rule.
  • Fig. 5 shows a schematic representation of an example of an encoded bitstream and index values describing probability estimation parameter values according to embodiments of the invention.
  • the bitstream 202 comprises a signaling in the form of a flag indication F or in other words a flag F.
  • the signaling may be transmitted in any suitable way.
  • the decoder may evaluate flag F in order to determine from which set of useable parameter values or from which set of useable tuples of parameter values the one or more probability estimation parameters are elected.
  • the decoder may be configured to evaluate the signaling indication which mapping rule out of a plurality of mapping rules should be used to map an encoded value representing one or more probability estimation parameters onto one or more probability estimation parameters.
  • the decoder's choice of sets, tuples and or mappings according to Fig. 4 may be based on the signaling, e.g. in the form of an encoded flag F.
  • Optional bitstream disassembly unit 240 may for example, disassemble bitstream 202 in the flag for indicating the set, tuple and/or mapping to be used for decoding and in an information about the probability estimation parameter values.
  • the index values q may be an encoded representation of one or more probability estimation parameters.
  • the decoder may be configured to decode the one or more index values q, for example using the signaling.
  • the one or more index values q may be associated with one or more context models c q i. and the decoder may be configured to decode the index values q, using the context models c q i.
  • the index values may be represented by one or more bins.
  • the decoder may be configured to decode the first bin and the optional additional bins. Any of these bins may be associated with a context, for example individually.
  • context c qi of index value q may be associated with the first bin of the index value and the additional bins may be decoded with a fixed length per bin.
  • the decoder may be configured to decode the one or more index values using a unary code decoding, or using a truncated unary code decoding, or using a variable length code decoding, wherein, for example, the code lengths are chosen according to probabilities of occurrence of different index values.
  • any suitable coding technique may be applied, for example individually for different index values, providing improved flexibility and coding efficiency.
  • the decoder may be configured to vary a number of bins or a maximum number of bins used for decoding the one or more probability estimation parameters in dependence on a quantization mode used for quantizing the one or more probability estimation parameters and/or in dependence on a number of parameters of a layer of the neural network, or in dependence on a number of neural network parameters to be decoded using the one or more probability estimation parameters, or in dependence on a number of elements of a layer parameter.
  • the decoder may be configured to switch between different sets of usable parameter values associated with the one or more probability estimation parameters, or between different sets of tuples of useable parameter values associated with a plurality of probability estimation parameters, or between different mapping rules for mapping an encoded value representing one or more probability estimation parameters onto one or more probability estimation parameters.
  • the decoder may switch between sets 410 (and/or between sets 412, 414) and 420 and/or between tuples 430, 440 and/or different mappings 450, 460 as shown in Fig. 4.
  • the beforementioned variation of number of bins or maximum number of bins may be performed by the decoder in accordance with a switching between different sets, tuples and/or mapping rules.
  • encoded bitstream 202 may be an encoded representation of weight parameters of a neural network, comprising a plurality of encoded weight parameters of the neural network in the form of the integer multiples n and an encoded representation of one or more probability estimation parameters, namely the index values q i .
  • Fig. 6 shows a schematic block diagram of methods according to embodiments of the invention.
  • Fig. 6 shows methods 600, 700 for decoding weight parameters of a neural network.
  • the methods 600 and 700 comprise obtaining 610, 710 a plurality of neural network parameters, e.g., at least one of entries Wi of matrix W, b, ⁇ , ⁇ 2 , ⁇ , ⁇ , and/or ⁇ , of the neural network on the basis of an encoded bitstream, a decoding 620, 720 the neural network parameters of the neural network, e.g., a quantized version thereof, using a context-dependent arithmetic decoding, e.g., using a context-adaptive binary arithmetic decoding (CABAC).
  • CABAC context-adaptive binary arithmetic decoding
  • Methods 600, 700 further comprise obtaining 630, 730 a probability estimate, e.g. P(t) or p k , which may, for example, be associated with a context, for a, optionally arithmetic, decoding of a bin of a number representation of a neural network parameter, e.g. on the basis of one or more previously decoded neural network parameters or bins thereof, using one or more probability estimation parameters, e.g., probability estimator parameters, e.g. N,
  • Method 600 comprises in addition using 640 different probability estimation parameter values for a decoding of different neural network parameters and/or using different probability estimation parameter values for a decoding of bins associated with different context models, e.g. Ck.
  • method 700 comprises in addition using 740 different probability estimation parameter values for a decoding of neural network parameters associated with different layers of the neural network.
  • the parameters W, b, m, ⁇ 2 , g, and b shall collectively be denoted parameters of a layer or layer parameters. They usually need to be signaled in a bitstream (e.g. in an encoded video representation, for example, if the neural network is used in a video decoder). For example, they could be represented as 32 bit floating point numbers or they could, for example, be quantized to an integer representation, also denoted as quantization indices. Note that e is usually not signaled in the bitstream.
  • a particularly efficient approach for encoding such parameters employs a uniform reconstruction quantizer (URQ) where, for example, each value is represented as integer multiple of a so-called quantization step size value.
  • the corresponding floating point number can, for example, be reconstructed by multiplying the integer with the quantization step size, which is usually (but not necessarily) a single floating point number.
  • efficient implementations for neural network inference that is, calculating the output of the neural network for an input
  • each value is, for example, represented as integer multiple of a quantization step size value.
  • each quantizer in the set employs a disjoint set of integer multiples of the quantization step size parameter as applicable reconstruction values, but two or more quantizers may share one or more reconstruction values.
  • the applied quantizer depends, for example, on the values of previous quantization indices in coding order.
  • the corresponding floating point number can, for example, be reconstructed by multiplying the integer with the quantization step size, which is usually, for example, a floating point number which depends on the chosen quantizer.
  • TCQ trellis coded quantization
  • DQ dependent quantization
  • a set of two quantizers is used.
  • the first quantizer employs, for example, all even multiples of the quantization step size including zero, and the second quantizer employs all the even multiples of the quantization step size including zero.
  • Entropy Coding and Probability Estimation (examples, details are optional)
  • the quantization indices that are output, for example, by the quantization method are then entropy coded using a suitable entropy coding method.
  • a particularly suitable entropy coding method for encoding such quantization indices is Context-based Adaptive Binary Arithmetic Coding, also denoted as CABAC.
  • CABAC Context-based Adaptive Binary Arithmetic Coding
  • each quantization index is, for example, decomposed into a sequence of binary decisions, so-called bins.
  • each bin is associated with a probability model, also denoted as context model, which models the statistics of the associated bins, for example, using a probability estimation method.
  • a probability model also denoted as context model, which models the statistics of the associated bins, for example, using a probability estimation method.
  • a probability estimator is an apparatus, that models the probability P(t) for a bin being equal to x, where x e ⁇ 0,1 ⁇ , for example, based on already coded bins associated with the probability estimator.
  • probability estimators have several parameters, denoted as probability estimator parameters or estimator parameters (or also as probability estimation parameters), that affect the probability estimates, e.g. the adaptation rate.
  • those estimator parameters are, for example, chosen globally, depending on the application scenario, e.g. encoding of neural network parameters.
  • each neural network parameter applies the same set of estimator parameters.
  • the compression efficiency can be improved by selecting optimized estimator parameters for a current neural network parameter.
  • the basic idea is to select suitable estimator parameters out of a set of parameters, which are then signaled to the decoder.
  • each context model c k one or more state variables are maintained with N 3 1.
  • Each state variable is implemented, for example, as signed integer value and represents, for example, a probability value
  • the probability estimate p k of a context model c k shall be defined, for example, as weighted sum of the probability values of all state variables of the context model.
  • State variables shall preferably but not necessarily have the following properties:
  • negative state variables may, for example, correspond to ⁇ 0.5.
  • P(-) for each state variable of each context model.
  • a state representation that is used in neural network compression can be achieved with the following equation: is a weighting factor, ⁇ is a parameter with 0 ⁇ ⁇ ⁇ 1.
  • all state variables are optionally initialized with sane values, denoted as initV , that may, for example, be optimized to the compression application.
  • a probability estimate is derived from the state variables of a context model.
  • Three alternative approaches are presented in the following as examples. Approach 1 yields more accurate results than approach 2 and approach 3, but also has a higher computational complexity.
  • each state variable of a context model is converted into a probability value .
  • the probability estimate p k is derived as weighted sum of the probability values
  • Step 1
  • a lookup table LUT1 is employed for converting a state variable into the corresponding probability value , for example according to Eq. (1 ).
  • LUT 1 is a lookup table containing probability values. is a weighting factor that adapts to the size of LUT 1.
  • the probability estimate p k is derived from the probability values , for example according to: f is a weighting factor that controls the influence of the individual state variables.
  • a weighted sum s k of the state variables is derived, for example, according to: is a weighting factor that controls the influence of each state variable.
  • the probability estimate p k is derived from the weighted sum of state variables for example according to:
  • LUT 2 is a lookup table containing probability estimates.
  • ⁇ k is a weighting factor that adapts s k to the size of LUT 2.
  • the weighted sum s k of the state variables is derived, for example, as in approach 2.
  • the probability estimate p k is derived from the weighted sum of state variables s k , for example according to:
  • Li/7’2 is a lookup table containing probability estimates.
  • a further approach uses a linear relation between the state values and the probability P(x, i, k).
  • the derivation of the probability estimate is, for example, using the approach of equation (2).
  • An example of approach 4 is the probability estimation scheme used in the current draft of Versatile Video Coding (VVC).
  • the method of approach 3 is used, for example, with for all k.
  • the look-up table containing the probability estimates is, for example,:
  • LUT 2 ⁇ 0.5000,0.4087,0.3568,0.3116,0.2721,0.2375,0.2074,0.1811, 0.1581,0.1381,0.1206,0.1053,0.0919,0.0803,0.0701,0.0612, 0.0534,0.0466,0.0407,0.0356,0.0310,0.0271,0.0237,0.0207, 0.0180,0.0158,0.0138,0,0120,0.0105,0.0092,0.0080,0.0070 ⁇ Update of state variables (examples, details are optional)
  • one or more state variables of a context model may be updated in order to track the statistical behaviour of the symbol sequence.
  • the update is, for example, carried out as follows: If symbol to be encoded is 1. If symbol to be encoded is 0.
  • A is a lookup table storing, for example, integer values. and are weighting factors that control, for example, the update ‘agility’.
  • the factors can be written, for example, according to t where shf also denoted as adaptation parameter.
  • z is an offset that ensures, for example, that look table A is accessed only with non-negative values.
  • lookup table A can, for example, be chosen so that stays in a particular given interval.
  • look-up A approximate, for example, an update function.
  • the estimation method of WC applies update functions for the state update and uses bit shifts, which, for example, determine the ‘agility’ of the update. This corresponds, for example, to the adaptation parameters described above.
  • the invention (see below) can be applied to those in the same manner.
  • the look-up table A is, for example:
  • A ⁇ 157, 143, 129, 115, 101 , 87, 73, 59, 45, 35, 29, 23, 17, 13, 9, 5, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 0 ⁇ .
  • probability estimator parameters i.e. N, initV and any other parameter related to the probability estimator (context model) shall be collectively denoted as probability estimator parameters or estimator parameters (or probability estimation parameters).
  • the probability estimation can, for example, be improved, if the parameters are chosen individually for each parameter or a subset of parameters of a layer (i.e. W, b, m, ⁇ 2 , g, and b) and/or context model c k .
  • the estimator parameter to be used is, for example, determined among the parameters of a set of parameters, which can, for example, be the base set or any subset of the base set.
  • Each parameter of the set may, for example, be associated with an integer index q.
  • one parameter of the set may be denoted as default parameter.
  • the default parameter is, for example, associated with an integer index equal to zero.
  • the index associated with the chosen estimator parameter is then, for example, signaled to the decoder.
  • the index q e [0, q MAX ] to be encoded is, for example, decomposed into a sequence of bins, which are then encoded.
  • Each bin may, for example, be coded using a context model or using a fixed probability.
  • the encoding procedure may, for example, be according to one of the following schemes:
  • a series of bins is encoded, which denote, for example, the index of the chosen parameter minus one (q - 1), indexMinusOne.
  • the number of bins encoded for index is, for example, equal to ⁇ log 2 (setLength - 1)1, where setLength , denotes the number of elements of the set.
  • This procedure applies a truncated unary code, which is, for example, identical to the unary code used in encoding method 2. , except for the case where the index to encode q is equal to q MAX .
  • the index to encode q is equal to q MAX .
  • the bin greaterThan_(q MAX - 1) no further bins are encoded.
  • the value of q is inferred to be q MAX , if greaterThan_ ( q MAX - 1) is equal to one.
  • This procedure uses a variable length code, where the code lengths are chosen according to the probability of occurrence of a symbol, for example a Huffman code.
  • an estimator applies, for example, a base set of adaptation parameters, which are N-tuple of adaptation parameters . Then a subset of the base set is chosen. One parameter out of the subset is signaled.
  • the configuration is, for example, equal to the previous preferred embodiment, but an estimator is used, which is configured such is identical to the estimator used the current draft of MPEG-7 part 17 standard for compression of neural networks for multimedia content description and analysis and the base set contains, for example, the following 28 pairs for
  • the subset of size 3 is defined and ordered, for example, such that the indexes q according to Table 2 are assigned, for example, in the case all parameters of a layer are quantized with DQ.
  • one parameter out of the subset is signaled, for example, by encoding q according to encoding scheme 1.
  • the bin useNotDefault is encoded using a context model and all other bins are encoded with a fixed length of one bit per bin.
  • the configuration is identical to the previous preferred embodiment, except for the assigned adaptation parameter pairs and the size of the chosen subset (Table 3), which is equal to 5.
  • Table 11 Preferred subset of adaptation parameters for set size 5
  • the configuration is identical to the previous preferred embodiment, except for assigned adaptation parameter pairs (Table 4):
  • Table 12 Second preferred subset of adaptation parameters for set size 5
  • the configuration is identical to the previous preferred embodiment, except for the assigned adaptation parameter pairs and the size of the chosen subset (Table 5), which is equal to 9.
  • Table 13 Preferred subset of adaptation parameters for set size 9
  • the configuration is identical to the previous preferred embodiment, except for the assigned adaptation parameter pairs (Table 6).
  • the configuration is identical to the previous preferred embodiment, except for assigned adaptation parameter pairs (Table 7), the size of the chosen subset (5), and the used quantization method, which uses URQ:
  • Table 15 Preferred subset of adaptation parameters for set size 5 and URQ
  • an estimator is used, which is configured such that it is identical to the estimator used the current draft of MPEG-7 part 17 standard for compression of neural networks for multimedia content description and analysis and the base set of Table 1 is used. This is denoted as base configuration.
  • base configuration Whenever a layer parameter is quantized with DQ, the subset (of size 9) of parameter pairs in Table 5 is applied. If a layer parameter is quantized with URQ the subset in Table 8 is used.
  • Table 16 Preferred subset of adaptation parameters for set size 9
  • the subset with size 3 of parameter pairs for example, in Table 2, denoted as first subset
  • the subset with size 9 for example, in Table 5, denoted as second subset
  • the configuration is identical to the previous preferred embodiment, but instead of using a threshold, a flag (for example, useSecondSubset) is encoded, which determines, for example, the subset to be used. For example, if the flag is equal to zero, the first subset is used. If the flag is equal to one, the second subset is used.
  • aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.
  • Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps may be executed by such an apparatus.
  • embodiments of the invention can be implemented in hardware or in software.
  • the implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.
  • Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
  • embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer.
  • the program code may for example be stored on a machine readable carrier.
  • inventions comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
  • an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
  • a further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein.
  • the data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitionary.
  • a further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein.
  • the data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.
  • a further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
  • a processing means for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
  • a further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
  • a further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver.
  • the receiver may, for example, be a computer, a mobile device, a memory device or the like.
  • the apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.
  • a programmable logic device for example a field programmable gate array
  • a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein.
  • the methods are preferably performed by any hardware apparatus.
  • the apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.
  • the apparatus described herein, or any components of the apparatus described herein, may be implemented at least partially in hardware and/or in software.
  • the methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.
  • the methods described herein, or any components of the apparatus described herein, may be performed at least partially by hardware and/or by software.
  • the present disclosure describes, in general, explicitly or implicitly, features usable in a video encoder (apparatus for providing an encoded representation of an input video signal) and in a video decoder (apparatus for providing a decoded representation of a video signal on the basis of an encoded representation), and in an audio encoder and in an audio decoder.
  • a video encoder apparatus for providing an encoded representation of an input video signal
  • a video decoder apparatus for providing a decoded representation of a video signal on the basis of an encoded representation
  • audio encoder and in an audio decoder any of the features described herein can be used in the context of a video encoder and in the context of a video decoder and in the context of an audio encoder and in the context of an audio decoder.
  • features and functionalities disclosed herein relating to a method can also be used in an apparatus (configured to perform such functionality).
  • any features and functionalities disclosed herein with respect to an apparatus can also be used in a corresponding method.
  • the methods disclosed herein can be supplemented by any of the features and functionalities described with respect to the apparatuses.
  • any of the features and functionalities described herein can be implemented in hardware or in software, or using a combination of hardware and software, as will be described in the section “implementation alternatives”.
  • any of the features and syntax elements described herein can optionally be introduced into a video bit stream, both individually and taken in combination.
  • a context derivation in a decoder may be analog to a context derivation in an encoder, wherein decoded valued may take the role of values to be encoded.
  • decoders are designed such that the context used in the decoder corresponds to the context used in the encoder, to keep the encoder and the decoder in synchronism.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Neurology (AREA)
  • Algebra (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
PCT/EP2021/059594 2020-04-14 2021-04-13 Decoder for decoding weight parameters of a neural network, encoder, methods and encoded representation using probability estimation parameters Ceased WO2021209470A1 (en)

Priority Applications (6)

Application Number Priority Date Filing Date Title
KR1020227039550A KR20230005868A (ko) 2020-04-14 2021-04-13 신경망의 가중치 파라미터들을 디코딩하기 위한 디코더, 인코더, 방법들 및 확률 추정 파라미터들을 사용하는 인코딩된 표현
EP21717116.4A EP4136756A1 (en) 2020-04-14 2021-04-13 Decoder for decoding weight parameters of a neural network, encoder, methods and encoded representation using probability estimation parameters
CN202180042614.4A CN115699585A (zh) 2020-04-14 2021-04-13 用于解码神经网络的权重参数的解码器、编码器、方法和使用概率估计参数的编码表示
JP2022562944A JP7695954B2 (ja) 2020-04-14 2021-04-13 ニューラルネットワークの重みパラメーターを復号化するデコーダー、エンコーダー、方法、及び確率推定パラメーターを使用する符号化表現
US18/046,227 US20230141029A1 (en) 2020-04-14 2022-10-13 Decoder for decoding weight parameters of a neural network, encoder, methods and encoded representation using probability estimation parameters
JP2025030998A JP7817477B2 (ja) 2020-04-14 2025-02-28 ニューラルネットワークの重みパラメーターを復号化するデコーダー、エンコーダー、方法、及び確率推定パラメーターを使用する符号化表現

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP20169498 2020-04-14
EP20169498.1 2020-04-14

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/046,227 Continuation US20230141029A1 (en) 2020-04-14 2022-10-13 Decoder for decoding weight parameters of a neural network, encoder, methods and encoded representation using probability estimation parameters

Publications (1)

Publication Number Publication Date
WO2021209470A1 true WO2021209470A1 (en) 2021-10-21

Family

ID=70390747

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2021/059594 Ceased WO2021209470A1 (en) 2020-04-14 2021-04-13 Decoder for decoding weight parameters of a neural network, encoder, methods and encoded representation using probability estimation parameters

Country Status (6)

Country Link
US (1) US20230141029A1 (https=)
EP (1) EP4136756A1 (https=)
JP (2) JP7695954B2 (https=)
KR (1) KR20230005868A (https=)
CN (1) CN115699585A (https=)
WO (1) WO2021209470A1 (https=)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115699020A (zh) * 2020-06-05 2023-02-03 华为技术有限公司 用于神经网络的量化
US20230050279A1 (en) * 2021-08-12 2023-02-16 Taiwan Semiconductor Manufacturing Company, Ltd. Integrated circuit and method of operating same
CN121488473A (zh) * 2023-06-30 2026-02-06 抖音视界有限公司 用于可视数据处理的方法、装置和介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9088796B2 (en) * 2011-11-07 2015-07-21 Sharp Kabushiki Kaisha Video decoder with enhanced CABAC decoding
JP6159240B2 (ja) * 2013-12-09 2017-07-05 日本電信電話株式会社 二値算術符号化装置、二値算術符号化方法及び二値算術符号化プログラム
US10225555B2 (en) * 2015-05-19 2019-03-05 Mediatek Inc. Method and apparatus for multi-table based context adaptive binary arithmetic coding
CN107736027B (zh) * 2015-06-12 2021-06-01 松下知识产权经营株式会社 图像编码方法、图像解码方法、图像编码装置及图像解码装置
EP3562162A1 (en) * 2018-04-27 2019-10-30 InterDigital VC Holdings, Inc. Method and apparatus for video encoding and decoding based on neural network implementation of cabac
CN112689960B (zh) * 2018-07-06 2025-10-17 弗劳恩霍夫应用研究促进协会 算术编码器、算术解码器、视频编码器、视频解码器、编码方法、解码方法和计算机程序
JP2020022145A (ja) * 2018-08-03 2020-02-06 日本放送協会 符号化装置、復号装置、学習装置及びプログラム

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
ANONYMOUS: "Context-adaptive binary arithmetic coding", INTERNET CITATION, 14 November 2018 (2018-11-14), XP002795705, Retrieved from the Internet <URL:https://en.wikipedia.org/w/index.php?title=Context-adaptive_binary_arithmetic_coding&oldid=868799469> [retrieved on 20191118] *
See also references of EP4136756A1
SHARAN CHETLUR ET AL.: "cuDNN: Efficient Primitives for Deep Learning", ARXIV: 1410.0759, 2014
WIEDEMANN SIMON ET AL: "DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks", 27 July 2019 (2019-07-27), pages 1 - 18, XP055815074, Retrieved from the Internet <URL:https://arxiv.org/pdf/1907.11900.pdf> [retrieved on 20210617] *

Also Published As

Publication number Publication date
JP2023522887A (ja) 2023-06-01
EP4136756A1 (en) 2023-02-22
JP2025081713A (ja) 2025-05-27
JP7695954B2 (ja) 2025-06-19
US20230141029A1 (en) 2023-05-11
JP7817477B2 (ja) 2026-02-18
CN115699585A (zh) 2023-02-03
KR20230005868A (ko) 2023-01-10

Similar Documents

Publication Publication Date Title
US20230141029A1 (en) Decoder for decoding weight parameters of a neural network, encoder, methods and encoded representation using probability estimation parameters
US20220004844A1 (en) Methods and apparatuses for compressing parameters of neural networks
US20250343764A1 (en) Concepts for Coding Neural Networks Parameters
US8736467B2 (en) Adaptive binarization for arithmetic coding
TWI827010B (zh) 算術編碼器、算術解碼器、視訊編碼器、視訊解碼器、編碼方法、解碼方法及電腦程式
US11265561B2 (en) Method and apparatus for range derivation in context adaptive binary arithmetic coding
CN110720222B (zh) 用于数字数据压缩的方法和设备
US20240046100A1 (en) Apparatus, method and computer program for decoding neural network parameters and apparatus, method and computer program for encoding neural network parameters using an update model
WO2016025281A1 (en) Method for coding pulse vectors using statistical properties
WO2016025283A1 (en) Method for coding pulse vectors using statistical properties
US20240364362A1 (en) Concepts for encoding and decoding neural network parameters
JP2026071331A (ja) ニューラルネットワークの重みパラメーターを復号化するデコーダー、エンコーダー、方法、及び確率推定パラメーターを使用する符号化表現
EP3180863A1 (en) Method for coding pulse vectors using statistical properties
Cui et al. An optimized probability estimation model for binary arithmetic coding
Tuan et al. Index Assignment Optimization for Digital Communication Systems Using M-ary Modulation Schemes
TW202606302A (zh) 算術編碼器、算術解碼器、視訊編碼器、視訊解碼器、編碼方法、解碼方法和電腦程式
WO2025181171A1 (en) Apparatus and method for refined probability estimation of transform coefficients for video coding
JP2016521927A (ja) 特に「レンジコーダ」又は算術圧縮による，特に圧縮画像を符号化する方法
WO2006134521A1 (en) Adaptive encoding and decoding of a stream of signal values

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21717116

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
ENP Entry into the national phase

Ref document number: 2022562944

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 202237058813

Country of ref document: IN

ENP Entry into the national phase

Ref document number: 20227039550

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021717116

Country of ref document: EP

Effective date: 20221114

WWD Wipo information: divisional of initial pct application

Ref document number: 202538121389

Country of ref document: IN

Ref document number: 202538121392

Country of ref document: IN

Ref document number: 202538121393

Country of ref document: IN

Ref document number: 202538121394

Country of ref document: IN

WWP Wipo information: published in national office

Ref document number: 202538121394

Country of ref document: IN

Ref document number: 202538121393

Country of ref document: IN

Ref document number: 202538121392

Country of ref document: IN

Ref document number: 202538121389

Country of ref document: IN