WO2023051335A1 - 数据编码方法、数据解码方法以及数据处理装置 - Google Patents

数据编码方法、数据解码方法以及数据处理装置 Download PDF

Info

Publication number
WO2023051335A1
WO2023051335A1 PCT/CN2022/120095 CN2022120095W WO2023051335A1 WO 2023051335 A1 WO2023051335 A1 WO 2023051335A1 CN 2022120095 W CN2022120095 W CN 2022120095W WO 2023051335 A1 WO2023051335 A1 WO 2023051335A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
reversible flow
auxiliary
parameter
layer
Prior art date
Application number
PCT/CN2022/120095
Other languages
English (en)
French (fr)
Inventor
张世枫
康宁
里德尔·汤姆
李震国
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023051335A1 publication Critical patent/WO2023051335A1/zh

Links

Images

Classifications

    • HELECTRICITY
    • H03ELECTRONIC CIRCUITRY
    • H03MCODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Definitions

  • the embodiments of the present application relate to the field of artificial intelligence (AI), and in particular, to a data encoding method, a data decoding method, and a data processing device.
  • AI artificial intelligence
  • Lossless compression is a technique for compressing data.
  • the length of the compressed data obtained after lossless compression of the original data is smaller than the length of the original data, and the compressed data can be completely restored to the original data through decompression.
  • the core of lossless compression is to find the internal distribution law of the original data, and compress according to the internal law of the original data. For example, if the frequency of the letter e in the original data is much higher than that of the letter z, then e is represented by shorter bits, so that the aforementioned original data can be represented by compressed data with a shorter length.
  • a commonly used model that uses AI technology to find the internal distribution of data is flow-based models.
  • the flow model transforms the original data into hidden variables, and the hidden variables have a certain probability distribution, and can be compressed according to the probability distribution of the hidden variables.
  • decompression process decompression is performed according to the probability distribution of the aforementioned hidden variables, and the original data is restored through the aforementioned flow model.
  • an integer discrete flow (IDF) model is generally used.
  • the input and output of the integer flow model need to be integers, and integer addition and subtraction operations are used to avoid floating point errors, so as to ensure that the input and output of the flow model are reversible.
  • the model representation ability is poor, and the probability distribution estimation of the model is poor, which in turn leads to low compression efficiency of lossless compression.
  • Embodiments of the present application provide a data encoding method, a data decoding method, and a data processing device, which are used to improve the compression efficiency of lossless compression.
  • the present application provides a data encoding method, which is used in a data processing device as a compression end.
  • the data processing device obtains the data to be coded; then, uses the reversible flow model to perform forward transformation processing on the data to be coded to obtain hidden variables; then, performs code processing to the hidden variables to obtain the corresponding encoded data.
  • the reversible flow model is a new flow model proposed by this application.
  • the reversible flow model includes a target reversible flow layer, and the operation corresponding to the target reversible flow layer is a reversible operation.
  • the target reversible flow layer includes model parameters, and the model parameters are used to constrain the auxiliary variables generated during the forward transformation process, and the operations corresponding to the target reversible flow layer include multiplication and division operations determined based on the model parameters , the auxiliary variable being the increment of the product of the multiplication operation or the remainder produced by the division operation.
  • model parameters in the target reversible flow layer may be pre-configured parameters, or may be parameters calculated by the target reversible flow layer based on a certain algorithm, which are not specifically limited here.
  • the data to be encoded may be original data such as images, videos, or texts. At this time, the data to be encoded is generally integer data.
  • the data to be encoded may also be preprocessed data based on original data such as images, videos, or texts. At this time, the data to be encoded may be a fixed-point number with a certain precision.
  • the target reversible flow layer since the reversible flow model used to perform forward transformation processing on the data to be encoded includes a target reversible flow layer, the target reversible flow layer not only includes multiplication and division operations determined based on the model parameters, but the target reversible flow layer
  • the model parameters in can also constrain the auxiliary variable generated during the forward transformation process, the auxiliary variable being the increment of the product of the multiplication operation or the remainder generated by the division operation. That is to say, this application can not only use multiplication, but also realize reversibility by using auxiliary variables when performing division. Therefore, compared with the integer flow model in the traditional technology, the model representation ability is improved, which is conducive to improving the probability distribution of the model
  • the estimation accuracy rate can improve the compression efficiency of lossless compression.
  • the reversible flow model includes N reversible flow layers, the N reversible flow layers include the target reversible flow layer, and N is an integer greater than or equal to 1.
  • the target reversible flow layer is used to perform forward transformation processing on the first data input to the target reversible flow layer to obtain second data output from the target reversible flow layer, the first data is the multiplier of the multiplication operation, and the second data is output from the target reversible flow layer.
  • the second data is the integer quotient of the division operation.
  • the first data is used as the multiplier of the multiplication operation, and another multiplier of the multiplication operation is determined by the model parameters; because , the auxiliary variable can be used as an increment of the product of the multiplication operation, therefore, the intermediate data can be obtained based on the product of the aforementioned first data and another multiplier, plus the aforementioned auxiliary variable.
  • the second data is used as an integer quotient of the division operation, the divisor of the division operation is intermediate data, and the dividend is determined by the model parameters. Therefore, after the first data is input into the target reversible flow layer, the second data will be output after multiplication and division operations, and the second data is the output of the target reversible flow layer.
  • the first data is the data to be encoded or the data output after being processed by at least one reversible flow layer
  • the second data is the hidden variable or the output data after the second data is processed by at least one reversible flow layer is Cain variable.
  • the aforementioned at least one reversible flow layer refers to n reversible flow layers, wherein n is an integer greater than or equal to 1 and less than N, however, N is an integer greater than 1 at this time.
  • the target reversible flow layer is the first reversible flow layer in the reversible flow model
  • the data to be encoded input to the reversible flow model is the first data.
  • the target reversible flow layer After forward transformation, the target reversible flow layer Output the second data.
  • the second data output by the target reversible flow layer is a hidden variable. If there are multiple reversible flow layers in the reversible flow model, the second data output by the target reversible flow layer will be used as the input of other reversible flow layers, so that other reversible flow layers can process the second data to
  • the output data after being processed by at least one reversible flow layer is the latent variable.
  • the output of the data to be encoded input to the reversible flow model after being processed by at least one of the reversible flow layers is the aforementioned first data
  • the first The data is used as the input of the target reversible flow layer
  • the target reversible flow layer outputs the second data. Since the target reversible flow layer is the last reversible flow layer in the reversible flow model, the second data output by the aforementioned target reversible flow layer is a hidden variable.
  • the reversible flow model of the present application has stronger model representation capabilities, which is conducive to improving the estimation accuracy of the probability distribution of the model, thereby improving the compression efficiency of lossless compression.
  • the model parameters include a first parameter and a second parameter, both of the first parameter and the second parameter are positive integers, and the first parameter and the second parameter are not equal; the auxiliary variable A first auxiliary variable is included which is the increment of the product of the multiplication operation.
  • the data processing device performs forward transformation processing on the first data input to the target reversible flow layer, and the process of obtaining the second data output from the target reversible flow layer may specifically be as follows:
  • the intermediate data is used as a divisor and the second parameter is used as a dividend to perform a division and rounding operation to obtain the second data.
  • the data processing device adds the product obtained by multiplying the first data and the first parameter (that is, the first product data) to the first auxiliary variable to obtain the intermediate data; then, the data processing device converts the intermediate The data is used as a divisor and the second parameter is used as a dividend to perform a division operation, and the obtained integer quotient is the second data.
  • the first data is first transformed into intermediate data based on multiplication and addition operations, and then the intermediate data is transformed into second data based on division and rounding operations.
  • the first auxiliary variable participates in the process of determining the intermediate data, that is, the first auxiliary variable is used as the increment of the product of the multiplication operation, the integer quotient of the division and rounding operation based on the intermediate data is the second data, therefore, avoid The problem of irreversible operation caused by only multiplication is avoided, and the problem of poor representation ability of the model caused by only addition and subtraction is also avoided.
  • the auxiliary variable further includes a second auxiliary variable, and the second auxiliary variable is a remainder generated by the division operation.
  • the data processing device will also perform division and remainder operation with the intermediate data as the divisor and the second parameter as the dividend to obtain the second auxiliary variable.
  • the second auxiliary variable is used as an increment of the product of the multiplication operation in the inverse transformation processing corresponding to the forward transformation processing
  • the first auxiliary variable is used as a remainder generated by the division operation in the inverse transformation processing.
  • the target reversible flow model will also generate a second auxiliary variable in the forward transformation process, and the second auxiliary variable is used as the product of the multiplication in the inverse transformation process corresponding to the forward transformation process increment. That is to say, in the inverse transformation process, the data processing device restores the second data to the intermediate data based on the second auxiliary variable, and then restores the intermediate data to the first data. Therefore, in the forward transformation process, the second auxiliary variable is calculated, which is beneficial to restore the second data to the first data.
  • the process of the data processing device determining the first auxiliary variable based on the first parameter may be:
  • the data processing device first obtains the first auxiliary data input into the target reversible flow layer; then, the data processing device determines the first auxiliary variable based on the first auxiliary data and the first parameter.
  • the data processing device will use the first auxiliary data input to the target reversible flow layer in the process of determining the first auxiliary variable based on the first parameter.
  • the first auxiliary data may be a binary code stream.
  • the data processing device may encode and decode the auxiliary data input to the target reversible flow layer based on entropy coding (for example, uniform distribution entropy coding), so as to generate or store the aforementioned auxiliary variables.
  • entropy coding for example, uniform distribution entropy coding
  • the process of the data processing device determining the first auxiliary variable based on the first auxiliary data and the first parameter may be:
  • the first auxiliary data is decoded by using the first uniform distribution to obtain the first auxiliary variable and second auxiliary data, where the value of the first auxiliary variable is between 0 and the first parameter.
  • the first uniform distribution determined by the first parameter as the probability distribution of entropy encoding, and then decode the first auxiliary data based on the first uniform distribution to obtain the first auxiliary variable and the second auxiliary data.
  • the data processing device uses the first uniform distribution to decode the first auxiliary data to obtain the first auxiliary variable and the second auxiliary data, it may be implemented through the following steps:
  • the second auxiliary data is used to encode the second auxiliary variable in the process of forward transformation, in This first auxiliary variable is encoded during the inverse transformation.
  • the data processing device after generating the second auxiliary variable, the data processing device will also use entropy coding (for example, uniform distribution entropy coding) to encode the second auxiliary variable, so that the data processing device can directly
  • entropy coding for example, uniform distribution entropy coding
  • the data processing device uses the second auxiliary data obtained by decoding the first auxiliary data as auxiliary data for encoding the second auxiliary variable. Specifically, the data processing device will encode the second auxiliary variable generated by the target reversible flow layer through the following steps:
  • the third auxiliary data is stored, the third auxiliary data is used for decoding the second auxiliary variable in the inverse transformation process.
  • the data processing device since the data processing device uses the second auxiliary data to encode the second auxiliary variable to obtain the third auxiliary data encoded with the second auxiliary variable, it is beneficial for the data processing device to store the third auxiliary data, so as to facilitate the inverse transformation During processing, the data processing device uses the third auxiliary data to decode the second auxiliary variable. Since the storage space occupied by storing the third auxiliary data is much smaller than the storage space required for storing the second auxiliary variable, compared with the scheme in which the data processing device directly stores the second auxiliary variable, it is beneficial to save storage space and improve storage efficiency .
  • the third auxiliary data is determined based on the encoding of the second auxiliary variable by the second auxiliary data, and the second auxiliary data is obtained in the process of decoding the first auxiliary variable by using the first auxiliary data, therefore, the third auxiliary data is stored
  • the auxiliary data indirectly stores the corresponding relationship between the second auxiliary variable and the first auxiliary variable (equivalent to correspondingly storing the first auxiliary variable and the second auxiliary variable).
  • the product obtained by multiplying the second auxiliary data by the second parameter is added to the second auxiliary variable to obtain the third auxiliary data.
  • the target reversible flow layer is also used to: obtain the first interval in the domain of the first monotone continuous function and the corresponding value of the first interval on the value domain of the first monotone continuous function
  • the first monotone continuous function is a nonlinear function
  • an interpolation method or a binary search method is used to calculate the model parameters.
  • the model parameters include a first parameter and a second parameter.
  • the target reversible flow layer can be constructed based on a nonlinear function (for example, the first monotone continuous function), then the data processing device can split the nonlinear function into multiple segments, and approximate each segment as a straight line To calculate the aforementioned first parameter and second parameter.
  • the data processing device may calculate the aforementioned first parameter and second parameter by using an interpolation algorithm or a binary search method.
  • the N reversible flow layers include at least two reversible flow layers
  • the N reversible flow layers are arranged in series.
  • the output of the i-1th reversible flow layer is used as the input of the i-th reversible flow layer, where i is a positive integer greater than 1 and not greater than N
  • the input of the first reversible flow layer is the data to be encoded
  • the output of the Nth reversible flow layer is the hidden variable.
  • the data to be encoded is data obtained by performing inverse encoding processing on original data by using noise data, and the data to be encoded is fixed-point number data with a first preset precision.
  • the i-th reversible flow layer is also used to process the fixed-point number data based on the first preset precision into integer data before the forward transform processing, and process the integer data after the forward transform processing is the fixed-point data with the first preset precision as the output of the ith reversible flow layer.
  • the aforementioned noise data may be a bit stream of compressed data in the data processing device, or may be a bit stream pre-stored in the data processing device, which is not specifically limited here.
  • the noise data since the noise data is used to perform inverse encoding processing on the original data, the value of the data calculated by the data processing device can be increased, which is conducive to improving the processing accuracy of the subsequent transformation process and also improving the compression rate.
  • the N reversible flow layers further include a reversible convolution layer, the reversible convolution layer includes a convolution kernel and the target reversible flow layer, the convolution kernel is the channel of the data to be encoded The number is a square matrix of order;
  • the reversible convolution layer is used for LU (LU Factorization) decomposition of the convolution kernel to obtain the target matrix, the target matrix is a diagonal matrix, and the non-zero elements on the diagonal of the diagonal matrix are used to determine the reversible
  • the model parameters for this target reversible flow layer in the convolutional layer are used for LU (LU Factorization) decomposition of the convolution kernel to obtain the target matrix, the target matrix is a diagonal matrix, and the non-zero elements on the diagonal of the diagonal matrix are used to determine the reversible
  • the model parameters for this target reversible flow layer in the convolutional layer are used for LU (LU Factorization) decomposition of the convolution kernel to obtain the target matrix, the target matrix is a diagonal matrix, and the non-zero elements on the diagonal of the diagonal matrix are used to determine the reversible
  • the model parameters for this target reversible flow layer in the convolutional layer are used for LU (LU Factorization) decomposition of
  • the 1x1 convolutional layer is converted into a matrix multiplication operation of continuous upper triangular matrix, diagonal matrix, lower triangular matrix, and scrambled matrix, and iterative calculations are used for the four matrix multiplications.
  • the N reversible flow layers further include a coupling layer, and the coupling layer includes the target reversible flow layer;
  • the coupling layer is used to divide the data input into the coupling layer into at least two parts of data according to preset dimensions, and process at least part of the at least two parts of data as the input of the target reversible flow layer to obtain the coupling layer Output.
  • the coupling layer may include a neural network, and the neural network is used to process the data input to the coupling layer in different dimensions.
  • the neural network can include various types. For example, convolutional neural network (convolutional neural network, CNN), deep neural network (deep neural network, DNN), recurrent neural network (recurrent neural networks, RNN) or other neural networks, etc. Specifically, there is no limitation here.
  • the coupling layer can divide the data input into the coupling layer according to dimensions, and only use the conversion method of the target reversible flow layer to transform the data of some of the dimensions. Therefore, the diversity of data transformation can be realized, and the representation ability of the reversible flow model can be improved.
  • the present application provides a data decoding method, which is used in a data processing device as a decompression end.
  • the data processing device obtains coded data; then, decodes the coded data to obtain hidden variables; and then uses a reversible flow model to perform inverse transformation processing on the hidden variables to obtain decoded outputs.
  • the reversible flow model is a new flow model proposed by this application. It should be understood that the data encoding method and the data decoding method use the same reversible flow model.
  • the reversible flow model includes a target reversible flow layer, the target reversible flow layer includes model parameters, and the model parameters are used to constrain the auxiliary variables generated during the inverse transformation process, and the operation corresponding to the target reversible flow layer includes For a determined multiplication and division operation, the auxiliary variable is the increment of the product of the multiplication operation or the remainder produced by the division operation.
  • model parameters in the target reversible flow layer may be pre-configured parameters, or may be parameters calculated by the target reversible flow layer based on a certain algorithm, which are not specifically limited here.
  • the data to be encoded may be original data such as images, videos, or texts. At this time, the data to be encoded is generally integer data.
  • the data to be encoded may also be preprocessed data based on original data such as images, videos, or texts. At this time, the data to be encoded may be a fixed-point number with a certain precision.
  • the data processing device performs inverse transformation processing on the decoded latent variables.
  • the reversible flow model of the inverse transformation process includes a target reversible flow layer, and the target reversible flow layer includes not only multiplication operations determined based on the model parameters and In the division operation, the model parameters in the target reversible flow layer can also constrain the auxiliary variable generated during the inverse transformation process, and the auxiliary variable is the increment of the product of the multiplication operation or the remainder generated by the division operation.
  • the present application can use the auxiliary variable to perform division operation for the multiplication operation in the forward transform process, so as to restore the hidden variable to the data before the forward transform process (that is, the data to be encoded in the aforementioned data encoding method) . Therefore, not only can the reversible operation of the transformation process be guaranteed, but also the representation ability of the model can be improved, which is conducive to improving the estimation accuracy of the probability distribution of the model, thereby improving the compression efficiency of lossless compression.
  • the reversible flow model includes N reversible flow layers, the N reversible flow layers include the target reversible flow layer, and N is an integer greater than or equal to 1.
  • the target reversible flow layer is used to reversely transform the second data input to the target reversible flow layer to obtain the first data output from the target reversible flow layer, the second data is the multiplier of the multiplication operation, and the first Data is the integer quotient of this division operation.
  • the second data is used as the multiplier of the multiplication operation, and another multiplier of the multiplication operation is determined by the model parameters; because, The auxiliary variable can be used as an increment to the product of this multiplication operation, so that the intermediate data can be obtained based on the product of the aforementioned second data and another multiplier, plus the aforementioned auxiliary variable.
  • the first data is used as an integer quotient of the division operation, the divisor of the division operation is intermediate data, and the dividend is determined by the model parameters. Therefore, after the second data is input into the target reversible flow layer, the first data will be output after multiplication and division operations, and the first data is the output of the target reversible flow layer.
  • model parameters for example, the first parameter
  • auxiliary variables for example, the first auxiliary variable
  • the model parameters (eg, second parameter) and auxiliary variables (eg, second auxiliary variable) used in the division operation of the transformation process are applied to the multiplication operation of the inverse transformation process.
  • the second data is the latent variable or the data output after being processed by at least one of the reversible flow layers
  • the first data is the decoding output or the output of the first data after being processed by at least one of the reversible flow layers is the decoded output.
  • the aforementioned at least one reversible flow layer refers to n reversible flow layers, wherein n is an integer greater than or equal to 1 and less than N, however, N is an integer greater than 1 at this time.
  • the second data input to the target reversible flow layer will be multiplied, and the first data output from the target reversible flow layer will be obtained through division. Therefore, compared to the method that only uses integer addition and subtraction For the integer flow model, the reversible flow model of the present application has stronger model representation capabilities, which is conducive to improving the estimation accuracy of the probability distribution of the model, thereby improving the compression efficiency of lossless compression.
  • the model parameters include a first parameter and a second parameter, both of the first parameter and the second parameter are positive integers, and the first parameter and the second parameter are not equal; the auxiliary variable A second auxiliary variable is included that is the increment of the product of the multiplication operation.
  • the data processing device performs inverse transformation processing on the second data input to the target reversible flow layer, and the process of obtaining and outputting the first data of the target reversible flow layer may specifically be as follows:
  • the intermediate data is used as a divisor and the first parameter is used as a dividend to perform division and rounding operations to obtain the first data.
  • the data processing device adds the product obtained by multiplying the second data and the second parameter (that is, the second product data) to the second auxiliary variable to obtain the intermediate data; then, the data processing device converts the intermediate The data is used as a divisor and the first parameter is used as a dividend to perform a division operation, and the obtained integer quotient is the first data.
  • the second data is first transformed into intermediate data based on multiplication and addition operations, and then the intermediate data is transformed into first data based on division and rounding operations.
  • the second auxiliary variable participates in the process of determining the intermediate data, that is, the second auxiliary variable is used as the increment of the product of the multiplication operation, the integer quotient of the division and rounding operation based on the intermediate data is the first data, therefore, avoid The problem of irreversible operation caused by only multiplication is avoided, and the problem of poor representation ability of the model caused by only addition and subtraction is also avoided.
  • model parameters and auxiliary variables used in the reverse transformation process of the target reversible flow layer is opposite to the order of the model parameters and auxiliary variables used in the forward transformation process.
  • the auxiliary variable further includes a first auxiliary variable, and the first auxiliary variable is a remainder generated by the division operation.
  • the data processing device will also perform division and remainder operation with the intermediate data as the divisor and the first parameter as the dividend to obtain the first auxiliary variable.
  • the target reversible flow model will also generate a first auxiliary variable in the inverse transformation process, so that the data processing device can return the data related to the first auxiliary variable based on the first auxiliary variable (for example, the first supplementary data).
  • the process of the data processing device determining the second auxiliary variable based on the second parameter may be:
  • the data processing device first obtains the third auxiliary data input to the target reversible flow layer, and the third auxiliary data is the auxiliary data stored in the forward transformation processing corresponding to the inverse transformation processing; then, the data processing device based on the third auxiliary data The data and the second parameter determine the second auxiliary variable.
  • the data processing device determine the second auxiliary variable by using the third auxiliary data stored in the forward transformation processing corresponding to the inverse transformation processing, which is beneficial to restore the second auxiliary variable from the third auxiliary data, so as to It is ensured that the value of the second auxiliary variable used in the process of inverse transformation is consistent with the value of the second auxiliary variable used in the process of forward transformation.
  • the data processing device encodes and decodes the auxiliary data input to the target reversible flow layer based on entropy coding (for example, uniform distribution entropy coding), so as to Causes the aforementioned auxiliary variable to be generated or stored.
  • entropy coding for example, uniform distribution entropy coding
  • the process for the data processing device to determine the second auxiliary variable based on the third auxiliary data and the second parameter may be:
  • the third auxiliary data is decoded by using the second uniform distribution to obtain the second auxiliary variable and the second auxiliary data, where the value of the second auxiliary variable is between 0 and the second parameter.
  • the second uniform distribution determined by the second parameter as the probability distribution of entropy coding, and then decode the third auxiliary data based on the second uniform distribution to obtain the second auxiliary variable and the second auxiliary data.
  • the data processing device uses the second uniform distribution to decode the third auxiliary data to obtain the second auxiliary variable and the second auxiliary data, it may be implemented through the following steps:
  • the second auxiliary data is used to encode the second auxiliary variable in the forward transformation process, in This first auxiliary variable is encoded during the inverse transformation.
  • the data processing device after generating the second auxiliary variable, the data processing device will also use entropy coding (for example, uniform distribution entropy coding) to encode the second auxiliary variable, so that the data processing device can directly
  • entropy coding for example, uniform distribution entropy coding
  • the data processing device uses the second auxiliary data obtained by decoding the third auxiliary data as auxiliary data for encoding the first auxiliary variable. Specifically, the data processing device will encode the first auxiliary variable generated by the target reversible flow layer through the following steps:
  • the value of the first auxiliary variable is between 0 and the first parameter
  • the first auxiliary data is stored, and the first auxiliary data is auxiliary data input to the target reversible flow layer in the forward transformation processing corresponding to the inverse transformation processing.
  • the data processing device since the data processing device uses the second auxiliary data to encode the first auxiliary variable to obtain the first auxiliary data encoded with the first auxiliary variable, it is beneficial for the data processing device to store the first auxiliary data. Since the storage space occupied by storing the first auxiliary data is much smaller than the storage space required for storing the first auxiliary variable, it is beneficial to save storage space and improve storage efficiency compared to the data processing device directly storing the first auxiliary variable. . In addition, generating and storing the first auxiliary data is beneficial to recovering the first auxiliary data input to the target reversible flow layer during the forward transformation process.
  • the product obtained by multiplying the second auxiliary data by the first parameter is added to the first auxiliary variable to obtain the first auxiliary data.
  • the target reversible flow layer is also used for:
  • the first monotone continuous function is a nonlinear function; based on the first
  • the first interval and the second interval adopt interpolation method or binary search method to calculate the model parameters.
  • the foregoing model parameters include a first parameter and a second parameter.
  • the target reversible flow layer can be constructed based on a nonlinear function (for example, the first monotone continuous function), then the data processing device can split the nonlinear function into multiple segments, and approximate each segment as a straight line To calculate the aforementioned first parameter and second parameter.
  • the data processing device may calculate the aforementioned first parameter and second parameter by using an interpolation algorithm or a binary search method.
  • the N reversible flow layers include at least two reversible flow layers
  • the N reversible flow layers are arranged in series.
  • the output of the i-th reversible flow layer is used as the input of the i-1th reversible flow layer
  • the i is a positive integer greater than 1 and not greater than N
  • the input of the N-th reversible flow layer is the hidden variable
  • the output of the first reversible flow layer is the decoding output.
  • the multiple reversible flow layers in the reversible flow model are arranged in series, which is beneficial for each reversible flow layer in the reversible flow model to process data sequentially. It should be understood that the order in which data passes through each reversible flow layer during the forward transformation process is opposite to the order in which data passes through each reversible flow layer during the inverse transformation process.
  • the i-th reversible flow layer is also used to process the fixed-point number data based on the first preset precision into integer data before the inverse transformation process, and process the inverse
  • the integer data processed by the inverse transformation is processed into fixed-point number data with a first preset precision as an output of the ith reversible flow layer.
  • the method further includes: the data processing device performs de-inversion processing on the output of the first reversible flow layer to obtain original data and noise data.
  • the data processing device can separate the noise data and the original data from the decoded output through the de-de-coding process.
  • the N serial reversible flow layers further include a reversible convolution layer, the reversible convolution layer includes a convolution kernel and the target reversible flow layer, and the convolution kernel is based on the to-be-encoded
  • the number of channels of the data is a square matrix of orders
  • the reversible convolution layer is used to perform LU decomposition on the convolution kernel to obtain a target matrix, the target matrix is a diagonal matrix, and the non-zero elements on the diagonal of the diagonal matrix are used to determine the The model parameters for the target's reversible flow layer.
  • the 1x1 convolutional layer is converted into a matrix multiplication operation of continuous upper triangular matrix, diagonal matrix, lower triangular matrix, and scrambled matrix, and iterative calculations are used for the four matrix multiplications.
  • the N serial reversible flow layers also include a coupling layer, and the coupling layer includes the target reversible flow layer;
  • the coupling layer is used to divide the data input into the coupling layer into at least two parts of data according to preset dimensions, and process at least part of the at least two parts of data as the input of the target reversible flow layer to obtain the coupling layer Output.
  • the coupling layer may include a neural network, and the neural network is used to process the data input to the coupling layer in different dimensions.
  • the neural network can include various types. For example, convolutional neural network CNN, deep neural network DNN, recurrent neural network RNN or other neural networks, etc. Specifically, there is no limitation here.
  • the coupling layer can divide the data input into the coupling layer according to dimensions, and only use the conversion method of the target reversible flow layer to transform the data of some of the dimensions. Therefore, the diversity of data transformation can be realized, and the representation ability of the reversible flow model can be improved.
  • the present application provides a data processing device, where the data processing device is an encoding end.
  • the data processing device includes: an acquisition module, a transformation processing module and an encoding module.
  • the transformation processing module includes the reversible flow model proposed in this application, and the reversible flow model is used to perform forward transformation processing on the data input to the reversible flow model at the encoding end.
  • the specific functions of the foregoing functional modules are as follows:
  • An acquisition module configured to acquire data to be encoded
  • the transformation processing module is used to use the reversible flow model to perform forward transformation processing on the data to be encoded to obtain hidden variables; wherein, the reversible flow model includes a target reversible flow layer, and the target reversible flow layer includes model parameters, and the model parameters are used Due to constraints on the auxiliary variable generated during the forward transformation process, the operation corresponding to the target reversible flow layer includes multiplication and division operations determined based on the model parameters, and the auxiliary variable is the increment of the product of the multiplication operation or the the remainder produced by the division operation;
  • the encoding module is configured to encode the hidden variable to obtain encoded data corresponding to the data to be encoded.
  • the target reversible flow layer since the reversible flow model used to perform forward transformation processing on the data to be encoded includes a target reversible flow layer, the target reversible flow layer not only includes multiplication and division operations determined based on the model parameters, but the target reversible flow layer
  • the model parameters in can also constrain the auxiliary variable generated during the forward transformation process, the auxiliary variable being the increment of the product of the multiplication operation or the remainder generated by the division operation. That is to say, this application can not only use multiplication, but also realize reversibility by using auxiliary variables when performing division. Therefore, compared with the integer flow model in the traditional technology, the model representation ability is improved, which is conducive to improving the probability distribution of the model The estimated accuracy rate of , which in turn can improve the compression efficiency of lossless compression.
  • the reversible flow model includes N reversible flow layers, the N reversible flow layers include the target reversible flow layer, and N is an integer greater than or equal to 1.
  • the target reversible flow layer is used to perform forward transformation processing on the first data input to the target reversible flow layer to obtain second data output from the target reversible flow layer, the first data is the multiplier of the multiplication operation, and the second data is output from the target reversible flow layer.
  • the second data is the integer quotient of the division operation.
  • the first data is the data to be encoded or the output data processed by at least one of the reversible flow layers
  • the second data is the latent variable or the output data of the second data processed by at least one of the reversible flow layers is the hidden variable.
  • the model parameters include a first parameter and a second parameter, both of the first parameter and the second parameter are positive integers, and the first parameter and the second parameter are not equal; the auxiliary variable A first auxiliary variable is included which is the increment of the product of the multiplication operation.
  • the transformation processing module is specifically used for:
  • the intermediate data is used as a divisor and the second parameter is used as a dividend to perform a division and rounding operation to obtain the second data.
  • the auxiliary variable further includes a second auxiliary variable, and the second auxiliary variable is a remainder generated by the division operation.
  • the transformation processing module is also used to perform division and remainder operation with the intermediate data as the divisor and the second parameter as the dividend to obtain the second auxiliary variable, which is used as the reverse direction corresponding to the forward transformation processing. Increment of the product of the multiplication operation in the transformation process, the first auxiliary variable is used as a remainder resulting from the division operation in the inverse transformation process.
  • the transformation processing module is also used for:
  • the first auxiliary variable is determined based on the first auxiliary data and the first parameter.
  • the transformation processing module is specifically configured to:
  • the first auxiliary data is decoded by using the first uniform distribution to obtain the first auxiliary variable and second auxiliary data, where the value of the first auxiliary variable is between 0 and the first parameter.
  • the transformation processing module includes a first entropy codec module, the first entropy codec module stores a first uniform distribution, and the first entropy codec module is used for:
  • the second auxiliary data is used to encode the second auxiliary variable in the process of forward transformation, in This first auxiliary variable is encoded during the inverse transformation.
  • the transformation processing module is also used for:
  • the third auxiliary data is stored, the third auxiliary data is used for decoding the second auxiliary variable in the inverse transformation process.
  • the transformation processing module includes a second entropy codec module, the second entropy codec module stores a second uniform distribution, and the second entropy codec module is used for:
  • the product obtained by multiplying the second auxiliary data by the second parameter is added to the second auxiliary variable to obtain the third auxiliary data.
  • the target reversible flow layer is also used for:
  • the model parameters are calculated by using an interpolation method or a binary search method based on the first interval and the second interval.
  • the N reversible flow layers include at least two reversible flow layers, the N reversible flow layers are arranged in series;
  • the output of the i-1th reversible flow layer is used as the input of the i-th reversible flow layer, where i is a positive integer greater than 1 and not greater than N, and the input of the first reversible flow layer is the data to be encoded , the output of the Nth reversible flow layer is the hidden variable.
  • the data to be encoded is data obtained by performing inverse encoding processing on original data by using noise data, and the data to be encoded is fixed-point number data with a first preset precision;
  • the i-th reversible flow layer is also used to process the fixed-point number data based on the first preset precision into integer data before the forward transform processing, and process the integer data after the forward transform processing is the fixed-point data with the first preset precision as the output of the ith reversible flow layer.
  • the N reversible flow layers further include a reversible convolution layer, the reversible convolution layer includes a convolution kernel and the target reversible flow layer, the convolution kernel is the channel of the data to be encoded The number is a square matrix of order;
  • the reversible convolution layer is used to perform LU decomposition on the convolution kernel to obtain a target matrix, the target matrix is a diagonal matrix, and the non-zero elements on the diagonal of the diagonal matrix are used to determine the The model parameters for the target's reversible flow layer.
  • the N reversible flow layers further include a coupling layer, and the coupling layer includes the target reversible flow layer;
  • the coupling layer is used to divide the data input into the coupling layer into at least two parts of data according to preset dimensions, and process at least part of the at least two parts of data as the input of the target reversible flow layer to obtain the coupling layer Output.
  • the present application provides a data processing device, where the data processing device is a decoding end.
  • the data processing device includes: an acquisition module, a transformation processing module and a decoding module.
  • the transformation processing module includes the reversible flow model proposed in the present application, and the reversible flow model is used to reversely transform the data input to the reversible flow model at the decoding end.
  • the specific functions of the foregoing functional modules are as follows:
  • An acquisition module used to acquire encoded data
  • a decoding module configured to decode the encoded data to obtain hidden variables
  • the transformation processing module is used to perform inverse transformation processing on the latent variable by using a reversible flow model to obtain a decoding output; wherein, the reversible flow model includes a target reversible flow layer, and the target reversible flow layer includes model parameters, and the model parameters are used to constrain
  • the auxiliary variable generated during the inverse transformation process, the operation corresponding to the target reversible flow layer includes the multiplication operation and division operation determined based on the model parameters, the auxiliary variable is the increment of the product of the multiplication operation or the result of the division operation remainder of .
  • the data processing device performs inverse transformation processing on the decoded latent variables.
  • the reversible flow model of the inverse transformation process includes a target reversible flow layer, and the target reversible flow layer includes not only multiplication operations determined based on the model parameters and In the division operation, the model parameters in the target reversible flow layer can also constrain the auxiliary variable generated during the inverse transformation process, and the auxiliary variable is the increment of the product of the multiplication operation or the remainder generated by the division operation.
  • the present application can use the auxiliary variable to perform division operation for the multiplication operation in the forward transform process, so as to restore the hidden variable to the data before the forward transform process (that is, the data to be encoded in the aforementioned data encoding method) . Therefore, not only can the reversible operation of the transformation process be guaranteed, but also the representation ability of the model can be improved, which is conducive to improving the estimation accuracy of the probability distribution of the model, thereby improving the compression efficiency of lossless compression.
  • the reversible flow model includes N reversible flow layers, the N reversible flow layers include the target reversible flow layer, and N is an integer greater than or equal to 1.
  • the target reversible flow layer is used to reversely transform the second data input to the target reversible flow layer to obtain the first data output from the target reversible flow layer, the second data is the multiplier of the multiplication operation, and the first Data is the integer quotient of this division operation.
  • the second data is the latent variable or the data output after being processed by at least one of the reversible flow layers, the first data is the decoding output or the output of the first data after being processed by at least one of the reversible flow layers is the decoded output.
  • the model parameters include a first parameter and a second parameter, both of the first parameter and the second parameter are positive integers, and the first parameter and the second parameter are not equal; the auxiliary variable A second auxiliary variable is included that is the increment of the product of the multiplication operation.
  • the transformation processing module is specifically used for:
  • the intermediate data is used as a divisor and the first parameter is used as a dividend to perform division and rounding operations to obtain the first data.
  • the auxiliary variable further includes a first auxiliary variable, where the first auxiliary variable is a remainder generated by the division operation;
  • the transform processing module is also used to:
  • the transformation processing module is specifically configured to:
  • the second auxiliary variable is determined based on the third auxiliary data and the second parameter.
  • the transformation processing module is specifically configured to:
  • the third auxiliary data is decoded by using the second uniform distribution to obtain the second auxiliary variable and the second auxiliary data, where the value of the second auxiliary variable is between 0 and the second parameter.
  • the transform processing module includes a second entropy codec module, the second entropy codec module stores a second uniform distribution, and the second entropy codec module is used for:
  • the second auxiliary data is used to encode the second auxiliary variable in the forward transformation process, in This first auxiliary variable is encoded during the inverse transformation.
  • the transformation processing module is also used for:
  • the value of the first auxiliary variable is between 0 and the first parameter
  • the first auxiliary data is stored, and the first auxiliary data is auxiliary data input to the target reversible flow layer in the forward transformation processing corresponding to the inverse transformation processing.
  • the transformation processing module includes a first entropy codec module, the first entropy codec module stores a first uniform distribution, and the first entropy codec module is used for:
  • the product obtained by multiplying the second auxiliary data by the first parameter is added to the first auxiliary variable to obtain the first auxiliary data.
  • the target reversible flow layer is also used for:
  • the model parameters are calculated by using an interpolation method or a binary search method based on the first interval and the second interval.
  • the N reversible flow layers include at least two reversible flow layers, the N reversible flow layers are arranged in series;
  • the output of the i-th reversible flow layer is used as the input of the i-1th reversible flow layer, the i is a positive integer greater than 1 and not greater than N, and the input of the N-th reversible flow layer is the hidden variable,
  • the output of the first reversible flow layer is the decoding output.
  • the i-th reversible flow layer is also used to process the fixed-point number data based on the first preset precision into integer data before the inverse transformation process, and process the inverse
  • the integer data processed by the inverse transformation is processed into fixed-point number data with a first preset precision as an output of the ith reversible flow layer.
  • the decoding module is further configured to perform de-inverse encoding processing on the output of the first reversible flow layer to obtain original data and noise data.
  • the N serial reversible flow layers further include a reversible convolution layer, the reversible convolution layer includes a convolution kernel and the target reversible flow layer, and the convolution kernel is based on the to-be-encoded
  • the number of channels of the data is a square matrix of orders
  • the reversible convolution layer is used to perform LU decomposition on the convolution kernel to obtain a target matrix, the target matrix is a diagonal matrix, and the non-zero elements on the diagonal of the diagonal matrix are used to determine the The model parameters for the target's reversible flow layer.
  • the N serial reversible flow layers further include a coupling layer, and the coupling layer includes the target reversible flow layer;
  • the coupling layer is used to divide the data input into the coupling layer into at least two parts of data according to preset dimensions, and process at least part of the at least two parts of data as the input of the target reversible flow layer to obtain the coupling layer Output.
  • the present application provides a data processing device, where the data processing device is an encoding end.
  • the data processing device includes a storage medium, a processing circuit, and a bus system; wherein, the storage medium is used to store instructions, and the processing circuit is used to execute instructions in the memory to perform any one of the above-mentioned first aspect and the first aspect
  • the data encoding method introduced in the implementation mode is not limited to.
  • the present application provides a data processing device, where the data processing device is a decoding end.
  • the data processing device includes a storage medium, a processing circuit, and a bus system; wherein, the storage medium is used to store instructions, and the processing circuit is used to execute instructions in the memory to perform any one of the above-mentioned second aspect and the second aspect
  • the data decoding method introduced in the implementation mode is a sixth aspect.
  • the embodiment of the present application provides a computer-readable storage medium, including instructions, when the instructions are run on the computer, so that the computer executes the above-mentioned first aspect or the second aspect, and each of the above-mentioned various aspects.
  • the embodiment of the present application provides a computer program product containing instructions, which, when run on a computer, enables the computer to execute the various implementations of the foregoing first aspect or the second aspect, as well as the foregoing aspects The method described in any one of the implementations.
  • the present application provides a system-on-a-chip, which includes a processor, configured to support an execution device or a training device to implement the functions involved in the above aspect, for example, to send or process the data and and/or information.
  • the chip system further includes a memory, and the memory is used for storing necessary program instructions and data of the execution device or the training device.
  • the system-on-a-chip may consist of chips, or may include chips and other discrete devices.
  • the target reversible flow layer not only includes the multiplication and division operations determined based on the model parameters, the target reversible flow layer
  • the model parameters in can also constrain the auxiliary variable generated during the forward transformation process, the auxiliary variable being the increment of the product of the multiplication operation or the remainder generated by the division operation. That is to say, this application can not only use multiplication, but also realize reversibility by using auxiliary variables when performing division. Therefore, compared with the integer flow model in the traditional technology, the model representation ability is improved, which is conducive to improving the probability distribution of the model
  • the estimation accuracy rate can improve the compression efficiency of lossless compression.
  • the data processing device will perform inverse transformation processing on the decoded latent variables.
  • the reversible flow model of the inverse transformation process includes a target reversible flow layer, and the target reversible flow layer not only includes multiplication operations determined based on the model parameters and In the division operation, the model parameters in the target reversible flow layer can also constrain the auxiliary variable generated during the inverse transformation process, and the auxiliary variable is the increment of the product of the multiplication operation or the remainder generated by the division operation.
  • the present application can use the auxiliary variable to perform division operation for the multiplication operation in the forward transform process, so as to restore the hidden variable to the data before the forward transform process (that is, the data to be encoded in the aforementioned data encoding method) . Therefore, not only can the reversible operation of the transformation process be guaranteed, but also the representation ability of the model can be improved, which is conducive to improving the estimation accuracy of the probability distribution of the model, thereby improving the compression efficiency of lossless compression.
  • Fig. 1 is a schematic structural diagram of the artificial intelligence subject framework provided by the present application.
  • Fig. 2 is a schematic diagram of the principles of the compression process and the decompression process provided by the present application
  • Fig. 3 is a flowchart of the data encoding method in the present application.
  • Fig. 4A is a schematic structural diagram of the reversible flow model in this application.
  • Fig. 4B is another structural schematic diagram of the reversible flow model in this application.
  • Fig. 5 is a flowchart of the data decoding method in the present application.
  • Figure 6A is a schematic structural diagram of the reversible flow model in this application.
  • Fig. 6B is another structural schematic diagram of the reversible flow model in this application.
  • Fig. 7 is a flow chart of the target reversible flow layer in the present application when performing forward transformation
  • Fig. 8A is another flow chart of the target reversible flow layer in this application when performing forward transformation
  • Figure 8B is an example diagram of the target reversible flow layer in this application when performing forward transformation
  • Fig. 9 is a flow chart of the target reversible flow layer in the present application during reverse transformation
  • Fig. 10A is another flow chart of the target reversible flow layer in this application when performing reverse transformation
  • Fig. 10B is an example diagram of the target reversible flow layer in the present application during inverse transformation
  • Fig. 11 is a schematic diagram of an embodiment of a data processing device in the present application.
  • Fig. 12 is a schematic diagram of another embodiment of the data processing device in the present application.
  • Fig. 13 is a schematic diagram of another embodiment of the data processing device in the present application.
  • Fig. 14 is a schematic diagram of an embodiment when the data processing device in the present application is used as an encoding end;
  • FIG. 15 is a schematic diagram of an embodiment of the data processing device in this application as a decoding end.
  • the data encoding method and data decoding method provided in this application can be applied to the field of artificial intelligence AI and the storage field related to artificial intelligence AI.
  • Figure 1 it is a schematic structural diagram of the main framework of artificial intelligence.
  • the following is an introduction to the above-mentioned artificial intelligence theme framework from the two dimensions of "intelligent information chain” (horizontal axis) and “IT value chain” (vertical axis).
  • the "intelligent information chain” reflects a series of processes from data acquisition to processing.
  • the foregoing processing process may be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output.
  • IT value chain refers to the industrial ecological process from the underlying infrastructure of human intelligence, information (provision and processing technology realization) to the system, reflecting the value brought by artificial intelligence to the information technology industry.
  • the infrastructure is used to provide computing power support for the artificial intelligence system, realize communication with the outside world, and realize support through the basic platform.
  • the foregoing infrastructure may be a processing circuit or an intelligent chip with computing capabilities.
  • the aforementioned smart chip includes a central processing unit (central processing unit, CPU), a neural network processor (neural-network processing unit, NPU), a graphics processing unit (graphics processing unit, GPU), an application specific integrated circuit (application specific integrated circuit, ASIC), field programmable logic gate array (field programmable gate array, FPGA) and other hardware acceleration chips.
  • the foregoing infrastructure may also be a basic platform that provides cloud computing or cloud storage functions.
  • the basic platform includes related platform guarantees and supports such as distributed computing framework and network, which may include cloud storage and computing, interconnection and intercommunication network, etc.
  • the sensor communicates with the outside to obtain data, and the data is provided to the smart chip in the distributed computing system provided by the basic platform for calculation.
  • Data from the upper layer of the infrastructure is used to represent data sources in the field of artificial intelligence.
  • the data involves graphics, images, voice, text, and IoT data of traditional equipment, including business data of existing systems and sensory data such as force, displacement, liquid level, temperature, and humidity.
  • Data processing usually includes data training, machine learning, deep learning, search, reasoning, and decision-making.
  • machine learning and deep learning can symbolize and formalize intelligent information modeling, extraction, preprocessing and training of data.
  • Reasoning refers to the process of simulating human intelligent reasoning in a computer or intelligent system, and using formalized information to carry out machine thinking and solve problems according to reasoning control strategies.
  • the typical functions are search and matching.
  • Decision-making refers to the process of decision-making after intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.
  • the aforementioned general capability may be an algorithm or a general system, such as translation, text analysis, computer vision processing, speech recognition, image recognition, and so on.
  • Intelligent products and industry applications refer to the products and applications of artificial intelligence systems in various fields. It is the packaging of the overall solution of artificial intelligence, which commercializes intelligent information decision-making and realizes landing applications. Its application fields mainly include: intelligent terminals, intelligent manufacturing, Smart transportation, smart home, smart medical care, smart security, autonomous driving, safe city, etc.
  • the data encoding method and data decoding method provided in the present application can be applied to scenarios where lossless compression needs to be realized in the above-mentioned various application fields. Specifically, it can be applied to scenarios of lossless compression of data such as images, videos, and texts.
  • the data processing device can obtain the data to be encoded, and the data to be encoded can be an original file or a preprocessed file of data such as images, videos, and texts.
  • this example takes the aforementioned data to be encoded as images to be compressed and stored (hereinafter referred to as images to be compressed) as an example for introduction.
  • the image to be compressed may be a photo taken by a camera or a frame captured from a video.
  • the data processing device can process the acquired image to be compressed through the reversible flow model, transform and process the image data into hidden variables, and generate probability estimates for each point in the hidden variables.
  • the encoder can encode the extracted hidden variables through the probability estimation of each point in the hidden variables, reduce the coding redundancy of hidden variables, further reduce the amount of data transmission in the image compression process, and encode the encoded data as data files
  • the form is saved in the corresponding storage location.
  • the data processing device can obtain and load the above saved file in the corresponding storage location, and obtain the decoded hidden variable based on the decoder, and reverse the hidden variable through the reversible flow model. Transform processing to obtain a reconstructed image (that is, a decoded output) after inverse transform processing.
  • the data processing apparatus involved in this application may be a terminal device or a server.
  • the terminal device may be a mobile phone, a tablet, a notebook computer, and a smart wearable device, etc., and the terminal device may perform compression processing on acquired data (such as image data, video data, or text data, etc.).
  • the terminal device may be a virtual reality (virtual reality, VR) device or an augmented reality (augmented reality, AR).
  • VR virtual reality
  • AR augmented reality
  • the embodiments of the present application may also be applied to other scenarios where data compression is required, and details are not described here. In this embodiment and subsequent embodiments, the title of data processing device is used for introduction.
  • Step 301 acquire data to be encoded.
  • the data to be encoded refers to data before compression encoding is performed.
  • the data to be encoded may be integer data or fixed-point data (also called floating-point data).
  • the data to be encoded may be original data such as images, videos, or texts.
  • the data to be encoded is generally integer data.
  • the data to be encoded may be an image captured by the terminal device through a camera, or an image obtained from inside the terminal device (for example, an image stored in the photo album of the terminal device, or , the image obtained by the terminal device from the cloud).
  • the above image may be an image that requires image compression, and this application does not limit the source of the image to be processed.
  • the data to be encoded may also be preprocessed data based on original data such as images, videos, or texts.
  • the data to be encoded may be a fixed-point number with a certain precision.
  • the data processing device may use the noise data to perform inverse encoding processing on the original data to obtain the aforementioned data to be encoded.
  • the data to be encoded is fixed-point number data with a first preset precision.
  • the data processing device can use the inverse coding technique to decode the noise data based on the probability distribution of the noise data, and obtain the decoded noise data u, which is a fixed-point number with a first preset precision data (for example, the first preset precision is the aforementioned precision k).
  • the data processing device sums the aforementioned original data and the decoded noise data (that is, calculates Among them, x is the original data, u is the noise data after decoding), and the data to be encoded is obtained
  • the data to be encoded is fixed-point number data with a first preset precision.
  • the aforementioned noise data may be a bit stream of compressed data in the data processing device, or may be a bit stream pre-stored in the data processing device, which is not specifically limited here.
  • the noise data since the noise data is used to perform inverse encoding processing on the original data, the value of the data calculated by the data processing device can be increased, which is conducive to improving the processing accuracy of the subsequent transformation process and also improving the compression rate.
  • the data to be encoded is video data
  • the size of the video does not match the input size of the model
  • the video needs to be cut into several video blocks, and the input size of each block is the same as the input size of the model (reversible flow model); if the video If the length is greater than the length required by the model, it will be cut into multiple video inputs. If the input size or video length is insufficient, it can be filled to the input size or specific length with patches of a specific color.
  • the compression process needs to convert the text into a vector first.
  • w) is the prior of w, generally the word frequency of w.
  • Step 302 using a reversible flow model to perform forward transformation processing on the data to be encoded to obtain hidden variables; wherein, the reversible flow model includes a target reversible flow layer, and the target reversible flow layer includes model parameters, and the model parameters are used to constrain the
  • the auxiliary variable generated during the forward transformation process, the operation corresponding to the target reversible flow layer includes the multiplication operation and division operation determined based on the model parameters, and the auxiliary variable is the increment of the product of the multiplication operation or the result of the division operation remainder of .
  • the reversible flow model is a new flow model proposed by this application.
  • the target reversible flow layer is a layer specific to the reversible flow model proposed in this application. Since the target reversible flow layer can realize reversible operation, the reversible flow model composed of the target reversible flow layer and/or the deformation layer of the target reversible flow layer can realize reversible operation.
  • the reversible flow model includes multiple reversible flow layers, and the multiple reversible flow layers are sequentially arranged to process data sequentially.
  • the aforementioned target reversible flow layer may be one of the aforementioned multiple reversible flow layers, or may be a component of one of the aforementioned multiple reversible flow layers. The following will introduce the two situations respectively:
  • the target reversible flow layer is a reversible flow layer among many reversible flow layers in the reversible flow model.
  • the reversible flow model may include N reversible flow layers, and the N reversible flow layers include the aforementioned target reversible flow layer, where N is an integer greater than or equal to 1.
  • the first N reversible flow layers include at least two reversible flow layers
  • the aforementioned N reversible flow layers may be arranged in series.
  • the reversible flow model includes N serial reversible flow layers, where N is an integer greater than 1.
  • N is an integer greater than 1.
  • the output of the i-1th reversible flow layer is used as the input of the i-th reversible flow layer
  • the output of the i-th reversible flow layer is used as the input of the i+1th reversible flow layer.
  • the i is greater than A positive integer equal to 1 and not greater than N.
  • the input of the first reversible flow layer (i.e., reversible flow layer 1) is the data to be encoded
  • the output of the first reversible flow layer (i.e., reversible flow layer 1) is used as the second reversible flow layer (i.e. The input of the reverse flow layer 2)
  • the output of the second reversible flow layer (that is, the reverse flow layer 2) is used as the input of the third reversible flow layer (that is, the reverse flow layer 3)
  • the Nth can be
  • the output of the counterflow layer (ie, the counterflow layer N) is the hidden variable.
  • the N serial reversible flow layers include the target reversible flow layer. In the example shown in FIG.
  • the target reversible flow layer may be one of the N reversible flow layers in FIG. 4A , for example, the reversible flow layer 2 is the target reversible flow layer.
  • the aforementioned N reversible flow layers may include multiple target reversible flow layers, for example, the reversible flow layer 2, the reversible flow layer 4 and the reversible flow layer 6 are all target reversible flow layers, that is, the aforementioned N reversible flow layers include 3 A target reversible flow layer.
  • the target reversible flow layer is a component of one reversible flow layer among the many reversible flow layers in the reversible flow model.
  • the target reversible flow layer may be a component of a certain reversible flow layer in FIG. 4A .
  • the reversible flow layer in the reversible flow model may include other models and/or operators in addition to the aforementioned target reversible flow layer, so as to form the deformation layer of the target reversible flow layer.
  • the deformation layer of the target reversible flow layer may be a reversible convolutional layer or a coupling layer.
  • the reversible flow model includes not only the target reversible flow layer, but also a reversible convolutional layer and/or a coupling layer.
  • the specific structure of the reversible convolutional layer and the coupling layer will be introduced in the following figure, and will not be repeated here.
  • the reversible flow model may include at least one target reversible flow layer, at least one reversible convolution layer, and at least one coupling layer.
  • the reversible flow model may be constructed in a way that the target reversible flow layer is interleaved with the coupling layer and the reversible convolution layer respectively.
  • the reversible flow model includes a target reversible flow layer, a coupling layer and a reversible convolution layer.
  • the data to be encoded is used as the input of the target reversible flow layer 1
  • the output of the target reversible flow layer 1 is used as the input of the coupling layer
  • the output of the coupling layer is used as the input of the target reversible flow layer 2
  • the output of the target reversible flow layer 2 is used as The input of the reversible convolutional layer, and so on, is processed by the target reversible flow layer n to output hidden variables.
  • Fig. 4B is only an example of stacking many reversible flow layers. In practical applications, the stacking order of each reversible flow layer can be adjusted based on actual needs. This application does not limit the stacking order of the foregoing reversible flow layers, and No more examples.
  • the target reversible flow layer is used to implement the data input to the target reversible flow layer (for ease of introduction, hereinafter referred to as the first data) and the data output by the target reversible flow layer (for ease of introduction, hereinafter referred to as first data) is called the reversible operation between the second data).
  • the first data is the data to be encoded or output after being processed by at least one of the reversible flow layers (for example, n reversible flow layers among the N reversible flow layers, where n is an integer greater than 1 and less than N).
  • the second data is the latent variable or the second data is processed by at least one reversible flow layer and the output data is the latent variable.
  • reversible means that the data after the forward transformation process can be restored to the data before the forward transformation process under the action of the reverse transformation process; The output of is consistent with the data before the forward transformation process.
  • the second data will be obtained after the first data is subjected to the forward transformation process of the target reversible flow layer, and the second data will be restored to the first data after being subjected to the inverse transformation process of the target reversible flow layer.
  • the reversible operation refers to the operation involved in the aforementioned transformation process, the operation can not only obtain output data through input data, but also deduce input data through output data.
  • the target reversible flow layer involves at least a multiplication operation and a division operation during the transformation process.
  • the target reversible flow layer includes model parameters used to constrain the auxiliary variables generated during the forward transformation process. That is to say, the target reversible flow layer can generate an auxiliary variable based on the aforementioned model parameters, and the auxiliary variable is used to assist the multiplied data to realize the reversible operation through the division operation. Since the calculation principle of the multiplication and division operation process of the computer is different from that of the mathematical theory, the data after the multiplication operation may not be recovered through the division operation. For example, if data and a certain coefficient (or parameter) can be multiplied to obtain a certain product, the quotient obtained by dividing the product with the aforementioned coefficient is not necessarily the aforementioned data.
  • the auxiliary variable generated by the target reversible flow layer can assist the data after multiplication to achieve reversibility through division. Specifically, it will be introduced in detail later.
  • the auxiliary variable is constrained by the model parameters in the target reversible flow layer, which means that the value range of the auxiliary variable is restricted by the value range of the model parameters.
  • the model parameters can be preset parameters, parameters generated based on neural network, or parameters determined by the reversible flow model based on other algorithms (for example, interpolation method or binary search method). limited.
  • the target reversible flow layer is used to perform forward transformation processing on the first data input to the target reversible flow layer to obtain second data output from the target reversible flow layer, and the first data is the multiplier of the multiplication operation , the second data is the integer quotient of the division operation. That is to say, in the forward transformation process, in the multiplication operation determined based on the model parameters, the first data is used as the multiplier of the multiplication operation, and another multiplier of the multiplication operation is determined by the model parameters; because , the auxiliary variable can be used as an increment of the product of the multiplication operation, therefore, the intermediate data can be obtained based on the product of the aforementioned first data and another multiplier, plus the aforementioned auxiliary variable.
  • the second data is used as an integer quotient of the division operation, the divisor of the division operation is intermediate data, and the dividend is determined by the model parameters. Therefore, after the first data is input into the target reversible flow layer, the second data will be output after multiplication and division operations, and the second data is the output of the target reversible flow layer. Specifically, please refer to the embodiments corresponding to FIG. 7 and FIG. 8A below.
  • the data processing device after the data processing device uses the reversible flow model to transform and process the data to be encoded into hidden variables, the data processing device will execute step 303 .
  • Step 303 encoding the hidden variable to obtain encoded data corresponding to the data to be encoded.
  • the hidden variable z may be represented by a probability distribution p Z (z), and the hidden variable z may be encoded according to the probability distribution p Z (z) to obtain encoded data.
  • the process of encoding hidden variables may be an entropy encoding process, and the entropy encoding process may use an existing entropy encoding technology, which will not be described in detail in this application.
  • the encoded data may be sent to a device for decompression.
  • the device for decompression can decompress (or call it decoding) the data.
  • the terminal equipment used for compression can store the coded data in the storage device, and when needed, the terminal device can obtain the coded data from the storage device, and can decompress the coded data.
  • the target reversible flow layer since the reversible flow model used to perform forward transformation processing on the data to be encoded includes a target reversible flow layer, the target reversible flow layer not only includes multiplication and division operations determined based on the model parameters, but the target reversible flow layer
  • the model parameters in can also constrain the auxiliary variable generated during the forward transformation process, the auxiliary variable being the increment of the product of the multiplication operation or the remainder generated by the division operation. That is to say, this application can not only use multiplication, but also realize reversibility by using auxiliary variables when performing division. Therefore, compared with the integer flow model in the traditional technology, the model representation ability is improved, which is conducive to improving the probability distribution of the model
  • the estimation accuracy rate can improve the compression efficiency of lossless compression.
  • Step 501 acquire coded data.
  • the encoded data refers to the data obtained by using the data encoding method introduced in the embodiment corresponding to FIG. 3 above.
  • the encoded data may be integer data.
  • the data processing device can obtain the coded data first, and then perform corresponding decoding processing based on the coded data.
  • the data processing device in the embodiment corresponding to FIG. 3 and the data processing device in the embodiment corresponding to FIG. 5 are located in the same device, or, the data processing device in the embodiment corresponding to FIG. 3
  • the data processing device in the embodiment corresponding to FIG. 5 is the same device.
  • the data processing device is a terminal device with a storage function, and the terminal device compresses and encodes the data to be encoded according to the method in the embodiment corresponding to FIG. 3 , and stores the obtained encoded data in the memory.
  • the terminal device reads the aforementioned encoded data from the memory, and decompresses the encoded data according to the method in the embodiment corresponding to FIG. 5 .
  • the data processing device in the embodiment corresponding to FIG. 3 and the data processing device in the embodiment corresponding to FIG. 5 are located in different devices, or, the data processing device in the embodiment corresponding to FIG. 3
  • the data processing device in the embodiment corresponding to the device in FIG. 5 is a different device.
  • the sending end will send the encoded data to the receiving end according to the method corresponding to the embodiment in Figure 3, and the receiving end will decode the encoded data according to the method corresponding to the embodiment in Figure 5 to obtain the compressed data.
  • Step 502 decode the coded data to obtain hidden variables.
  • the data processing device may decode the coded data to obtain hidden variables.
  • the encoded data may be decoded using the entropy decoding technology in the prior art to obtain reconstructed hidden variables.
  • Step 503 use the reversible flow model to perform inverse transformation processing on the latent variable, and obtain the decoding output;
  • the aforementioned reversible flow model includes a target reversible flow layer, and the target reversible flow layer includes model parameters, and the model parameters are used to constrain the inverse transformation
  • An auxiliary variable generated during processing, the operation corresponding to the target reversible flow layer includes a multiplication operation and a division operation determined based on the model parameters, and the auxiliary variable is the increment of the product of the multiplication operation or the remainder generated by the division operation.
  • the reversible flow model is a new flow model proposed by this application.
  • the target reversible flow layer is a layer specific to the reversible flow model proposed in this application.
  • the reversible flow model includes a plurality of reversible flow layers, and the plurality of reversible flow layers sequentially process data in a serial arrangement.
  • the aforementioned target reversible flow layer may be one of the aforementioned multiple reversible flow layers, or may be a component of one of the aforementioned multiple reversible flow layers.
  • FIG. 4A and FIG. 4B for the structure of the reversible flow model, reference may be made to the relevant descriptions corresponding to FIG. 4A and FIG. 4B above, and details are not repeated here.
  • the reversible flow model used by the data processing device in the data encoding method corresponding to FIG. 3 is the same as the reversible flow model used in the data decoding method corresponding to FIG.
  • the order of each reversible flow layer in the model is opposite to the order of data flowing through each reversible flow layer in the data decoding method. It can also be understood that the order of data flowing through the reversible flow layer during the reverse transformation process is the same as the above The order in which data flows through the reversible flow layer is reversed during the forward transformation.
  • the data processing order of each reversible flow layer in the reversible flow model is as shown in Figure 4A
  • the reverse transformation process the data processing order of each reversible flow layer in the reversible flow model
  • the sequence can be as shown in Figure 6A.
  • the reversible flow model includes N serial reversible flow layers, where N is an integer greater than 1.
  • the data flows through the i-1th reversible flow layer, the i-th reversible flow layer, and the i+1th reversible flow layer in turn, then in the reverse During the conversion process, as shown in the processing sequence on the right side of FIG. 6A , the data sequentially flows through the i+1th reversible flow layer, the i-th reversible flow layer, and the i-1th reversible flow layer.
  • the i is a positive integer greater than or equal to 1 and not greater than N.
  • the type of the reversible flow layer constituting the reversible flow model during the forward transformation process should be the same as the type of the reversible flow layer constituting the reversible flow model during the forward transformation process.
  • different types of reversible flow layers ie target reversible flow layer, reversible convolution layer and coupling layer
  • FIG. 4B different types of reversible flow layers (ie target reversible flow layer, reversible convolution layer and coupling layer) in the reversible flow model are combined and stacked according to the example shown in FIG. 4B .
  • the data to be encoded is used as the input of the target reversible flow layer 1, and the output of the target reversible flow layer 1 is used as the input of the coupling layer, and the output of the coupling layer As the input of the target reversible flow layer 2, the output of the target reversible flow layer 2 is used as the input of the reversible convolution layer, and so on, after the processing of the target reversible flow layer n, the hidden variable is output.
  • the hidden variable is used as the input of the target reversible flow layer n
  • the output of the target reversible flow layer n is the next reversible flow layer, and then enters the reversible convolution layer
  • the output of the reversible convolutional layer is used as the input of the target reversible flow layer 2
  • the output of the target reversible flow layer 2 is used as the input of the coupling layer
  • the output of the coupling layer is used as the input of the target reversible flow layer 1
  • the output is the data to be encoded described in the foregoing embodiments.
  • FIG. 6B is only an example of stacking many reversible flow layers. In practical applications, the stacking order of each reversible flow layer can be adjusted based on actual needs. This application does not limit the stacking order of the aforementioned reversible flow layers. No more examples.
  • the target reversible flow layer is used to transform and process the second data input into the target reversible flow layer into the first data introduced above.
  • the second data is the latent variable or the data output after being processed by at least one of the reversible flow layers
  • the first data is the decoding output or the output of the first data after being processed by at least one of the reversible flow layers is the decoded output.
  • the target reversible flow layer is used to inversely transform the second data input to the target reversible flow layer to obtain the first data output from the target reversible flow layer, the second data being the multiplier of the multiplication operation
  • the first data is an integer quotient of the division operation. That is to say, in the inverse transformation process, in the multiplication operation determined based on the model parameters, the second data is used as the multiplier of the multiplication operation, and another multiplier of the multiplication operation is determined by the model parameters; because,
  • the auxiliary variable can be used as an increment to the product of this multiplication operation, so that the intermediate data can be obtained based on the product of the aforementioned second data and another multiplier, plus the aforementioned auxiliary variable.
  • the first data is used as an integer quotient of the division operation, the divisor of the division operation is intermediate data, and the dividend is determined by the model parameters. Therefore, after the second data is input into the target reversible flow layer, the first data will be output after multiplication and division operations, and the first data is the output of the target reversible flow layer. Specifically, please refer to the embodiments corresponding to FIG. 9 and FIG. 10A below.
  • model parameters and auxiliary variables used in the inverse transformation process are generated or stored in the foregoing transformation process.
  • model parameters for example, the first parameter
  • auxiliary variables for example, the first auxiliary variable
  • the model parameters (eg, second parameter) and auxiliary variables (eg, second auxiliary variable) used in the division operation of the process are applied in the multiplication operation of the inverse transformation process.
  • model parameters and auxiliary variables reference may be made to the previous embodiment corresponding to FIG. 3 , which will not be repeated here.
  • the data processing device can directly output and code the aforementioned data.
  • the data before encoding is original data such as images, videos, or texts
  • the decoded output after the aforementioned inverse transformation processing is original data such as images, videos, or texts. If the data before encoding is preprocessed data based on original data such as images, videos or texts.
  • the data processing device may also restore the aforementioned decoded output to original data such as images, videos, or texts.
  • the data processing device may separate the noise data and the original data from the decoded output through the de-de-coding process.
  • the probability distribution of noise data is given (for example, the probability distribution of noise data u is q(u
  • x) ⁇ , and u ⁇ [0,1), where, ⁇ 2 -k , k represents the precision of fixed-point numbers)
  • the output of the reversible flow model is Then based on the output of the reversible flow model Carry out the lower integer operation (that is, calculate ) to get the original data.
  • the output of the reversible flow model Make a difference with the original data x (that is, calculate ), to get the noise data u.
  • x) ⁇ of the noise data u is used to encode u, so as to restore the code stream used before the inverse encoding process.
  • the data processing device performs inverse transformation processing on the decoded latent variables.
  • the reversible flow model of the inverse transformation process includes a target reversible flow layer, and the target reversible flow layer includes not only multiplication operations determined based on the model parameters and In the division operation, the model parameters in the target reversible flow layer can also constrain the auxiliary variable generated during the inverse transformation process, and the auxiliary variable is the increment of the product of the multiplication operation or the remainder generated by the division operation.
  • the present application can use the auxiliary variable to perform division operation for the multiplication operation in the forward transform process, so as to restore the hidden variable to the data before the forward transform process (that is, the data to be encoded in the aforementioned data encoding method) . Therefore, not only can the reversible operation of the transformation process be guaranteed, but also the representation ability of the model can be improved, which is conducive to improving the estimation accuracy of the probability distribution of the model, thereby improving the compression efficiency of lossless compression.
  • the target reversible flow layer will obtain auxiliary variables based on the model parameters, and then, based on the monotone continuous function determined by the auxiliary variables and the model parameters, the first data input to the target reversible flow layer will be transformed to obtain the output Second data of the target's reversible flow layer.
  • the first monotone continuous function is a function determined by the aforementioned auxiliary variables and model parameters, and the target reversible flow layer will use the aforementioned monotone continuous function to process the first data input to the target reversible flow layer during the transformation process, so as to This makes it possible to output the aforementioned second data after a series of processing.
  • the model parameters include a first parameter and a second parameter, both of the first parameter and the second parameter are positive integers, and the first parameter and the second parameter are not equal.
  • the auxiliary variable includes a first auxiliary variable and a second auxiliary variable.
  • the first auxiliary variable corresponds to the first parameter
  • the second auxiliary variable corresponds to the second parameter. It can be understood that the first auxiliary variable is determined based on the first parameter, and the second auxiliary variable is determined based on the second parameter.
  • the data processing device will perform the following steps in the forward transformation process through the target reversible flow layer:
  • Step 701 Obtain a first auxiliary variable based on a first parameter, the first parameter is used to determine the slope of the first linear function of one variable, and the first auxiliary variable is used to determine the intercept of the first linear function of one variable.
  • the data processing device uses auxiliary data to encode and decode the auxiliary variable, and the data processing device may obtain the aforementioned first auxiliary variable based on the probability distribution determined by the first parameter and the first auxiliary data.
  • the auxiliary data can be a code stream (for example, a binary code stream) pre-stored in the data processing device, or a code stream randomly generated by the reversible flow model in the data processing device, or a code stream obtained by the data processing device from an external storage device.
  • the obtained code stream may also be a code stream formed by compressed data, which is not specifically limited here.
  • the state vector bs can be set to store the binary code stream.
  • auxiliary data encoding auxiliary variables is as follows:
  • auxiliary data i.e., c3 and bs
  • the process of decoding auxiliary variables based on auxiliary data is as follows:
  • K and M are integers greater than 0.
  • the value of K may be 32, and the value of M may be 4. It should be understood that in practical applications, the aforementioned values of K and/or M may be adjusted based on calculation needs, which are not specifically limited here.
  • the data processing device when the data processing device obtains the first auxiliary variable based on the probability distribution determined by the first parameter and the first auxiliary data, the data processing device will determine the first uniform distribution based on the first parameter, and the first The endpoints of the uniform distribution are 0 and the first parameter. Then, the data processing device uses the first uniform distribution to decode the first auxiliary data to obtain the first auxiliary variable and second auxiliary data, where the value of the first auxiliary variable is between 0 and the first parameter.
  • the second auxiliary data is data obtained by decoding the first auxiliary data
  • the second auxiliary data is used to encode the first auxiliary variable during the inverse transformation process.
  • the process of decoding the first auxiliary data by using the first uniform distribution to obtain the first auxiliary variable and the second auxiliary data may specifically be: the data processing device takes the first auxiliary data as a divisor and the first parameter is The dividend is subjected to division and remainder operation to obtain the first auxiliary variable, and the first auxiliary data is used as the divisor and the first parameter is the dividend to perform division and round operation to obtain the second auxiliary data.
  • the data processing device will store the second auxiliary data in order to use the aforementioned second auxiliary data during the inverse transformation process.
  • the data processing device determines a first value range based on the first parameter, and the first value range is from 0 to the first parameter. Then, the data processing device randomly selects an integer from the aforementioned first value range as the first auxiliary variable.
  • the data processing device will also store the first auxiliary variable, so that the data processing device can perform inverse transformation processing based on the first auxiliary variable during subsequent inverse transformation processing.
  • Step 702 bring the first data into the first unary linear function for calculation to obtain intermediate data.
  • the data processing device performs an addition operation on a product obtained by multiplying the first parameter and the first data with the first auxiliary variable to obtain intermediate data.
  • the first auxiliary variable is the increment of the product of the multiplication operation.
  • the data processing device multiplies the first data as a multiplier of the first parameter to obtain the first product data; then, adds the first auxiliary variable to the increment of the first product data, get intermediate data.
  • the first auxiliary variable is also used as the remainder generated by the division operation in the inverse transformation process.
  • the first auxiliary variable participates in the process of determining the intermediate data, that is, the first auxiliary variable is used as the increment of the product of the multiplication operation
  • the integer quotient of the division and rounding operation based on the intermediate data is the second Data, therefore, avoids the problem of irreversible operations caused by only multiplication operations, and also avoids the problem of poor model representation ability caused by only addition and subtraction operations.
  • Step 703 Determine the second data based on the intermediate data and the second parameter, and output the second data.
  • the data processing device will use the intermediate data as a divisor and the second parameter as a dividend, perform division and rounding operations to obtain the second data, and output the second data.
  • Step 704 determining a second auxiliary variable based on the intermediate data and the second parameter.
  • the data processing device will use the intermediate data as a divisor and the second parameter as a dividend to perform division and remainder operation to obtain the second auxiliary variable.
  • the second parameter and the second auxiliary variable are used to determine a second unary linear function
  • the second unary linear function is a function used in the inverse transformation process corresponding to the aforementioned forward transformation process.
  • the second parameter is used to determine the slope of the second linear function of one variable
  • the second auxiliary variable is used to determine the intercept of the second linear function of one variable.
  • the second auxiliary variable is the remainder generated by the division operation.
  • the second auxiliary variable is also used as an increment of the product of the multiplication in the corresponding inverse transform process of the forward transform process.
  • the second auxiliary variable since the target reversible flow model is in the forward transformation process, the second auxiliary variable will also be generated, and the second auxiliary variable is used as the product of the multiplication operation in the inverse transformation process corresponding to the forward transformation process increment. That is to say, in the inverse transformation process, the data processing device restores the second data to the intermediate data based on the second auxiliary variable, and then restores the intermediate data to the first data. Therefore, in the forward transformation process, the second auxiliary variable is calculated, which is beneficial to restore the second data to the first data.
  • step 703 and step 704. The data processing device may first perform step 703 and then step 704, or the data processing device may first perform step 704 and then perform step 703.
  • the data The processing device may also execute step 703 and step 704 simultaneously, which is not specifically limited here.
  • Step 705 storing the first auxiliary variable and the second auxiliary variable.
  • step 705 should be performed after step 704 .
  • the data processing device may directly store the first auxiliary variable and the second auxiliary variable, and the data processing device may also store the first auxiliary variable and the second auxiliary variable indirectly by encoding.
  • the data processing device may directly store the first auxiliary variable and the second auxiliary variable.
  • the data processing apparatus may store the first auxiliary variable after generating the first auxiliary variable in step 701, and then, the data processing apparatus may store the second auxiliary variable after generating the second auxiliary variable in step 704.
  • the data processing device may store the first auxiliary variable and the second auxiliary variable correspondingly, so that during the inverse transformation process, the data processing device may obtain the first auxiliary variable corresponding to the second auxiliary variable, In turn, it is beneficial to improve the efficiency of the inverse transformation process.
  • the data processing device may correspondingly store the first auxiliary variable and the second auxiliary variable in an array or a table, which is not specifically limited here.
  • the data processing device may store the auxiliary data encoded with the auxiliary variable instead of directly storing the auxiliary variable. That is to say, the data processing device can temporarily store the second auxiliary data obtained by decoding the first auxiliary variable in the aforementioned step 701, and after step 704, the data processing device can encode the second auxiliary data based on the same principle with reference to the following manner two auxiliary variables, and store the auxiliary data encoded with the second auxiliary variable (the third auxiliary data to be introduced later), so as to realize the indirect storage of the first auxiliary variable and the second auxiliary variable.
  • the data processing apparatus may encode the second auxiliary variable based on the uniform distribution entropy encoding principle introduced in the foregoing step 701 .
  • the data processing device may obtain temporarily stored second auxiliary data (ie, the second auxiliary data obtained by decoding the first auxiliary variable), and at the same time, determine the second uniform distribution based on the second parameter, and the second uniform distribution The endpoints are 0 and the second parameter.
  • the data processing device uses the second uniform distribution and the second auxiliary data to encode the second auxiliary variable to obtain third auxiliary data, the value of the second auxiliary variable is between 0 and the second parameter.
  • the data processing device then stores the third auxiliary data, which is used for decoding the second auxiliary variable during the inverse transformation.
  • the process of encoding the second auxiliary variable by using the second uniform distribution and the second auxiliary data to obtain the third auxiliary data may specifically be: the data processing device performs a multiplication operation on the second auxiliary data and the second parameter The obtained product is added to the second auxiliary variable to obtain the third auxiliary data.
  • the second data input to the target reversible flow layer will be multiplied, and the first data output from the target reversible flow layer will be obtained through division. Therefore, compared to the method that only uses integer addition and subtraction For the integer flow model, the reversible flow model of the present application has stronger model representation capabilities, which is conducive to improving the estimation accuracy of the probability distribution of the model, thereby improving the compression efficiency of lossless compression.
  • Step 801 acquire first auxiliary data, where the first auxiliary data is used to decode a first auxiliary variable.
  • Step 802 decoding the first auxiliary data by using the first uniform distribution to obtain the first auxiliary variable and the second auxiliary data.
  • the first uniform distribution is determined based on the first parameter.
  • the related introduction please refer to the related introduction in the embodiment corresponding to FIG. 7 above.
  • Step 803 adding the first auxiliary variable to the first product data obtained by multiplying the first data and the first parameter to obtain intermediate data.
  • step 804 the intermediate data is used as a divisor and the second parameter is used as a dividend to perform a division and rounding operation to obtain the second data.
  • Step 805 Perform division and remainder operation with the intermediate data as the divisor and the second parameter as the dividend to obtain a second auxiliary variable.
  • step 804 and step 805. The data processing device may first perform step 804 and then step 805, or the data processing device may first perform step 805 and then perform step 804. The data The processing device may also execute step 804 and step 805 respectively at the same time, which is not specifically limited here.
  • Step 806 Encode the second auxiliary variable with the second auxiliary data to obtain third auxiliary data.
  • step 806 should be performed after step 805 .
  • Step 807 storing third auxiliary data.
  • the transformation process (step 701 to step 705) of the target reversible flow layer will be introduced below with reference to specific examples.
  • FIG. 8B it is a target reversible flow layer (for example, reversible flow layer i) in the reversible flow model.
  • x represents the data of the input target reversible flow layer (i.e. the first data), which is the output of the previous reversible flow layer (for example, reversible flow layer i-1);
  • z represents the data of the output target reversible flow layer (i.e.
  • the second data is the input of the next reversible flow layer (for example, reversible flow layer i+1); b1 represents the first auxiliary data, b2 represents the second auxiliary data, and b3 represents the third auxiliary data.
  • the first parameter is R and the second parameter is S
  • the first uniform distribution determined based on the first parameter R is U(0, R)
  • the second uniform distribution determined based on the second parameter S is U(0, S ).
  • the data processing device will store the aforementioned third auxiliary data b3, so as to decode the second auxiliary variable r e and the second auxiliary data b2 based on the third auxiliary data b3 during the inverse transformation process, and, based on the second auxiliary data b2 Encodes the first auxiliary variable rd .
  • the i-th reversible flow layer (i.e. Layer i) is also used to process the fixed-point number data based on the first preset precision into integer data; and, after performing the transformation process shown in FIG.
  • the fixed-point data with the first preset precision is processed as an output of the ith reversible flow layer (ie, reversible flow layer i).
  • the target reversible flow layer can encode and decode auxiliary variables based on model parameters (for example, first parameter and second parameter) and auxiliary data (for example, first auxiliary data, second auxiliary data, and third auxiliary data) (for example, the first auxiliary variable and the second auxiliary variable), the auxiliary variable can realize the reversibility of the data subjected to the multiplication operation through the division operation. Since the target reversible flow layer involves not only addition operations, but also multiplication operations, division operations, rounding operations, and remainder operations, the diversity of transformation processing can be improved and the representation ability of the model can be improved.
  • model parameters for example, first parameter and second parameter
  • auxiliary data for example, first auxiliary data, second auxiliary data, and third auxiliary data
  • the auxiliary variable can realize the reversibility of the data subjected to the multiplication operation through the division operation. Since the target reversible flow layer involves not only addition operations, but also multiplication operations, division operations, rounding operations, and remainder operations, the diversity of transformation processing can be improved and
  • first parameter and the second parameter in the aforementioned target reversible flow layer may be preset parameters, parameters generated based on a neural network, or determined by the data processing device based on an interpolation method or a binary search algorithm. parameters.
  • R can be defined as the first parameter
  • S can be defined as the second parameter.
  • round(x) means to return the rounded integer value of x.
  • the data processing device may split the nonlinear function into multiple segments, and divide each A segment is approximately a straight line, and the aforementioned first parameter and second parameter are calculated by using an interpolation algorithm or a binary search method. Specifically, the data processing device may acquire a first interval in the domain of the first monotone continuous function and a second interval corresponding to the first interval on the value domain of the first monotone continuous function. Then, the data processing device will calculate the first parameter and the second parameter based on the first interval and the second interval by using an interpolation method or a binary search method.
  • the first interval in the domain of the first monotone continuous function is [x l , x h )
  • the first interval is in the value
  • the corresponding second interval on the domain is [z l , z h ), where x l , x h , z l , and z h are all fixed-point numbers (ie floating-point numbers) with a precision of k.
  • the data processing apparatus may use any one of the following implementation manners to determine the aforementioned first interval and the second interval.
  • the data processing device may perform uniform interpolation based on the domain of definition, and this implementation manner is applicable to situations where the value domain is larger than the domain of definition.
  • the reverse interpolation interval needs to be determined by searching, and the determination process is as follows:
  • the data processing device may perform linear interpolation based on the value range, and this implementation manner is applicable to a situation where the definition domain is larger than the value range.
  • the forward interpolation interval is determined by searching, and the determination process is as follows:
  • the target reversible flow layer is a flow layer constructed based on a linear function
  • the target reversible flow layer can be called a linear reversible flow layer (or a linear flow layer); if the aforementioned target reversible flow layer is based on a nonlinear function
  • the constructed flow layer, then the target reversible flow layer can be called a nonlinear reversible flow layer (or nonlinear flow layer).
  • the target reversible flow layer is a nonlinear reversible flow layer
  • the nonlinear function is divided into multi-segment linear functions by using interpolation algorithm or binary search algorithm
  • the numerical reversible operation of the nonlinear flow layer is converted into the numerical reversible operation of the linear flow layer question. It is beneficial to realize the diversity of the target reversible flow layer, and then diversify each reversible flow layer in the reversible flow model, which is conducive to improving the representation ability of the reversible flow model.
  • the reversible flow layer in this application can be not only the aforementioned target reversible flow layer, but also a deformable layer of the target reversible flow layer, and the deformable layer of the target reversible flow layer can be a reversible convolution layer or a coupling layer.
  • the reversible flow model includes not only the target reversible flow layer, but also a reversible convolutional layer and/or coupling layer.
  • one or more of the N reversible flow layers is a reversible convolution layer
  • the reversible convolution layer includes a convolution kernel and the target reversible flow layer
  • the convolution The kernel is a square matrix whose order is the number of channels of the data to be encoded.
  • the reversible convolution layer is used for LU decomposition (LU Factorization) of the convolution kernel to obtain the target matrix, the target matrix is a diagonal matrix, and the non-zero elements on the diagonal of the diagonal matrix are used to determine the reversible The model parameters for this target reversible flow layer in the convolutional layer.
  • LU decomposition LU Factorization
  • the reversible convolutional layer is 1x1 convolutional layer as an example for introduction.
  • the parameter of the reversible convolution layer is a convolution kernel W, and the convolution kernel W satisfies W ⁇ R c ⁇ c , where c represents the dimension of data x input to the reversible convolution layer.
  • the matrix W is a displace matrix
  • the matrix L is a lower triangular matrix (LTM)
  • the matrix U is an upper triangular matrix (upper triangular matrix) , UTM)
  • matrix ⁇ is a diagonal matrix (diagonal matrix).
  • each element ⁇ i in the main diagonal in the diagonal matrix ⁇ can be represented by a fraction, (that is, ⁇ i can be expressed as the division of two integers), and the numerator of the fraction can be used as the first parameter R, and the fraction The denominator of is used as the second parameter S to construct the target reversible flow layer. Then, using h i as the first data input to the target reversible flow layer in the reversible convolution layer, calculate and output the second data m i of the target reversible flow layer in the reversible convolution layer.
  • n the product of the data m and the lower triangular matrix L is calculated.
  • n Lm
  • l ij is the element of row i and column j of L
  • m i is the i-th element of m
  • n i is the i-th element of n
  • the 1x1 convolutional layer is converted into a matrix multiplication operation of continuous upper triangular matrix, diagonal matrix, lower triangular matrix, and scrambled matrix, and iterative calculations are used for the four matrix multiplications.
  • one or more of the N reversible flow layers is a coupling layer, and the coupling layer includes the target reversible flow layer.
  • the coupling layer is used to divide the data input into the coupling layer into at least two parts of data according to preset dimensions, and process at least a part of the at least two parts of data as the input of the target reversible flow layer to obtain the coupling layer Output.
  • the coupling layer may include a neural network, and the neural network is used to process the data input to the coupling layer in different dimensions.
  • the neural network can include various types. For example, convolutional neural network (CNN), deep neural network (DNN), recurrent neural network (RNN) or other neural networks. Specifically, there is no limitation here.
  • the coupling layer can divide the data input into the coupling layer according to dimensions, and only use the conversion method of the target reversible flow layer to transform the data of some of the dimensions. Therefore, the diversity of data transformation can be realized, and the representation ability of the reversible flow model can be improved.
  • reversible flow layers ie, target reversible flow layer, reversible convolution layer and coupling layer
  • target reversible flow layer ie, target reversible flow layer, reversible convolution layer and coupling layer
  • coupling layer ie, reversible convolution layer and coupling layer
  • the target reversible flow layer will obtain auxiliary variables based on the model parameters, and then, based on the monotone continuous function determined by the auxiliary variables and the model parameters, the second data input to the target reversible flow layer will be transformed to obtain the output The first data of the target reversible flow layer.
  • the monotone continuous function is a function determined by the aforementioned auxiliary variables and model parameters
  • the target reversible flow layer will use the aforementioned first monotone continuous function to process the second data input to the target reversible flow layer during the transformation process, so as to After a series of processing, the aforementioned first data can be output, that is, the aforementioned second data can be restored to the aforementioned first data.
  • the model parameters include a first parameter and a second parameter, both of the first parameter and the second parameter are positive integers, and the first parameter and the second parameter are not equal.
  • the auxiliary variable includes a first auxiliary variable and a second auxiliary variable.
  • the first auxiliary variable corresponds to the first parameter
  • the second auxiliary variable corresponds to the second parameter. It can be understood that the first auxiliary variable is determined based on the first parameter, and the second auxiliary variable is determined based on the second parameter.
  • the data processing device will perform the following steps in the inverse transformation process through the target reversible flow layer:
  • Step 901 acquire a second auxiliary variable, the second auxiliary variable is determined based on a second parameter, and the second auxiliary variable and the second parameter are used to determine a second unary linear function.
  • the second auxiliary variable is an auxiliary variable generated during the forward transformation.
  • the second auxiliary variable is determined based on a second parameter, and the second auxiliary variable and the second parameter are used to determine a second linear function of one variable.
  • the second parameter is used to determine the slope of the second unary linear function
  • the second auxiliary variable is used to determine the intercept of the second unary linear function.
  • the data processing device uses auxiliary data to encode and decode auxiliary variables, and stores auxiliary data (for example, third auxiliary data) used for encoding and decoding auxiliary variables, Then, during the inverse transformation process, the data processing device will first obtain the third auxiliary data, and then decode the second auxiliary variable based on the third auxiliary data.
  • auxiliary data for example, third auxiliary data
  • auxiliary data decoding auxiliary variables for the calculation principle of auxiliary data decoding auxiliary variables, reference may be made to the relevant introduction in step 701 above, and details are not described here.
  • the data processing device when the data processing device acquires the second auxiliary variable based on the probability distribution determined by the second parameter and the third auxiliary data, the data processing device will determine the second uniform distribution based on the second parameter, and the second uniform distribution The endpoint is 0 and the second parameter. Then, decoding the third auxiliary data by using the second uniform distribution to obtain the second auxiliary variable and the second auxiliary data, where the value of the second auxiliary variable is between 0 and the second parameter.
  • the second auxiliary data is data obtained by decoding the third auxiliary data.
  • the process of decoding the third auxiliary data by using the second uniform distribution to obtain the second auxiliary variable and the second auxiliary data may specifically be: the data processing device takes the third auxiliary data as a divisor and the second parameter is the dividend, perform division and remainder operation to obtain the second auxiliary variable, and take the third auxiliary data as the divisor and the second parameter as the dividend, perform division and round operation to obtain the second auxiliary data.
  • the data processing device may temporarily store the aforementioned second auxiliary data, so as to facilitate the use of the aforementioned second auxiliary data in a subsequent transformation process.
  • the first auxiliary variable is encoded during subsequent processing using the second auxiliary data.
  • the data processing device directly stores the second auxiliary variable without using auxiliary data, then the data processing device will directly store the second auxiliary variable from the position gets the aforementioned second auxiliary variable.
  • Step 902 bring the second data into the second unary linear function for calculation to obtain intermediate data.
  • the data processing device performs an addition operation on a product obtained by multiplying the second parameter and the second data with the second auxiliary variable to obtain intermediate data.
  • the second auxiliary variable is the increment of the product of the multiplication operation.
  • the data processing device multiplies the second data as a multiplier of the second parameter to obtain the second product data; then, adds the second auxiliary variable to the increment of the second product data, get intermediate data.
  • the second auxiliary variable participates in the process of determining the intermediate data, that is, the second auxiliary variable is used as the increment of the product of the multiplication operation, the integer quotient of the division and rounding operation based on the intermediate data is the first Data, therefore, avoids the problem of irreversible operations caused by only multiplication operations, and also avoids the problem of poor model representation ability caused by only addition and subtraction operations.
  • Step 903 Determine the first data based on the intermediate data and the first parameter, and output the first data.
  • the data processing device will use the intermediate data as a divisor and the first parameter as a dividend, perform division and rounding operations to obtain the first data, and output the first data.
  • the first data is output data of the target reversible flow layer after inverse transformation processing of the target reversible flow layer.
  • Step 904 determine a first auxiliary variable based on the intermediate data and the first parameter.
  • the data processing device will use the intermediate data as a divisor and the first parameter as a dividend to perform division and remainder operation to obtain the first auxiliary variable.
  • the first parameter and the first auxiliary variable are used to determine the first unary linear function
  • the first unary linear function is the next forward transformation process (that is, the next transformation process corresponding to this inverse transformation process)
  • the function used, the first parameter is used to determine the slope of the first linear function of one variable, and the first auxiliary variable is used to determine the intercept of the first linear function of one variable.
  • the first auxiliary variable is the remainder generated by the division operation.
  • step 903 and step 904. The data processing device may first perform step 903 and then step 904, or the data processing device may first perform step 904 and then perform step 903.
  • the data The processing device may also execute step 903 and step 904 respectively simultaneously, which is not specifically limited here.
  • Step 905 storing the first auxiliary variable.
  • the data processing device may directly store the first auxiliary variable.
  • the data processing apparatus may store the first auxiliary variable after generating the first auxiliary variable in step 904 .
  • the data processing device may store the first auxiliary variable and the originally stored second auxiliary variable correspondingly, so that in the subsequent transformation process, the data processing device may obtain the first auxiliary variable corresponding to the second auxiliary variable.
  • Auxiliary variables which in turn help to improve the efficiency of the subsequent transformation process.
  • the data processing device may store the auxiliary data encoded with the auxiliary variable instead of directly storing the auxiliary variable.
  • the data processing apparatus may use the second auxiliary data generated in step 901 to encode the first auxiliary variable to obtain the first auxiliary data, and store the first auxiliary data.
  • the data processing apparatus may encode the first auxiliary variable based on the uniform distribution entropy encoding principle introduced in step 701 in the embodiment corresponding to FIG. 7 above.
  • the data processing device may acquire the second auxiliary data generated in step 901, and at the same time, determine a first uniform distribution based on the first parameter, the endpoint of the first uniform distribution is 0 and the first parameter.
  • the data processing device uses the first uniform distribution and the second auxiliary data to encode the first auxiliary variable to obtain first auxiliary data, where the value of the first auxiliary variable is between 0 and the first parameter.
  • the data processing means stores the first auxiliary data.
  • the process of encoding the first auxiliary variable by using the first uniform distribution and the second auxiliary data to obtain the first auxiliary data may specifically be: the data processing device performs a multiplication operation on the second auxiliary data and the first parameter The obtained product is added to the first auxiliary variable to obtain the first auxiliary data.
  • the target reversible flow layer can output the first auxiliary data again during the reverse transformation process, that is, the target
  • the reversible stream layer can restore the code stream used in the compression encoding process (ie, the forward transformation process), so that the used code stream can be reversible.
  • Step 1001 acquire third auxiliary data, the third auxiliary data is used to decode the second auxiliary variable.
  • auxiliary data is auxiliary data stored by the data processing device during the forward transformation process.
  • Step 1002 decode the third auxiliary data by using the second uniform distribution to obtain the second auxiliary variable and the second auxiliary data.
  • the second uniform distribution is determined based on the second parameter.
  • step 1003 the second product data obtained by multiplying the second data and the second parameter is added to the second auxiliary variable to obtain intermediate data.
  • Step 1004 performing division and rounding operation with the intermediate data as the divisor and the first parameter as the dividend, to obtain the first data.
  • Step 1005 performing division and remainder operation with the intermediate data as the divisor and the first parameter as the dividend, to obtain the first auxiliary variable.
  • step 1004 and step 1005. The data processing device may first perform step 1004 and then step 1005, or the data processing device may first perform step 1005 and then perform step 1004.
  • the data The processing device may also execute step 1004 and step 1005 respectively at the same time, which is not specifically limited here.
  • Step 1006 encoding the first auxiliary variable with the second auxiliary data to obtain the first auxiliary data.
  • Step 1007 storing the first auxiliary data.
  • step 901 to step 905 the conversion process (step 901 to step 905) of the target reversible flow layer will be introduced below with reference to specific examples.
  • FIG. 10B it is a certain target reversible flow layer (for example, reversible flow layer i) in the reversible flow model.
  • z represents the data of the input target reversible flow layer (i.e. the second data), which is the output of the previous reversible flow layer (for example, reversible flow layer i+1);
  • x represents the data of the output target reversible flow layer (i.e.
  • the first data is the input of the next reversible flow layer (eg, reversible flow layer i-1); b1 represents the first auxiliary data, b2 represents the second auxiliary data and b3 represents the third auxiliary data.
  • the first uniform distribution determined based on the first parameter R is U(0, R)
  • the second uniform distribution determined based on the second parameter S is U(0, S ).
  • the data processing device may store the aforementioned first auxiliary data b1, and restore the code stream in the storage device (for example, a memory in the data processing device).
  • the ith can
  • the reverse flow layer (that is, the reverse flow layer i) is also used to process the fixed-point number data based on the first preset precision into integer data; and, after performing the transformation process shown in FIG.
  • the transformed integer data is processed into fixed-point number data with the first preset precision as an output of the ith reversible flow layer (ie, reversible flow layer i).
  • the target reversible flow layer can encode and decode auxiliary variables based on model parameters (for example, first parameter and second parameter) and auxiliary data (for example, first auxiliary data, second auxiliary data, and third auxiliary data) (for example, the first auxiliary variable and the second auxiliary variable), the auxiliary variable can realize the reversibility of the data subjected to the multiplication operation through the division operation. Since the target reversible flow layer involves not only addition operations, but also multiplication operations, division operations, rounding operations, and remainder operations, the diversity of transformation processing can be improved and the representation ability of the model can be improved.
  • model parameters for example, first parameter and second parameter
  • auxiliary data for example, first auxiliary data, second auxiliary data, and third auxiliary data
  • the auxiliary variable can realize the reversibility of the data subjected to the multiplication operation through the division operation. Since the target reversible flow layer involves not only addition operations, but also multiplication operations, division operations, rounding operations, and remainder operations, the diversity of transformation processing can be improved and
  • the data encoding method and data decoding method provided by this application can be compressed and decompressed. Effectively improve the compression efficiency, specifically, the compression efficiency can be increased by 5 times.
  • the embodiments of the present application achieve good compression ratios on various image data sets, and the embodiments of the present application can be effectively applied to image lossless compression tasks.
  • the optimal lossless compression rate has been achieved in CIFAR10, ImageNet32, ImageNet and other data sets.
  • An important reason for the good compression ratio is that the used reversible flow model has a strong ability to fit the data distribution.
  • the reversible flow model in the embodiment of the present application has good generalization performance, and can use a single model to complete the compression of various types and sizes of images.
  • this application proposes a new numerically reversible flow model calculation method (that is, the reverse flow model), and proposes corresponding numerically reversible calculation methods for flow layers such as unary linear/nonlinear flow, 1x1 convolutional layer, and coupling layer. , to eliminate the numerical error of the flow model.
  • This method is simple and efficient. Compared with the local bits-back cocding (LBB) model in the traditional technology, it does not require a complicated local encoding and decoding process, and can greatly improve the efficiency of lossless compression.
  • LBB local bits-back cocding
  • FIG. 11 it is a schematic structural diagram of a data processing device 110 provided in the embodiment of the present application.
  • the data processing apparatus 110 may be a server, a terminal device or other devices that require compressed storage, which is not specifically limited here.
  • the steps in the above embodiments corresponding to FIG. 3 , FIG. 5 , FIG. 7 , FIG. 8A , FIG. 9 and FIG. 10A may be executed by the data processing device 110 .
  • the data processing device 110 acts as an encoding end, the data processing device 110 will execute the steps in the embodiments corresponding to FIG. 3 , FIG. 7 and FIG. 8A .
  • the data processing device 110 acts as a decoder, the data processing device 110 will execute the steps in the embodiments corresponding to FIG. 5 , FIG. 9 and FIG. 10A .
  • the data processing device 110 includes at least one processor 1101 and at least one memory 1102 .
  • the foregoing processor 1101 and the foregoing memory 1102 are interconnected through lines. It should be understood that FIG. 11 only shows one processor 1101 and one memory 1102 .
  • the processor 1101 may be a general-purpose central processing unit (central processing unit, CPU), a microprocessor, a network processor (network processor, NP) or a specific application-specific integrated circuit (application-specific integrated circuit), or one or more An integrated circuit for controlling program execution of the scheme of this application.
  • the foregoing processor 1101 may be a single-core (single-CPU) processor, or may be a multi-core (multi-CPU) processor.
  • Processor 1101 may refer to one or more devices, circuits, and/or processing cores for processing data (eg, computer program instructions).
  • the processor 1101 can be an independent semiconductor chip, or can be integrated with other circuits to form a semiconductor chip.
  • a system on a chip (system-on-a-chip, SoC), or can also be integrated in the described ASIC as a built-in processor of a special application integrated circuit (application specific integrated circuit, ASIC), and the ASIC integrated with the processor can Packaged separately or together with other circuits.
  • SoC system-on-a-chip
  • ASIC application specific integrated circuit
  • the aforementioned memory 1102 may be a read-only memory (read-only memory, ROM), or other types of static storage devices that can store static information and instructions, or a random access memory (random access memory, RAM). , can also be other types of dynamic storage devices that can store information and instructions, and can also be electrically erasable programmable read-only memory (electrically erasable programmable read-only memory, EEPROM), which is not limited here.
  • the memory 1102 may exist independently, but is connected to the aforementioned processor 1101 .
  • the memory 1102 may also be integrated with the aforementioned processor 1101 . For example, integrated within one or more chips.
  • the memory 1102 is also used to store program codes for executing the technical solutions of the embodiments of the present application.
  • the foregoing program codes may be executed under the control of the processor 1101 , and various types of computer program codes to be executed may also be regarded as drivers of the processor 1101 . Therefore, the aforementioned processor 1101 can obtain model parameters such as the first parameter and the second parameter, and determine auxiliary variables (for example, the first auxiliary variable and the second auxiliary variable) based on the model parameters, and the input can be determined based on the model parameters and the auxiliary variables.
  • the data of the countercurrent model is transformed.
  • the processor 1101 in the data processing device 110 invokes the program code stored in the memory 1102 to run the reversible flow model.
  • the processor 1101 in the data processing device 110 invokes the program code stored in the memory 1102 to run a uniform entropy coder (the uniform entropy coder can use The second uniform distribution determined by the parameters).
  • the processor 1101 reads the data to be encoded from the memory 1102 (for example, various types of original data in FIG. 12 ), uses the reversible flow model to perform forward transformation processing on the data to be encoded to obtain hidden variables, and then, The encoder compresses and encodes the latent variable to obtain encoded data (ie, compressed data).
  • the processor 1101 stores the generated encoded data in the memory 1102 .
  • the processor 1101 reads the encoded data (ie compressed data) from the memory 1102 and performs decoding processing to obtain hidden variables, and then uses the reversible flow model to inversely transform the hidden variables to obtain the decoded output (that is, the data before encoding, that is, the data to be encoded).
  • the processor 1101 stores the generated data to be encoded in the memory 1102, or directly outputs the data before encoding (that is, the data to be encoded).
  • the data processing apparatus 110 further includes a communication interface 1103, which is used for communicating with other servers or network devices.
  • the data processing device 110 can receive instructions or data from other devices through the communication interface 1103 .
  • the communication interface 1103 is a transceiver, for example, a wireless transceiver or an optical transceiver.
  • the data processing device 110 may receive the first parameter and the second parameter set by the user through the communication interface 1103 .
  • the data processing apparatus 110 may receive auxiliary data, for example, first auxiliary data or third auxiliary data, etc. through the communication interface 1103 .
  • the data processing device 110 may acquire the first auxiliary data through the communication interface 1103, and then, the processor 1101 in the data processing device 110 decodes the first auxiliary data using the first uniform distribution determined based on the first parameter to obtain the first Auxiliary variables and second auxiliary data.
  • the processor 1101 uses the second uniform distribution determined based on the second parameter and the aforementioned second auxiliary data to encode the second auxiliary variable to obtain third auxiliary data.
  • the data processing device 110 may output and store the third auxiliary data to an external storage medium through the communication interface 1103 , or directly store the third auxiliary data in the memory 1102 of the data processing device 110 . If the data processing device 110 outputs and stores the third auxiliary data to an external storage medium through the communication interface 1103, then during the inverse transformation process, the data processing device 110 will also read the third auxiliary data from the external storage medium through the communication interface 1103. data.
  • the data processing apparatus 110 may also send instructions or data to other devices through the communication interface 1103 .
  • the data processing device 110 serving as the encoding end may send encoded data and model parameters (for example, the first parameter and the second parameter) to other devices serving as the decoding end through the communication interface 1103 , so that the decoding end can perform decoding processing and inverse transformation processing based on the encoded data and model parameters to obtain the decoding output.
  • the embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a program for constructing a reversible flow model, and when the aforementioned computer program is run on a computer, the computer can execute the 3.
  • the aforementioned data processing device 110 shown in FIG. 11 is a chip, which is also called a digital processing chip.
  • the chip includes a processing unit and a communication unit.
  • the processing unit obtains program instructions through the communication unit, and the program instructions are executed by the processing unit, so that the processing unit executes the method steps described in the corresponding embodiments of the preceding figures 3, 5, 7, 8A, 9 and 10A .
  • the processing unit is a circuit that integrates the above-mentioned processor 1101 or is used to realize the functions of the above-mentioned processor 1101
  • the communication unit is a circuit or an interface that integrates the above-mentioned communication interface 1103 or is used to realize the functions of the above-mentioned communication interface 1103 .
  • the storage unit may be a storage device such as a memory.
  • the processing unit in the chip can call the program code from the storage unit to implement the method steps described in the aforementioned embodiments corresponding to FIG. 3 , FIG. 5 , FIG. 7 , FIG. 8A , FIG. 9 and FIG. 10A .
  • the chip can be connected to an external storage device such as an external memory through the aforementioned communication unit, so that the program code can be obtained from the aforementioned external memory to realize the aforementioned Figure 3, Figure 5, Figure 7, FIG. 8A , FIG. 9 and FIG. 10A correspond to the method steps introduced in the embodiment.
  • FIG. 13 is a kind of structural schematic diagram of the chip that the embodiment of the present application provides, and this chip can be represented as neural network processor (neural-network processing units, NPU) 130, and this neural network processor 130 is mounted on as coprocessor On the main CPU (Host CPU), tasks are assigned by the Host CPU.
  • the core part of the neural network processor 130 is an operation circuit 1301 .
  • the neural network processor 130 controls the operation circuit 1301 through the controller 1302 to obtain the matrix data in the memory (for example, the weight memory and the input memory described later) and perform multiplication.
  • the operation circuit 1301 includes multiple processing units (process engine, PE).
  • the arithmetic circuit 1301 is a two-dimensional systolic array.
  • the operation circuit 1301 may also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition.
  • the operation circuit 1301 is a general matrix processor. Specifically, the embodiment of the present application does not limit the specific form of the aforementioned computing circuit 1301 .
  • an input matrix A is stored in the input memory 1304
  • a weight matrix B is stored in the weight memory 1303 .
  • the operation circuit 1301 will fetch the data corresponding to the matrix B from the weight memory 1303 and store it in each PE in the operation circuit 1301 .
  • the operation circuit 1301 will also obtain the matrix A from the input memory 1304, and then use the aforementioned matrix A and matrix B to perform matrix operations to obtain the output matrix C.
  • the output matrix C can be a partial result or a final result, and the output matrix C will be saved in an accumulator (accumulator) 1305 .
  • the neural network processor 130 also includes a unified memory 1306 for storing input data and output data.
  • the input data can be transferred to the unified memory 1306 through a storage unit access controller (direct memory access controller, DMAC) 1307 .
  • DMAC direct memory access controller
  • Part of the input data or output data can be transferred to the weight storage 1303 through the storage unit access controller 1307 .
  • externally input weight data for example, weight matrix
  • the neural network processor 130 has a bus interface unit (bus interface unit, BIU) 1308, which is used to access the controller 1307 and the instruction fetch buffer (instruction fetch buffer, IFB) through the bus protocol (advanced extensible interface, AXI) and the storage unit 1309 interactions.
  • BIU bus interface unit
  • the bus interface unit 1308 is also used to obtain instructions from the external memory through the instruction fetch memory 1309, and is also used to obtain input data from the external memory through the storage unit access controller 1307, for example, the original input matrix A in the previous example Data or the original data of the weight matrix B, etc.
  • the neural network processor 130 further includes a vector calculation unit 1310 .
  • the vector calculation unit 1310 includes a plurality of operation processing units. If necessary, the vector computing unit 1310 can further process the output of the computing circuit 1301 . For example, vector multiplication, vector addition, exponential operations, logarithmic operations, or size comparisons.
  • the vector calculation unit 1310 is also used for non-convolutional/fully connected layer network calculations in neural networks, such as batch normalization (batch normalization) (also known as batch normalization), pixel-level summation, and feature map Upsampling and other processing.
  • the aforementioned vector calculation unit 1310 can store the processed and output vectors in the unified memory 1306, so that the operation circuit 1301 can call the vectors, functions and other data in the unified memory 1306 in the subsequent calculation process .
  • the vector calculation unit 1310 can store the linear function and/or the nonlinear function in the unified memory 1306, so that the vector calculation unit can call the linear function and/or the nonlinear function from the unified memory 1306 to perform the operation on the output of the operation circuit 1301. Further process, and store the processing result in the unified memory 1306 or the accumulator 1305.
  • the vector calculation unit 1310 may perform linear transformation or nonlinear transformation on the data to be processed.
  • the vector calculation unit 1310 may also be used to generate a normalized value, a pixel-level summed value, or both.
  • the processed output vector can be used as an activation input to the operation circuit 1301 so that the output vector can be used in subsequent hidden layers in the neural network.
  • controller 1302 is also connected to an instruction fetch memory 1309 , so that the instruction fetch memory 1309 can store instructions used by the controller 1302 .
  • weight memory 1303 input memory 1304 , unified memory 1306 and instruction fetch memory 1309 are all on-chip (On-Chip) memories.
  • external memory is private to the neural network processor hardware architecture.
  • each layer in the neural network can be performed by the operation circuit 1301 or the vector calculation unit 1310 .
  • the processor mentioned in any of the above-mentioned places can be a general-purpose central processing unit, microprocessor, ASIC, or one or more for controlling the above-mentioned Fig. 3, Fig. 5, Fig. 7, Fig. 8A, Fig. 9 and
  • the integrated circuit executed by the program of the method in FIG. 10A is not specifically limited here.
  • FIG. 14 it is a schematic structural diagram of a data processing device 140 provided by an embodiment of the present application.
  • the data processing device 140 runs the following modules to realize the functions of the data processing device in the method embodiments corresponding to FIG. 3 , FIG. 7 and FIG. 8A .
  • the data processing device 140 includes: an acquisition module 1401 , a transformation processing module 1402 and an encoding module 1403 .
  • the transformation processing module 1402 includes the reversible flow model proposed in this application, and the reversible flow model is used to perform forward transformation processing on the data input to the reversible flow model at the encoding end.
  • the specific functions of the foregoing functional modules are as follows:
  • the transformation processing module 1402 is used to perform forward transformation processing on the data to be encoded by using a reversible flow model to obtain hidden variables; wherein, the reversible flow model includes a target reversible flow layer, and the target reversible flow layer includes model parameters, and the model parameters Used to constrain the auxiliary variable generated during the forward transformation process, the operation corresponding to the target reversible flow layer includes the multiplication operation and division operation determined based on the model parameters, and the auxiliary variable is the increment of the product of the multiplication operation or the remainder resulting from the division operation;
  • the encoding module 1403 is configured to perform encoding processing on the latent variable to obtain encoded data corresponding to the data to be encoded.
  • the target reversible flow layer since the reversible flow model used to perform forward transformation processing on the data to be encoded includes a target reversible flow layer, the target reversible flow layer not only includes multiplication and division operations determined based on the model parameters, but the target reversible flow layer
  • the model parameters in can also constrain the auxiliary variable generated during the forward transformation process, the auxiliary variable being the increment of the product of the multiplication operation or the remainder generated by the division operation. That is to say, this application can not only use multiplication, but also realize reversibility by using auxiliary variables when performing division. Therefore, compared with the integer flow model in the traditional technology, the model representation ability is improved, which is conducive to improving the probability distribution of the model
  • the estimation accuracy rate can improve the compression efficiency of lossless compression.
  • the reversible flow model includes N reversible flow layers, the N reversible flow layers include the target reversible flow layer, and N is an integer greater than or equal to 1.
  • the target reversible flow layer is used to perform forward transformation processing on the first data input to the target reversible flow layer to obtain second data output from the target reversible flow layer, the first data is the multiplier of the multiplication operation, and the second data is output from the target reversible flow layer.
  • the second data is the integer quotient of the division operation.
  • the first data is the data to be encoded or the output data processed by at least one of the reversible flow layers
  • the second data is the latent variable or the output data of the second data processed by at least one of the reversible flow layers is the hidden variable.
  • the model parameters include a first parameter and a second parameter, both of the first parameter and the second parameter are positive integers, and the first parameter and the second parameter are not equal; the auxiliary variable A first auxiliary variable is included which is the increment of the product of the multiplication operation.
  • the transformation processing module 1402 is specifically used for:
  • the intermediate data is used as a divisor and the second parameter is used as a dividend to perform a division and rounding operation to obtain the second data.
  • the data processing device 140 adds the product obtained by multiplying the first data and the first parameter (that is, the first product data) to the first auxiliary variable to obtain intermediate data; then, the data processing device 140 performs a division operation with the intermediate data as the divisor and the second parameter as the dividend, and the obtained integer quotient is the second data. It can be seen that, in the aforementioned forward transformation process, the first data is first transformed into intermediate data based on multiplication and addition operations, and then the intermediate data is transformed into second data based on division and rounding operations.
  • the first auxiliary variable participates in the process of determining the intermediate data, that is, the first auxiliary variable is used as the increment of the product of the multiplication operation, the integer quotient of the division and rounding operation based on the intermediate data is the second data, therefore, avoid The problem of irreversible operation caused by only multiplication is avoided, and the problem of poor representation ability of the model caused by only addition and subtraction is also avoided.
  • the auxiliary variable further includes a second auxiliary variable, and the second auxiliary variable is a remainder generated by the division operation.
  • the transformation processing module 1402 is further configured to use the intermediate data as the divisor and the second parameter as the dividend to perform division and remainder operation to obtain the second auxiliary variable, which is used as the corresponding forward transformation processing Increment of the product of the multiplication operation in the inverse transformation process, the first auxiliary variable is used as a remainder generated by the division operation in the inverse transformation process.
  • the target reversible flow model will also generate a second auxiliary variable in the forward transformation process, and the second auxiliary variable is used as the product of the multiplication in the inverse transformation process corresponding to the forward transformation process increment. That is to say, in the inverse transformation process, the data processing device 140 restores the second data to the intermediate data based on the second auxiliary variable, and then restores the intermediate data to the first data. Therefore, in the forward transformation process, the second auxiliary variable is calculated, which is beneficial to restore the second data to the first data.
  • the transformation processing module 1402 is also configured to:
  • the first auxiliary variable is determined based on the first auxiliary data and the first parameter.
  • the transformation processing module 1402 is specifically configured to:
  • the first auxiliary data is decoded by using the first uniform distribution to obtain the first auxiliary variable and second auxiliary data, where the value of the first auxiliary variable is between 0 and the first parameter.
  • the transformation processing module 1402 includes a first entropy codec module 14021, the first entropy codec module 14021 stores a first uniform distribution, and the first entropy codec module 14021 is used for:
  • the second auxiliary data is used to encode the second auxiliary variable in the process of forward transformation, in This first auxiliary variable is encoded during the inverse transformation.
  • the transformation processing module 1402 is also configured to:
  • the third auxiliary data is stored, the third auxiliary data is used for decoding the second auxiliary variable in the inverse transformation process.
  • the data processing device 140 since the data processing device 140 uses the second auxiliary data to encode the second auxiliary variable to obtain the third auxiliary data encoded with the second auxiliary variable, it is beneficial for the data processing device 140 to store the third auxiliary data, so that During the inverse transformation process, the data processing device 140 uses the third auxiliary data to decode the second auxiliary variable. Since the storage space occupied by storing the third auxiliary data is much smaller than the storage space required for storing the second auxiliary variable, compared with the data processing device 140 directly storing the second auxiliary variable, it is beneficial to save storage space and improve storage capacity. efficiency.
  • the third auxiliary data is determined based on the encoding of the second auxiliary variable by the second auxiliary data, and the second auxiliary data is obtained in the process of decoding the first auxiliary variable by using the first auxiliary data, therefore, the third auxiliary data is stored
  • the auxiliary data indirectly stores the corresponding relationship between the second auxiliary variable and the first auxiliary variable (equivalent to correspondingly storing the first auxiliary variable and the second auxiliary variable).
  • the transformation processing module 1402 includes a second entropy codec module 14022, the second entropy codec module 14022 stores a second uniform distribution, the second entropy codec Module 14022 is used to:
  • the product obtained by multiplying the second auxiliary data by the second parameter is added to the second auxiliary variable to obtain the third auxiliary data.
  • the target reversible flow layer is also used for:
  • the model parameters are calculated by using an interpolation method or a binary search method based on the first interval and the second interval.
  • the N reversible flow layers include at least two reversible flow layers, the N reversible flow layers are arranged in series;
  • the output of the i-1th reversible flow layer is used as the input of the i-th reversible flow layer, where i is a positive integer greater than 1 and not greater than N, and the input of the first reversible flow layer is the data to be encoded , the output of the Nth reversible flow layer is the hidden variable.
  • the data to be encoded is data obtained by inversely encoding original data with noise data, and the data to be encoded is fixed-point data with a first preset precision;
  • the reverse flow layer is also used to process the fixed-point number data based on the first preset precision into integer data before the forward transform processing, and process the integer data after the forward transform processing to have the first
  • the fixed-point number data with preset precision is used as the output of the ith reversible flow layer.
  • the aforementioned noise data may be a bit stream of compressed data in the data processing device 140 , or a bit stream pre-stored in the data processing device 140 , which is not specifically limited here.
  • the noise data since the noise data is used to perform inverse encoding processing on the original data, the value of the data calculated by the data processing device 140 can be increased, which is conducive to improving the processing accuracy of the subsequent transformation process and also improving the compression rate.
  • the N reversible flow layers further include a reversible convolution layer, the reversible convolution layer includes a convolution kernel and the target reversible flow layer, the convolution kernel is the channel of the data to be encoded The number is a square matrix of order; the reversible convolution layer is used for LU decomposition of the convolution kernel to obtain the target matrix, the target matrix is a diagonal matrix, and the non-zero elements on the diagonal of the diagonal matrix are used to determine the model parameters of the target reversible flow layer in the reversible convolutional layer.
  • the 1x1 convolutional layer is converted into a matrix multiplication operation of continuous upper triangular matrix, diagonal matrix, lower triangular matrix, and scrambled matrix, and iterative calculations are used for the four matrix multiplications.
  • the N reversible flow layers further include a coupling layer, the coupling layer includes the target reversible flow layer; the coupling layer is used to divide the data input into the coupling layer into at least two Part of the data, and processing at least a part of the at least two parts of the data as the input of the target reversible flow layer to obtain the output of the coupling layer.
  • the coupling layer may include a neural network, and the neural network is used to process the data input to the coupling layer in different dimensions.
  • the neural network can include various types. For example, convolutional neural network (CNN), deep neural network (DNN), recurrent neural network (RNN) or other neural networks. Specifically, there is no limitation here.
  • the coupling layer can divide the data input into the coupling layer according to dimensions, and only use the conversion method of the target reversible flow layer to transform the data of some of the dimensions. Therefore, the diversity of data transformation can be realized, and the representation ability of the reversible flow model can be improved.
  • FIG. 15 it is a schematic structural diagram of a data processing device 150 provided by an embodiment of the present application.
  • the data processing device 150 runs the following modules to realize the functions of the data processing device in the method embodiments corresponding to FIG. 5 , FIG. 9 and FIG. 10A .
  • the data processing device 150 includes: an acquisition module 1501 , a decoding module 1502 and a transformation processing module 1503 .
  • the transformation processing module 1502 includes the reversible flow model proposed in this application, and the reversible flow model is used to perform inverse transformation processing on the hidden variables input to the reversible flow model at the decoding end.
  • the specific functions of the foregoing functional modules are as follows:
  • Decoding module 1502 configured to decode the encoded data to obtain hidden variables
  • the transformation processing module 1503 is configured to use a reversible flow model to perform inverse transformation processing on the latent variable to obtain a decoding output; wherein, the reversible flow model includes a target reversible flow layer, and the target reversible flow layer includes model parameters, and the model parameters are used for Constrain the auxiliary variable generated during the inverse transformation process, the operation corresponding to the target reversible flow layer includes the multiplication operation and division operation determined based on the model parameters, the auxiliary variable is the increment of the product of the multiplication operation or the division operation The resulting remainder.
  • the data processing device 150 performs inverse transformation processing on the decoded latent variables.
  • the reversible flow model of the inverse transformation process includes a target reversible flow layer, and the target reversible flow layer includes not only multiplication operations determined based on the model parameters and division operation, the model parameters in the target reversible flow layer can also constrain the auxiliary variable generated during the inverse transformation process, the auxiliary variable being the increment of the product of the multiplication operation or the remainder generated by the division operation. That is to say, the present application can use the auxiliary variable to perform division operation for the multiplication operation in the forward transform process, so as to restore the hidden variable to the data before the forward transform process (that is, the data to be encoded in the aforementioned data encoding method) . Therefore, not only can the reversible operation of the transformation process be guaranteed, but also the representation ability of the model can be improved, which is conducive to improving the estimation accuracy of the probability distribution of the model, thereby improving the compression efficiency of lossless compression.
  • the reversible flow model includes N reversible flow layers, the N reversible flow layers include the target reversible flow layer, and N is an integer greater than or equal to 1.
  • the target reversible flow layer is used to reversely transform the second data input to the target reversible flow layer to obtain the first data output from the target reversible flow layer, the second data is the multiplier of the multiplication operation, and the first Data is the integer quotient of this division operation.
  • the second data is the latent variable or the data output after being processed by at least one of the reversible flow layers, the first data is the decoding output or the output of the first data after being processed by at least one of the reversible flow layers is the decoded output.
  • the model parameters include a first parameter and a second parameter, both of the first parameter and the second parameter are positive integers, and the first parameter and the second parameter are not equal; the auxiliary variable A second auxiliary variable is included that is the increment of the product of the multiplication operation.
  • the transformation processing module 1503 is specifically used for:
  • the intermediate data is used as a divisor and the first parameter is used as a dividend to perform division and rounding operations to obtain the first data.
  • the data processing device 150 adds the product obtained by multiplying the second data and the second parameter (that is, the second product data) to the second auxiliary variable to obtain intermediate data; then, the data processing device 150 performs a division operation with the intermediate data as the divisor and the first parameter as the dividend, and the obtained integer quotient is the first data.
  • the second data is first transformed into intermediate data based on multiplication and addition operations, and then the intermediate data is transformed into first data based on division and rounding operations.
  • the second auxiliary variable participates in the process of determining the intermediate data, that is, the second auxiliary variable is used as the increment of the product of the multiplication operation, the integer quotient of the division and rounding operation based on the intermediate data is the first data, therefore, avoid The problem of irreversible operation caused by only multiplication is avoided, and the problem of poor representation ability of the model caused by only addition and subtraction is also avoided.
  • the auxiliary variable further includes a first auxiliary variable, where the first auxiliary variable is a remainder generated by the division operation;
  • the transformation processing module 1503 is also used for:
  • the transformation processing module 1503 is specifically configured to:
  • the second auxiliary variable is determined based on the third auxiliary data and the second parameter.
  • the data processing device 150 determines the second auxiliary variable by using the third auxiliary data stored in the forward transformation processing corresponding to the inverse transformation processing, which is beneficial to restore the second auxiliary variable from the third auxiliary data, To ensure that the value of the second auxiliary variable used in the reverse transformation process is consistent with the value of the second auxiliary variable used in the forward transformation process.
  • the transformation processing module 1503 is specifically configured to:
  • the third auxiliary data is decoded by using the second uniform distribution to obtain the second auxiliary variable and the second auxiliary data, where the value of the second auxiliary variable is between 0 and the second parameter.
  • the transformation processing module 1503 includes a second entropy codec module 15032, the second entropy codec module 15032 stores a second uniform distribution, and the second entropy codec module 15032 is used for:
  • the second auxiliary data is used to encode the second auxiliary variable in the forward transformation process, in This first auxiliary variable is encoded during the inverse transformation.
  • the transformation processing module 1503 is also configured to:
  • the value of the first auxiliary variable is between 0 and the first parameter
  • the first auxiliary data is stored, and the first auxiliary data is auxiliary data input to the target reversible flow layer in the forward transformation processing corresponding to the inverse transformation processing.
  • the data processing device 150 uses the second auxiliary data to encode the first auxiliary variable to obtain the first auxiliary data encoded with the first auxiliary variable, it is beneficial for the data processing device 150 to store the first auxiliary data. Since the storage space occupied by storing the first auxiliary data is much smaller than the storage space required for storing the first auxiliary variable, compared with the scheme in which the data processing device 150 directly stores the first auxiliary variable, it is beneficial to save storage space and improve storage capacity. efficiency. In addition, generating and storing the first auxiliary data is beneficial to recovering the first auxiliary data input to the target reversible flow layer during the forward transformation process.
  • the transformation processing module 1503 includes a first entropy codec module 15031, the first entropy codec module 15031 stores a first uniform distribution, and the first entropy codec Module 15031 is used to:
  • the product obtained by multiplying the second auxiliary data by the first parameter is added to the first auxiliary variable to obtain the first auxiliary data.
  • the target reversible flow layer is also used for:
  • the model parameters are calculated by using an interpolation method or a binary search method based on the first interval and the second interval.
  • the N reversible flow layers include at least two reversible flow layers, the N reversible flow layers are arranged in series;
  • the output of the i-th reversible flow layer is used as the input of the i-1th reversible flow layer, the i is a positive integer greater than 1 and not greater than N, and the input of the N-th reversible flow layer is the hidden variable,
  • the output of the first reversible flow layer is the decoding output.
  • the i-th reversible flow layer is also used to process the fixed-point number data based on the first preset precision into integer data before the inverse transformation process, and process the inverse
  • the transformed integer data is processed into fixed-point number data with a first preset precision as an output of the ith reversible flow layer.
  • the decoding module is further configured to perform de-inverse encoding processing on the output of the first reversible flow layer to obtain original data and noise data.
  • the N serial reversible flow layers further include a reversible convolution layer, the reversible convolution layer includes a convolution kernel and the target reversible flow layer, and the convolution kernel is based on the to-be-encoded
  • the number of channels of the data is a square matrix of order; the reversible convolution layer is used for LU decomposition of the convolution kernel to obtain the target matrix, the target matrix is a diagonal matrix, and the non- The zero element is used to determine the model parameters of the target reversible flow layer in the reversible convolutional layer.
  • the 1x1 convolutional layer is converted into a matrix multiplication operation of continuous upper triangular matrix, diagonal matrix, lower triangular matrix, and scrambled matrix, and iterative calculations are used for the four matrix multiplications.
  • the N serial reversible flow layers further include a coupling layer, the coupling layer includes the target reversible flow layer; the coupling layer is used to divide the data input into the coupling layer according to preset dimensions at least two parts of data, and processing at least one part of the at least two parts of data as the input of the target reversible flow layer to obtain the output of the coupling layer.
  • the coupling layer may include a neural network, and the neural network is used to process the data input to the coupling layer in different dimensions.
  • the neural network can include various types. For example, convolutional neural network CNN, deep neural network DNN, recurrent neural network RNN or other neural networks, etc. Specifically, there is no limitation here.
  • the coupling layer can divide the data input into the coupling layer according to dimensions, and only use the conversion method of the target reversible flow layer to transform the data of some of the dimensions. Therefore, the diversity of data transformation can be realized, and the representation ability of the reversible flow model can be improved.
  • the device embodiments described above are only illustrative.
  • the units described as separate components may or may not be physically separated.
  • Components shown as units may or may not be physical units, may be located in one place, or may be distributed over multiple network units. In practical applications, some or all of the modules/units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • the connection relationship between the modules indicates that there is a communication connection between them, and the specific implementation may be one or more communication buses or signal lines.
  • the essence of the technical solution of this application or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a readable storage medium, such as a floppy disk of a computer , U disk, mobile hard disk, read only memory (read only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk, etc., including several instructions to make a computer device (which can be a personal A computer, server, or network device, etc.) executes the methods described in various embodiments of the present application.
  • a computer device which can be a personal A computer, server, or network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

本申请涉及人工智能领域,公开了数据编码方法、数据解码方法以及数据处理装置,其中,数据编码方法和数据解码方法均涉及可逆流模型。其中,可逆流模型包括目标可逆流层,目标可逆流层中的模型参数用于约束在逆向变换处理过程中生成的辅助变量,目标可逆流层对应的运算包括基于模型参数确定的乘法运算和除法运算,该辅助变量为乘法运算的乘积的增量或除法运算产生的余数。由于,在数据编码和数据解码过程中,不仅能够使用乘法运算,还可以在进行除法运算时通过使用辅助变量实现可逆,因此,相比于传统技术中的整数流模型提高了模型表征能力,有利于提升模型概率分布的估计准确率,进而能够提升无损压缩的压缩效率。

Description

数据编码方法、数据解码方法以及数据处理装置
本申请要求于2021年09月30日提交中国国家知识产权局、申请号为202111163722.5、申请名称为“数据编码方法、数据解码方法以及数据处理装置”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及人工智能(artificial intelligence,AI)领域,尤其涉及数据编码方法、数据解码方法以及数据处理装置。
背景技术
无损压缩(Lossless Compression)是一种对数据进行压缩的技术。原始数据经过无损压缩之后获得的压缩数据的长度小于原始数据的长度,并且,压缩数据通过解压能够完全恢复为原始数据。目前,无损压缩的核心是寻找原始数据内部的分布规律,根据原始数据内部的规律进行压缩。例如,若原始数据中字母e出现频率远高于字母z,则对e用更短的比特进行表示,就可以采用长度更短的压缩数据来表示前述原始数据。
随着人工智能AI技术的发展,研究人员希望利用AI技术寻找数据内部的更优的分布规律,利用数据内部的分布规律进行压缩,以期望获得更优的无损压缩率。目前,一种常用的利用AI技术寻找数据内部的分布规律的模型为流模型(flow-based models)。具体地,在压缩流程中,流模型将原始数据变换为隐变量,该隐变量具有一定的概率分布,可以根据该隐变量的概率分布进行压缩。在解压流程中,根据前述隐变量的概率分布进行解压,并通过前述流模型恢复成原始数据。
在传统方案中,为了实现无损压缩,一般采用整数流(integer discrete flow,IDF)模型。该整数流模型的输入和输出需要均为整数,并且,使用整数加减运算避免产生浮点数误差,以保证流模型的输入和输出是可逆的。然而,由于前述流模型仅限于加减运算,导致模型表征能力较差,模型概率分布估计较差,进而导致无损压缩的压缩效率较低。
发明内容
本申请实施例提供了一种数据编码方法、数据解码方法以及数据处理装置,用于提升无损压缩的压缩效率。
第一方面,本申请提供了一种数据编码方法,该数据编码方法用在作为压缩端的数据处理装置中。在该方法中,数据处理装置获取待编码数据;然后,采用可逆流模型对该待编码数据进行正向变换处理,得到隐变量;然后,对该隐变量进行编码处理,得到该待编码数据对应的编码数据。
其中,可逆流模型是本申请提出的新的流模型。该可逆流模型包括目标可逆流层,该目标可逆流层对应的运算为可逆运算。其中,该目标可逆流层包括模型参数,该模型参数用于约束在该正向变换处理过程中生成的辅助变量,该目标可逆流层对应的运算包括基于该模型 参数确定的乘法运算和除法运算,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。
应理解,该目标可逆流层中的模型参数可以为预配置的参数,也可以是该目标可逆流层基于某种算法计算出的参数,具体此处不做限定。
应理解,该待编码数据可以是图像、视频或者文本的等原始数据。此时,该待编码数据一般为整数数据。该待编码数据也可以为基于图像、视频或者文本的等原始数据进行预处理后的数据。此时,该待编码数据可能是具有一定精度的定点数。
本实施例中,由于用于对待编码数据进行正向变换处理的可逆流模型包括目标可逆流层,该目标可逆流层不仅包括基于该模型参数确定的乘法运算和除法运算,该目标可逆流层中的模型参数还能够约束在所述正向变换处理过程中生成的辅助变量,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。也就是说,本申请不仅能够使用乘法运算,还可以在进行除法运算时通过使用辅助变量实现可逆,因此,相比于传统技术中的整数流模型提高了模型表征能力,有利于提升模型概率分布的估计准确率,进而能够提升无损压缩的压缩效率。
在一种可能的实施方式中,该可逆流模型包括N个可逆流层,该N个可逆流层包括该目标可逆流层,该N为大于或等于1的整数。该目标可逆流层用于对输入该目标可逆流层的第一数据进行正向变换处理,得到输出该目标可逆流层的第二数据,该第一数据为该乘法运算的乘数,该第二数据为该除法运算的整数商。
也就是说,在正向变换处理过程中,在基于该模型参数确定的乘法运算中,该第一数据作为该乘法运算的乘数,该乘法运算的另一个乘数由该模型参数确定;由于,辅助变量可以作为该乘法运算的乘积的增量,因此,基于前述第一数据与另一个乘数的乘积,再加上前述辅助变量可以获得中间数据。在基于该模型参数确定的除法运算中,该第二数据作为该除法运算的整数商,该除法运算的除数为中间数据,被除数由该模型参数确定。因此,第一数据在输入目标可逆流层之后,经过乘法运算和除法运算等运算处理后,将输出第二数据,该第二数据为目标可逆流层的输出。
其中,该第一数据为该待编码数据或者经过至少一个可逆流层处理后输出的数据,该第二数据为该隐变量或者该第二数据经过至少一个该可逆流层处理后输出的数据为该隐变量。应理解,前述至少一个可逆流层指n个可逆流层,其中,n为大于或等于1且小于N的整数,但是,此时N为大于1的整数。
也就是说,若目标可逆流层为可逆流模型中的第一个可逆流层,则输入该可逆流模型的待编码数据即为第一数据,经过正向变换处理之后,该目标可逆流层输出第二数据。若该可逆流模型仅有一个可逆流层,则前述目标可逆流层输出的第二数据即为隐变量。若该可逆流模型中有多个可逆流层,则前述目标可逆流层输出的第二数据将作为其他的可逆流层的输入,以使得其他的可逆流层对该第二数据进行处理,以使得经过至少一个该可逆流层处理后输出的数据为该隐变量。
类似的,若目标可逆流层为可逆流模型中的最后一个可逆流层,则输入该可逆流模型的待编码数据经过至少一个该可逆流层处理后的输出为前述第一数据,该第一数据作为目标可逆流层的输入,经过正向变换处理之后,该目标可逆流层输出第二数据。由于,该目标可逆 流层为可逆流模型中的最后一个可逆流层,则前述目标可逆流层输出的第二数据即为隐变量。
本实施方式中,提出输入该目标可逆流层的第一数据会经过乘法运算,输出该目标可逆流层的第二数据是经过除法运算获得的,因此,相比于仅使用整数加减法的整数流模型来说,本申请的可逆流模型具备更强的模型表征能力,有利于提升模型概率分布的估计准确率,进而能够提升无损压缩的压缩效率。
在一种可能的实施方式中,该模型参数包括第一参数和第二参数,该第一参数和该第二参数均为正整数,该第一参数与该第二参数不相等;该辅助变量包括第一辅助变量,该第一辅助变量为该乘法运算的乘积的增量。
具体地,该数据处理装置对输入该目标可逆流层的第一数据进行正向变换处理,得到输出该目标可逆流层的第二数据的过程具体可以为:
将该第一数据作为该第一参数的乘数进行乘法运算,得到第一乘积数据;
将该第一辅助变量作用该第一乘积数据的增量进行加法运算,得到中间数据;
将该中间数据作为除数且该第二参数作为被除数进行除法取整运算,得到该第二数据。
也就是说,数据处理装置将第一数据与第一参数进行乘法运算而获得的乘积(即第一乘积数据)与第一辅助变量进行加法运算,得到中间数据;然后,该数据处理装置将中间数据作为除数且该第二参数作为被除数进行除法运算,得到的整数商为第二数据。由此可见,在前述正向变换过程中,第一数据先基于乘法运算和加法运算变换为中间数据,再由该中间数据基于除法取整运算变换为第二数据。由于,第一辅助变量参与了确定中间数据的过程,即第一辅助变量作为该乘法运算的乘积的增量,基于该中间数据进行除法取整运算的整数商才为第二数据,因此,避免了仅用乘法运算而造成的运算不可逆的问题,也避免了仅用加法运算和减法运算而造成的模型表征能力差的问题。
在一种可能的实施方式中,该辅助变量还包括第二辅助变量,该第二辅助变量为该除法运算产生的余数。
具体地,该数据处理装置在输出第二数据的同时,还将以该中间数据为除数且该第二参数为被除数进行除法取余运算,得到该第二辅助变量。其中,该第二辅助变量用于作为该正向变换处理对应的逆向变换处理中的乘法运算的乘积的增量,该第一辅助变量用于作为该逆向变换处理中的除法运算产生的余数。
本实施方式中,提出该目标可逆流模型在正向变换处理中,还将生成第二辅助变量,该第二辅助变量用于作为该正向变换处理对应的逆向变换处理中的乘法运算的乘积的增量。也就是说,在逆向变换处理中,数据处理装置将基于第二辅助变量将第二数据恢复为中间数据,进而将中间数据恢复为第一数据。因此,在正向变换处理中,计算出第二辅助变量,有利于实现将第二数据恢复为第一数据。
在一种可能的实施方式中,该数据处理装置基于第一参数确定第一辅助变量的过程可以为:
数据处理装置先获取输入该目标可逆流层的第一辅助数据;然后,数据处理装置基于该第一辅助数据和该第一参数确定该第一辅助变量。
本实施方式中,提出数据处理装置在基于第一参数确定第一辅助变量的过程中会用到输入该目标可逆流层的第一辅助数据。其中,该第一辅助数据可以为二进制码流。
在一种可能的实施方式中,该数据处理装置可以基于熵编码的方式(例如,均匀分布熵编码)对输入目标可逆流层的辅助数据进行编解码,以使得生成或存储前述辅助变量。
具体地,该数据处理装置基于该第一辅助数据和该第一参数确定该第一辅助变量的过程可以为:
基于该第一参数确定第一均匀分布,该第一均匀分布的端点为0和该第一参数;
采用该第一均匀分布解码该第一辅助数据,得到该第一辅助变量和第二辅助数据,该第一辅助变量的取值位于0到该第一参数之间。
本实施方式中,提出采用第一参数确定的第一均匀分布作为熵编码的概率分布,然后,基于该第一均匀分布解码第一辅助数据,以得到第一辅助变量和第二辅助数据。
在一种可能的实施方式中,该数据处理装置在采用该第一均匀分布解码该第一辅助数据,得到该第一辅助变量和第二辅助数据的过程中,可以通过如下步骤实现:
以该第一辅助数据为除数且该第一参数为被除数,进行除法取余运算,得到该第一辅助变量;
以该第一辅助数据为除数且该第一参数为被除数,进行除法取整运算,得到该第二辅助数据,该第二辅助数据用于在正向变换过程中编码该第二辅助变量,在逆向变换过程中编码该第一辅助变量。
在一种可能的实施方式中,该数据处理装置在生成第二辅助变量后,还将采用熵编码的方式(例如,均匀分布熵编码)编码第二辅助变量,以使得该数据处理装置可以直接存储编码第二辅助变量后获得的辅助数据,而不用直接存储第二辅助变量。
本实施例中,该数据处理装置将采用解码第一辅助数据而获得的第二辅助数据作为编码第二辅助变量的辅助数据。具体地,该数据处理装置将通过如下步骤编码目标可逆流层生成的第二辅助变量:
基于该第二参数确定第二均匀分布,该第二均匀分布的端点为0和该第二参数;
采用该第二均匀分布和该第二辅助数据编码该第二辅助变量,得到第三辅助数据,该第二辅助变量的取值位于0到该第二参数之间;
存储该第三辅助数据,该第三辅助数据用于在该逆向变换处理中解码该第二辅助变量。
本实施方式中,由于数据处理装置采用第二辅助数据编码第二辅助变量,以获得编码了第二辅助变量的第三辅助数据,有利于数据处理装置存储第三辅助数据,以便于在逆向变换处理过程中,数据处理装置采用第三辅助数据解码第二辅助变量。由于,存储第三辅助数据占用的存储空间远小于存储第二辅助变量需要的存储空间,因此,相比于该数据处理装置直接存储第二辅助变量的方案,有利于节省存储空间,提高存储效率。另外,由于第三辅助数据是基于第二辅助数据编码第二辅助变量确定的,而第二辅助数据又是在采用第一辅助数据解码第一辅助变量的过程获得的,因此,存储了第三辅助数据,便间接存储了第二辅助变量和第一辅助变量的对应关系(相当于将第一辅助变量和第二辅助变量对应存储)。
在一种可能的实施方式中,该数据处理装置在采用该第二均匀分布和该第二辅助数据编码该第二辅助变量,得到第三辅助数据的过程中,可以通过如下步骤实现:
将该第二辅助数据与该第二参数进行乘法运算而获得的乘积,与该第二辅助变量进行加法运算,得到该第三辅助数据。
在一种可能的实施方式中,该目标可逆流层还用于:获取第一单调连续函数的定义域中的第一区间和该第一区间在该第一单调连续函数的值域上对应的第二区间,该第一单调连续函数为非线性函数;基于该第一区间和该第二区间采用插值法或二分查找法计算该模型参数。其中,模型参数包括第一参数和第二参数。
本实施方式中,提出目标可逆流层可以是基于非线性函数(例如,第一单调连续函数)构建的,则该数据处理装置可以将该非线性函数拆分为多段,将每一段近似为直线以计算前述第一参数和第二参数。可选的,该数据处理装置可以采用插值算法或二分查找法计算前述第一参数和第二参数。
在一种可能的实施方式中,当该N个可逆流层包括至少两个可逆流层时,该N个可逆流层采用串行的方式排布。其中,第i-1个可逆流层的输出用于作为第i个可逆流层的输入,该i为大于1且不大于N的正整数,第1个可逆流层的输入为该待编码数据,该第N个可逆流层的输出为该隐变量。
本实施方式中,提出可逆流模型中的多个可逆流层是串行排布的,有利于该可逆流模型中各个可逆流层依次对数据进行处理,进而保证在逆向变换处理过程的顺序。
在一种可能的实施方式中,该待编码数据为采用噪声数据对原始数据进行反编码处理而获得的数据,该待编码数据为具有第一预设精度的定点数数据。
该第i个可逆流层还用于在该正向变换处理之前,将基于该具有第一预设精度的定点数数据处理为整数数据,以及将该经该正向变换处理后的整数数据处理为具有该第一预设精度的定点数数据以作为该第i个可逆流层的输出。
应理解,前述噪声数据可以是数据处理装置中已压缩的数据的比特流,还可以是数据处理装置中预存的比特流,具体此处不做限定。本实施方式中,由于采用噪声数据对原始数据进行反编码处理,能够增大数据处理装置计算的数据的数值,有利于提升后续变换过程的处理精度,也有利于提升压缩率。
在一种可能的实施方式中,该N个可逆流层还包括可逆卷积层,该可逆卷积层包括卷积核和该目标可逆流层,该卷积核为以该待编码数据的通道数为阶数的方阵;
该可逆卷积层用于对该卷积核进行LU(LU Factorization)分解,得到目标矩阵,该目标矩阵为对角阵,该对角阵的对角线上的非零元素用于确定该可逆卷积层中该目标可逆流层的模型参数。
本实施方式中,提出在可逆流模型中设置可逆卷积层,采用1x1卷积层与目标可逆流层结合。具体地,将1x1卷积层转化成连续上三角阵、对角阵、下三角阵和置乱阵的矩阵乘法运算,对四种矩阵乘法分别使用迭代计算、基于目标可逆流层的数值可逆线性流变换处理、迭代计算、元素重排。因此,能够实现数据变换的多样性,提高可逆流模型的表征能力。
在一种可能的实施方式中,该N个可逆流层还包括耦合层,该耦合层包括该目标可逆流层;
该耦合层用于将输入该耦合层的数据按预设维度划分为至少两部分数据,并将该至少两部分数据中的至少一部分数据作为该目标可逆流层的输入进行处理,得到该耦合层的输出。
应理解,耦合层可以包含神经网络,该神经网络用于分维度对输入该耦合层的数据进行处理。该神经网络可以包括多种类型。例如,卷积神经网络(convolutional neural network, CNN)、深度神经网络(deep neural network,DNN)、循环神经网络(recurrent neural networks,RNN)或其他神经网络等。具体此处不做限定。
本实施方式中,提出在可逆流模型中设置耦合层,该耦合层能够将输入该耦合层的数据按照维度划分,并仅对其中一部分维度的数据采用目标可逆流层的变换方式进行变换处理。因此,能够实现数据变换的多样性,提高可逆流模型的表征能力。
第二方面,本申请提供了一种数据解码方法,该数据解码方法用在作为解压端的数据处理装置中。在该方法中,数据处理装置获取编码数据;然后,对该编码数据进行解码处理,得到隐变量;然后,采用可逆流模型对该隐变量进行逆向变换处理,得到解码输出。
其中,可逆流模型是本申请提出的新的流模型。应理解,数据编码方法和数据解码方法采用的是相同的可逆流模型。该可逆流模型包括目标可逆流层,该目标可逆流层包括模型参数,该模型参数用于约束在该逆向变换处理过程中生成的辅助变量,该目标可逆流层对应的运算包括基于该模型参数确定的乘法运算和除法运算,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。
应理解,该目标可逆流层中的模型参数可以为预配置的参数,也可以是该目标可逆流层基于某种算法计算出的参数,具体此处不做限定。
应理解,该待编码数据可以是图像、视频或者文本的等原始数据。此时,该待编码数据一般为整数数据。该待编码数据也可以为基于图像、视频或者文本的等原始数据进行预处理后的数据。此时,该待编码数据可能是具有一定精度的定点数。
本实施例中,数据处理装置将对解码后的隐变量进行逆向变换处理,该逆向变换处理的可逆流模型包括目标可逆流层,该目标可逆流层不仅包括基于该模型参数确定的乘法运算和除法运算,该目标可逆流层中的模型参数还能够约束在所述逆向变换处理过程中生成的辅助变量,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。也就是说,本申请能够针对正向变换处理过程中的乘法运算,采用辅助变量进行除法运算,以将隐变量恢复为正向变换处理前的数据(即前述数据编码方法中的待编码数据)。因此,不仅能够保证变换处理的可逆运算,还能够提高模型表征能力,有利于提升模型概率分布的估计准确率,进而能够提升无损压缩的压缩效率。
在一种可能的实施方式中,该可逆流模型包括N个可逆流层,该N个可逆流层包括该目标可逆流层,该N为大于或等于1的整数。该目标可逆流层用于对输入该目标可逆流层的第二数据进行逆向变换处理,得到输出该目标可逆流层的第一数据,该第二数据为该乘法运算的乘数,该第一数据为该除法运算的整数商。
也就是说,在逆向变换处理过程中,在基于该模型参数确定的乘法运算中,该第二数据作为该乘法运算的乘数,该乘法运算的另一个乘数由该模型参数确定;由于,辅助变量可以作为该乘法运算的乘积的增量,因此,基于前述第二数据与另一个乘数的乘积,再加上前述辅助变量可以获得中间数据。在基于该模型参数确定的除法运算中,该第一数据作为该除法运算的整数商,该除法运算的除数为中间数据,被除数由该模型参数确定。因此,第二数据在输入目标可逆流层之后,经过乘法运算和除法运算等运算处理后,将输出第一数据,该第一数据为目标可逆流层的输出。
应理解,在正向变换处理过程的乘法运算所使用的模型参数(例如,第一参数)和辅助 变量(例如,第一辅助变量)将应用于逆向变换处理过程的除法运算中;在正向变换处理过程的除法运算所使用的模型参数(例如,第二参数)和辅助变量(例如,第二辅助变量)将应用于逆向变换处理过程的乘法运算中。
其中,该第二数据为该隐变量或者经过至少一个该可逆流层处理后输出的数据,该第一数据为该解码输出或者该第一数据经过至少一个该可逆流层处理后的输出为该解码输出。应理解,前述至少一个可逆流层指n个可逆流层,其中,n为大于或等于1且小于N的整数,但是,此时N为大于1的整数。
本实施方式中,提出输入该目标可逆流层的第二数据会经过乘法运算,输出该目标可逆流层的第一数据是经过除法运算获得的,因此,相比于仅使用整数加减法的整数流模型来说,本申请的可逆流模型具备更强的模型表征能力,有利于提升模型概率分布的估计准确率,进而能够提升无损压缩的压缩效率。
在一种可能的实施方式中,该模型参数包括第一参数和第二参数,该第一参数和该第二参数均为正整数,该第一参数与该第二参数不相等;该辅助变量包括第二辅助变量,该第二辅助变量为该乘法运算的乘积的增量。
具体地,该数据处理装置对输入该目标可逆流层的第二数据进行逆向变换处理,得到输出该目标可逆流层的第一数据的过程具体可以为:
将该第二数据作为该第二参数的乘数进行乘法运算,得到第二乘积数据;
将该第二辅助变量作用该第二乘积数据的增量进行加法运算,得到中间数据;
将该中间数据作为除数且该第一参数作为被除数进行除法取整运算,得到该第一数据。
也就是说,数据处理装置将第二数据与第二参数进行乘法运算而获得的乘积(即第二乘积数据)与第二辅助变量进行加法运算,得到中间数据;然后,该数据处理装置将中间数据作为除数且该第一参数作为被除数进行除法运算,得到的整数商为第一数据。由此可见,在前述正向变换过程中,第二数据先基于乘法运算和加法运算变换为中间数据,再由该中间数据基于除法取整运算变换为第一数据。由于,第二辅助变量参与了确定中间数据的过程,即第二辅助变量作为该乘法运算的乘积的增量,基于该中间数据进行除法取整运算的整数商才为第一数据,因此,避免了仅用乘法运算而造成的运算不可逆的问题,也避免了仅用加法运算和减法运算而造成的模型表征能力差的问题。
应理解,目标可逆流层在逆向变换处理过程中使用的模型参数和辅助变量的顺序,与正向变换处理过程中使用的模型参数和辅助变量的顺序相反。
在一种可能的实施方式中,该辅助变量还包括第一辅助变量,该第一辅助变量为该除法运算产生的余数。
具体地,该数据处理装置在输出第二数据的同时,还将以该中间数据为除数且该第一参数为被除数进行除法取余运算,得到该第一辅助变量。
本实施方式中,提出该目标可逆流模型在逆向变换处理中,还将生成第一辅助变量,以使得数据处理装置基于第一辅助变量回复与该第一辅助变量相关的数据(例如,第一辅助数据)。
在一种可能的实施方式中,该数据处理装置基于第二参数确定第二辅助变量的过程可以为:
数据处理装置先获取输入该目标可逆流层的第三辅助数据,该第三辅助数据为在该逆向变换处理对应的正向变换处理中存储的辅助数据;然后,数据处理装置基于该第三辅助数据和该第二参数确定该第二辅助变量。
本实施例中,提出数据处理装置采用在该逆向变换处理对应的正向变换处理中存储的第三辅助数据确定第二辅助变量,有利于从第三辅助数据中恢复出第二辅助变量,以确保在逆向变换处理过程中使用的第二辅助变量的取值与在正向变换处理过程中使用的第二辅助变量的取值是一致的。
在一种可能的实施方式中,若在正向变换处理过程中,该数据处理装置是基于熵编码的方式(例如,均匀分布熵编码)对输入目标可逆流层的辅助数据进行编解码,以使得生成或存储前述辅助变量。此时,在正向变换处理过程中,该数据处理装置将采用相同的均匀分布编解码辅助数据。
具体地,该数据处理装置基于该第三辅助数据和该第二参数确定该第二辅助变量的过程可以为:
基于该第二参数确定第二均匀分布,该第二均匀分布的端点为0和该第二参数;
采用该第二均匀分布解码该第三辅助数据,得到该第二辅助变量和该第二辅助数据,该第二辅助变量的取值位于0到该第二参数之间。
本实施方式中,提出采用第二参数确定的第二均匀分布作为熵编码的概率分布,然后,基于该第二均匀分布解码第三辅助数据,以得到第二辅助变量和第二辅助数据。
在一种可能的实施方式中,该数据处理装置在采用该第二均匀分布解码该第三辅助数据,得到该第二辅助变量和该第二辅助数据的过程中,可以通过如下步骤实现:
以该第三辅助数据为除数且该第二参数为被除数,进行除法取余运算,得到该第二辅助变量;
以该第三辅助数据为除数且该第二参数为被除数,进行除法取整运算,得到该第二辅助数据,该第二辅助数据用于在正向变换过程中编码该第二辅助变量,在逆向变换过程中编码该第一辅助变量。
在一种可能的实施方式中,该数据处理装置在生成第二辅助变量后,还将采用熵编码的方式(例如,均匀分布熵编码)编码第二辅助变量,以使得该数据处理装置可以直接存储编码第二辅助变量后获得的辅助数据,而不用直接存储第二辅助变量。
本实施例中,该数据处理装置将采用解码第三辅助数据而获得的第二辅助数据作为编码第一辅助变量的辅助数据。具体地,该数据处理装置将通过如下步骤编码目标可逆流层生成的第一辅助变量:
基于该第一参数确定第一均匀分布,该第一均匀分布的端点为0和该第一参数;
采用该第一均匀分布和该第二辅助数据编码该第一辅助变量,得到第一辅助数据,该第一辅助变量的取值位于0到该第一参数之间;
存储该第一辅助数据,该第一辅助数据为该逆向变换处理对应的正向变换处理中输入该目标可逆流层的辅助数据。
本实施方式中,由于数据处理装置采用第二辅助数据编码第一辅助变量,以获得编码了第一辅助变量的第一辅助数据,有利于数据处理装置存储第一辅助数据。由于,存储第一辅 助数据占用的存储空间远小于存储第一辅助变量需要的存储空间,因此,相比于该数据处理装置直接存储第一辅助变量的方案,有利于节省存储空间,提高存储效率。另外,生成并存储第一辅助数据,有利于恢复在正向变换处理过程中输入该目标可逆流层的第一辅助数据。
在一种可能的实施方式中,该数据处理装置在采用该第一均匀分布和该第二辅助数据编码该第一辅助变量,得到第一辅助数据的过程中,可以通过如下步骤实现:
将该第二辅助数据与该第一参数进行乘法运算而获得的乘积,与该第一辅助变量进行加法运算,得到该第一辅助数据。
在一种可能的实施方式中,该目标可逆流层还用于:
获取第一单调连续函数的定义域中的第一区间和该第一区间在该第一单调连续函数的值域上对应的第二区间,该第一单调连续函数为非线性函数;基于该第一区间和该第二区间采用插值法或二分查找法计算该模型参数。其中,前述模型参数包括第一参数和第二参数。
本实施方式中,提出目标可逆流层可以是基于非线性函数(例如,第一单调连续函数)构建的,则该数据处理装置可以将该非线性函数拆分为多段,将每一段近似为直线以计算前述第一参数和第二参数。可选的,该数据处理装置可以采用插值算法或二分查找法计算前述第一参数和第二参数。
在一种可能的实施方式中,当该N个可逆流层包括至少两个可逆流层时,该N个可逆流层采用串行的方式排布。其中,第i个可逆流层的输出用于作为第i-1个可逆流层的输入,该i为大于1且不大于N的正整数,第N个可逆流层的输入为该隐变量,该第1个可逆流层的输出为该解码输出。
本实施方式中,提出可逆流模型中的多个可逆流层是串行排布的,有利于该可逆流模型中各个可逆流层依次对数据进行处理。应理解,正向变换处理过程中数据经过各个可逆流层的顺序,与逆向变换处理过程中数据经过各个可逆流层的顺序相反。
在一种可能的实施方式中,该第i个可逆流层还用于在该逆向变换处理之前,将基于该具有第一预设精度的定点数数据处理为整数数据,以及将该经该逆向逆变换处理后的整数数据处理为具有第一预设精度的定点数数据以作为该第i个可逆流层的输出。
在一种可能的实施方式中,该方法还包括:数据处理装置对该第1个可逆流层的输出进行解反编码处理,得到原始数据和噪声数据。
本实施例中,提出若解码输出为采用噪声数据对原始数据进行反编码处理而获得的数据,则该数据处理装置可以通过解反编码处理从前述解码输出分离出噪声数据和原始数据。
在一种可能的实施方式中,该N个串行的可逆流层还包括可逆卷积层,该可逆卷积层包括卷积核和该目标可逆流层,该卷积核为以该待编码数据的通道数为阶数的方阵;
该可逆卷积层用于对该卷积核进行LU分解,得到目标矩阵,该目标矩阵为对角阵,该对角阵的对角线上的非零元素用于确定该可逆卷积层中该目标可逆流层的模型参数。
本实施方式中,提出在可逆流模型中设置可逆卷积层,采用1x1卷积层与目标可逆流层结合。具体地,将1x1卷积层转化成连续上三角阵、对角阵、下三角阵和置乱阵的矩阵乘法运算,对四种矩阵乘法分别使用迭代计算、基于目标可逆流层的数值可逆线性流变换处理、迭代计算、元素重排。因此,能够实现数据变换的多样性,提高可逆流模型的表征能力。
在一种可能的实施方式中,该N个串行的可逆流层还包括耦合层,该耦合层包括该目标 可逆流层;
该耦合层用于将输入该耦合层的数据按预设维度划分为至少两部分数据,并将该至少两部分数据中的至少一部分数据作为该目标可逆流层的输入进行处理,得到该耦合层的输出。
应理解,耦合层可以包含神经网络,该神经网络用于分维度对输入该耦合层的数据进行处理。该神经网络可以包括多种类型。例如,卷积神经网络CNN、深度神经网络DNN、循环神经网络RNN或其他神经网络等。具体此处不做限定。
本实施方式中,提出在可逆流模型中设置耦合层,该耦合层能够将输入该耦合层的数据按照维度划分,并仅对其中一部分维度的数据采用目标可逆流层的变换方式进行变换处理。因此,能够实现数据变换的多样性,提高可逆流模型的表征能力。
第三方面,本申请提供了一种数据处理装置,该数据处理装置为编码端。具体地,该数据处理装置包括:获取模块、变换处理模块以及编码模块。其中,变换处理模块包括本申请提出的可逆流模型,在编码端该可逆流模型用于对输入该可逆流模型的数据进行正向变换处理。具体地,前述各个功能模块的具体功能如下:
获取模块,用于获取待编码数据;
变换处理模块,用于采用可逆流模型对该待编码数据进行正向变换处理,得到隐变量;其中,该可逆流模型包括目标可逆流层,该目标可逆流层包括模型参数,该模型参数用于约束在该正向变换处理过程中生成的辅助变量,该目标可逆流层对应的运算包括基于该模型参数确定的乘法运算和除法运算,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数;
编码模块,用于对该隐变量进行编码处理,得到该待编码数据对应的编码数据。
本实施例中,由于用于对待编码数据进行正向变换处理的可逆流模型包括目标可逆流层,该目标可逆流层不仅包括基于该模型参数确定的乘法运算和除法运算,该目标可逆流层中的模型参数还能够约束在所述正向变换处理过程中生成的辅助变量,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。也就是说,本申请不仅能够使用乘法运算,还可以在进行除法运算时通过使用辅助变量实现可逆,因此,相比于传统技术中的整数流模型提高了模型表征能力,有利于提升模型概率分布的估计准确率,进而能够提升无损压缩的压缩效率。
在一种可能的实施方式中,该可逆流模型包括N个可逆流层,该N个可逆流层包括该目标可逆流层,该N为大于或等于1的整数。
该目标可逆流层用于对输入该目标可逆流层的第一数据进行正向变换处理,得到输出该目标可逆流层的第二数据,该第一数据为该乘法运算的乘数,该第二数据为该除法运算的整数商。其中,该第一数据为该待编码数据或者经过至少一个该可逆流层处理后输出的数据,该第二数据为该隐变量或者该第二数据经过至少一个该可逆流层处理后输出的数据为该隐变量。
在一种可能的实施方式中,该模型参数包括第一参数和第二参数,该第一参数和该第二参数均为正整数,该第一参数与该第二参数不相等;该辅助变量包括第一辅助变量,该第一辅助变量为该乘法运算的乘积的增量。
该变换处理模块,具体用于:
将该第一数据作为该第一参数的乘数进行乘法运算,得到第一乘积数据;
将该第一辅助变量作用该第一乘积数据的增量进行加法运算,得到中间数据;
将该中间数据作为除数且该第二参数作为被除数进行除法取整运算,得到该第二数据。
在一种可能的实施方式中,该辅助变量还包括第二辅助变量,该第二辅助变量为该除法运算产生的余数。
该变换处理模块,还用于以该中间数据为除数且该第二参数为被除数进行除法取余运算,得到该第二辅助变量,该第二辅助变量用于作为该正向变换处理对应的逆向变换处理中的乘法运算的乘积的增量,该第一辅助变量用于作为该逆向变换处理中的除法运算产生的余数。
在一种可能的实施方式中,该变换处理模块,还用于:
获取输入该目标可逆流层的第一辅助数据;
基于该第一辅助数据和该第一参数确定该第一辅助变量。
在一种可能的实施方式中,该变换处理模块,具体用于:
基于该第一参数确定第一均匀分布,该第一均匀分布的端点为0和该第一参数;
采用该第一均匀分布解码该第一辅助数据,得到该第一辅助变量和第二辅助数据,该第一辅助变量的取值位于0到该第一参数之间。
在一种可能的实施方式中,该变换处理模块包括第一熵编解码模块,该第一熵编解码模块存储有第一均匀分布,该第一熵编解码模块用于:
以该第一辅助数据为除数且该第一参数为被除数,进行除法取余运算,得到该第一辅助变量;
以该第一辅助数据为除数且该第一参数为被除数,进行除法取整运算,得到该第二辅助数据,该第二辅助数据用于在正向变换过程中编码该第二辅助变量,在逆向变换过程中编码该第一辅助变量。
在一种可能的实施方式中,该变换处理模块,还用于:
基于该第二参数确定第二均匀分布,该第二均匀分布的端点为0和该第二参数;
采用该第二均匀分布和该第二辅助数据编码该第二辅助变量,得到第三辅助数据,该第二辅助变量的取值位于0到该第二参数之间;
存储该第三辅助数据,该第三辅助数据用于在该逆向变换处理中解码该第二辅助变量。
在一种可能的实施方式中,该变换处理模块,该变换处理模块包括第二熵编解码模块,该第二熵编解码模块存储有第二均匀分布,该第二熵编解码模块用于:
将该第二辅助数据与该第二参数进行乘法运算而获得的乘积,与该第二辅助变量进行加法运算,得到该第三辅助数据。
在一种可能的实施方式中,该目标可逆流层还用于:
获取第一单调连续函数的定义域中的第一区间和该第一区间在该第一单调连续函数的值域上对应的第二区间,该第一单调连续函数为非线性函数;
基于该第一区间和该第二区间采用插值法或二分查找法计算该模型参数。
在一种可能的实施方式中,当该N个可逆流层包括至少两个可逆流层时,该N个可逆流层采用串行的方式排布;
其中,第i-1个可逆流层的输出用于作为第i个可逆流层的输入,该i为大于1且不大于 N的正整数,第1个可逆流层的输入为该待编码数据,该第N个可逆流层的输出为该隐变量。
在一种可能的实施方式中,该待编码数据为采用噪声数据对原始数据进行反编码处理而获得的数据,该待编码数据为具有第一预设精度的定点数数据;
该第i个可逆流层还用于在该正向变换处理之前,将基于该具有第一预设精度的定点数数据处理为整数数据,以及将该经该正向变换处理后的整数数据处理为具有该第一预设精度的定点数数据以作为该第i个可逆流层的输出。
在一种可能的实施方式中,该N个可逆流层还包括可逆卷积层,该可逆卷积层包括卷积核和该目标可逆流层,该卷积核为以该待编码数据的通道数为阶数的方阵;
该可逆卷积层用于对该卷积核进行LU分解,得到目标矩阵,该目标矩阵为对角阵,该对角阵的对角线上的非零元素用于确定该可逆卷积层中该目标可逆流层的模型参数。
在一种可能的实施方式中,该N个可逆流层还包括耦合层,该耦合层包括该目标可逆流层;
该耦合层用于将输入该耦合层的数据按预设维度划分为至少两部分数据,并将该至少两部分数据中的至少一部分数据作为该目标可逆流层的输入进行处理,得到该耦合层的输出。
需要说明的是,本申请实施例还有多种具体其他实施方式,具体可参见第一方面的具体实施方式和其有益效果,在此不再赘述。
第四方面,本申请提供了一种数据处理装置,该数据处理装置为解码端。具体地,该数据处理装置包括:获取模块、变换处理模块以及解码模块。其中,变换处理模块包括本申请提出的可逆流模型,在解码端该可逆流模型用于对输入该可逆流模型的数据进行逆向变换处理。具体地,前述各个功能模块的具体功能如下:
获取模块,用于获取编码数据;
解码模块,用于对该编码数据进行解码处理,得到隐变量;
变换处理模块,用于采用可逆流模型对该隐变量进行逆向变换处理,得到解码输出;其中,该可逆流模型包括目标可逆流层,该目标可逆流层包括模型参数,该模型参数用于约束在该逆向变换处理过程中生成的辅助变量,该目标可逆流层对应的运算包括基于该模型参数确定的乘法运算和除法运算,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。
本实施例中,数据处理装置将对解码后的隐变量进行逆向变换处理,该逆向变换处理的可逆流模型包括目标可逆流层,该目标可逆流层不仅包括基于该模型参数确定的乘法运算和除法运算,该目标可逆流层中的模型参数还能够约束在所述逆向变换处理过程中生成的辅助变量,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。也就是说,本申请能够针对正向变换处理过程中的乘法运算,采用辅助变量进行除法运算,以将隐变量恢复为正向变换处理前的数据(即前述数据编码方法中的待编码数据)。因此,不仅能够保证变换处理的可逆运算,还能够提高模型表征能力,有利于提升模型概率分布的估计准确率,进而能够提升无损压缩的压缩效率。
在一种可能的实施方式中,该可逆流模型包括N个可逆流层,该N个可逆流层包括该目标可逆流层,该N为大于或等于1的整数。
该目标可逆流层用于对输入该目标可逆流层的第二数据进行逆向变换处理,得到输出该 目标可逆流层的第一数据,该第二数据为该乘法运算的乘数,该第一数据为该除法运算的整数商。其中,该第二数据为该隐变量或者经过至少一个该可逆流层处理后输出的数据,该第一数据为该解码输出或者该第一数据经过至少一个该可逆流层处理后的输出为该解码输出。
在一种可能的实施方式中,该模型参数包括第一参数和第二参数,该第一参数和该第二参数均为正整数,该第一参数与该第二参数不相等;该辅助变量包括第二辅助变量,该第二辅助变量为该乘法运算的乘积的增量。
该变换处理模块,具体用于:
将该第二数据作为该第二参数的乘数进行乘法运算,得到第二乘积数据;
将该第二辅助变量作用该第二乘积数据的增量进行加法运算,得到中间数据;
将该中间数据作为除数且该第一参数作为被除数进行除法取整运算,得到该第一数据。
在一种可能的实施方式中,该辅助变量还包括第一辅助变量,该第一辅助变量为该除法运算产生的余数;
该变换处理模块,还用于:
以该中间数据为除数且该第一参数为被除数进行除法取余运算,得到该第一辅助变量。
在一种可能的实施方式中,该变换处理模块,具体用于:
获取输入该目标可逆流层的第三辅助数据,该第三辅助数据为在该逆向变换处理对应的正向变换处理中存储的辅助数据;
基于该第三辅助数据和该第二参数确定该第二辅助变量。
在一种可能的实施方式中,该变换处理模块,具体用于:
基于该第二参数确定第二均匀分布,该第二均匀分布的端点为0和该第二参数;
采用该第二均匀分布解码该第三辅助数据,得到该第二辅助变量和该第二辅助数据,该第二辅助变量的取值位于0到该第二参数之间。
在一种可能的实施方式中,该变换处理模块包括第二熵编解码模块,该第二熵编解码模块存储有第二均匀分布,该第二熵编解码模块用于:
以该第三辅助数据为除数且该第二参数为被除数,进行除法取余运算,得到该第二辅助变量;
以该第三辅助数据为除数且该第二参数为被除数,进行除法取整运算,得到该第二辅助数据,该第二辅助数据用于在正向变换过程中编码该第二辅助变量,在逆向变换过程中编码该第一辅助变量。
在一种可能的实施方式中,该变换处理模块,还用于:
基于该第一参数确定第一均匀分布,该第一均匀分布的端点为0和该第一参数;
采用该第一均匀分布和该第二辅助数据编码该第一辅助变量,得到第一辅助数据,该第一辅助变量的取值位于0到该第一参数之间;
存储该第一辅助数据,该第一辅助数据为该逆向变换处理对应的正向变换处理中输入该目标可逆流层的辅助数据。
在一种可能的实施方式中,该变换处理模块,该变换处理模块包括第一熵编解码模块,该第一熵编解码模块存储有第一均匀分布,该第一熵编解码模块用于:
将该第二辅助数据与该第一参数进行乘法运算而获得的乘积,与该第一辅助变量进行加 法运算,得到该第一辅助数据。
在一种可能的实施方式中,该目标可逆流层还用于:
获取第一单调连续函数的定义域中的第一区间和该第一区间在该第一单调连续函数的值域上对应的第二区间,该第一单调连续函数为非线性函数;
基于该第一区间和该第二区间采用插值法或二分查找法计算该模型参数。
在一种可能的实施方式中,当该N个可逆流层包括至少两个可逆流层时,该N个可逆流层采用串行的方式排布;
其中,第i个可逆流层的输出用于作为第i-1个可逆流层的输入,该i为大于1且不大于N的正整数,第N个可逆流层的输入为该隐变量,该第1个可逆流层的输出为该解码输出。
在一种可能的实施方式中,该第i个可逆流层还用于在该逆向变换处理之前,将基于该具有第一预设精度的定点数数据处理为整数数据,以及将该经该逆向逆变换处理后的整数数据处理为具有第一预设精度的定点数数据以作为该第i个可逆流层的输出。
在一种可能的实施方式中,该解码模块,还用于对该第1个可逆流层的输出进行解反编码处理,得到原始数据和噪声数据。
在一种可能的实施方式中,该N个串行的可逆流层还包括可逆卷积层,该可逆卷积层包括卷积核和该目标可逆流层,该卷积核为以该待编码数据的通道数为阶数的方阵;
该可逆卷积层用于对该卷积核进行LU分解,得到目标矩阵,该目标矩阵为对角阵,该对角阵的对角线上的非零元素用于确定该可逆卷积层中该目标可逆流层的模型参数。
在一种可能的实施方式中,该N个串行的可逆流层还包括耦合层,该耦合层包括该目标可逆流层;
该耦合层用于将输入该耦合层的数据按预设维度划分为至少两部分数据,并将该至少两部分数据中的至少一部分数据作为该目标可逆流层的输入进行处理,得到该耦合层的输出。
需要说明的是,本申请实施例还有多种具体其他实施方式,具体可参见第二方面的具体实施方式和其有益效果,在此不再赘述。
第五方面,本申请提供了一种数据处理装置,该数据处理装置为编码端。该数据处理装置包括存储介质、处理电路以及总线系统;其中,所述存储介质用于存储指令,所述处理电路用于执行存储器中的指令,以执行上述第一方面及第一方面任意一种实施方式所介绍的数据编码方法。
第六方面,本申请提供了一种数据处理装置,该数据处理装置为解码端。该数据处理装置包括存储介质、处理电路以及总线系统;其中,所述存储介质用于存储指令,所述处理电路用于执行存储器中的指令,以执行上述第二方面及第二方面任意一种实施方式所介绍的数据解码方法。
第七方面,本申请实施例提供了一种计算机可读存储介质,包括指令,当该指令在计算机上运行时,以使得计算机执行如前述第一方面或第二方面,以及前述各个方面的各种实施方式中的任一种实施方式所介绍的方法。
第八方面,本申请实施例提供了一种包含指令的计算机程序产品,当其在计算机上运行时,使得该计算机执行如前述第一方面或第二方面,以及前述各个方面的各种实施方式中的任一种实施方式所介绍的方法。
第九方面,本申请提供了一种芯片系统,该芯片系统包括处理器,用于支持执行设备或训练设备实现上述方面中所涉及的功能,例如,发送或处理上述方法中所涉及的数据和/或信息。在一种可能的设计中,所述芯片系统还包括存储器,所述存储器,用于保存执行设备或训练设备必要的程序指令和数据。该芯片系统,可以由芯片构成,也可以包括芯片和其他分立器件。
从以上技术方案可以看出,本申请实施例具有以下优点:
对于数据编码方法,由于用于对待编码数据进行正向变换处理的可逆流模型包括目标可逆流层,该目标可逆流层不仅包括基于该模型参数确定的乘法运算和除法运算,该目标可逆流层中的模型参数还能够约束在所述正向变换处理过程中生成的辅助变量,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。也就是说,本申请不仅能够使用乘法运算,还可以在进行除法运算时通过使用辅助变量实现可逆,因此,相比于传统技术中的整数流模型提高了模型表征能力,有利于提升模型概率分布的估计准确率,进而能够提升无损压缩的压缩效率。
对于数据解码方法,数据处理装置将对解码后的隐变量进行逆向变换处理,该逆向变换处理的可逆流模型包括目标可逆流层,该目标可逆流层不仅包括基于该模型参数确定的乘法运算和除法运算,该目标可逆流层中的模型参数还能够约束在所述逆向变换处理过程中生成的辅助变量,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。也就是说,本申请能够针对正向变换处理过程中的乘法运算,采用辅助变量进行除法运算,以将隐变量恢复为正向变换处理前的数据(即前述数据编码方法中的待编码数据)。因此,不仅能够保证变换处理的可逆运算,还能够提高模型表征能力,有利于提升模型概率分布的估计准确率,进而能够提升无损压缩的压缩效率。
附图说明
为了更清楚地说明本申请实施例的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例。
图1为本申请提供的人工智能主体框架的一种结构示意图;
图2为本申请提供的压缩流程和解压流程的一个原理示意图;
图3为本申请中数据编码方法的一个流程图;
图4A为本申请中可逆流模型的一个结构示意图;
图4B为本申请中可逆流模型的另一个结构示意图;
图5为本申请中数据解码方法的一个流程图;
图6A为本申请中可逆流模型的一个结构示意图;
图6B为本申请中可逆流模型的另一个结构示意图;
图7为本申请中目标可逆流层在进行正向变换时的一个流程图;
图8A为本申请中目标可逆流层在进行正向变换时的另一个流程图;
图8B为本申请中目标可逆流层在进行正向变换时的一个示例图;
图9为本申请中目标可逆流层在进行逆向变换时的一个流程图;
图10A为本申请中目标可逆流层在进行逆向变换时的另一个流程图;
图10B为本申请中目标可逆流层在进行逆向变换时的一个示例图;
图11为本申请中数据处理装置的一个实施例示意图;
图12为本申请中数据处理装置的另一个实施例示意图;
图13为本申请中数据处理装置的另一个实施例示意图;
图14为本申请中数据处理装置作为编码端时的一个实施例示意图;
图15为本申请中数据处理装置作为解码端时的一个实施例示意图。
具体实施方式
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本申请一部分实施例,而不是全部的实施例。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象,而不必用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换,以便这里描述的实施例能够以除了在这里图示或描述的内容以外的顺序实施。此外,术语“包括”和“具有”以及他们的任何变形,意图在于覆盖不排他的包含,例如,包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元,而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
本申请提供的数据编码方法和数据解码方法可以应用于人工智能AI领域以及涉及人工智能AI的存储领域。如图1所示,为人工智能主体框架的一种结构示意图,下面从“智能信息链”(水平轴)和“IT价值链”(垂直轴)两个维度对上述人工智能主题框架进行介绍。其中,“智能信息链”反映从数据的获取到处理的一列过程。示例性的,前述处理过程可以是智能信息感知、智能信息表示与形成、智能推理、智能决策、智能执行与输出的一般过程。在前述过程中,数据经历了“数据—信息—知识—智慧”的凝练过程。此外,“IT价值链”表示从人智能的底层基础设施、信息(提供和处理技术实现)到系统的产业生态过程,反映了人工智能为信息技术产业带来的价值。
下面分别对前述主体框架中的各个部分进行介绍:
(1)基础设施
基础设施用于为人工智能系统提供计算能力支持,实现与外部世界的沟通,并通过基础平台实现支撑。具体地,前述基础设施可以是具备计算能力的处理电路或智能芯片。示例性的,前述智能芯片包括中央处理器(central processing unit,CPU)、神经网络处理器(neural-network processing unit,NPU)、图形处理器(graphics processing unit,GPU)、专用集成电路(application specific integrated circuit,ASIC)、现场可编程逻辑门阵列(field programmable gate array,FPGA)等硬件加速芯片。此外,前述基础设施还可以是提供云计算或云存储功能的基础平台。具体地,基础平台包括分布式计算框架及网络等相关的平台保障和支持,可以包括云存储和计算、互联互通网络等。示例性的,传感器和外部沟通获取数据,这些数据提供给基础平台提供的分布式计算系统中的智能芯片进行计算。
(2)数据
基础设施的上一层的数据用于表示人工智能领域的数据来源。其中,数据涉及到图形、 图像、语音、文本,还涉及到传统设备的物联网数据,包括已有系统的业务数据以及力、位移、液位、温度、湿度等感知数据。
(3)数据处理
数据处理通常包括数据训练、机器学习、深度学习、搜索、推理以及决策等方式。
其中,机器学习和深度学习可以对数据进行符号化和形式化的智能信息建模、抽取、预处理以及训练等。
推理是指在计算机或智能系统中,模拟人类的智能推理方式,依据推理控制策略,利用形式化的信息进行机器思维和求解问题的过程,典型的功能是搜索与匹配。
决策是指智能信息经过推理后进行决策的过程,通常提供分类、排序以及预测等功能。
(4)通用能力
对数据经过上面提到的数据处理后,进一步基于数据处理的结果可以形成一些通用的能力。例如,前述通用的能力可以是算法或者一个通用系统,例如,翻译,文本的分析,计算机视觉的处理,语音识别,图像的识别等等。
(5)智能产品及行业应用
智能产品及行业应用指人工智能系统在各领域的产品和应用,是对人工智能整体解决方案的封装,将智能信息决策产品化、实现落地应用,其应用领域主要包括:智能终端、智能制造、智能交通、智能家居、智能医疗、智能安防、自动驾驶、平安城市等。
本申请提供的数据编码方法和数据解码方法可以应用于对上述各种应用领域中需要实现无损压缩的场景中。具体地,可以应用于对图像、视频、文本等数据的无损压缩场景中。
为便于理解,以如图2为例对本申请的应用场景进行介绍。如图2所示,数据处理装置可以获取到待编码数据,该待编码数据可以图像、视频、文本等数据的原始文件或经过预处理后的文件。为便于理解,本示例以前述待编码数据为待压缩存储的图像(后文简称待压缩图像)为例进行介绍。其中,待压缩图像可以是相机拍摄的照片或是从视频中截取的一帧画面。数据处理装置可以通过可逆流模型对获取到的待压缩图像进行处理,将图像数据变换处理为隐变量,且产生隐变量中各点的概率估计。编码器可以通过隐变量中各点的概率估计对提取获得的隐变量进行编码,降低隐变量的编码冗余,进一步降低图像压缩过程中的数据传输量,并将编码得到的编码数据以数据文件的形式保存在对应的存储位置。当用户需要获取上述存储位置中保存的文件时,数据处理装置可以在相应的存储位置获取并加载上述保存的文件,并基于解码器获取到解码的隐变量,通过可逆流模型对隐变量进行逆变换处理,得到经过逆变换处理后重构的图像(即解码输出)。
应理解,本申请所涉及的数据处理装置可以为终端设备或者服务器。具体地,该终端设备可以为手机、平板、笔记本电脑以及智能穿戴设备等,该终端设备可以对获取到的数据(例如图像数据、视频数据或文本数据等)进行压缩处理。示例性的,终端设备可以为虚拟现实(virtual reality,VR)设备或者增强现实(augmented reality,AR)。应理解,本申请实施例还可以应用于其他需要进行数据压缩的场景中,此处不予赘述。本实施例以及后续实施例中,采用数据处理装置这一称谓进行介绍。
基于前述应用场景,下面将先分别对前述数据编码方法和数据解码方法的主要流程进行介绍,再分别对可逆流模型分别在数据编码方法和数据解码方法中对数据的变化处理过程进 行介绍:
下面将结合图3对本申请提供的数据编码方法的流程进行介绍,此时,前述数据处理装置作为编码端将执行如下步骤:
步骤301,获取待编码数据。
其中,待编码数据指进行压缩编码前的数据。该待编码数据可以是整数数据,也可以是定点数数据(也被称为浮点数数据)。
在一种可能的实施方式中,该待编码数据可以是图像、视频或者文本的等原始数据。此时,该待编码数据一般为整数数据。示例性的,若待编码数据为图像数据,该待编码数据可以是终端设备通过摄像头拍摄到的图像,也可以是从终端设备内部获得的图像(例如,终端设备的相册中存储的图像,或者,终端设备从云端获取的图像)。应理解,上述图像可以是具有图像压缩需求的图像,本申请并不对待处理图像的来源进行限定。
在另一种可能的实施方式中,该待编码数据也可以为基于图像、视频或者文本的等原始数据进行预处理后的数据。此时,该待编码数据可能是具有一定精度的定点数。示例性的,数据处理装置可以在获取的原始数据之后,采用噪声数据对原始数据进行反编码处理以获得前述待编码数据。此时,该待编码数据为具有第一预设精度的定点数数据。
示例性的,若给定噪声数据的概率分布(例如,噪声数据u的概率分布为q(u|x)δ,且u∈[0,1),其中,δ=2 -k,k表示定点数的精度),数据处理装置可以采用反编码技术基于该噪声数据的概率分布解码该噪声数据,得到解码后的噪声数据u,该解码后的噪声数据u为具有第一预设精度的定点数数据(例如,第一预设精度为前述精度k)。然后,该数据处理装置将前述原始数据与解码后的噪声数据求和(即计算
Figure PCTCN2022120095-appb-000001
其中,x为原始数据,u为解码后的噪声数据),得到待编码数据
Figure PCTCN2022120095-appb-000002
该待编码数据为具有第一预设精度的定点数数据。
应理解,前述噪声数据可以是数据处理装置中已压缩的数据的比特流,还可以是数据处理装置中预存的比特流,具体此处不做限定。本实施方式中,由于采用噪声数据对原始数据进行反编码处理,能够增大数据处理装置计算的数据的数值,有利于提升后续变换过程的处理精度,也有利于提升压缩率。
应注意,若待编码数据为视频数据,且视频的大小与模型输入大小不匹配,则需要将视频切割成若干视频块,每块输入大小与模型(可逆流模型)的输入大小相同;如果视频长度大于模型要求的长度,则切割成多段视频输入。如果输入大小或视频长度不足,可以用特定颜色的色块填充到输入大小或特定长度。
应注意,若待编码数据为文本数据,则需要对文本中的字符或者单词构造词向量表示,即压缩过程需要先将文本转换成向量。示例性的,设输入数据w(单词或字符),对应的词向量表示为d维向量x=μ(w)。构造概率分布p(x|w)=N(μ(w),σ 2(w))δ(δ=2 -dk),
Figure PCTCN2022120095-appb-000003
(p(w)为w的先验,一般为w的词频)。在数据预处理过程中,给定输入w,利用p(x|w)解码x,利用p(w|x)编码,得到待编码数据。
步骤302,采用可逆流模型对该待编码数据进行正向变换处理,得到隐变量;其中,该可逆流模型包括目标可逆流层,该目标可逆流层包括模型参数,该模型参数用于约束在该正向变换处理过程中生成的辅助变量,该目标可逆流层对应的运算包括基于该模型参数确定的乘法运算和除法运算,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。
其中,可逆流模型是本申请提出的新的流模型。目标可逆流层是本申请提出的可逆流模型特有的层。由于,目标可逆流层能够实现可逆运算,以使得该目标可逆流层和/或该目标可逆流层的变形层组成的可逆流模型便能够实现可逆运算。
本实施例中,该可逆流模型包括多个可逆流层,多个可逆流层多个可逆流层采用串行的排布方式依次对数据进行处理。而前述目标可逆流层可以是前述多个可逆流层中的一个可逆流层,也可以是前述多个可逆流层中的一个可逆流层的组成部分。下面将分别对两种情况进行介绍:
在一种可能的实施方式中,该目标可逆流层为可逆流模型中众多可逆流层中的一个可逆流层。具体地,该可逆流模型可以包括N个可逆流层,该N个可逆流层包括前述目标可逆流层,其中,N为大于或等于1的整数。可选的,当前N个可逆流层包括至少两个可逆流层时,前述N个可逆流层可以采用串行的排布方式。
示例性的,如图4A所示,该可逆流模型包括N个串行的可逆流层,该N为大于1的整数。其中,第i-1个可逆流层的输出用于作为第i个可逆流层的输入,第i个可逆流层的输出用于作为第i+1个可逆流层的输入,该i为大于等于1且不大于N的正整数。例如,第1个可逆流层(即可逆流层1)的输入为该待编码数据,第1个可逆流层(即可逆流层1)的输出用于作为第2个可逆流层(即可逆流层2)的输入,第2个可逆流层(即可逆流层2)的输出用于作为第3个可逆流层(即可逆流层3)的输入,以此类推,该第N个可逆流层(即可逆流层N)的输出为该隐变量。该N个串行的可逆流层包括该目标可逆流层。在图4A所示示例中,该目标可逆流层可以是图4A中N个可逆流层中的一个可逆流层,例如,可逆流层2为目标可逆流层。当然,前述N个可逆流层可以包括多个目标可逆流层,例如,可逆流层2、可逆流层4和可逆流层6均为目标可逆流层,即前述N个可逆流层中包含3个目标可逆流层。
在另一种可能的实施方式中,该目标可逆流层为可逆流模型中的众多可逆流层中的一个可逆流层的组成部分。示例性的,该目标可逆流层可以是图4A中某一个可逆流层中组成部分。此时,可逆流模型中的可逆流层除了包含前述目标可逆流层之外,还可以包括其他模型和/或算子,以构成该目标可逆流层的变形层。具体地,该目标可逆流层的变形层可以为可逆卷积层或耦合层。此时,可逆流模型除了包括目标可逆流层之外,还包括可逆卷积层和/或耦合层。关于可逆卷积层和耦合层的具体结构将会在后文图进行介绍,此处不予赘述。
应理解,前述不同种类的可逆流层(即目标可逆流层、可逆卷积层和耦合层)可以进行组合堆叠,以提高可逆流模型的变换处理能力。具体地,该可逆流模型可以包括至少一个目标可逆流层、至少一个可逆卷积层以及至少一个耦合层。示例性的,可以采取目标可逆流层分别与耦合层和可逆卷积层交错堆叠的方式构建可逆流模型。以图4B为例,该可逆流模型包括目标可逆流层、耦合层和可逆卷积层。其中,待编码数据作为目标可逆流层1的输入,该目标可逆流层1的输出作为耦合层的输入,该耦合层的输出作为目标可逆流层2的输入,目标可逆流层2的输出作为可逆卷积层的输入,以此类推,经过目标可逆流层n的处理,输出隐变量。
应理解,图4B仅仅为众多可逆流层堆叠的一种示例,在实际应用中,可以基于实际需求调整各个可逆流层的堆叠顺序,本申请不对前述各个可逆流层的堆叠顺序进行限定,也不再一一举例介绍。
本实施例中,该目标可逆流层用于实现输入该目标可逆流层的数据(为便于介绍,后文称为第一数据)与该目标可逆流层输出的数据(为便于介绍,后文称为第二数据)之间的可逆运算。其中,该第一数据为该待编码数据或者经过至少一个该可逆流层(例如,N个可逆流层中的n个可逆流层,其中,n为大于1且小于N的整数)处理后输出的数据,该第二数据为该隐变量或者该第二数据经过至少一个该可逆流层处理后输出的数据为该隐变量。
其中,可逆指经正向变换处理后的数据,能够在逆向变换处理作用下恢复为前述正向变换处理之前的数据;也可以理解为,经正向变换处理后的数据再经逆向变换处理后的输出,与前述正向变换处理之前的数据一致。示例性的,前述第一数据经过目标可逆流层的正向变换处理后将获得第二数据,该第二数据再经过前述目标可逆流层的逆向变换处理后将恢复为前述第一数据。此外,可逆运算指前述变换过程中涉及的运算,该运算既可以通过输入数据得到输出数据,也可以通过输出数据反推出输入数据。例如,设输入数据为x,输出数据为z,正向运算为z=f(x),则可以通过逆向运算(例如,x=f -1(z))从输出数据z中恢复x。该目标可逆流层在变换处理过程中至少涉及乘法运算和除法运算。
进一步地,该目标可逆流层包括模型参数,该模型参数用于约束在该正向变换处理过程中生成的辅助变量。也就是说,该目标可逆流层能够基于前述模型参数生成辅助变量,该辅助变量用于辅助经过乘法运算的数据通过除法运算实现可逆运算。由于计算机的乘除运算过程与数学理论的乘除运算过程的计算原理不同,因此,经过乘法运算的数据不一定能够通过除法运算恢复。例如,若数据与某个系数(或参数)在经过乘法运算之后能够获得某个乘积,该乘积再与前述系数进行除法运算而获得的商不一定是前述数据。而本申请中,该目标可逆流层生成的辅助变量能够辅助经过乘法运算的数据通过除法运算实现可逆。具体地,后文将进行详细介绍。
应理解,该辅助变量受目标可逆流层中的模型参数的约束,指该辅助变量的取值范围受模型参数的取值范围的约束。该模型参数可以是预设的参数,也可以是基于神经网络生成的参数,还可以是该可逆流模型基于其他的算法(例如,插值法或二分查找法)确定的参数,具体此处不做限定。
具体地,该目标可逆流层用于对输入该目标可逆流层的第一数据进行正向变换处理,得到输出该目标可逆流层的第二数据,该第一数据为该乘法运算的乘数,该第二数据为该除法运算的整数商。也就是说,在正向变换处理过程中,在基于该模型参数确定的乘法运算中,该第一数据作为该乘法运算的乘数,该乘法运算的另一个乘数由该模型参数确定;由于,辅助变量可以作为该乘法运算的乘积的增量,因此,基于前述第一数据与另一个乘数的乘积,再加上前述辅助变量可以获得中间数据。在基于该模型参数确定的除法运算中,该第二数据作为该除法运算的整数商,该除法运算的除数为中间数据,被除数由该模型参数确定。因此,第一数据在输入目标可逆流层之后,经过乘法运算和除法运算等运算处理后,将输出第二数据,该第二数据为目标可逆流层的输出。具体地,请参阅后文图7和图8A对应的实施例。
本实施例中,当数据处理装置采用可逆流模型将待编码数据变换处理为隐变量之后,该数据处理装置将执行步骤303。
步骤303,对该隐变量进行编码处理,得到该待编码数据对应的编码数据。
本实施例中,隐变量z可以由概率分布p Z(z)表示,可以根据概率分布p Z(z)对隐变量z编码, 以得到编码数据。可选的,对隐变量进行编码处理的过程可以是熵编码过程,该熵编码过程可采用现有的熵编码技术,本申请对此不再赘述。
本实施例中,在得到编码数据之后,可以将编码数据发送给用于解压缩的设备。此时,用于解压缩的设备可以对该数据进行解压缩(或者称之为解码)。或者,用于压缩的终端设备可以将编码数据存储在存储设备中,在需要时,该终端设备可以从存储设备中获取编码数据,并可以对该编码数据进行解压缩。
本实施例中,由于用于对待编码数据进行正向变换处理的可逆流模型包括目标可逆流层,该目标可逆流层不仅包括基于该模型参数确定的乘法运算和除法运算,该目标可逆流层中的模型参数还能够约束在所述正向变换处理过程中生成的辅助变量,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。也就是说,本申请不仅能够使用乘法运算,还可以在进行除法运算时通过使用辅助变量实现可逆,因此,相比于传统技术中的整数流模型提高了模型表征能力,有利于提升模型概率分布的估计准确率,进而能够提升无损压缩的压缩效率。
下面将结合图5对本申请提供的数据解码方法的流程进行介绍,此时,前述数据处理装置作为解码端将执行如下步骤:
步骤501,获取编码数据。
其中,编码数据指采用前述图3对应实施例所介绍的数据编码方法而获得的数据。该编码数据可以是整数数据。
本实施例中,若数据处理装置需要获取压缩前的数据(即前述实施例所介绍的待编码数据),则该数据处理装置可以先获取编码数据,再基于前述编码数据进行相应的解码处理。
在一种可能的实施方式中,图3对应的实施例中的数据处理装置与图5对应的实施例中的数据处理装置位于同一设备中,或者,图3对应的实施例中的数据处理装置与图5对应的实施例中的数据处理装置为同一设备。例如,数据处理装置为具有存储功能的终端设备,该终端设备在按照前述图3对应实施例的方法将待编码数据进行压缩编码,并将获得的编码数据存储至存储器。当需要使用前述待编码数据时,该终端设备从存储器中读取前述编码数据,并按照图5对应实施例的方法对该编码数据进行解压缩。
在另一种可能的实施方式中,图3对应的实施例中的数据处理装置与图5对应的实施例中的数据处理装置位于不同设备中,或者,图3对应的实施例中的数据处理装置与图5对应的实施例中的数据处理装置为不同的设备。例如,在需要传输压缩数据的场景,发送端将按照图3对应实施例的方法将编码数据发送给接收端,接收端按照图5对应实施例的方法将编码数据进行解码,以获得压缩前的数据。
应注意,本申请不限定编码端和解码端是否位于同一设备中。
步骤502,对该编码数据进行解码处理,得到隐变量。
本实施例中,数据处理装置可以对编码数据进行解码,以得到隐变量。具体的,可以利用现有技术中的熵解码技术对编码数据进行解码,得到重建的隐变量。
步骤503,采用可逆流模型对隐变量进行逆向变换处理,得到解码输出;其中,前述可逆流模型包括目标可逆流层,该目标可逆流层包括模型参数,该模型参数用于约束在该逆向 变换处理过程中生成的辅助变量,该目标可逆流层对应的运算包括基于该模型参数确定的乘法运算和除法运算,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。
其中,可逆流模型是本申请提出的新的流模型。目标可逆流层是本申请提出的可逆流模型特有的层。该可逆流模型包括多个可逆流层,多个可逆流层多个可逆流层采用串行的排布方式依次对数据进行处理。而前述目标可逆流层可以是前述多个可逆流层中的一个可逆流层,也可以是前述多个可逆流层中的一个可逆流层的组成部分。具体地,该可逆流模型的结构可以参阅前文图4A和图4B对应的相关描述,具体此处不予赘述。应注意,数据处理装置在图3对应的数据编码方法中使用的可逆流模型和图5对应的数据解码方法中使用的可逆流模型是相同的,但是,在数据编码方法中数据流经可逆流模型中各个可逆流层的顺序与在数据解码方法中数据流经可逆流模型中各个可逆流层的顺序相反,也可以理解为,在逆向变换处理过程中数据流经可逆流层的顺序与前述正向变换过程中数据流经可逆流层的顺序相反。
示例性的,若在正向变换过程中,可逆流模型中各个可逆流层对数据的处理顺序如图4A所示,则在逆向变换过程中,可逆流模型中各个可逆流层对数据处理的顺序可以如图6A所示。其中,该可逆流模型包括N个串行的可逆流层,该N为大于1的整数。若在正向变换处理过程中,如图6A左侧的处理顺序,数据依次流经第i-1个可逆流层、第i个可逆流层以及第i+1个可逆流层,则在逆变换处理过程中,如图6A右侧的处理顺序,数据依次流经第i+1个可逆流层、第i个可逆流层以及第i-1个可逆流层。其中,该i为大于等于1且不大于N的正整数。
此外,在正向变换过程中构成可逆流模型的可逆流层的种类,与在正向变换过程中构成可逆流模型的可逆流层的种类,应当相同。示例性的,可逆流模型中不同种类的可逆流层(即目标可逆流层、可逆卷积层和耦合层)按照图4B所示示例进行组合堆叠。例如,在正向变换过程中,如图6B左侧的数据处理顺序,待编码数据作为目标可逆流层1的输入,该目标可逆流层1的输出作为耦合层的输入,该耦合层的输出作为目标可逆流层2的输入,目标可逆流层2的输出作为可逆卷积层的输入,以此类推,经过目标可逆流层n的处理,输出隐变量。那么,在逆向变换过程中,如图6B右侧的数据处理顺序,隐变量作为目标可逆流层n的输入,该目标可逆流层n的输出下一个可逆流层,随后,进入可逆卷积层,可逆卷积层的输出作为目标可逆流层2的输入,目标可逆流2的输出作为耦合层的输入,耦合层的输出作为目标可逆流层1的输入,最后,目标可逆流层1的解码输出即为前述实施例所介绍的待编码数据。
应理解,图6B仅仅为众多可逆流层堆叠的一种示例,在实际应用中,可以基于实际需求调整各个可逆流层的堆叠顺序,本申请不对前述各个可逆流层的堆叠顺序进行限定,也不再一一举例介绍。
该目标可逆流层在逆变换处理过程中,用于将输入该目标可逆流层的第二数据变换处理为前文所介绍的第一数据。其中,该第二数据为该隐变量或者经过至少一个该可逆流层处理后输出的数据,该第一数据为该解码输出或者该第一数据经过至少一个该可逆流层处理后的输出为该解码输出。
具体地,该目标可逆流层用于对输入该目标可逆流层的第二数据进行逆向变换处理,得到输出该目标可逆流层的第一数据,该第二数据为该乘法运算的乘数,该第一数据为该除法 运算的整数商。也就是说,在逆向变换处理过程中,在基于该模型参数确定的乘法运算中,该第二数据作为该乘法运算的乘数,该乘法运算的另一个乘数由该模型参数确定;由于,辅助变量可以作为该乘法运算的乘积的增量,因此,基于前述第二数据与另一个乘数的乘积,再加上前述辅助变量可以获得中间数据。在基于该模型参数确定的除法运算中,该第一数据作为该除法运算的整数商,该除法运算的除数为中间数据,被除数由该模型参数确定。因此,第二数据在输入目标可逆流层之后,经过乘法运算和除法运算等运算处理后,将输出第一数据,该第一数据为目标可逆流层的输出。具体地,请参阅后文图9和图10A对应的实施例。
应理解,在逆变换处理过程所使用的模型参数和辅助变量等为前述变换处理过程中生成或存储的。例如,在正向变换处理过程的乘法运算所使用的模型参数(例如,第一参数)和辅助变量(例如,第一辅助变量)将应用于逆向变换处理过程的除法运算中;在正向变换处理过程的除法运算所使用的模型参数(例如,第二参数)和辅助变量(例如,第二辅助变量)将应用于逆向变换处理过程的乘法运算中。具体地,关于模型参数和辅助变量的介绍可以参阅前文图3对应的实施例,此处不予赘述。
本实施例中,当数据处理装置采用可逆流模型对隐变量进行变换处理之后,该数据处理装置可以直接输出编码前述的数据。应理解,若编码前的数据为图像、视频或者文本的等原始数据,则经前述逆向变换处理之后的解码输出为图像、视频或者文本的等原始数据。若编码前的数据为基于图像、视频或者文本的等原始数据进行预处理后的数据。数据处理装置还可以将前述解码输出恢复为图像、视频或者文本的等原始数据。
示例性的,若解码输出为采用噪声数据对原始数据进行反编码处理而获得的数据,则该数据处理装置可以通过解反编码处理从前述解码输出分离出噪声数据和原始数据。若给定噪声数据的概率分布(例如,噪声数据u的概率分布为q(u|x)δ,且u∈[0,1),其中,δ=2 -k,k表示定点数的精度),该可逆流模型的输出为
Figure PCTCN2022120095-appb-000004
则基于该可逆流模型的输出
Figure PCTCN2022120095-appb-000005
进行下取整运算(即计算
Figure PCTCN2022120095-appb-000006
),得到原始数据。可选的,将可逆流模型的输出
Figure PCTCN2022120095-appb-000007
与原始数据x作差(即计算
Figure PCTCN2022120095-appb-000008
),得到噪声数据u。可选的,采用噪声数据u的概率分布q(u|x)δ编码u,以恢复进行反编码处理之前所使用的码流。
本实施例中,数据处理装置将对解码后的隐变量进行逆向变换处理,该逆向变换处理的可逆流模型包括目标可逆流层,该目标可逆流层不仅包括基于该模型参数确定的乘法运算和除法运算,该目标可逆流层中的模型参数还能够约束在所述逆向变换处理过程中生成的辅助变量,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。也就是说,本申请能够针对正向变换处理过程中的乘法运算,采用辅助变量进行除法运算,以将隐变量恢复为正向变换处理前的数据(即前述数据编码方法中的待编码数据)。因此,不仅能够保证变换处理的可逆运算,还能够提高模型表征能力,有利于提升模型概率分布的估计准确率,进而能够提升无损压缩的压缩效率。
下面将结合前述模型参数对该目标可逆流层的正向变换处理过程的原理进行介绍:
具体地,该目标可逆流层将基于该模型参数获取辅助变量,然后,基于由该辅助变量和该模型参数确定的单调连续函数对输入该目标可逆流层的第一数据进行变换处理,得到输出该目标可逆流层的第二数据。其中,第一单调连续函数为由前述辅助变量和模型参数确定的 函数,该目标可逆流层在变换处理过程中将使用前述单调连续函数对输入该目标可逆流层的第一数据进行处理,以使得经过一系列处理后能够输出前述第二数据。
其中,该模型参数包括第一参数和第二参数,该第一参数和该第二参数均为正整数,该第一参数与该第二参数不相等。该辅助变量包括第一辅助变量和第二辅助变量。其中,第一辅助变量与第一参数对应,第二辅助变量与第二参数对应。可以理解为,第一辅助变量是基于第一参数确定的,第二辅助变量是基于第二参数确定的。该第一单调连续函数可以是一元线性函数,也被称为一元一次函数(linear function of one variable),即形如y=kx+b(k≠0)的函数。
进一步地,如图7所示,该数据处理装置将通过目标可逆流层在正向变换处理过程中执行如下步骤:
步骤701,基于第一参数获取第一辅助变量,该第一参数用于确定第一一元线性函数的斜率,该第一辅助变量用于确定第一一元线性函数的截距。
在一种可选的实施方式中,数据处理装置采用辅助数据编解码辅助变量,则该数据处理装置可以基于第一参数确定的概率分布和第一辅助数据获取前述第一辅助变量。该辅助数据可以是数据处理装置中预存的码流(例如,二进制码流),也可以是该数据处理装置中的可逆流模型随机生成的码流,还可以是数据处理装置从外部存储装置中获取的码流,还可以是已压缩的数据形成的码流,具体此处不做限定。
为便于理解,下面先对辅助数据编解码辅助变量的原理进行介绍:
以均匀分布U(0,A)为例进行介绍。由均匀分布的定义可知,对于任意符号s∈{0,1,…,A-1},满足p(s)=1/A。在编码过程中,编码前的码流为整数c(其中,c采用k位的二进制码流表示),则对前述任意符号s编码后获得的码流为c′=c·A+s(其中,c’采用k位的二进制码流表示)。在解码过程中,根据编码后的码流(即解码前的码流)c′采用除法取余运算恢复前述任意符号s,即采用公式s=c′mod A计算前述任意符号s,与此同时,采用除法取整运算获得编码前的码流c,即采用公式
Figure PCTCN2022120095-appb-000009
计算前述编码前的码流c。
可选的,为避免码流过长而影响算法实现效率,在实际应用中,可以设置状态向量bs存储二进制码流。
具体地,辅助数据编码辅助变量的过程如下所示:
(1)c2←c1·A+s;
(2)如果c2≥2 M+K,则将c2的低K位(c2 mod 2 K)存储到bs的尾部,
Figure PCTCN2022120095-appb-000010
否则,c3←c2;
(3)存储辅助数据(即c3和bs)。
相应的,当给定辅助数据(即c3和bs)时,基于辅助数据解码辅助变量的过程如下所示:
(1)如果c3<2 M·A,则从bs的尾部读取并弹出K位数r(即(c2 mod 2 K)),c2←2 K·c3+r;否则c2←c3;
(2)s←c2 modA;
(3)
Figure PCTCN2022120095-appb-000011
(4)输出解压的符号s。
其中,K和M均为大于0的整数。示例性的,K的取值可以为32,M的取值可以为4。 应理解,在实际应用中,可以基于计算需要调整前述K和/或M的取值,具体此处不做限定。
由前述计算原理可知,当数据处理装置基于第一参数确定的概率分布和第一辅助数据获取前述第一辅助变量时,该数据处理装置将基于该第一参数确定第一均匀分布,该第一均匀分布的端点为0和该第一参数。然后,该数据处理装置采用该第一均匀分布解码该第一辅助数据,得到该第一辅助变量和第二辅助数据,该第一辅助变量的取值位于0到该第一参数之间。其中,第二辅助数据是第一辅助数据经解码而获得的数据,该第二辅助数据用于在逆向变换过程中编码该第一辅助变量。
其中,该采用该第一均匀分布解码该第一辅助数据,得到该第一辅助变量和第二辅助数据的过程具体可以为:数据处理装置以该第一辅助数据为除数且该第一参数为被除数,进行除法取余运算,得到该第一辅助变量,并且,以该第一辅助数据为除数且该第一参数为被除数,进行除法取整运算,得到该第二辅助数据。此外,该数据处理装置将存储该第二辅助数据,比便于在逆向变换过程中使用前述第二辅助数据。
本实施方式中,提出采用基于第一参数确定的均匀分布的熵编码算法编解码辅助变量,有利于提高存储辅助变量的效率。相比于传统技术中的动态熵编码技术rANS(range-based Asymmetric Numerical System)而言,有利于提升目标可逆流层的吞吐率。
在另一种可选的实施方式中,该数据处理装置基于第一参数确定第一取值范围,该第一取值范围为0到第一参数。然后,该数据处理装置从前述第一取值范围中随机选择一个整数作为第一辅助变量。可选的,该数据处理装置还将存储第一辅助变量,以便于在后续逆向变换处理过程中,该数据处理装置能够基于第一辅助变量进行逆向变换处理。
步骤702,将该第一数据带入该第一一元线性函数进行计算,得到中间数据。
具体地,数据处理装置将采用该第一参数与第一数据进行乘法运算而得的乘积,与该第一辅助变量进行加法运算,得到中间数据。
在本步骤中,即在正向变换处理中,该第一辅助变量为该乘法运算的乘积的增量。具体地,数据处理装置将该第一数据作为该第一参数的乘数进行乘法运算,得到第一乘积数据;然后,将该第一辅助变量作用该第一乘积数据的增量进行加法运算,得到中间数据。
应理解,该第一辅助变量还用于作为该逆向变换处理中的除法运算产生的余数。
本实施例中,由于,第一辅助变量参与了确定中间数据的过程,即第一辅助变量作为该乘法运算的乘积的增量,基于该中间数据进行除法取整运算的整数商才为第二数据,因此,避免了仅用乘法运算而造成的运算不可逆的问题,也避免了仅用加法运算和减法运算而造成的模型表征能力差的问题。
步骤703,基于该中间数据和该第二参数确定该第二数据,并输出该第二数据。
具体地,数据处理装置将以该中间数据为除数且该第二参数为被除数,进行除法取整运算,得到该第二数据,并输出该第二数据。
步骤704,基于该中间数据和第二参数确定第二辅助变量。
具体地,数据处理装置将以该中间数据为除数且该第二参数为被除数进行除法取余运算,得到该第二辅助变量。其中,该第二参数和该第二辅助变量用于确定第二一元线性函数,该第二一元线性函数为前述正向变换处理对应的逆向变换处理所使用的函数。该第二参数用于确定第二一元线性函数的斜率,该第二辅助变量用于确定该第二一元线性函数的截距。
在本步骤中,即在正向变换处理中,该第二辅助变量为该除法运算产生的余数。
应理解,该第二辅助变量还用于作为该正向变换处理对应的逆向变换处理中的乘法运算的乘积的增量。
本实施例中,由于该目标可逆流模型在正向变换处理中,还将生成第二辅助变量,该第二辅助变量用于作为该正向变换处理对应的逆向变换处理中的乘法运算的乘积的增量。也就是说,在逆向变换处理中,数据处理装置将基于第二辅助变量将第二数据恢复为中间数据,进而将中间数据恢复为第一数据。因此,在正向变换处理中,计算出第二辅助变量,有利于实现将第二数据恢复为第一数据。
应注意,步骤703和步骤704之间没有明确的时间先后顺序的限定,该数据处理装置可以先执行步骤703再执行步骤704,该数据处理装置也可以先执行步骤704再执行步骤703,该数据处理装置还可以分别同时执行步骤703和步骤704,具体此处不做限定。
步骤705,存储第一辅助变量和第二辅助变量。
应理解,步骤705应当在步骤704之后执行。
本实施例中,数据处理装置可以直接存储第一辅助变量和第二辅助变量,该数据处理装置也可以采用编码的方式间接存储第一辅助变量和第二辅助变量。
在一种可选的实施方式中,该数据处理装置可以直接存储第一辅助变量和第二辅助变量。具体地,该数据处理装置可以在步骤701中生成第一辅助变量后便存储第一辅助变量,随后,该数据处理装置可以在步骤704中生成第二辅助变量后便存储第二辅助变量。可选的,该数据处理装置可以将第一辅助变量和第二辅助变量对应存储,以便于在逆向变换处理过程中,该数据处理装置可以获取到与第二辅助变量对应的第一辅助变量,进而有利于提高逆向变换处理过程的效率。示例性的,该数据处理装置可以采用数组或表格等方式对应存储前述第一辅助变量和第二辅助变量,具体此处不做限定。
在另一种可选的实施方式中,若数据处理装置采用辅助数据编解码辅助变量,此时,数据处理装置可以存储编码了辅助变量的辅助数据,而不用直接存储辅助变量。也就是说,该数据处理装置可以在前述步骤701过程将解码第一辅助变量而获得的第二辅助数据进行暂存,在步骤704之后,该数据处理装置可以基于同样的原理参照如下方式编码第二辅助变量,并将编码了第二辅助变量的辅助数据(后文将介绍的第三辅助数据)进行存储,以实现间接存储第一辅助变量和第二辅助变量。
具体地,该数据处理装置可以基于前述步骤701所介绍的均匀分布熵编码原理编码第二辅助变量。首先,数据处理装置可以获取暂存的第二辅助数据(即解码第一辅助变量而获得的第二辅助数据),与此同时,基于该第二参数确定第二均匀分布,该第二均匀分布的端点为0和该第二参数。然后,该数据处理装置采用该第二均匀分布和该第二辅助数据编码该第二辅助变量,得到第三辅助数据,该第二辅助变量的取值位于0到该第二参数之间。然后,该数据处理装置存储该第三辅助数据,该第三辅助数据用于在逆向变换过程中解码该第二辅助变量。
其中,该采用该第二均匀分布和该第二辅助数据编码该第二辅助变量,得到第三辅助数据的过程具体可以为:数据处理装置将该第二辅助数据与该第二参数进行乘法运算而获得的乘积,与该第二辅助变量进行加法运算,得到该第三辅助数据。
本实施例中,提出输入该目标可逆流层的第二数据会经过乘法运算,输出该目标可逆流层的第一数据是经过除法运算获得的,因此,相比于仅使用整数加减法的整数流模型来说,本申请的可逆流模型具备更强的模型表征能力,有利于提升模型概率分布的估计准确率,进而能够提升无损压缩的压缩效率。
如图8A所示,为目标可逆流层在正向变换处理过程的具体计算流程:
步骤801,获取第一辅助数据,该第一辅助数据用于解码第一辅助变量。
步骤802,采用第一均匀分布解码第一辅助数据,得到第一辅助变量和第二辅助数据。
其中,第一均匀分布是基于第一参数确定,具体请参阅前述图7对应实施例中的相关介绍。
步骤803,将第一数据与第一参数进行乘法运算而获得的第一乘积数据,加上第一辅助变量,得到中间数据。
步骤804,将中间数据作为除数且该第二参数作为被除数进行除法取整运算,得到该第二数据。
步骤805,以中间数据为除数且该第二参数为被除数进行除法取余运算,得到第二辅助变量。
应注意,步骤804和步骤805之间没有明确的时间先后顺序的限定,该数据处理装置可以先执行步骤804再执行步骤805,该数据处理装置也可以先执行步骤805再执行步骤804,该数据处理装置还可以分别同时执行步骤804和步骤805,具体此处不做限定。
步骤806,采用第二辅助数据编码第二辅助变量,得到第三辅助数据。
应理解,步骤806应当在步骤805之后执行。
步骤807,存储第三辅助数据。
为便于理解,下面将结合具体示例对目标可逆流层的变换过程(步骤701到步骤705)进行介绍。如图8B所示,为可逆流模型中的某一个目标可逆流层(例如,可逆流层i)。其中,x表示输入目标可逆流层的数据(即第一数据),是上一个可逆流层(例如,可逆流层i-1)的输出;z表示输出目标可逆流层的数据(即第二数据),是下一个可逆流层(例如,可逆流层i+1)的输入;b1表示第一辅助数据,b2表示第二辅助数据,b3表示第三辅助数据。假设第一参数为R,第二参数为S,则基于第一参数R确定的第一均匀分布为U(0,R),基于第二参数S确定的第二均匀分布为U(0,S)。当第一数据x输入目标可逆流层之后,该数据处理装置将获取第一辅助数据b1,并基于第一均匀分布U(0,R)解码第一辅助数据b1,得到第一辅助变量r d和第二辅助数据b2。然后,该数据处理装置采用基于第一参数R和第一辅助变量r d确定第一一元线性函数(即y=R·x+r d)计算中间数据y。然后,采用中间数据y和第二参数S进行除法取整运算(即计算
Figure PCTCN2022120095-appb-000012
),得到第二数据z。此外,该数据处理装置还将基于中间数据y和第二参数S进行除法取余运算(即计算r e=ymodS(即r e=mod(y,S))),得到第二辅助变量r e。然后,该数据处理装置将获取在解码第一辅助变量r d的过程中获得的第二辅助数据b2,并基于第二均匀分布U(0,S)编码第二辅助变量r e,得到编码后的第三辅助数据b3。该数据处理装置将存储前述第三辅助数据b3,以便于在逆变换处理过程中基于第三辅助数据b3解码出第二辅助变量r e和第二辅助数据b2,以及,基于第二辅助数据b2编码 第一辅助变量r d
应注意,若输入该可逆流模型的待编码数据为具有第一预设精度的定点数数据时,在进行前述图8B所示的变换处理过程之前,该第i个可逆流层(即可逆流层i)还用于在将基于该具有第一预设精度的定点数数据处理为整数数据;以及,在进行前述图8B所示的变换处理过程之后,将该经该变换处理后的整数数据处理为具有该第一预设精度的定点数数据以作为该第i个可逆流层(即可逆流层i)的输出。
本实施方式中,由于目标可逆流层可以基于模型参数(例如,第一参数和第二参数)和辅助数据(例如,第一辅助数据、第二辅助数据以及第三辅助数据)编解码辅助变量(例如,第一辅助变量和第二辅助变量),该辅助变量可以实现进行乘法运算的数据通过除法运算实现可逆。由于,该目标可逆流层不仅涉及加法运算,还涉及乘法运算、除法运算、取整运算和取余运算,因此,可以提升变换处理的多样性,提升模型的表征能力。
应注意,前述目标可逆流层中的第一参数和第二参数可以是预设的参数,也可以是基于神经网络生成的参数,也可以是该数据处理装置基于插值法或二分查找法算法确定的参数。
在一种可能的实施方式中,若目标可逆流层是基于一元线性函数(例如,函数z=f(x)=a·x(a>0))构建的,可以基于该一元线性函数的系数a确定第一参数和第二参数。具体地,可以将前述系数a变形为分数形式,即a=R/S,此时,可以定义R为第一参数,S为第二参数。可选的,可以设置S为较大值,则R=round(a·S)。其中,round(x)表示返回x的四舍五入的整数值。
在另一种可能的实施方式中,若目标可逆流层是基于非线性函数(例如,第一单调连续函数)构建的,则该数据处理装置可以将该非线性函数拆分为多段,将每一段近似为直线,采用插值算法或二分查找法计算前述第一参数和第二参数。具体地,该数据处理装置可以获取第一单调连续函数的定义域中的第一区间和该第一区间在该第一单调连续函数的值域上对应的第二区间。然后,该数据处理装置将基于该第一区间和该第二区间采用插值法或二分查找法计算该第一参数和第二参数。
示例性的,下面将以插值算法为例对确定第一参数和第二参数的过程进行介绍:
假设,第一单调连续函数为单调递增函数z=f(x),并且,该第一单调连续函数中的定义域中的第一区间为[x l,x h),该第一区间在值域上对应的第二区间为[z l,z h),其中,x l,x h,z l,z h均为精度为k的定点数(即浮点数)。
此时,可以先设置一个较大的整数S作为第二参数,然后,基于插值公式(例如,线性插值公式
Figure PCTCN2022120095-appb-000013
)计算第一参数R。
应注意,该数据处理装置可以采用如下任意一种实施方式确定前述第一区间和第二区间。
在一种可能的实施方式中,该数据处理装置可以基于定义域做均匀插值,这种实施方式适用于值域比定义域大的情形。
考虑插值区间大小2 -h(h<<k),正向插值区间确定过程如下:
(1)
Figure PCTCN2022120095-appb-000014
(2)
Figure PCTCN2022120095-appb-000015
(3)输出x l,x h,z l,z h
反向插值区间需要通过搜索确定,确定过程如下:
(1)x′←f -1(z);
(2)
Figure PCTCN2022120095-appb-000016
x′ l=x′ m-2 -h,x′ h=x′ m+2 -h
(3)
Figure PCTCN2022120095-appb-000017
(4)如果z<z′ m则x l=x′ l,x h=x′ m,z l=z′ l,z h=z m′;否则x l=x′ m,x h=x′ h,z l=z′ m,z h=z′ h
(5)输出x l,x h,z l,z h
在另一种可能的实施方式中,该数据处理装置可以基于值域做线性插值,这种实施方式适用于定义域比值域大的情形。
考虑插值区间大小2 -h(h<<k),正向插值区间通过搜索确定,确定过程如下:
(1)z′←f(x);
(2)
Figure PCTCN2022120095-appb-000018
z′ l=z′ m-2 -h,z′ h=z′ m+2 -h
(3)
Figure PCTCN2022120095-appb-000019
(4)如果x<x′ m则x l=x′ l,x h=x′ m,z l=z′ l,z h=z m′;否则x l=x′ m,x h=x′ h,z l=z′ m,z h=z′ h
(5)输出x l,x h,z l,z h
反向插值区间直接计算得到:
(1)
Figure PCTCN2022120095-appb-000020
z h=z l+2 -h
(2)
Figure PCTCN2022120095-appb-000021
(3)输出x l,x h,z l,z h
应注意,若前述目标可逆流层是基于线性函数构建的流层,则该目标可逆流层可以被称为线性可逆流层(或线性流层);若前述目标可逆流层是基于非线性函数构建的流层,则该目标可逆流层可以被称为非线性可逆流层(或非线性流层)。当该目标可逆流层为非线性可逆流层时,由于采用插值算法或二分查找算法将非线性函数划分为多段线性函数,即将非线性流层的数值可逆运算转换成线性流层的数值可逆运算问题。有利于实现目标可逆流层的多样性,进而使得可逆流模型中的各个可逆流层多样化,有利于提升可逆流模型的表征能力。
此外,本申请中的可逆流层除了可以是前述目标可逆流层之外,还可以是目标可逆流层的变形层,该目标可逆流层的变形层可以为可逆卷积层或耦合层。此时,可逆流模型除了包 括目标可逆流层之外,还包括可逆卷积层和/或耦合层。
在一种可能的实施方式中,该N个可逆流层中的某一个或多个可逆流层为可逆卷积层,该可逆卷积层包括卷积核和该目标可逆流层,该卷积核为以该待编码数据的通道数为阶数的方阵。该可逆卷积层用于对该卷积核进行LU分解(LU Factorization),得到目标矩阵,该目标矩阵为对角阵,该对角阵的对角线上的非零元素用于确定该可逆卷积层中该目标可逆流层的模型参数。
示例性的,以可逆卷积层为1x1卷积层为例进行介绍。该可逆卷积层的参数为卷积核W,该卷积核W满足W∈R c×c,其中,c表示输入该可逆卷积层的数据x的维度。此时,输出该可逆卷积层的数据z可以表示为z=Wx。对矩阵W进行LU分解,可得W=PLΛU,其中,矩阵P为置乱阵(displace matrix),矩阵L为下三角阵(lower triangular matrix,LTM),矩阵U为上三角阵(upper triangular matrix,UTM),矩阵Λ为对角阵(diagonal matrix)。此时,输出该可逆卷积层的数据z可以表示为z=PLΛUx,因此,可以分别计算数据x与矩阵U、矩阵Λ、矩阵L以及矩阵P的乘积。
首先,计算数据x与上三角阵U的乘积h。此时,设h=Ux,u ij为U的第i行第j列的元素,x i为x的第i个元素,h i为h的第i个元素,则采用如下公式可以算得h:
Figure PCTCN2022120095-appb-000022
然后,计算数据h与上对角阵Λ的乘积m。此时,设m=Λh,λ ij为Λ的第i行第j列的元素,h i为h的第i个元素,m i为m的第i个元素,则m i=λ i·h i,其中。此时,λ i用于确定模型参数(例如,前述第一参数和第二参数)。具体地,可以将对角阵Λ中主对角线中每个元素λ i用分数表示,(即将λ i表示为两个整数相除),将该分数的分子作为第一参数R,该分数的分母作为第二参数S,构建目标可逆流层。然后,将h i作为输入该可逆卷积层中的目标可逆流层的第一数据,计算输出该可逆卷积层中的目标可逆流层的第二数据m i
然后,计算数据m与下三角阵L的乘积n。此时,设n=Lm,l ij为L的第i行第j列的元素,m i为m的第i个元素,n i为n的第i个元素,则采用如下公式可以算得n:
Figure PCTCN2022120095-appb-000023
然后,计算数据n与置换阵P的乘积,得到输出该可逆卷积层的数据z。此时,设z=Pn,z根据矩阵P对x的元素重排得到。
本实施方式中,提出在可逆流模型中设置可逆卷积层,采用1x1卷积层与目标可逆流层结合。具体地,将1x1卷积层转化成连续上三角阵、对角阵、下三角阵和置乱阵的矩阵乘法运算,对四种矩阵乘法分别使用迭代计算、基于目标可逆流层的数值可逆线性流变换处理、迭代计算、元素重排。因此,能够实现数据变换的多样性,提高可逆流模型的表征能力。
在另一种可能的实施方式中,该N个可逆流层中的某一个或多个可逆流层为耦合层,该耦合层包括该目标可逆流层。该耦合层用于将输入该耦合层的数据按预设维度划分为至少两部分数据,并将该至少两部分数据中的至少一部分数据作为该目标可逆流层的输入进行处理, 得到该耦合层的输出。
示例性的,以数据处理装置将输入该耦合层的数据按照预设维度划分为两部分数据进行介绍。假设,输入该耦合层的数据为x,将数据x按维度分解成两部分x=[x a,x b],则耦合层为
Figure PCTCN2022120095-appb-000024
其中,
Figure PCTCN2022120095-appb-000025
为以x a为参数的一元单调函数(例如,一元线性函数),将数据x b带入由一元单调函数
Figure PCTCN2022120095-appb-000026
确定的模型参数构建的目标可逆流层中进行计算,得到
Figure PCTCN2022120095-appb-000027
Figure PCTCN2022120095-appb-000028
作为耦合层的其中一个维度的输出,即z b。此外,将x a直接作为耦合层的另一个维度的输出,即z a
应理解,耦合层可以包含神经网络,该神经网络用于分维度对输入该耦合层的数据进行处理。该神经网络可以包括多种类型。例如,卷积神经网络(convolutional neural network,CNN)、深度神经网络(deep neural network,DNN)、循环神经网络(recurrent neural networks,RNN)或其他神经网络等。具体此处不做限定。
本实施方式中,提出在可逆流模型中设置耦合层,该耦合层能够将输入该耦合层的数据按照维度划分,并仅对其中一部分维度的数据采用目标可逆流层的变换方式进行变换处理。因此,能够实现数据变换的多样性,提高可逆流模型的表征能力。
应理解,前述不同种类的可逆流层(即目标可逆流层、可逆卷积层和耦合层)可以进行组合堆叠,以提高可逆流模型的变换处理能力。具体地,请参阅前文图4A和图4B对应的相关介绍,此处不予赘述。
下面将结合前述模型参数对该目标可逆流层的逆向变换处理过程的原理进行介绍:
具体地,该目标可逆流层将基于该模型参数获取辅助变量,然后,基于由该辅助变量和该模型参数确定的单调连续函数对输入该目标可逆流层的第二数据进行变换处理,得到输出该目标可逆流层的第一数据。其中,单调连续函数为由前述辅助变量和模型参数确定的函数,该目标可逆流层在变换处理过程中将使用前述第一单调连续函数对输入该目标可逆流层的第二数据进行处理,以使得经过一系列处理后能够输出前述第一数据,即能够将前述第二数据恢复为前述第一数据。
其中,该模型参数包括第一参数和第二参数,该第一参数和该第二参数均为正整数,该第一参数与该第二参数不相等。该辅助变量包括第一辅助变量和第二辅助变量。其中,第一辅助变量与第一参数对应,第二辅助变量与第二参数对应。可以理解为,第一辅助变量是基于第一参数确定的,第二辅助变量是基于第二参数确定的。该第一单调连续函数可以是一元线性函数,也被称为一元一次函数(linear function of one variable),即形如y=kx+b(k≠0)的函数。
进一步地,如图9所示,该数据处理装置将通过目标可逆流层在逆向变换处理过程中执行如下步骤:
步骤901,获取第二辅助变量,该第二辅助变量是基于第二参数确定的,该第二辅助变量和第二参数用于确定第二一元线性函数。
其中,该第二辅助变量是在正向变换过程中生成的辅助变量。该第二辅助变量是基于第 二参数确定的,该第二辅助变量和第二参数用于确定第二一元线性函数。其中,该第二参数用于确定该第二一元线性函数的斜率,该第二辅助变量用于确定该第二一元线性函数的截距。
在一种可选的实施方式中,若在正向变换过程中,数据处理装置采用辅助数据编解码辅助变量,并存储了编解码辅助变量所使用的辅助数据(例如,第三辅助数据),则在逆向变换过程中,该数据处理装置将先获取第三辅助数据,再基于该第三辅助数据解码第二辅助变量。
具体地,辅助数据解码辅助变量的计算原理可以参阅前文步骤701中的相关介绍,具体此处不予赘述。
具体地,当数据处理装置基于第二参数确定的概率分布和第三辅助数据获取前述第二辅助变量时,该数据处理装置将基于该第二参数确定第二均匀分布,该第二均匀分布的端点为0和该第二参数。然后,采用该第二均匀分布解码该第三辅助数据,得到该第二辅助变量和该第二辅助数据,该第二辅助变量的取值位于0到该第二参数之间。其中,第二辅助数据是第三辅助数据经解码而获得的数据。
其中,该采用该第二均匀分布解码该第三辅助数据,得到该第二辅助变量和该第二辅助数据的过程具体可以为:数据处理装置以该第三辅助数据为除数且该第二参数为被除数,进行除法取余运算,得到该第二辅助变量,并且,以该第三辅助数据为除数且该第二参数为被除数,进行除法取整运算,得到该第二辅助数据。可选的,该数据处理装置可以暂存前述第二辅助数据,比便于在后续变换处理过程中使用前述第二辅助数据。例如,使用该第二辅助数据在后续处理过程中编码该第一辅助变量。
本实施方式中,提出采用基于第二参数确定的均匀分布的熵编码算法编解码辅助变量,有利于提高存储辅助变量的效率。相比于传统技术中的动态熵编码技术rANS而言,有利于提升目标可逆流层的吞吐率。
在另一种可选的实施方式中,若在正向变换过程中,数据处理装置直接存储的第二辅助变量,没有使用辅助数据,则该数据处理装置将直接从存储第二辅助变量的存储位置获取前述第二辅助变量。
步骤902,将该第二数据带入该第二一元线性函数进行计算,得到中间数据。
具体地,数据处理装置将采用该第二参数与第二数据进行乘法运算而得的乘积,与该第二辅助变量进行加法运算,得到中间数据。
在本步骤中,即在逆向变换处理中,该第二辅助变量为该乘法运算的乘积的增量。具体地,数据处理装置将该第二数据作为该第二参数的乘数进行乘法运算,得到第二乘积数据;然后,将该第二辅助变量作用该第二乘积数据的增量进行加法运算,得到中间数据。
本实施例中,由于,第二辅助变量参与了确定中间数据的过程,即第二辅助变量作为该乘法运算的乘积的增量,基于该中间数据进行除法取整运算的整数商才为第一数据,因此,避免了仅用乘法运算而造成的运算不可逆的问题,也避免了仅用加法运算和减法运算而造成的模型表征能力差的问题。
步骤903,基于该中间数据和该第一参数确定该第一数据,并输出该第一数据。
具体地,数据处理装置将以该中间数据为除数且该第一参数为被除数,进行除法取整运算,得到该第一数据,并输出该第一数据。该第一数据为经目标可逆流层的逆向变换处理后,输出该目标可逆流层的数据。
步骤904,基于该中间数据和第一参数确定第一辅助变量。
具体地,数据处理装置将以该中间数据为除数且该第一参数为被除数进行除法取余运算,得到该第一辅助变量。其中,该第一参数和该第一辅助变量用于确定第一一元线性函数,该第一一元线性函数为下一次正向变换处理(即本次逆向变换处理对应的下一次变换处理)所使用的函数,该第一参数用于确定第一一元线性函数的斜率,该第一辅助变量用于确定该第一一元线性函数的截距。
在本步骤中,即在逆向变换处理中,该第一辅助变量为该除法运算产生的余数。
应注意,步骤903和步骤904之间没有明确的时间先后顺序的限定,该数据处理装置可以先执行步骤903再执行步骤904,该数据处理装置也可以先执行步骤904再执行步骤903,该数据处理装置还可以分别同时执行步骤903和步骤904,具体此处不做限定。
步骤905,存储第一辅助变量。
在一种可选的实施方式中,该数据处理装置可以直接存储第一辅助变量。具体地,该数据处理装置可以在步骤904中生成第一辅助变量后便存储第一辅助变量。可选的,该数据处理装置可以将第一辅助变量和原本存储的第二辅助变量对应存储,以便于在后续变换处理过程中,该数据处理装置可以获取到与第二辅助变量对应的第一辅助变量,进而有利于提高后续变换处理过程的效率。
在另一种可选的实施方式中,若数据处理装置采用辅助数据编解码辅助变量,此时,数据处理装置可以存储编码了辅助变量的辅助数据,而不用直接存储辅助变量。具体地,该数据处理装置可以采用步骤901中产生的第二辅助数据编码第一辅助变量而获得第一辅助数据,并存储该第一辅助数据。
具体地,该数据处理装置可以基于前文图7对应实施例中步骤701所介绍的均匀分布熵编码原理编码第一辅助变量。首先,数据处理装置可以获取在步骤901中产生的第二辅助数据,与此同时,基于该第一参数确定第一均匀分布,该第一均匀分布的端点为0和该第一参数。然后,该数据处理装置采用该第一均匀分布和该第二辅助数据编码该第一辅助变量,得到第一辅助数据,该第一辅助变量的取值位于0到该第一参数之间。然后,该数据处理装置存储该第一辅助数据。其中,该采用该第一均匀分布和该第二辅助数据编码该第一辅助变量,得到第一辅助数据的过程具体可以为:数据处理装置将该第二辅助数据与该第一参数进行乘法运算而获得的乘积,与该第一辅助变量进行加法运算,得到该第一辅助数据。
本实施例中,由于,该第一辅助数据也是在正向变换过程中输入该目标可逆流层的数据,在逆向变换过程中该目标可逆流层能够再次输出该第一辅助数据,即该目标可逆流层能够恢复在压缩编码过程(即正向变换过程)中使用的码流,以使得使用的码流实现可逆。
如图10A所示,为目标可逆流层在逆向变换处理过程的具体计算流程:
步骤1001,获取第三辅助数据,该第三辅助数据用于解码第二辅助变量。
应理解,该第三辅助数据为正向变换处理过程中,该数据处理装置存储的辅助数据。
步骤1002,采用第二均匀分布解码第三辅助数据,得到第二辅助变量和第二辅助数据。
其中,第二均匀分布是基于第二参数确定,具体请参阅前述图9对应实施例中的相关介绍。
步骤1003,将第二数据与第二参数进行乘法运算而获得的第二乘积数据,加上第二辅助变量,得到中间数据。
步骤1004,将中间数据作为除数且该第一参数作为被除数进行除法取整运算,得到该第一数据。
步骤1005,以中间数据为除数且该第一参数为被除数进行除法取余运算,得到第一辅助变量。
应注意,步骤1004和步骤1005之间没有明确的时间先后顺序的限定,该数据处理装置可以先执行步骤1004再执行步骤1005,该数据处理装置也可以先执行步骤1005再执行步骤1004,该数据处理装置还可以分别同时执行步骤1004和步骤1005,具体此处不做限定。
步骤1006,采用第二辅助数据编码第一辅助变量,得到第一辅助数据。
步骤1007,存储第一辅助数据。
为便于理解,下面将结合具体示例对目标可逆流层的变换过程(步骤901到步骤905)进行介绍。如图10B所示,为可逆流模型中的某一个目标可逆流层(例如,可逆流层i)。其中,z表示输入目标可逆流层的数据(即第二数据),是上一个可逆流层(例如,可逆流层i+1)的输出;x表示输出目标可逆流层的数据(即第一数据),是下一个可逆流层(例如,可逆流层i-1)的输入;b1表示第一辅助数据,b2表示第二辅助数据以及b3表示第三辅助数据。假设第一参数为R,第二参数为S,则基于第一参数R确定的第一均匀分布为U(0,R),基于第二参数S确定的第二均匀分布为U(0,S)。当第二数据z输入目标可逆流层之后,该数据处理装置将获取存储的第三辅助数据b3,并基于第二均匀分布U(0,S)解码第三辅助数据b3,得到第二辅助变量r e和第二辅助数据b2。然后,该数据处理装置采用基于第二参数S和第二辅助变量r e确定第一一元线性函数(即y=S·z+r e)计算中间数据y。然后,采用中间数据y和第一参数R进行除法取整运算(即计算
Figure PCTCN2022120095-appb-000029
),得到第一数据x。此外,该数据处理装置还将基于中间数据y和第一参数R进行除法取余运算(即计算r d=ymodR(即r d=mod(y,R))),得到第一辅助变量r d。然后,该数据处理装置基于第一均匀分布U(0,R)和第二辅助数据b2编码第一辅助变量r d,得到编码后的第一辅助数据b1。该数据处理装置可以存储前述第一辅助数据b1,已恢复存储装置(例如,数据处理装置中的存储器)中的码流。
在一种可能的实施方式中,若输入该可逆流模型的待编码数据为具有第一预设精度的定点数数据时,在进行前述图10B所示的变换处理过程之前,该第i个可逆流层(即可逆流层i)还用于在将基于该具有第一预设精度的定点数数据处理为整数数据;以及,在进行前述图10B所示的变换处理过程之后,将该经该变换处理后的整数数据处理为具有该第一预设精度的定点数数据以作为该第i个可逆流层(即可逆流层i)的输出。
本实施方式中,由于目标可逆流层可以基于模型参数(例如,第一参数和第二参数)和辅助数据(例如,第一辅助数据、第二辅助数据以及第三辅助数据)编解码辅助变量(例如,第一辅助变量和第二辅助变量),该辅助变量可以实现进行乘法运算的数据通过除法运算实现可逆。由于,该目标可逆流层不仅涉及加法运算,还涉及乘法运算、除法运算、取整运算和取余运算,因此,可以提升变换处理的多样性,提升模型的表征能力。
应理解,采用本申请提供的数据编码方法和数据解码方法进行无损压缩,相比于传统技 术中的无损压缩方法,能够在保证更快的压缩吞吐率前提下,取得最优的压缩率。
如下表1所示,与目前最优的基于流模型无损压缩方法局部反编码模型(local bits-back cocding,LBB)相比,采用本申请提供的数据编码方法和数据解码方法进行压缩和解压缩能够有效提升压缩效率,具体地,压缩效率可以提升5倍。
表1
Figure PCTCN2022120095-appb-000030
以待编码数据为图像数据为例,如表2所示,本申请实施例在各类图像数据集上取得了良好的压缩率,本申请实施例能够有效应用在图像无损压缩任务。在CIFAR10、ImageNet32、ImageNet等数据集均取得了最优的无损压缩率。取得良好压缩率的重要原因是使用的可逆流模型的拟合数据分布能力强。
表2
Figure PCTCN2022120095-appb-000031
此外,如下表3所示,本申请实施例中的可逆流模型具有良好的泛化性能,能够使用单一模型,完成各类、各种大小图像的压缩。用ImageNet64数据集训练可逆流模型(输入大小64x64),自然图像做无损压缩测试(自然图像切块成64x64大小,对于小于64x64的图像用合适像素值的色块填充图像到64x64大小),取得了超过3.1倍的压缩率,该压缩率远远超过现有的无损压缩方法。
表3
Figure PCTCN2022120095-appb-000032
Figure PCTCN2022120095-appb-000033
由此可见,本申请提出全新的数值可逆的流模型计算方法(即可逆流模型),对一元线性/非线性流、1x1卷积层、耦合层等流层都提出了相应的数值可逆计算方法,消除流模型的数值误差。该方法简单高效,相对于传统技术中的局部反编码模型(local bits-back cocding,LBB),不需要复杂的局部编解码过程,能够大幅度提升无损压缩的效率。
如图11所示,为本申请实施例提供的一种数据处理装置110的结构示意图。该数据处理装置110可以是服务器、终端设备或其他有压缩存储需求的设备,具体此处不做限定。前述图3、图5、图7、图8A、图9以及图10A对应的实施例中的步骤可以由该数据处理装置110执行。当该数据处理装置110作为编码端时,该数据处理装置110将执行图3、图7以及图8A对应的实施例中的步骤。当该数据处理装置110作为解码端时,该数据处理装置110将执行图5、图9以及图10A对应的实施例中的步骤。
该数据处理装置110包括至少一个处理器1101和至少一个存储器1102。前述处理器1101和前述存储器1102通过线路互联。应当理解的是,图11仅示出了一个处理器1101和一个存储器1102。
其中,该处理器1101可以是通用中央处理单元(central processing unit,CPU)、微处理器、网络处理器(network processor,NP)或特定应用集成电路(application-specific integrated circuit),或一个或多个用于控制本申请方案的程序执行的集成电路。前述处理器1101可以是一个单核(single-CPU)处理器,也可以是一个多核(multi-CPU)处理器。处理器1101可以指一个或多个装置、电路和/或用于处理数据(例如计算机程序指令)的处理核。此外,该处理器1101可以是个单独的半导体芯片,也可以跟其他电路一起集成为一个半导体芯片,例如,可以跟其他电路(如编解码电路、硬件加速电路或各种总线和接口电路)构成一个片上系统(system-on-a-chip,SoC),或者也可以作为一个特殊应用集成电路(application specific integrated circuit,ASIC)的内置处理器集成在所述ASIC当中,该集成了处理器的ASIC可以单独封装或者也可以跟其他电路封装在一起。
此外,前述存储器1102可以是只读存储器(read-only memory,ROM),也可以是可存储静态信息和指令的其他类型的静态存储设备,也可以是随机存取存储器(random access memory,RAM),也可以是可存储信息和指令的其他类型的动态存储设备,还可以是电可擦可编程只读存储器(electrically erasable programmable read-only memory,EEPROM),具体此处不做限定。该存储器1102可以是独立存在,但与前述处理器1101相连。可选的,该存储器1102也可以和前述处理器1101集成于一体。例如,集成于一个或多个芯片之内。
此外,该存储器1102还用于存储执行本申请实施例的技术方案的程序代码。前述程序代码可以由处理器1101来控制执行,被执行的各类计算机程序代码也可被视为是处理器1101的驱动程序。于是,前述处理器1101可以获取例如第一参数和第二参数等模型参数,并基于模型参数确定辅助变量(例如,第一辅助变量和第二辅助变量),基于模型参数和辅助变量对输入可逆流模型的数据进行变换处理。
以图12为例,该数据处理装置110中的处理器1101调用存储在存储器1102中的程序代码以运行可逆流模型。可选的,该数据处理装置110中的处理器1101调用存储在存储器1102中的程序代码以运行均匀熵编码器(该均匀熵编码器可以采用基于第一参数确定第一均匀分布以及基于第二参数确定的第二均匀分布)。在压缩编码过程中,该处理器1101从存储器1102中读取待编码数据(例如图12中的各类原始数据),采用可逆流模型对待编码数据进行正向变换处理以得到隐变量,然后,编码器对该隐变量进行压缩编码,以得到编码数据(即压缩数据)。该处理器1101将生成的编码数据存储于存储器1102中。在解码解压缩过程中,该处理器1101从存储器1102中读取编码数据(即压缩数据)先进行解码处理以获得隐变量,然后,采用可逆流模型对隐变量进行逆向变换处理以得到解码输出(即编码前的数据,即待编码数据)。该处理器1101将生成的待编码数据存储于存储器1102中,或直接将编码前的数据(即待编码数据)输出。
可选的,该数据处理装置110还包括通信接口1103,该通信接口1103用于与其他服务器或网络设备进行通信。数据处理装置110可以通过该通信接口1103接收来自其他设备的指令或数据。示例性的,该通信接口1103为收发器,例如,无线收发器或光收发器等。
例如,数据处理装置110可以通过该通信接口1103接收来自用户设置的第一参数和第二参数。又例如,该数据处理装置110可以通过该通信接口1103接收辅助数据,例如,第一辅助数据或第三辅助数据等。具体地,该数据处理装置110可以通过通信接口1103获取第一辅助数据,然后,该数据处理装置110中的处理器1101采用基于第一参数确定的第一均匀分布解码第一辅助数据得到第一辅助变量和第二辅助数据。此外,当该处理器1101计算出第二辅助变量之后,该处理器1101采用基于第二参数确定的第二均匀分布和前述第二辅助数据编码第二辅助变量,得到第三辅助数据。该数据处理装置110可以通过通信接口1103将第三辅助数据输出存储至外部存储介质,也可以直接将第三辅助数据存储至该数据处理装置110中的存储器1102中。若该数据处理装置110通过通信接口1103将第三辅助数据输出存储至外部存储介质,则在逆向变换处理过程中,该数据处理装置110还将通过通信接口1103从外部存储介质读取第三辅助数据。
此外,数据处理装置110还可以通过该通信接口1103向其他设备发送指令或数据。例如,当编码端和解码端不是同一设备时,该作为编码端的数据处理装置110可以通过该通信接口1103向作为解码端的其他设备发送编码数据和模型参数(例如,第一参数和第二参数),以使得解码端可以基于编码数据和模型参数进行解码处理和逆向变换处理,以获得解码输出。
本申请实施例中还提供了一种计算机可读存储介质,该计算机可读存储介质中存储有用于构建可逆流模型的程序,当前述计算机程序在计算机上运行时,可以使得计算机执行如前述图3、图5、图7、图8A、图9以及图10A所示实施例介绍的方法。
在一种可能的实现方式中,前述图11所示的数据处理装置110为芯片,该芯片也被称为数字处理芯片。该芯片包括处理单元和通信单元。其中,处理单元通过通信单元获取程序指令,该程序指令被处理单元执行,以使得该处理单元执行前述图3、图5、图7、图8A、图9以及图10A对应实施例介绍的方法步骤。具体地,该处理单元为集成了上述处理器1101或者用于实现上述处理器1101的功能的电路,该通信单元为集成了上述通信接口1103或者用于实现上述通信接口1103的功能的电路或接口。
可选的,当该芯片中集成了存储单元时,该存储单元可以是存储器等存储装置。此时,该芯片中的处理单元可以从该存储单元中调用程序代码以实现前述图3、图5、图7、图8A、图9以及图10A对应实施例所介绍的方法步骤。当该芯片未集成存储单元时,该芯片可以通过前述通信单元与外置的存储器等存储装置连接,以使得从前述外置的存储器中获取程序代码以实现前述图3、图5、图7、图8A、图9以及图10A对应实施例所介绍的方法步骤。
示例性地,前述图11所示的数据处理装置110为芯片时,该芯片的具体接口可以如图13所示。该图13为本申请实施例提供的芯片的一种结构示意图,该芯片可以表现为神经网络处理器(neural-network processing units,NPU)130,该神经网络处理器130作为协处理器挂载到主CPU(Host CPU)上,由Host CPU分配任务。神经网络处理器130的核心部分为运算电路1301。该神经网络处理器130通过控制器1302控制运算电路1301获取存储器(例如,后文介绍的权重存储器和输入存储器等)中的矩阵数据并进行乘法运算。
其中,前述运算电路1301有多种可能的实现方式。在一种可能的实现方式中,该运算电路1301内部包括多个处理单元(process engine,PE)。在另一种可能的实现方式中,运算电路1301是二维脉动阵列。在另一种可能的实现方式中,运算电路1301还可以是一维脉动阵列或者能够执行例如乘法和加法等数学运算的其它电子线路。在另一种可能的实现方式中,该运算电路1301是通用的矩阵处理器。具体本申请实施例不对前述运算电路1301的具体形态进行限定。
示例性的,假设输入存储器1304中存储有输入矩阵A,权重存储器1303中存储有权重矩阵B。运算电路1301将从权重存储器1303中取矩阵B相应的数据,并缓存在运算电路1301中每一个PE上。此外,该运算电路1301还将从输入存储器1304中获取矩阵A,然后,采用前述矩阵A和矩阵B进行矩阵运算,得到输出矩阵C。该输出矩阵C可以是部分结果或最终结果,该输出矩阵C将被保存在累加器(accumulator)1305中。
此外,该神经网络处理器130还包括统一存储器1306,用于存放输入数据以及输出数据。一般地,输入数据可以通过存储单元访问控制器(direct memory access controller,DMAC)1307搬运到统一存储器1306中。部分输入数据或输出数据可以通过存储单元访问控制器1307搬运到权重存储器1303中。例如,外部输入的权重数据(例如,权重矩阵)可以通过存储单元访问控制器1307搬运到前述权重存储器1303中。
此外,该神经网络处理器130总线接口单元(bus interface unit,BIU)1308,用于通过总线协议(advanced extensible interface,AXI)与存储单元访问控制器1307和取指存储器(instruction fetch buffer,IFB)1309交互。此外,该总线接口单元1308,还用于通过取指存储器1309从外部存储器获取指令,还用于通过存储单元访问控制器1307从外部存储器获取输入数据,例如,前述示例中的输入矩阵A的原数据或者权重矩阵B的原数据等。
可选的,该神经网络处理器130还包括向量计算单元1310。该向量计算单元1310包括多个运算处理单元。在需要的情况下,该向量计算单元1310可以对运算电路1301的输出做进一步处理。例如,向量乘、向量加、指数运算、对数运算或者大小比较等。此外,该向量计算单元1310还用于神经网络中非卷积/全连接层网络计算,如批量归一化(batch normalization)(也被称为批量标准化)、像素级求和以及对特征图进行上采样等处理。
在一种可能的实现方式中,前述向量计算单元1310能将经处理之后输出的向量存储到统一存储器1306中,以使得运算电路1301在后续计算过程中调用统一存储器1306中的向量、函数等数据。例如,向量计算单元1310可以将线性函数和/或非线性函数存于统一存储器1306中,以使得该向量计算单元可以从统一存储器1306调用线性函数和/或非线性函数对运算电路1301的输出进行进一步处理,并将处理结果存储至统一存储器1306或累加器1305中。又例如,该向量计算单元1310可以对待处理数据进行线性变换或非线性变换。在另一种可能的实现方式中,向量计算单元1310还可以用于生成归一化的值、像素级求和的值,或二者均有。在另一种可能的实现方式中,处理过的输出的向量能够用作到运算电路1301的激活输入,以使得输出的向量能够在神经网络中的后续隐含层中使用。
此外,前述控制器1302还与取指存储器1309连接,以使得该取指存储器1309能够存储控制器1302使用的指令。
应当理解的是,前述权重存储器1303、输入存储器1304、统一存储器1306以及取指存储器1309均为片上(On-Chip)存储器。此外,外部存储器私有于该神经网络处理器硬件架构。
其中,神经网络中各层的运算可以由运算电路1301或向量计算单元1310执行。
其中,上述任一处提到的处理器,可以是一个通用中央处理器、微处理器、ASIC,或一个或多个用于控制上述图3、图5、图7、图8A、图9以及图10A的方法的程序执行的集成电路,具体此处不做限定。
如图14所示,为本申请一个实施例提供的一种数据处理装置140的结构示意图。
当数据处理装置140作为编码端,该数据处理装置140运行如下模块以实现前述图3、图7以及图8A对应的方法实施例中的数据处理装置的功能。
具体地,该数据处理装置140包括:获取模块1401、变换处理模块1402以及编码模块1403。其中,变换处理模块1402包括本申请提出的可逆流模型,在编码端该可逆流模型用于对输入该可逆流模型的数据进行正向变换处理。具体地,前述各个功能模块的具体功能如下:
获取模块1401,用于获取待编码数据;
变换处理模块1402,用于采用可逆流模型对该待编码数据进行正向变换处理,得到隐变量;其中,该可逆流模型包括目标可逆流层,该目标可逆流层包括模型参数,该模型参数用于约束在该正向变换处理过程中生成的辅助变量,该目标可逆流层对应的运算包括基于该模型参数确定的乘法运算和除法运算,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数;
编码模块1403,用于对该隐变量进行编码处理,得到该待编码数据对应的编码数据。
本实施例中,由于用于对待编码数据进行正向变换处理的可逆流模型包括目标可逆流层,该目标可逆流层不仅包括基于该模型参数确定的乘法运算和除法运算,该目标可逆流层中的模型参数还能够约束在所述正向变换处理过程中生成的辅助变量,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。也就是说,本申请不仅能够使用乘法运算,还可以在进行除法运算时通过使用辅助变量实现可逆,因此,相比于传统技术中的整数流模型提高了模型表征能力,有利于提升模型概率分布的估计准确率,进而能够提升无损压缩的压缩效率。
在一种可能的实施方式中,该可逆流模型包括N个可逆流层,该N个可逆流层包括该目标可逆流层,该N为大于或等于1的整数。
该目标可逆流层用于对输入该目标可逆流层的第一数据进行正向变换处理,得到输出该目标可逆流层的第二数据,该第一数据为该乘法运算的乘数,该第二数据为该除法运算的整数商。其中,该第一数据为该待编码数据或者经过至少一个该可逆流层处理后输出的数据,该第二数据为该隐变量或者该第二数据经过至少一个该可逆流层处理后输出的数据为该隐变量。
在一种可能的实施方式中,该模型参数包括第一参数和第二参数,该第一参数和该第二参数均为正整数,该第一参数与该第二参数不相等;该辅助变量包括第一辅助变量,该第一辅助变量为该乘法运算的乘积的增量。
该变换处理模块1402,具体用于:
将该第一数据作为该第一参数的乘数进行乘法运算,得到第一乘积数据;
将该第一辅助变量作用该第一乘积数据的增量进行加法运算,得到中间数据;
将该中间数据作为除数且该第二参数作为被除数进行除法取整运算,得到该第二数据。
本实施方式中,数据处理装置140将第一数据与第一参数进行乘法运算而获得的乘积(即第一乘积数据)与第一辅助变量进行加法运算,得到中间数据;然后,该数据处理装置140将中间数据作为除数且该第二参数作为被除数进行除法运算,得到的整数商为第二数据。由此可见,在前述正向变换过程中,第一数据先基于乘法运算和加法运算变换为中间数据,再由该中间数据基于除法取整运算变换为第二数据。由于,第一辅助变量参与了确定中间数据的过程,即第一辅助变量作为该乘法运算的乘积的增量,基于该中间数据进行除法取整运算的整数商才为第二数据,因此,避免了仅用乘法运算而造成的运算不可逆的问题,也避免了仅用加法运算和减法运算而造成的模型表征能力差的问题。
在一种可能的实施方式中,该辅助变量还包括第二辅助变量,该第二辅助变量为该除法运算产生的余数。
该变换处理模块1402,还用于以该中间数据为除数且该第二参数为被除数进行除法取余运算,得到该第二辅助变量,该第二辅助变量用于作为该正向变换处理对应的逆向变换处理中的乘法运算的乘积的增量,该第一辅助变量用于作为该逆向变换处理中的除法运算产生的余数。
本实施方式中,提出该目标可逆流模型在正向变换处理中,还将生成第二辅助变量,该第二辅助变量用于作为该正向变换处理对应的逆向变换处理中的乘法运算的乘积的增量。也就是说,在逆向变换处理中,数据处理装置140将基于第二辅助变量将第二数据恢复为中间 数据,进而将中间数据恢复为第一数据。因此,在正向变换处理中,计算出第二辅助变量,有利于实现将第二数据恢复为第一数据。
在一种可能的实施方式中,该变换处理模块1402,还用于:
获取输入该目标可逆流层的第一辅助数据;
基于该第一辅助数据和该第一参数确定该第一辅助变量。
在一种可能的实施方式中,该变换处理模块1402,具体用于:
基于该第一参数确定第一均匀分布,该第一均匀分布的端点为0和该第一参数;
采用该第一均匀分布解码该第一辅助数据,得到该第一辅助变量和第二辅助数据,该第一辅助变量的取值位于0到该第一参数之间。
在一种可能的实施方式中,该变换处理模块1402包括第一熵编解码模块14021,该第一熵编解码模块14021存储有第一均匀分布,该第一熵编解码模块14021用于:
以该第一辅助数据为除数且该第一参数为被除数,进行除法取余运算,得到该第一辅助变量;
以该第一辅助数据为除数且该第一参数为被除数,进行除法取整运算,得到该第二辅助数据,该第二辅助数据用于在正向变换过程中编码该第二辅助变量,在逆向变换过程中编码该第一辅助变量。
在一种可能的实施方式中,该变换处理模块1402,还用于:
基于该第二参数确定第二均匀分布,该第二均匀分布的端点为0和该第二参数;
采用该第二均匀分布和该第二辅助数据编码该第二辅助变量,得到第三辅助数据,该第二辅助变量的取值位于0到该第二参数之间;
存储该第三辅助数据,该第三辅助数据用于在该逆向变换处理中解码该第二辅助变量。
本实施方式中,由于数据处理装置140采用第二辅助数据编码第二辅助变量,以获得编码了第二辅助变量的第三辅助数据,有利于数据处理装置140存储第三辅助数据,以便于在逆向变换处理过程中,数据处理装置140采用第三辅助数据解码第二辅助变量。由于,存储第三辅助数据占用的存储空间远小于存储第二辅助变量需要的存储空间,因此,相比于该数据处理装置140直接存储第二辅助变量的方案,有利于节省存储空间,提高存储效率。另外,由于第三辅助数据是基于第二辅助数据编码第二辅助变量确定的,而第二辅助数据又是在采用第一辅助数据解码第一辅助变量的过程获得的,因此,存储了第三辅助数据,便间接存储了第二辅助变量和第一辅助变量的对应关系(相当于将第一辅助变量和第二辅助变量对应存储)。
在一种可能的实施方式中,该变换处理模块1402,该变换处理模块1402包括第二熵编解码模块14022,该第二熵编解码模块14022存储有第二均匀分布,该第二熵编解码模块14022用于:
将该第二辅助数据与该第二参数进行乘法运算而获得的乘积,与该第二辅助变量进行加法运算,得到该第三辅助数据。
在一种可能的实施方式中,该目标可逆流层还用于:
获取第一单调连续函数的定义域中的第一区间和该第一区间在该第一单调连续函数的值域上对应的第二区间,该第一单调连续函数为非线性函数;
基于该第一区间和该第二区间采用插值法或二分查找法计算该模型参数。
在一种可能的实施方式中,当该N个可逆流层包括至少两个可逆流层时,该N个可逆流层采用串行的方式排布;
其中,第i-1个可逆流层的输出用于作为第i个可逆流层的输入,该i为大于1且不大于N的正整数,第1个可逆流层的输入为该待编码数据,该第N个可逆流层的输出为该隐变量。
在一种可能的实施方式中,该待编码数据为采用噪声数据对原始数据进行反编码处理而获得的数据,该待编码数据为具有第一预设精度的定点数数据;该第i个可逆流层还用于在该正向变换处理之前,将基于该具有第一预设精度的定点数数据处理为整数数据,以及将该经该正向变换处理后的整数数据处理为具有该第一预设精度的定点数数据以作为该第i个可逆流层的输出。
应理解,前述噪声数据可以是数据处理装置140中已压缩的数据的比特流,还可以是数据处理装置140中预存的比特流,具体此处不做限定。本实施方式中,由于采用噪声数据对原始数据进行反编码处理,能够增大数据处理装置140计算的数据的数值,有利于提升后续变换过程的处理精度,也有利于提升压缩率。
在一种可能的实施方式中,该N个可逆流层还包括可逆卷积层,该可逆卷积层包括卷积核和该目标可逆流层,该卷积核为以该待编码数据的通道数为阶数的方阵;该可逆卷积层用于对该卷积核进行LU分解,得到目标矩阵,该目标矩阵为对角阵,该对角阵的对角线上的非零元素用于确定该可逆卷积层中该目标可逆流层的模型参数。
本实施方式中,提出在可逆流模型中设置可逆卷积层,采用1x1卷积层与目标可逆流层结合。具体地,将1x1卷积层转化成连续上三角阵、对角阵、下三角阵和置乱阵的矩阵乘法运算,对四种矩阵乘法分别使用迭代计算、基于目标可逆流层的数值可逆线性流变换处理、迭代计算、元素重排。因此,能够实现数据变换的多样性,提高可逆流模型的表征能力。
在一种可能的实施方式中,该N个可逆流层还包括耦合层,该耦合层包括该目标可逆流层;该耦合层用于将输入该耦合层的数据按预设维度划分为至少两部分数据,并将该至少两部分数据中的至少一部分数据作为该目标可逆流层的输入进行处理,得到该耦合层的输出。
应理解,耦合层可以包含神经网络,该神经网络用于分维度对输入该耦合层的数据进行处理。该神经网络可以包括多种类型。例如,卷积神经网络(convolutional neural network,CNN)、深度神经网络(deep neural network,DNN)、循环神经网络(recurrent neural networks,RNN)或其他神经网络等。具体此处不做限定。
本实施方式中,提出在可逆流模型中设置耦合层,该耦合层能够将输入该耦合层的数据按照维度划分,并仅对其中一部分维度的数据采用目标可逆流层的变换方式进行变换处理。因此,能够实现数据变换的多样性,提高可逆流模型的表征能力。
如图15所示,为本申请一个实施例提供的一种数据处理装置150的结构示意图。
当数据处理装置150作为解码端,该数据处理装置150运行如下模块以实现前述图5、图9以及图10A对应的方法实施例中的数据处理装置的功能。
具体地,该数据处理装置150包括:获取模块1501、解码模块1502以及变换处理模块1503。其中,变换处理模块1502包括本申请提出的可逆流模型,在解码端该可逆流模型用于 对输入该可逆流模型的隐变量进行逆向变换处理。具体地,前述各个功能模块的具体功能如下:
获取模块1501,用于获取编码数据;
解码模块1502,用于对该编码数据进行解码处理,得到隐变量;
变换处理模块1503,用于采用可逆流模型对该隐变量进行逆向变换处理,得到解码输出;其中,该可逆流模型包括目标可逆流层,该目标可逆流层包括模型参数,该模型参数用于约束在该逆向变换处理过程中生成的辅助变量,该目标可逆流层对应的运算包括基于该模型参数确定的乘法运算和除法运算,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。
本实施例中,数据处理装置150将对解码后的隐变量进行逆向变换处理,该逆向变换处理的可逆流模型包括目标可逆流层,该目标可逆流层不仅包括基于该模型参数确定的乘法运算和除法运算,该目标可逆流层中的模型参数还能够约束在所述逆向变换处理过程中生成的辅助变量,该辅助变量为该乘法运算的乘积的增量或该除法运算产生的余数。也就是说,本申请能够针对正向变换处理过程中的乘法运算,采用辅助变量进行除法运算,以将隐变量恢复为正向变换处理前的数据(即前述数据编码方法中的待编码数据)。因此,不仅能够保证变换处理的可逆运算,还能够提高模型表征能力,有利于提升模型概率分布的估计准确率,进而能够提升无损压缩的压缩效率。
在一种可能的实施方式中,该可逆流模型包括N个可逆流层,该N个可逆流层包括该目标可逆流层,该N为大于或等于1的整数。
该目标可逆流层用于对输入该目标可逆流层的第二数据进行逆向变换处理,得到输出该目标可逆流层的第一数据,该第二数据为该乘法运算的乘数,该第一数据为该除法运算的整数商。其中,该第二数据为该隐变量或者经过至少一个该可逆流层处理后输出的数据,该第一数据为该解码输出或者该第一数据经过至少一个该可逆流层处理后的输出为该解码输出。
在一种可能的实施方式中,该模型参数包括第一参数和第二参数,该第一参数和该第二参数均为正整数,该第一参数与该第二参数不相等;该辅助变量包括第二辅助变量,该第二辅助变量为该乘法运算的乘积的增量。
该变换处理模块1503,具体用于:
将该第二数据作为该第二参数的乘数进行乘法运算,得到第二乘积数据;
将该第二辅助变量作用该第二乘积数据的增量进行加法运算,得到中间数据;
将该中间数据作为除数且该第一参数作为被除数进行除法取整运算,得到该第一数据。
本实施方式中,数据处理装置150将第二数据与第二参数进行乘法运算而获得的乘积(即第二乘积数据)与第二辅助变量进行加法运算,得到中间数据;然后,该数据处理装置150将中间数据作为除数且该第一参数作为被除数进行除法运算,得到的整数商为第一数据。由此可见,在前述正向变换过程中,第二数据先基于乘法运算和加法运算变换为中间数据,再由该中间数据基于除法取整运算变换为第一数据。由于,第二辅助变量参与了确定中间数据的过程,即第二辅助变量作为该乘法运算的乘积的增量,基于该中间数据进行除法取整运算的整数商才为第一数据,因此,避免了仅用乘法运算而造成的运算不可逆的问题,也避免了仅用加法运算和减法运算而造成的模型表征能力差的问题。
在一种可能的实施方式中,该辅助变量还包括第一辅助变量,该第一辅助变量为该除法运算产生的余数;
该变换处理模块1503,还用于:
以该中间数据为除数且该第一参数为被除数进行除法取余运算,得到该第一辅助变量。
在一种可能的实施方式中,该变换处理模块1503,具体用于:
获取输入该目标可逆流层的第三辅助数据,该第三辅助数据为在该逆向变换处理对应的正向变换处理中存储的辅助数据;
基于该第三辅助数据和该第二参数确定该第二辅助变量。
本实施例中,提出数据处理装置150采用在该逆向变换处理对应的正向变换处理中存储的第三辅助数据确定第二辅助变量,有利于从第三辅助数据中恢复出第二辅助变量,以确保在逆向变换处理过程中使用的第二辅助变量的取值与在正向变换处理过程中使用的第二辅助变量的取值是一致的。
在一种可能的实施方式中,该变换处理模块1503,具体用于:
基于该第二参数确定第二均匀分布,该第二均匀分布的端点为0和该第二参数;
采用该第二均匀分布解码该第三辅助数据,得到该第二辅助变量和该第二辅助数据,该第二辅助变量的取值位于0到该第二参数之间。
在一种可能的实施方式中,该变换处理模块1503包括第二熵编解码模块15032,该第二熵编解码模块15032存储有第二均匀分布,该第二熵编解码模块15032用于:
以该第三辅助数据为除数且该第二参数为被除数,进行除法取余运算,得到该第二辅助变量;
以该第三辅助数据为除数且该第二参数为被除数,进行除法取整运算,得到该第二辅助数据,该第二辅助数据用于在正向变换过程中编码该第二辅助变量,在逆向变换过程中编码该第一辅助变量。
在一种可能的实施方式中,该变换处理模块1503,还用于:
基于该第一参数确定第一均匀分布,该第一均匀分布的端点为0和该第一参数;
采用该第一均匀分布和该第二辅助数据编码该第一辅助变量,得到第一辅助数据,该第一辅助变量的取值位于0到该第一参数之间;
存储该第一辅助数据,该第一辅助数据为该逆向变换处理对应的正向变换处理中输入该目标可逆流层的辅助数据。
本实施方式中,由于数据处理装置150采用第二辅助数据编码第一辅助变量,以获得编码了第一辅助变量的第一辅助数据,有利于数据处理装置150存储第一辅助数据。由于,存储第一辅助数据占用的存储空间远小于存储第一辅助变量需要的存储空间,因此,相比于该数据处理装置150直接存储第一辅助变量的方案,有利于节省存储空间,提高存储效率。另外,生成并存储第一辅助数据,有利于恢复在正向变换处理过程中输入该目标可逆流层的第一辅助数据。
在一种可能的实施方式中,该变换处理模块1503,该变换处理模块1503包括第一熵编解码模块15031,该第一熵编解码模块15031存储有第一均匀分布,该第一熵编解码模块15031用于:
将该第二辅助数据与该第一参数进行乘法运算而获得的乘积,与该第一辅助变量进行加法运算,得到该第一辅助数据。
在一种可能的实施方式中,该目标可逆流层还用于:
获取第一单调连续函数的定义域中的第一区间和该第一区间在该第一单调连续函数的值域上对应的第二区间,该第一单调连续函数为非线性函数;
基于该第一区间和该第二区间采用插值法或二分查找法计算该模型参数。
在一种可能的实施方式中,当该N个可逆流层包括至少两个可逆流层时,该N个可逆流层采用串行的方式排布;
其中,第i个可逆流层的输出用于作为第i-1个可逆流层的输入,该i为大于1且不大于N的正整数,第N个可逆流层的输入为该隐变量,该第1个可逆流层的输出为该解码输出。
在一种可能的实施方式中,该第i个可逆流层还用于在该逆向变换处理之前,将基于该具有第一预设精度的定点数数据处理为整数数据,以及将该经该逆向变换处理后的整数数据处理为具有第一预设精度的定点数数据以作为该第i个可逆流层的输出。
在一种可能的实施方式中,该解码模块,还用于对该第1个可逆流层的输出进行解反编码处理,得到原始数据和噪声数据。
在一种可能的实施方式中,该N个串行的可逆流层还包括可逆卷积层,该可逆卷积层包括卷积核和该目标可逆流层,该卷积核为以该待编码数据的通道数为阶数的方阵;该可逆卷积层用于对该卷积核进行LU分解,得到目标矩阵,该目标矩阵为对角阵,该对角阵的对角线上的非零元素用于确定该可逆卷积层中该目标可逆流层的模型参数。
本实施方式中,提出在可逆流模型中设置可逆卷积层,采用1x1卷积层与目标可逆流层结合。具体地,将1x1卷积层转化成连续上三角阵、对角阵、下三角阵和置乱阵的矩阵乘法运算,对四种矩阵乘法分别使用迭代计算、基于目标可逆流层的数值可逆线性流变换处理、迭代计算、元素重排。因此,能够实现数据变换的多样性,提高可逆流模型的表征能力。
在一种可能的实施方式中,该N个串行的可逆流层还包括耦合层,该耦合层包括该目标可逆流层;该耦合层用于将输入该耦合层的数据按预设维度划分为至少两部分数据,并将该至少两部分数据中的至少一部分数据作为该目标可逆流层的输入进行处理,得到该耦合层的输出。
应理解,耦合层可以包含神经网络,该神经网络用于分维度对输入该耦合层的数据进行处理。该神经网络可以包括多种类型。例如,卷积神经网络CNN、深度神经网络DNN、循环神经网络RNN或其他神经网络等。具体此处不做限定。
本实施方式中,提出在可逆流模型中设置耦合层,该耦合层能够将输入该耦合层的数据按照维度划分,并仅对其中一部分维度的数据采用目标可逆流层的变换方式进行变换处理。因此,能够实现数据变换的多样性,提高可逆流模型的表征能力。
另外需说明的是,以上所描述的装置实施例仅仅是示意性的。其中所述作为分离部件说明的单元可以是物理上分开的,也可以不是物理上分开的。作为单元显示的部件可以是物理单元,也可以不是物理单元,即可以位于一个地方,也可以分布到多个网络单元上。在实际应用中,可以根据实际的需要选择其中的部分或者全部模块/单元来实现本实施例方案的目的。 另外,本申请提供的装置实施例的附图中,模块之间的连接关系表示它们之间具有通信连接,具体实现可以为一条或多条通信总线或信号线。
通过以上的实施方式的描述,所属领域的技术人员可以清楚地了解到本申请可借助软件加必需的通用硬件的方式来实现,当然也可以通过专用硬件包括专用集成电路、专用CPU、专用存储器以及专用元器件等来实现。一般情况下,凡由计算机程序完成的功能都可以很容易地用相应的硬件来实现,而且,用来实现同一功能的具体硬件结构也可以是多种多样的,例如模拟电路、数字电路或专用电路等。但是,对本申请而言更多情况下软件程序实现是更佳的实施方式。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在可读取的存储介质中,如计算机的软盘、U盘、移动硬盘、只读存储器(read only memory,ROM)、随机存取存储器(random access memory,RAM)、磁碟或者光盘等,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本申请各个实施例所述的方法。
在本申请实施例的描述中,除非另有说明,“/”表示或的意思,例如,A/B可以表示A或B;文本中的“和/或”仅仅是一种描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况,另外,在本申请实施例的描述中,“多个”是指两个或多于两个。
本文中“第一”、“第二”……之类的描述仅仅用来将一个对象或者操作与另一个对象或操作区分开来,而不一定要求或者暗示这些对象或操作之间存在任何这种实际的关系或者顺序。
所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的系统,装置和单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。
以上实施例仅用以说明本申请的技术方案,而非对其限制;尽管参照前述实施例对本申请进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (33)

  1. 一种数据编码方法,其特征在于,包括:
    获取待编码数据;
    采用可逆流模型对所述待编码数据进行正向变换处理,得到隐变量;其中,所述可逆流模型包括目标可逆流层,所述目标可逆流层包括模型参数,所述模型参数用于约束在所述正向变换处理过程中生成的辅助变量,所述目标可逆流层对应的运算包括基于所述模型参数确定的乘法运算和除法运算,所述辅助变量为所述乘法运算的乘积的增量或所述除法运算产生的余数;
    对所述隐变量进行编码处理,得到所述待编码数据对应的编码数据。
  2. 根据权利要求1所述的方法,其特征在于,所述可逆流模型包括N个可逆流层,所述N个可逆流层包括所述目标可逆流层,所述N为大于或等于1的整数;
    所述目标可逆流层用于对输入所述目标可逆流层的第一数据进行正向变换处理,得到输出所述目标可逆流层的第二数据,所述第一数据为所述乘法运算的乘数,所述第二数据为所述除法运算的整数商;
    其中,所述第一数据为所述待编码数据或者经过至少一个所述可逆流层处理后输出的数据,所述第二数据为所述隐变量或者所述第二数据经过至少一个所述可逆流层处理后输出的数据为所述隐变量。
  3. 根据权利要求2所述的方法,其特征在于,所述模型参数包括第一参数和第二参数,所述第一参数和所述第二参数均为正整数,所述第一参数与所述第二参数不相等;所述辅助变量包括第一辅助变量,所述第一辅助变量为所述乘法运算的乘积的增量;
    所述对输入所述目标可逆流层的第一数据进行正向变换处理,得到输出所述目标可逆流层的第二数据,包括:
    将所述第一数据作为所述第一参数的乘数进行乘法运算,得到第一乘积数据;
    将所述第一辅助变量作用所述第一乘积数据的增量进行加法运算,得到中间数据;
    将所述中间数据作为除数且所述第二参数作为被除数进行除法取整运算,得到所述第二数据。
  4. 根据权利要求3所述的方法,其特征在于,所述辅助变量还包括第二辅助变量,所述第二辅助变量为所述除法运算产生的余数;
    所述方法还包括:
    以所述中间数据为除数且所述第二参数为被除数进行除法取余运算,得到所述第二辅助变量,所述第二辅助变量用于作为所述正向变换处理对应的逆向变换处理中的乘法运算的乘积的增量,所述第一辅助变量用于作为所述逆向变换处理中的除法运算产生的余数。
  5. 根据权利要求3或4所述的方法,其特征在于,所述方法还包括:
    获取输入所述目标可逆流层的第一辅助数据;
    基于所述第一辅助数据和所述第一参数确定所述第一辅助变量。
  6. 根据权利要求5所述的方法,其特征在于,所述基于所述第一辅助数据和所述第一参数确定所述第一辅助变量,包括:
    基于所述第一参数确定第一均匀分布,所述第一均匀分布的端点为0和所述第一参数;
    采用所述第一均匀分布解码所述第一辅助数据,得到所述第一辅助变量和第二辅助数据,所述第一辅助变量的取值位于0到所述第一参数之间。
  7. 根据权利要求6所述的方法,其特征在于,所述采用所述第一均匀分布解码所述第一辅助数据,得到所述第一辅助变量和第二辅助数据,包括:
    以所述第一辅助数据为除数且所述第一参数为被除数,进行除法取余运算,得到所述第一辅助变量;
    以所述第一辅助数据为除数且所述第一参数为被除数,进行除法取整运算,得到所述第二辅助数据,所述第二辅助数据用于在正向变换过程中编码所述第二辅助变量,在逆向变换过程中编码所述第一辅助变量。
  8. 根据权利要求6或7所述的方法,其特征在于,所述方法还包括:
    基于所述第二参数确定第二均匀分布,所述第二均匀分布的端点为0和所述第二参数;
    采用所述第二均匀分布和所述第二辅助数据编码所述第二辅助变量,得到第三辅助数据,所述第二辅助变量的取值位于0到所述第二参数之间;
    存储所述第三辅助数据,所述第三辅助数据用于在所述逆向变换处理中解码所述第二辅助变量。
  9. 根据权利要求8所述的方法,其特征在于,所述采用所述第二均匀分布和所述第二辅助数据编码所述第二辅助变量,得到第三辅助数据,包括:
    将所述第二辅助数据与所述第二参数进行乘法运算而获得的乘积,与所述第二辅助变量进行加法运算,得到所述第三辅助数据。
  10. 根据权利要求2至9中任意一项所述的方法,其特征在于,所述目标可逆流层还用于:
    获取第一单调连续函数的定义域中的第一区间和所述第一区间在所述第一单调连续函数的值域上对应的第二区间,所述第一单调连续函数为非线性函数;
    基于所述第一区间和所述第二区间采用插值法或二分查找法计算所述模型参数。
  11. 根据权利要求2至10中任意一项所述的方法,其特征在于,当所述N个可逆流层包括至少两个可逆流层时,所述N个可逆流层采用串行的方式排布;
    其中,第i-1个可逆流层的输出用于作为第i个可逆流层的输入,所述i为大于1且不大于N的正整数,第1个可逆流层的输入为所述待编码数据,所述第N个可逆流层的输出为所述隐变量。
  12. 根据权利要求11所述的方法,其特征在于,所述待编码数据为采用噪声数据对原始数据进行反编码处理而获得的数据,所述待编码数据为具有第一预设精度的定点数数据;
    所述第i个可逆流层还用于在所述正向变换处理之前,将基于所述具有第一预设精度的定点数数据处理为整数数据,以及将所述经所述正向变换处理后的整数数据处理为具有所述第一预设精度的定点数数据以作为所述第i个可逆流层的输出。
  13. 根据权利要求2至12中任意一项所述的方法,其特征在于,所述N个可逆流层还包括可逆卷积层,所述可逆卷积层包括卷积核和所述目标可逆流层,所述卷积核为以所述待编码数据的通道数为阶数的方阵;
    所述可逆卷积层用于对所述卷积核进行LU分解,得到目标矩阵,所述目标矩阵为对角阵,所述对角阵的对角线上的非零元素用于确定所述可逆卷积层中所述目标可逆流层的模型 参数。
  14. 根据权利要求2至12中任意一项所述的方法,其特征在于,所述N个可逆流层还包括耦合层,所述耦合层包括所述目标可逆流层;
    所述耦合层用于将输入所述耦合层的数据按预设维度划分为至少两部分数据,并将所述至少两部分数据中的至少一部分数据作为所述目标可逆流层的输入进行处理,得到所述耦合层的输出。
  15. 一种数据解码方法,其特征在于,包括:
    获取编码数据;
    对所述编码数据进行解码处理,得到隐变量;
    采用可逆流模型对所述隐变量进行逆向变换处理,得到解码输出;其中,所述可逆流模型包括目标可逆流层,所述目标可逆流层包括模型参数,所述模型参数用于约束在所述逆向变换处理过程中生成的辅助变量,所述目标可逆流层对应的运算包括基于所述模型参数确定的乘法运算和除法运算,所述辅助变量为所述乘法运算的乘积的增量或所述除法运算产生的余数。
  16. 根据权利要求15所述的方法,其特征在于,所述可逆流模型包括N个可逆流层,所述N个可逆流层包括所述目标可逆流层,所述N为大于或等于1的整数;
    所述目标可逆流层用于对输入所述目标可逆流层的第二数据进行逆向变换处理,得到输出所述目标可逆流层的第一数据,所述第二数据为所述乘法运算的乘数,所述第一数据为所述除法运算的整数商;
    其中,所述第二数据为所述隐变量或者经过至少一个所述可逆流层处理后输出的数据,所述第一数据为所述解码输出或者所述第一数据经过至少一个所述可逆流层处理后的输出为所述解码输出。
  17. 根据权利要求16所述的方法,其特征在于,所述模型参数包括第一参数和第二参数,所述第一参数和所述第二参数均为正整数,所述第一参数与所述第二参数不相等;所述辅助变量包括第二辅助变量,所述第二辅助变量为所述乘法运算的乘积的增量;
    所述对输入所述目标可逆流层的第二数据进行逆向变换处理,得到输出所述目标可逆流层的第一数据,包括:
    将所述第二数据作为所述第二参数的乘数进行乘法运算,得到第二乘积数据;
    将所述第二辅助变量作用所述第二乘积数据的增量进行加法运算,得到中间数据;
    将所述中间数据作为除数且所述第一参数作为被除数进行除法取整运算,得到所述第一数据。
  18. 根据权利要求17所述的方法,其特征在于,所述辅助变量还包括第一辅助变量,所述第一辅助变量为所述除法运算产生的余数;
    所述方法还包括:
    以所述中间数据为除数且所述第一参数为被除数进行除法取余运算,得到所述第一辅助变量。
  19. 根据权利要求17或18所述的方法,其特征在于,所述方法还包括:
    获取输入所述目标可逆流层的第三辅助数据,所述第三辅助数据为在所述逆向变换处理 对应的正向变换处理中存储的辅助数据;
    基于所述第三辅助数据和所述第二参数确定所述第二辅助变量。
  20. 根据权利要求19所述的方法,其特征在于,所述基于所述第三辅助数据和所述第二参数确定所述第二辅助变量,包括:
    基于所述第二参数确定第二均匀分布,所述第二均匀分布的端点为0和所述第二参数;
    采用所述第二均匀分布解码所述第三辅助数据,得到所述第二辅助变量和所述第二辅助数据,所述第二辅助变量的取值位于0到所述第二参数之间。
  21. 根据权利要求20所述的方法,其特征在于,所述采用所述第二均匀分布解码所述第三辅助数据,得到所述第二辅助变量和所述第二辅助数据,包括:
    以所述第三辅助数据为除数且所述第二参数为被除数,进行除法取余运算,得到所述第二辅助变量;
    以所述第三辅助数据为除数且所述第二参数为被除数,进行除法取整运算,得到所述第二辅助数据,所述第二辅助数据用于在正向变换过程中编码所述第二辅助变量,在逆向变换过程中编码所述第一辅助变量。
  22. 根据权利要求20或21所述的方法,其特征在于,所述方法还包括:
    基于所述第一参数确定第一均匀分布,所述第一均匀分布的端点为0和所述第一参数;
    采用所述第一均匀分布和所述第二辅助数据编码所述第一辅助变量,得到第一辅助数据,所述第一辅助变量的取值位于0到所述第一参数之间;
    存储所述第一辅助数据,所述第一辅助数据为所述逆向变换处理对应的正向变换处理中输入所述目标可逆流层的辅助数据。
  23. 根据权利要求22所述的方法,其特征在于,所述采用所述第一均匀分布和所述第二辅助数据编码所述第一辅助变量,得到第一辅助数据,包括:
    将所述第二辅助数据与所述第一参数进行乘法运算而获得的乘积,与所述第一辅助变量进行加法运算,得到所述第一辅助数据。
  24. 根据权利要求16至23中任意一项所述的方法,其特征在于,所述目标可逆流层还用于:
    获取第一单调连续函数的定义域中的第一区间和所述第一区间在所述第一单调连续函数的值域上对应的第二区间,所述第一单调连续函数为非线性函数;
    基于所述第一区间和所述第二区间采用插值法或二分查找法计算所述模型参数。
  25. 根据权利要求16至24中任意一项所述的方法,其特征在于,当所述N个可逆流层包括至少两个可逆流层时,所述N个可逆流层采用串行的方式排布;
    其中,第i个可逆流层的输出用于作为第i-1个可逆流层的输入,所述i为大于1且不大于N的正整数,第N个可逆流层的输入为所述隐变量,所述第1个可逆流层的输出为所述解码输出。
  26. 根据权利要求25所述的方法,其特征在于,所述第i个可逆流层还用于在所述逆向变换处理之前,将基于所述具有第一预设精度的定点数数据处理为整数数据,以及将所述经所述逆向逆变换处理后的整数数据处理为具有第一预设精度的定点数数据以作为所述第i个可逆流层的输出。
  27. 根据权利要求25或26所述的方法,其特征在于,所述方法还包括:
    对所述第1个可逆流层的输出进行解反编码处理,得到原始数据和噪声数据。
  28. 根据权利要求16至27中任意一项所述的方法,其特征在于,所述N个串行的可逆流层还包括可逆卷积层,所述可逆卷积层包括卷积核和所述目标可逆流层,所述卷积核为以所述待编码数据的通道数为阶数的方阵;
    所述可逆卷积层用于对所述卷积核进行LU分解,得到目标矩阵,所述目标矩阵为对角阵,所述对角阵的对角线上的非零元素用于确定所述可逆卷积层中所述目标可逆流层的模型参数。
  29. 根据权利要求16至27中任意一项所述的方法,其特征在于,所述N个串行的可逆流层还包括耦合层,所述耦合层包括所述目标可逆流层;
    所述耦合层用于将输入所述耦合层的数据按预设维度划分为至少两部分数据,并将所述至少两部分数据中的至少一部分数据作为所述目标可逆流层的输入进行处理,得到所述耦合层的输出。
  30. 一种数据处理装置,其特征在于,包括处理器,所述处理器和存储器耦合,所述存储器存储有程序,当所述存储器存储的程序指令被所述处理器执行时使得所述数据处理装置实现权利要求1至14中任意一项所述的方法。
  31. 一种数据处理装置,其特征在于,包括处理器,所述处理器和存储器耦合,所述存储器存储有程序,当所述存储器存储的程序指令被所述处理器执行时使得所述数据处理装置实现权利要求15至29中任意一项所述的方法。
  32. 一种计算机可读存储介质,包括程序,当所述程序被处理单元所执行时,执行如权利要求1至14中任意一项所述的方法,或者,执行如权利要求15至29中任意一项所述的方法。
  33. 一种芯片,其特征在于,包括处理单元和通信接口,所述处理单元通过所述通信接口获取程序指令,当所述程序指令被所述处理单元执行时使得所述芯片实现权利要求1至14中任意一项所述的方法,或者,实现如权利要求15至29中任意一项所述的方法。
PCT/CN2022/120095 2021-09-30 2022-09-21 数据编码方法、数据解码方法以及数据处理装置 WO2023051335A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111163722.5 2021-09-30
CN202111163722.5A CN115913245A (zh) 2021-09-30 2021-09-30 数据编码方法、数据解码方法以及数据处理装置

Publications (1)

Publication Number Publication Date
WO2023051335A1 true WO2023051335A1 (zh) 2023-04-06

Family

ID=85767797

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/120095 WO2023051335A1 (zh) 2021-09-30 2022-09-21 数据编码方法、数据解码方法以及数据处理装置

Country Status (2)

Country Link
CN (1) CN115913245A (zh)
WO (1) WO2023051335A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116091873A (zh) * 2023-04-10 2023-05-09 宁德时代新能源科技股份有限公司 图像生成方法、装置、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102547295A (zh) * 2012-02-22 2012-07-04 上海大学 一种图像数据无损压缩的编码方法
US10122379B1 (en) * 2016-03-23 2018-11-06 EMC IP Holding Company LLC Content-aware compression of data with reduced number of class codes to be encoded
CN112487992A (zh) * 2020-12-02 2021-03-12 重庆邮电大学 一种基于流模型的人脸情绪图像的生成方法及设备
WO2021155832A1 (zh) * 2020-02-07 2021-08-12 华为技术有限公司 一种图像处理方法以及相关设备
US20210256358A1 (en) * 2020-02-06 2021-08-19 Royal Bank Of Canada Systems and methods for modeling continuous stochastic processes with dynamic normalizing flows
WO2022179588A1 (zh) * 2021-02-27 2022-09-01 华为技术有限公司 一种数据编码方法以及相关设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102547295A (zh) * 2012-02-22 2012-07-04 上海大学 一种图像数据无损压缩的编码方法
US10122379B1 (en) * 2016-03-23 2018-11-06 EMC IP Holding Company LLC Content-aware compression of data with reduced number of class codes to be encoded
US20210256358A1 (en) * 2020-02-06 2021-08-19 Royal Bank Of Canada Systems and methods for modeling continuous stochastic processes with dynamic normalizing flows
WO2021155832A1 (zh) * 2020-02-07 2021-08-12 华为技术有限公司 一种图像处理方法以及相关设备
CN112487992A (zh) * 2020-12-02 2021-03-12 重庆邮电大学 一种基于流模型的人脸情绪图像的生成方法及设备
WO2022179588A1 (zh) * 2021-02-27 2022-09-01 华为技术有限公司 一种数据编码方法以及相关设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116091873A (zh) * 2023-04-10 2023-05-09 宁德时代新能源科技股份有限公司 图像生成方法、装置、电子设备及存储介质
CN116091873B (zh) * 2023-04-10 2023-11-28 宁德时代新能源科技股份有限公司 图像生成方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN115913245A (zh) 2023-04-04

Similar Documents

Publication Publication Date Title
US10834415B2 (en) Devices for compression/decompression, system, chip, and electronic device
US10909728B1 (en) Learned lossy image compression codec
US20220375133A1 (en) Image processing method and related device
US11570477B2 (en) Data preprocessing and data augmentation in frequency domain
WO2022179588A1 (zh) 一种数据编码方法以及相关设备
US20220027715A1 (en) Artificial neural network processing methods and system
WO2023207836A1 (zh) 一种图像编码方法、图像解压方法以及装置
WO2022028197A1 (zh) 一种图像处理方法及其设备
EP3791331A1 (en) Efficient data encoding for deep neural network training
WO2023051335A1 (zh) 数据编码方法、数据解码方法以及数据处理装置
CN115661635A (zh) 基于Transformer融合卷积神经网络的高光谱图像重建方法
WO2022022176A1 (zh) 一种图像处理方法以及相关设备
US11569843B2 (en) Compressing device and method using parameters of quadtree method
CN114761968B (zh) 用于频域静态通道滤波的方法、系统和存储介质
WO2023174256A1 (zh) 一种数据压缩方法以及相关设备
TW202406344A (zh) 一種點雲幾何資料增強、編解碼方法、裝置、碼流、編解碼器、系統和儲存媒介
WO2021081854A1 (zh) 一种卷积运算电路和卷积运算方法
CN109034274B (zh) 提升高光谱图像分类精度的方法、装置、设备及存储介质
CN113554719B (zh) 一种图像编码方法、解码方法、存储介质及终端设备
CN114501031B (zh) 一种压缩编码、解压缩方法以及装置
Chandra Data bandwidth reduction in deep neural network SoCs using history buffer and Huffman coding
CN113949867B (zh) 一种图像处理的方法及装置
CN114154621A (zh) 一种基于fpga的卷积神经网络图像处理方法及装置
CN113949868B (zh) 一种熵编码方法及装置
CN110751274A (zh) 一种基于随机投影哈希的神经网络压缩方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22874722

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2022874722

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022874722

Country of ref document: EP

Effective date: 20240330