JP2020053820A - Quantization and encoder creation method, compressor creation method, compressor creation apparatus, and program - Google Patents

Abstract

To create an efficient compressor.SOLUTION: There is provided a quantization and encoder method for compressing data by irreversible compression, in which, in the quantization and encoding of the data, a computer is caused to execute a procedure of determining a sample space to be quantized by a predetermined function using information already decoded at the time of decoding. There is also provided a quantization and encoder method for compressing data by irreversible compression, in which, in the quantization and encoding of the data, a computer is caused to execute a procedure of determining a sample space to be quantized by a predetermined function using information already decoded at the time of decoding, and the predetermined function is a neural network, and an internal parameters of the neural network are optimized such that the compression performance in the irreversible compression is increased, thereby determining the sample space to be quantized.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a quantization and encoder creation method, a compressor creation method, a compressor creation device, and a program.

  Data compression is the task of how short a code length (ie, file size) can store data. The data compressor includes an encoder that encodes original data and converts it into compressed data, and a decoder that decodes compressed data and converts it into reconstructed data. Lossless data compression requires that the reconstructed data exactly match the original data, while irreversible data compression allows the reconstructed data to have some distortion from the original data. Thus, in the irreversible data compression, the code length can be significantly reduced. Here, the distortion is an amount indicating how much the reconstructed data has deteriorated from the original data.

  Many lossy image compressions extend the structure of a self-encoder (autoencoder) to quantize each element of an output array in the hidden layer (the output array is also called a feature map). The encoding generates a compressed representation of the image. Many irreversible image compressors, including the state-of-the-art DNN (Deep Neural Network) model, use a quantization-destination sample space when performing this quantization. A sample space prepared in advance inside the compressor is used.

International Publication No. 2012/109407 JP 2005-196040 A

Full Resolution Image Compression with Recurrent Neural Networks, arXiv: 1608.05148. End-to-end Optimized Image Compression, arXiv: 1611.01704. Real-Time Adaptive Image Compression, arXiv: 1705.05823.

  As described above, in the related art, when quantizing each element of the output array of the intermediate layer, a sample space prepared in advance (that is, a predetermined sample space) is used as a sample space to be quantized. For this reason, the sample space at the quantization destination is not always optimized, and an efficient compressor (and a decompressor corresponding to this compressor) may not be obtained. Here, an efficient compressor is a compressor in which the distortion between the data to be compressed and the data restored by the decompressor is small, and the code length of the quantized and encoded data is small. is there.

  An embodiment of the present invention has been made in view of the above points, and has as its object to create an efficient compressor.

  In order to achieve the above object, embodiments of the present invention focus on the above-described quantization part, and adaptively calculate a probability mass function used in encoding based on information of a part that has already been decoded at the time of decoding. In addition to the change, the sample space is adaptively changed up to the sample space used in the quantization (hereinafter, this is referred to as "ASAP (Adaptive Sample-space & Adaptive Probability) coding"). Then, the ASAP coding is realized as follows.

  A method for creating a quantizer and an encoder when compressing data by lossy compression, wherein in the quantization and encoding of the data, a sample space to be quantized is decoded using information already decoded at the time of decoding. The computer executes a procedure determined by a predetermined function.

  In addition, compression is made by optimizing the parameters in the ASAP coding from the viewpoint of trade-off between the distortion of the reconstructed data and the code length of the compressed data to be stored by converting the parameters in the neural network into neural networks. We propose NASAP (Neural-network-based ASAP) coding as a compressor that efficiently performs lossy compression based on the characteristics of the target data group. Then, this NASAP coding is realized as follows.

  A method for creating a quantizer and an encoder when compressing data by lossy compression, wherein in the quantization and encoding of the data, a sample space to be quantized is decoded using information already decoded at the time of decoding. The procedure of determining by a predetermined function, the computer executes, the predetermined function is a neural network, by optimizing the internal parameters of the neural network to increase the compression performance in the lossy compression, The sample space to be quantized is determined.

  An efficient compressor can be created.

It is a figure showing an example of functional composition of a converter creation device in an embodiment of the invention. FIG. 3 is a diagram illustrating an example of a detailed functional configuration of a quantization unit according to the embodiment of the present invention. FIG. 4 is a diagram illustrating an example of an outline of quantization by a quantization unit at the time of optimization. It is a flowchart which shows an example of an optimization process. It is a figure showing an example of a division method. It is a figure showing an example of an effect of the present invention. FIG. 2 is a diagram illustrating an example of a hardware configuration of a converter creation device according to an embodiment of the present invention.

  First, a specific example of ASAP coding will be described.

  In the JPEG (Joint Photographic Experts Group), a quantization table is prepared to divide an image into 8 × 8 sizes. Then, the following quantization / coding operation is performed for each color channel (channel) in each section. First, a discrete cosine transform is performed to create an 8 × 8 two-dimensional array. Then, it is quantized with a quantization width indicated in a predetermined quantization table.

  When the above-mentioned quantization and coding part of JPEG is replaced with ASAP coding, the following is obtained. In the ASAP coding, the quantization width is not determined in advance, but is determined adaptively from the already decoded part (hereinafter, this part is also referred to as “context”) at the time of decoding (decoding). Adaptively determining the quantization width is one of the methods for adaptively determining the sample space used for quantization. The function for determining the quantization width from the context may be determined heuristically, or may be determined using machine learning.

Here, ASAP coding is effective in the following points. For example, when an unquantized array is {z ₁ ,..., Z _N }, and this array is quantized and losslessly compressed, generally, the generated quantization error and the stored code length generally have a difference between them. There is a trade-off, and the code length can be reduced to allow a large quantization error. If the degree of quantization error allowable in the quantization of the n-th element z _n depends on the context (that is, the quantized array elements up to the (n−1) -th element that has already been decoded at the time of decoding), ASAP In coding, not only a probability mass function (PMF) but also a sample space is adaptively determined using context information. For this reason, in the ASAP coding, compared to the case where a common sample space is used, storage with a smaller code length can be expected with the same degree of distortion.

  Next, the NASAP coding will be described.

  A neural network is used as a function of a part that adaptively determines a sample space to be quantized in ASAP coding using context information, and parameters in the neural network are improved so that compression performance is improved (that is, distortion is small, and It is created by optimization (so that the code length of the quantized / encoded data is reduced). Hereinafter, a specific example of the NASAP coding will be described.

  In the embodiment of the present invention, a converter creation device 10 that creates a compressor that irreversibly compresses data (and a decompressor corresponding to the compressor) will be described. Here, the decompressor corresponding to the compressor is a decompressor that can decompress data compressed by the compressor. Since the compressor irreversibly compresses data, data to be compressed (hereinafter also referred to as “compression target data”) and decompressed data (hereinafter also referred to as “decompressed data”). Does not exactly match.

Hereinafter, as an example, the data to be compressed and the decompressed data are a three-dimensional array having a size of C ₀ × H ₀ × W _{0 where} the number of channels is C ₀ , the height size is H ₀ , and the width size is W _0. It is assumed that the image data is represented by However, the compression target data and the decompressed data are not limited to image data. As the data to be compressed and the decompressed data, data in any format (for example, text data, audio data, etc.) may be used.

  The converter creating apparatus 10 according to the embodiment of the present invention optimizes an analyzer (analyzer), an integrator (synthesizer), and a quantizer at the same time, thereby achieving an efficient compressor (and the compressor). ). Here, as described above, an efficient compressor is a compressor in which the distortion between the data to be compressed and the restored data is small, and the code length of the quantized and coded data is small. That is. Note that the compressor and the decompressor are examples of a converter that performs data conversion.

<Functional configuration of converter creation device 10>
First, a functional configuration of a converter creation device 10 according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of a functional configuration of a converter creation device 10 according to an embodiment of the present invention.

  As shown in FIG. 1, the converter creation device 10 according to the embodiment of the present invention includes an analysis unit 101, a quantization unit 102, an integration unit 103, and an optimization unit 104.

  The analysis unit 101 functions as an analyzer realized by a convolutional neural network (CNN). Hereinafter, this convolutional neural network is also referred to as “first CNN”.

The analysis unit 101 receives compression target data (image data represented by a three-dimensional array having a size of C ₀ × H ₀ × W ₀ ) and outputs intermediate data obtained by converting the compression target data. In the following, this intermediate data

と表す。ここで、Ｃは第１のＣＮＮの出力データのチャネル数、Ｈは第１のＣＮＮの出力データの高さのサイズ、Ｗは第１のＣＮＮの出力データの幅のサイズである。このように、中間データは、Ｃ×Ｈ×Ｗのサイズの３次元配列で表されるデータである。

It expresses. Here, C is the number of channels of the output data of the first CNN, H is the size of the height of the output data of the first CNN, and W is the size of the width of the output data of the first CNN. Thus, the intermediate data is data represented by a three-dimensional array having a size of C × H × W.

  The quantization unit 102 receives the intermediate data output by the analysis unit 101 and quantizes the intermediate data. Then, the quantization unit 102 outputs the quantized intermediate data. The detailed functional configuration of the quantization unit 102 will be described later.

  Hereinafter, quantized intermediate data (hereinafter, also referred to as “quantized intermediate data”) will be described.

と表す。このように、量子化中間データは、Ｃ×Ｈ×Ｗのサイズの３次元配列で表されるデータである。

It expresses. As described above, the quantized intermediate data is data represented by a three-dimensional array having a size of C × H × W.

  Further, the quantization unit 102 calculates the amount of information necessary to store the quantized intermediate data by arithmetic coding. This information amount is, as described later,

で表される。なお、この情報量は、量子化中間データの保存に必要なビット長又は符号長等と称されても良い。

It is represented by This information amount may be referred to as a bit length or a code length necessary for storing the quantized intermediate data.

  The integration unit 103 functions as a synthesizer realized by a convolutional neural network. Hereinafter, this convolutional neural network is also referred to as “second CNN”.

The integration unit 103 receives the quantized intermediate data and outputs restored data (image data represented by a three-dimensional array of C ₀ × H ₀ × W ₀ ) obtained by converting the quantized intermediate data.

  The optimizing unit 104 optimizes the analyzing unit 101, the quantizing unit 102, and the integrating unit 103. That is, the optimizing unit 104 performs the analysis unit 101 and the quantization unit 102 so that the distortion between the data to be compressed and the decompressed data is as small as possible and the information amount I expressed by the above equation 3 is as small as possible. And the parameters with the integration unit 103 are optimized. The parameters to be optimized include a first CNN parameter, a second CNN parameter, a third CNN parameter described later, and first to third parameters described later. is there.

  By optimizing these parameters, the converter creating apparatus 10 according to the embodiment of the present invention can create an efficient compressor and a decompressor corresponding to the compressor. Note that the compressor may be, for example, a converter including the optimized analysis unit 101 and quantization unit 102. The decompressor is, for example, a converter configured by a functional unit that performs a process reverse to that of an arithmetic coding unit 116 (described later) included in the quantization unit 102 and an integrated unit 103 after optimization. Good.

<Detailed functional configuration of quantization section 102>
Next, a detailed functional configuration of the quantization unit 102 will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of a detailed functional configuration of the quantization unit 102 according to the embodiment of the present invention.

  As shown in FIG. 2, the quantization unit 102 according to the embodiment of the present invention includes a division unit 111, a CNN calculation unit 112, an error calculation unit 113, a quantization calculation unit 114, and an information amount calculation unit 115. And an arithmetic coding unit 116.

The dividing unit 111 divides the intermediate data into K groups determined in advance. That is, the dividing unit 111 classifies each element x _i (i = 1,..., CHW) of the intermediate data into K groups. In the following, k (k = 1, ··· , K) th i-th element belonging to the group of the _"x ^{i (k)"} represents, and the same for each element _x ^{i (k)} for k (This data is also referred to as “k-th divided data”).

と表す。ここで、例えば分割データの全てが３次元配列で表すことができる場合、Ｎ^（ｋ）は、ｋ番目のグループに属する要素数であり、Ｎ^（ｋ）＝Ｃ^（ｋ）Ｈ^（ｋ）Ｗ^（ｋ）と表される。Ｃ^（ｋ）はｋ番目の分割データのチャネル数、Ｈ^（ｋ）はｋ番目の分割データの高さのサイズ、Ｗ^（ｋ）はｋ番目の分割データの幅のサイズである。

It expresses. Here, for example, when all of the divided data can be represented by a three-dimensional array, N ^(k) is the number of elements belonging to the k-th group, and N ^(k) = C ^(k) H ^(k) W ^(K) . C ^(k) is the number of channels of the k-th divided data, H ^(k) is the height size of the k-th divided data, and W ^(k) is the width size of the k-th divided data.

  Also, data obtained by quantizing the k-th divided data (hereinafter, also referred to as “k-th quantized divided data”).

と表す。

It expresses.

  The CNN calculation unit 112 is realized by a convolutional neural network. When k ≧ 2, the CNN calculation unit 112 inputs the first quantized divided data to the (k−1) -th quantized divided data, and obtains k-th predicted value data, The k-th index value data and the k-th quantization width data are calculated. Hereinafter, this convolutional neural network is also referred to as “third CNN”.

  In the following, the k-th predicted value data is

と表す。μ_ｉ ^（ｋ）は、ｘ_ｉ ^（ｋ）の予測値である。

It expresses. μ _i ^(k) is a predicted value of x _i ^(k) .

  Also, the k-th index value data

と表す。σ_ｉ ^（ｋ）は、予測値μ_ｉ ^（ｋ）が、真値ｘ_ｉ ^（ｋ）に対してどの程度近いかを表す指標値である。

It expresses. σ _i ^(k) is an index value indicating how close the predicted value μ _i ^(k) is to the true value x _i ^(k) .

  Also, the k-th quantization width data is

と表す。ｑ_ｉ ^（ｋ）は、量子化幅であり、

It expresses. q _i ^(k) is a quantization width,

を真値ｘ_ｉ ^（ｋ）にどの程度近付けるべきかを表す値（言い換えれば、真値ｘ_ｉ ^（ｋ）を量子化する際の量子化の細かさを決める値）である。

True value x _i value indicating whether to how close to ^_(k) (in other words, the true value x _{i ^(k)} value that determines fineness of quantization at the time of quantizing) the a.

  However, in the case of k = 1, the first prediction value data, the first index value data, and the first quantum value are calculated using parameters stored in the model (that is, in the compressor itself) in advance. Width data is created.

The error calculator 113 calculates k-th error data used for quantization of x _i ^(k) . k-th error data

と表す。後述するように、この誤差δ_ｉ ^（ｋ）と、量子化幅ｑ_ｉ ^（ｋ）とにより、ｘ_ｉ ^（ｋ）を量子化した際の量子化誤差が表される。

It expresses. As will be described later, the error δ _i ^(k) and the quantization width q _i ^(k) represent a quantization error when quantizing x _i ^(k) .

The quantization calculation unit 114 quantizes x _i ^(k) using the error δ _i ^(k) and the quantization width q _i ^(k) .

  The information amount calculation unit 115 calculates an information amount (k-th information amount) necessary to store the k-th quantized divided data by arithmetic coding. The k-th information amount is, as described later,

で表される。このとき、情報量計算部１１５は、ｋ番目の予測値データと、ｋ番目の指標値データとによって定義されるｋ番目の累積分布関数群

It is represented by At this time, the information amount calculation unit 115 calculates the k-th cumulative distribution function group defined by the k-th predicted value data and the k-th index value data.

を用いて、上記のｋ番目の情報量を計算する。ここで、累積分布関数Ｃｕｍ_ｉ ^（ｋ）は、例えば、予測値μ_ｉ ^（ｋ）と評価値σ_ｉ ^（ｋ）とによって決定される（例えば、μ_ｉ ^（ｋ）を平均、σ_ｉ ^（ｋ）の二乗を分散とする正規分布に対する累積分布関数等）。

Is used to calculate the k-th information amount. Here, the cumulative distribution function Cum _i ^(k) is determined by, for example, a predicted value μ _i ^(k) and an evaluation value σ _i ^(k) (for example, μ _i ^(k) is averaged, σ _i ^{(k) )} , The cumulative distribution function for a normal distribution with variance squared).

  The arithmetic coding unit 116 converts the quantized intermediate data (that is, the first quantized divided data to the K-th quantized divided data) into a discrete probability calculated from the corresponding cumulative distribution function group and quantization width data. Save by arithmetic coding according to distribution.

<Outline of quantization by quantization section 102 during optimization>
Next, an outline of quantization by the quantization unit 102 at the time of optimization will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of an outline of quantization by the quantization unit 102 at the time of optimization. When the number of divisions of the intermediate data by the division unit 111 is K, the quantization by the quantization unit 102 at the time of optimization is sequentially performed from k = 1 to k = K. FIG. 3 illustrates a case where the k-th quantization is executed with k ≧ 2.

  Here, in FIG. 3, the solid arrow indicates “backward” where the error (prediction error) is back-propagated by the error back propagation method, and the broken arrow indicates “no backward” where the error (prediction error) is not back-propagated. . Note that, as described above, in the k = first quantization, the first prediction value data, the first index value data, and the first quantization width data are stored using parameters stored in advance. The point created is different from the case where k ≧ 2.

  As shown in FIG. 3, in the quantization by the quantization unit 102, the first to the k−1th quantized divided data are input to the CNN calculating unit 112, and the k-th predicted value data and , K-th index value data and k-th quantization width data are calculated.

  Next, the k-th predicted value data and the k-th true value data (that is, the k-th divided data) are input to the error calculator 113, and the k-th error data is calculated. Next, the k-th error data, the k-th quantization width data, and the k-th true value data are input to the quantization calculator 114, and the k-th quantized divided data is calculated. The k-th quantized divided data is input to the CNN calculating unit 112 in the (k + 1) -th quantization.

Next, the k-th quantized width data and the k-th quantized divided data are input to the information amount calculator 115, and the k-th information amount I ^(k) is calculated by the k-th cumulative distribution function group. I do. Thereby, for each k = 1,..., K, the k-th information amount I ^(k) is obtained. The sum of the information amount I ^(k) with respect to k = 1,..., K is the information amount I shown in Equation 3 above. As will be described later, the analysis unit 101, the quantization unit 102, and the integration unit 103 are optimized so that the distortion between the data to be compressed and the decompressed data is as small as possible and the information amount I is as small as possible. You.

<Optimization processing>
Next, details of the optimization processing of the converter creation device 10 according to the embodiment of the present invention will be described with reference to FIG. FIG. 4 is a flowchart illustrating an example of the optimization processing.

Step S101: The analysis unit 101 inputs compression target data (image data represented by a three-dimensional array having a size of C ₀ × H ₀ × W ₀ ) and outputs intermediate data obtained by converting the compression target data. . The parameters of the first CNN that realizes the analysis unit 101 are parameters to be optimized by the optimization unit 104.

  Step S102: The dividing unit 111 of the quantization unit 102 divides the intermediate data into K groups. Here, a case where H = 16 and W = 16 and the intermediate data is divided into groups of K = 10 in the spatial direction (that is, with reference to the H × W plane) will be described with reference to FIG. FIG. 5 is a diagram illustrating an example of the dividing method.

As shown in FIG. 5 (a), for example, (H, W) = (1, 1), (1, 9), the (9,1), each element _{x i} is (9, 9) k = Classify into 1 group. In this case, N ⁽¹⁾ = C ⁽¹⁾ H ⁽¹⁾ W ⁽¹⁾ = C × 2 × 2 = 4C. The same applies to k = 2 to 4.

Also, as shown in FIG. 5B, for example, (H, W) = (1, 3), (1, 7), (1, 11), (1, 15), (5, 3), (5,7), (5,11), (5,15), (9,3), (9,7), (9,11), (9,15), (13,3), (13) , 7), (13, 11), (13, 15), each element _{x i} is classified into groups of k = 5. In this case, N ⁽⁵⁾ = C ⁽⁵⁾ H ⁽⁵⁾ W ⁽⁵⁾ = C × 4 × 4 = 16C. The same applies to k = 6 to 7. Note that x _i ^(k) (k = 1, 2, 3, 4) in FIG. 5B indicates that x _i ^(k) is one of k = 1 to 4 according to FIG. Represents that it was classified into a group.

Also, as shown in FIG. 5C, for example, (H, W) = (1, 2), (1, 4), (1, 6), (1, 8), (1, 10), (1,12), (1,14), (1,16), (3,2), (3,4), (3,6), (3,8), (3,10), (3 , 12), (3,14), (3,16), ..., (15,2), (15,4), (15,6), (15,8), (15,10), (15, 12), (15, 14) are classified into groups of k = 8 each element _{x i} is a (15, 16). In this case, N ⁽⁸⁾ = C ⁽⁸⁾ H ⁽⁸⁾ W ⁽⁸⁾ = C × 8 × 8 = 64C. The same applies to k = 9 to 10. It should be noted that x _i ^(k) (k = 1, 2,..., 7) in FIG. 5C indicates that x _i ^(k) is represented by FIGS. 5A and 5B. It indicates that the group was classified into any one of k = 1 to 7.

  Thus, the intermediate data is divided into K = 10 groups. In FIG. 5, for convenience of explanation, the intermediate data is divided in order from k = 1, but it is not always necessary to divide the intermediate data in order. The division unit 111 can simultaneously divide the intermediate data into groups of k = 1,..., K.

  Note that the division method illustrated in FIG. 5 is an example, and the division unit 111 may divide the intermediate data using various division methods other than this division method.

  The subsequent steps S103 to S106 are repeatedly executed in order from k = 1 to k = K.

  Step S103: The quantization unit 102 obtains k-th predicted value data, k-th index value data, and k-th quantization width data.

  Here, when k = 1, the quantization unit 102 divides the first prediction value data, the first index value data, and the first quantization width data using the parameters stored in advance. By creating these, these data are obtained.

  Specifically, the first parameter stored in advance

を空間方向にコピー（すなわち、Ｈ^（１）×Ｗ^（１）個の第１のパラメータを作成）することで、１番目の予測値データ

Is copied in the spatial direction (that is, H ⁽¹⁾ × W ⁽¹⁾ first parameters are created) to obtain the first predicted value data.

を作成する。これにより、１番目の予測値データが得られる。

Create Thereby, the first predicted value data is obtained.

  Similarly, a second parameter stored in advance

を空間方向にコピーすることで、１番目の指標値データ

Is copied in the spatial direction, the first index value data

を作成する。これにより、１番目の指標値データが得られる。

Create Thereby, the first index value data is obtained.

  Similarly, a third parameter stored in advance

を空間方向にコピーすることで、１番目の量子化幅データ

Is copied in the spatial direction to obtain the first quantization width data.

を作成する。これにより、１番目の量子化幅データが得られる。

Create As a result, the first quantization width data is obtained.

  As described above, when k = 1, the first prediction value data, the first index value data, and the first quantization width data are obtained using the first to third parameters. As described above, these first to third parameters have one value for each channel. Note that these first to third parameters are parameters to be optimized by the optimization unit 104.

  On the other hand, if k ≧ 2, the quantization unit 102 causes the CNN calculation unit 112 to calculate k-th predicted value data, k-th index value data, and k-th quantization width data. Obtain these data.

  Specifically, the CNN calculation unit 112 inputs the first quantized divided data to the (k-1) th quantized divided data, and inputs k-th predicted value data, k-th index value data, and k-th index value data. Calculate the data of the quantization width. Thereby, k-th predicted value data, k-th index value data, and k-th quantization width data are obtained. Note that the third CNN parameter for realizing the CNN calculation unit 112 is a parameter to be optimized by the optimization unit 104.

  Step S104: Next, the error calculation unit 113 of the quantization unit 102 calculates the k-th error data using the k-th prediction value data, the k-th quantization width data, and the k-th divided data. calculate.

Specifically, first, the error calculator 113 calculates, for each i = 1,..., N ^(k) ,

となるように

So that

を作成する。

Create

Next, the error calculator 113 calculates, for each i = 1,..., N ^(k) ,

となるように

So that

を作成する。これにより、ｋ番目の誤差データが計算される。

Create As a result, the k-th error data is calculated.

  Step S105: Next, the quantization calculation unit 114 of the quantization unit 102 uses the k-th divided data, the k-th error data, and the k-th quantization width data to perform the k-th quantization division. Calculate the data.

Specifically, the quantization unit 102 calculates, for each i = 1,..., N ^(k) ,

とする。これにより、ｋ番目の量子化分割データが計算される。このとき、δ_ｉ ^（ｋ）とｑ_ｉ ^（ｋ）との積が量子化誤差を表す。

And As a result, the k-th quantized divided data is calculated. At this time, the product of δ _i ^(k) and q _i ^(k) represents a quantization error.

  Here, the expression shown in the above Expression 23 is

と変形することができる。すなわち、

And can be transformed. That is,

は、予測値μ_ｉ ^（ｋ）と真値ｘ_ｉ ^（ｋ）との差ξ_ｉ ^（ｋ）ｑ_ｉ ^（ｋ）を、量子化幅ｑ_ｉ ^（ｋ）で量子化したものと言うことができる。また、上記の数２４により、

Is the difference between the predicted value mu _i ^(k) and the true value ^{_{x i (k) ξ i (}} k) q i (k), can be referred to those quantized by the quantization width _q ^{i (k)} . Further, according to the above Equation 24,

は、ｘ_ｉ ^（ｋ）に対して、区間［−０．５ｑ_ｉ ^（ｋ），０．５ｑ_ｉ ^（ｋ））の量子化誤差が入ったものと捉えることもできる。これにより、最適化部１０４による最適化では、量子化誤差を小さくするように（すなわち、量子化幅ｑ_ｉ ^（ｋ）を小さくするように）パラメータを最適化する力が働く。

Can be considered to include a quantization error in the interval [−0.5q _i ^(k) , 0.5q _i ^(k) ) with respect to x _i ^(k) . Accordingly, in the optimization performed by the optimization unit 104, a force for optimizing the parameters acts so as to reduce the quantization error (that is, to reduce the quantization width q _i ^(k)) .

Step S106: Next, the information amount calculation unit 115 of the quantization unit 102 calculates the k-th information amount I ^(k) necessary for storing the k-th quantized divided data by arithmetic coding.

Specifically, first, the information amount calculation unit 115 calculates the cumulative distribution function Cum _i ^(k)

で定義された関数とする。なお、累積分布関数は、一般に、確率密度関数を区間［−∞，ｘ］で積分した関数として定義される。したがって、上記の累積分布関数Ｃｕｍ_ｉ ^（ｋ）は、所定の確率密度関数（例えば、μ_ｉ ^（ｋ）を平均、σ_ｉ ^（ｋ）の二乗を分散とする正規分布に対する確率密度関数等）を上記の数２７に示す区間で積分した関数を意味する。

Function. Note that the cumulative distribution function is generally defined as a function obtained by integrating a probability density function in an interval [−∞, x]. Therefore, the cumulative distribution function Cum _i ^(k) is a predetermined probability density function (for example, a probability density function for a normal distribution having μ _i ^(k) as an average and σ _i ^(k) as a variance). It means a function integrated in the section shown in the above Expression 27.

  here,

は、ｘ_ｉ ^（ｋ）＝−１として、上記のステップＳ１０４及びステップＳ１０５と同様の方法で、ｘ_ｉ ^（ｋ）＝−１を量子化したものである。

_As ^{x i (k)} = -1, in the same manner as in steps S104 and S105 described above _is obtained by quantizing the ^{x i (k)} = -1.

  Also,

は、ｘ_ｉ ^（ｋ）＝１として、上記のステップＳ１０４及びステップＳ１０５と同様の方法で、ｘ_ｉ ^（ｋ）＝１を量子化したものである。

_As ^{x i (k)} = 1, in the same manner as in steps S104 and S105 described _above, the ^{x i (k)} = 1 is obtained by quantizing.

Note that the cumulative distribution function Cum _i ^(k) is determined by the predicted value μ _i ^(k) and the evaluation value σ _i ^(k) as described above. However, it may be determined only by the predicted value μ _i ^(k) .

At this time, the cumulative distribution function Cum _i ^(k) is an increasing function,

を満たす。

Meet.

  Next, the information amount calculation unit 115 calculates an upper bound and a lower bound required for range coding, which is one of arithmetic coding.

と設定する。

Set as

  At this time,

の出現予測確率は、

Is the appearance prediction probability of

で表される。これにより、最適化部１０４による最適化では、算術符号化による保存に必要な情報量を小さくするように（すなわち、量子化幅ｑ_ｉ ^（ｋ）を大きくするように）パラメータを最適化する力が働く。

It is represented by As a result, in the optimization by the optimizing unit 104, the power for optimizing the parameters to reduce the amount of information necessary for storage by arithmetic coding (that is, to increase the quantization width q _i ^(k)). Works.

Therefore, the information amount calculation unit 115 calculates the k-th information amount I ^(k)

により計算することができる。

Can be calculated by

By repeating the above steps S103 to S106 in order from k = 1 to k = K, each k-th quantized divided data and each information amount I ^(k) are obtained. That is, the quantized intermediate data and the information amount I = I ⁽¹⁾ +... + I ^(K) necessary for storing the quantized intermediate data are obtained. As a result, the arithmetic coding unit 116 can store the quantized intermediate data as the information amount I (more precisely, the information amount close to the information amount I) by arithmetic coding. However, at the time of optimization, the quantized intermediate data need not always be stored.

Step S107: Next, the integration unit 103 inputs the quantized intermediate data, and converts the quantized intermediate data into restored data (an image represented by a three-dimensional array having a size of C ₀ × H ₀ × W ₀ ). Output). Note that the parameters of the second CNN that realizes the integration unit 103 are the parameters to be optimized by the optimization unit 104.

  Step S108: Finally, the optimizing unit 104 sets the parameter of the first CNN, the parameter of the second CNN, the parameter of the third CNN, and the parameter of the third CNN such that the value of the predetermined loss function is minimized. The first parameter to the third parameter are optimized. Here, as the loss function, an arbitrary value having a smaller value as the code length is smaller, and a smaller value as the distortion between the restored data output from the integrating unit 103 and the original data (that is, the compression target data) is smaller. Can be used. For example, the following function can be used as the loss function.

ここで、α、β及びγは予め決められた定数（ただし、０以外の定数とする。）である。α、β及びγとしては、例えば、α＝β＝γ＝１とすれば良い。

Here, α, β, and γ are predetermined constants (provided that they are constants other than 0). α, β, and γ may be, for example, α = β = γ = 1.

  In addition, bit_length is the information amount I calculated by the information amount calculation unit 115. MSSSIM (multi-scale structural similarity) is an MSSSIM value between the data to be compressed and the decompressed data. The MSE (mean squared error) is an average of pixel values corresponding to each other in the data to be compressed and the decompressed data (that is, values of pixels at the same position (that is, values of pixels of the same height, width, and channel)). It is a square error. The gradient value of the loss function shown in Equation 35 can be calculated from the error (prediction error) backpropagated by the error backpropagation method. However, since the quantization operation is a non-differentiable operation, the gradient value of this loss function is approximated by ignoring the presence of quantization (that is, by using x instead of x ＾). It is calculated by various calculations.

Thereby, the parameters of the first CNN, the parameters of the second CNN, the parameters of the third CNN, and the first to third parameters are optimized. At this time, as described above, the power for optimizing parameters so as to reduce the quantization error (that is, to reduce the quantization width q _i ^(k)) and the information necessary for storage by arithmetic coding The power of optimizing the parameters works to reduce the quantity (that is, to increase the quantization width q _i ^(k)) . Thus, at the time of parameter optimization, two forces having a trade-off relationship act. With this trade-off, an appropriate quantization width q _i ^(k) can be learned. That is, by performing optimization so as to reduce the quantization error and the amount of information necessary for storage, an appropriate quantization width q _i ^(k) is learned, and the quantization width data is also optimized. You.

  By using the loss function shown in Equation 35 as the loss function, it is possible to reduce the time and effort required for optimizing the structure of the neural network, and to reduce the memory at the time of use. . In the case of ordinary distortion coding, when a certain code length constraint is set, a loss function specialized for the code length constraint is defined (that is, a different loss function is created for each different code length constraint). , The optimal neural network structure (for example, the number of layers, the number of channels, the number of units, etc.) differs for each loss function. Therefore, in normal distortion coding, it is necessary to optimize the neural network for each loss function. On the other hand, in the embodiment of the present invention, by using the loss function shown in Equation 35 above, the structure of the neural network is optimized so that the loss function is minimized, so that a wide range of code lengths is obtained. High performance (that is, small distortion and small code length of quantized / encoded data) under the constraint of Further, in this case, even when performing encoding and decoding using the neural network after learning, it is not necessary to read a different neural network for each constraint on the code length, and it is possible to reduce the memory required for use.

  As described above, the converter creating apparatus 10 according to the embodiment of the present invention can create an efficient compressor and a decompressor corresponding to the compressor. The optimization in step S108 may be performed, for example, after repeatedly performing steps S101 to S107 using a plurality of different samples (compression target data).

<Effect of the present invention>
The effect of the present invention will be described with reference to FIG. FIG. 6 is a diagram showing an example of the effect of the present invention. FIG. 6 shows a comparison result between the compressor obtained by the method of the present invention and the compressor obtained by the conventional method when the vertical axis is 1-MSSSIM and the horizontal axis is bpp (bits per pixel). . 1-MSSSIM indicates that the lower the value is, the smaller the distortion is, and the lower the value is, the higher the compression ratio is. Note that bpp is obtained by dividing the information amount I by H × W.

  As shown in FIG. 6, it can be seen that the compressor obtained by the method of the present invention has a smaller distortion and a higher compression ratio than the compressor obtained by the conventional method. In FIG. 6, the numbers in the graph indicating the method of the present invention are the numbers of channels of the first CNN and the second CNN.

  In addition, in the conventional method of FIG. 6, “Neural Multi-scale Image Compression” is “Neural Multi-scale Image Compression”. "Johstonton et al." Refers to a method described in the following Reference 1.

[Reference 1]
Nick Johnston, Damien Vincent, David Minnen, Michele Covell, Saurabh Singh, Troy Chinen, Sung Jin Hwang, Joel Shor, and George Toderici. Improved lossy image compression with priming and spatially adaptive bit rates for recurrent networks. 2017.
<Hardware configuration of converter creation device 10>
Lastly, a hardware configuration of the converter creation device 10 according to the embodiment of the present invention will be described with reference to FIG. FIG. 7 is a diagram illustrating an example of a hardware configuration of the converter creating device 10 according to the embodiment of the present invention.

  As shown in FIG. 7, the converter creation device 10 according to the embodiment of the present invention includes an input device 201, a display device 202, an external I / F 203, a communication I / F 204, and a random access memory (RAM) 205. , A ROM (Read Only Memory) 206, a processor 207, and an auxiliary storage device 208. These pieces of hardware are communicably connected by a bus 209.

  The input device 201 is, for example, a keyboard, a mouse, a touch panel, or the like, and is used by a user to input various operations. The display device 202 is, for example, a display or the like, and displays various processing results of the converter creation device 10.

  The external I / F 203 is an interface with an external device. The external device includes a recording medium 203a and the like. The converter creation device 10 can read and write the recording medium 203a and the like via the external I / F 203. On the recording medium 203a, one or more programs for realizing each functional unit (that is, the analyzing unit 101, the quantizing unit 102, the integrating unit 103, and the optimizing unit 104) included in the converter creating device 10 are recorded. Is also good.

  Examples of the recording medium 203a include a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

  The communication I / F 204 is an interface for connecting the converter creation device 10 to a communication network. One or more programs for realizing each functional unit of the converter creation device 10 may be obtained (downloaded) from a predetermined server device or the like via the communication I / F 204.

  The RAM 205 is a volatile semiconductor memory that temporarily stores programs and data. The ROM 206 is a nonvolatile semiconductor memory that can retain programs and data even when the power is turned off. The ROM 206 stores, for example, settings related to an OS (Operating System) and settings related to a communication network.

  The processor 207 is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like, and is an arithmetic device that reads a program or data from the ROM 206, the auxiliary storage device 208, or the like onto the RAM 205 and executes processing. Each functional unit of the converter creation device 10 is realized by, for example, a process in which one or more programs stored in the auxiliary storage device 208 cause the processor 207 to execute. The converter creation device 10 may have both the CPU and the GPU as the processor 207, or may have only one of the CPU and the GPU. Further, the converter creating device 10 may include a dedicated semiconductor chip such as an FPGA (field-programmable gate array).

  The auxiliary storage device 208 is, for example, a hard disk drive (HDD) or a solid state drive (SSD), and is a nonvolatile storage device that stores programs and data. The auxiliary storage device 208 stores, for example, an OS, various application software, one or more programs for realizing each functional unit of the converter creation device 10, and the like.

  The converter creation device 10 according to the embodiment of the present invention can realize the above-described various processes by having the hardware configuration shown in FIG. Note that the hardware configuration shown in FIG. 7 is an example, and another configuration may be used. For example, the converter creating device 10 may not have at least one of the input device 201 and the display device 202.

  The present invention is not limited to the above-described embodiments specifically disclosed, and various modifications and changes can be made without departing from the scope of the claims.

DESCRIPTION OF SYMBOLS 10 Transformer creation apparatus 101 Analysis part 102 Quantization part 103 Integration part 104 Optimization part 111 Division part 112 CNN calculation part 113 Error calculation part 114 Quantization calculation part 115 Information amount calculation part 116 Arithmetic coding part

Claims (11)

A method for creating a quantization and encoder when compressing data by lossy compression,
In the quantization and encoding of the data, a procedure for determining a sample space to be quantized by a predetermined function using information that has already been decoded at the time of decoding,
Is performed by a computer. The predetermined function is a neural network;
2. The quantization method according to claim 1, wherein the internal parameters of the neural network are optimized so that the compression performance in the lossy compression is improved, so that the quantization target sample space is determined. And an encoder creation method. An analysis procedure for creating intermediate data obtained by converting data to be compressed by an analyzer realized by a first neural network;
A quantization / encoder implemented by a second neural network determines a sample space to quantize the intermediate data, and uses the sample space to quantize / encode the intermediate data to obtain quantized data. The quantization / encoding procedure to create,
An integration procedure for creating restored data obtained by converting the quantized data by an integrator realized by a third neural network;
Parameters of the first neural network and the second neural network based on a distortion between the data to be compressed and the decompressed data and an amount of information required when the quantized data is stored by arithmetic coding. An optimization procedure for optimizing the parameters of the network and the parameters of the third neural network;
And a computer for executing the method. An analysis procedure for creating intermediate data obtained by converting data to be compressed by an analyzer realized by a first neural network;
A quantization procedure for creating quantized data obtained by quantizing the intermediate data with a quantizer,
An integration procedure for creating restored data obtained by converting the quantized data by an integrator realized by a second neural network;
Parameters of the first neural network, based on a distortion between the data to be compressed and the decompressed data and an amount of information required when the quantized data is stored by arithmetic coding, An optimization procedure for optimizing the parameters and the parameters of the second neural network;
And a computer for executing the method. The optimization procedure includes:
α, β, and γ are predetermined constants, the mean square error of the pixel values corresponding to each other in the data to be compressed and the decompressed data is MSE, the MSSSIM value between the data to be compressed and the decompressed data is MSSSIM, and the information amount. As bit_length indicating the bit length represented by

5. The method according to claim 4, wherein parameters of the first neural network, parameters of the quantizer, and parameters of the second neural network are optimized so as to minimize. How to make a compressor. The quantization procedure includes:
After dividing each element of the intermediate data into K groups, the k-th intermediate data composed of the elements belonging to the k-th (k = 1,. The k-th quantized data corresponding to the k-th intermediate data and the k-th information amount necessary for storing the k-th quantized data by arithmetic coding are sequentially created from k = 1 to k = K. By doing so, the information amount represented by the sum of the k-th information amount from k = 1 to k = K and the combination of the k-th quantized data from k = 1 to k = K are represented. The method according to claim 4, wherein the quantized data is generated. When k ≧ 2, the quantizer receives k-th to k−1th quantized data as inputs and predicts k-th predicted value data that predicts k-th intermediate data. , K-th index value data indicating the closeness between the k-th intermediate data and the k-th prediction value data, and k-th quantum indicating a quantization width when quantizing the k-th intermediate data And a third neural network for outputting the normalized width data and
The quantization procedure includes:
Using the k-th prediction value data, the k-th quantization width data, and the k-th intermediate data, creating the k-th quantization data,
Using the cumulative distribution function determined by the k-th predicted value data and the k-th index value data, the k-th quantized data, and the k-th quantized width data, 7. The method according to claim 6, wherein the information amount is created. When k = 1, the k-th prediction value data, the k-th index value data, and the k-th quantization width data are a first parameter and a second parameter stored in advance. And the third parameter, respectively,
The parameter of the quantizer includes a parameter of the third neural network, the first parameter, the second parameter, and the third parameter. 8. The method for producing a compressor according to item 7. Each element of the k-th quantized data is a difference between each element of the k-th predictive value data and each element of the k-th intermediate data by each element of the k-th quantization width data. Quantized,
The k-th information amount is a sum of appearance probabilities determined by each element of the k-th quantized data, each element of the k-th quantized data, and each element of the set of cumulative distribution functions. The method according to claim 7 or 8, wherein: Analysis means for creating intermediate data obtained by converting data to be compressed by an analyzer realized by a first neural network;
A quantization procedure for creating quantized data obtained by quantizing the intermediate data with a quantizer,
Integrating means for generating restored data obtained by converting the quantized data by an integrator realized by a second neural network;
Parameters of the first neural network, based on a distortion between the data to be compressed and the decompressed data and an amount of information required when the quantized data is stored by arithmetic coding, Optimizing means for optimizing parameters and parameters of the second neural network;
A compressor producing device, comprising: An analysis procedure for creating intermediate data obtained by converting data to be compressed by an analyzer realized by a first neural network;
A quantization procedure for creating quantized data obtained by quantizing the intermediate data with a quantizer,
An integration procedure for creating restored data obtained by converting the quantized data by an integrator realized by a second neural network;
Parameters of the first neural network, based on a distortion between the data to be compressed and the decompressed data and an amount of information required when the quantized data is stored by arithmetic coding, An optimization procedure for optimizing the parameters and the parameters of the second neural network;
Which causes a computer to execute the program.

Images