CN112101548A - Data compression method and device, data decompression method and device, and electronic device - Google Patents
Data compression method and device, data decompression method and device, and electronic device Download PDFInfo
- Publication number
- CN112101548A CN112101548A CN202011000571.7A CN202011000571A CN112101548A CN 112101548 A CN112101548 A CN 112101548A CN 202011000571 A CN202011000571 A CN 202011000571A CN 112101548 A CN112101548 A CN 112101548A
- Authority
- CN
- China
- Prior art keywords
- data
- numerical value
- compressed
- value
- values
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 79
- 238000013144 data compression Methods 0.000 title claims abstract description 44
- 230000006837 decompression Effects 0.000 title claims abstract description 37
- 238000003062 neural network model Methods 0.000 claims description 41
- 238000007906 compression Methods 0.000 claims description 18
- 230000006835 compression Effects 0.000 claims description 18
- 238000012163 sequencing technique Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000013528 artificial neural network Methods 0.000 description 16
- 238000004422 calculation algorithm Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000013500 data storage Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 238000013138 pruning Methods 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/94—Hardware or software architectures specially adapted for image or video understanding
- G06V10/955—Hardware or software architectures specially adapted for image or video understanding using specific electronic processors
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Multimedia (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Neurology (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The application provides a data compression method and device, a data decompression method and device and electronic equipment, and belongs to the technical field of data processing, wherein the data compression method comprises the following steps: acquiring data to be compressed; coding the numerical values in the data to be compressed based on the numerical value types contained in the data to be compressed and the occurrence frequency of each numerical value to obtain a coded value corresponding to each numerical value; the length of the code value corresponding to the numerical value is inversely related to the appearance frequency of the numerical value; and compressing the numerical values in the data to be compressed based on the corresponding relation between the numerical values and the encoding values. The method and the device are beneficial to reducing the difficulty of storing/operating data of the electronic equipment and the hardware cost limit.
Description
Technical Field
The application belongs to the technical field of data processing, and particularly relates to a data compression method and device, a data decompression method and device and electronic equipment.
Background
Hardware resources of the electronic device generally limit the data processing capability of the electronic device, and particularly for data needing to be subjected to a large amount of operations, such as a neural network model, hardware resources of internet-of-things embedded devices, such as mobile phones and computers, are occupied, so that the difficulty of transplanting the neural network model to the internet-of-things embedded devices with limited hardware resources for storage and operation is high, and the electronic device for operating the data is limited.
Disclosure of Invention
In order to overcome the problems in the related art at least to a certain extent, the application provides a data compression method and device, a data decompression method and device, and an electronic device, which can effectively realize lossless data compression and decompression, and are beneficial to reducing the difficulty of operating data of the electronic device and the limitation of hardware cost.
In order to achieve the purpose, the following technical scheme is adopted in the application:
in a first aspect, the present application provides a data compression method, including: acquiring data to be compressed; coding the numerical values in the data to be compressed based on the numerical value types contained in the data to be compressed and the occurrence frequency of each numerical value to obtain a coded value corresponding to each numerical value; wherein, the length of the code value corresponding to the numerical value is inversely related to the occurrence frequency of the numerical value; and compressing the numerical value in the data to be compressed based on the corresponding relation between the numerical value and the encoding value.
Further, the step of obtaining the data to be compressed includes: acquiring a trained neural network model; extracting weight parameters of the neural network model; and taking the extracted weight parameters as data to be compressed.
Further, the step of encoding the values in the data to be compressed based on the types of the values contained in the data to be compressed and the occurrence frequency of each value includes: constructing a tree structure based on the numerical value types and the occurrence frequency of each numerical value contained in the data to be compressed; for each value in the data to be compressed, encoding the value based on the position of the value in the tree structure.
Further, the step of constructing a tree structure based on the numerical value types and the occurrence frequency of each numerical value included in the data to be compressed includes: forming a binary tree set based on the numerical value types contained in the data to be compressed and the occurrence frequency of each numerical value; each binary tree corresponds to the occurrence frequency of a numerical value, each binary tree only comprises a root node represented by the occurrence frequency currently, and left and right subtrees of the root node are empty currently; repeatedly executing the following operations on the binary tree set until only one tree exists in the binary tree set; wherein the operations comprise: sequentially sequencing the binary trees in the binary tree set according to the occurrence frequency of the root node of each binary tree, respectively taking two adjacent binary trees with the minimum occurrence frequency as a left sub-tree and a right sub-tree, and constructing a new binary tree; wherein, the appearance frequency corresponding to the new binary tree is the sum of the appearance frequency corresponding to the left sub-tree and the appearance frequency corresponding to the right sub-tree; and deleting the two adjacent binary trees with the minimum occurrence frequency from the binary tree set, and adding the new binary tree into the binary tree set.
Further, the tree structure is a binary tree; the step of encoding the value based on its position in the tree structure comprises: setting the assignment of each branch in a path of the tree structure from a root node to a leaf node, wherein the assignments of all left branches in the path are first values, and the assignments of all right branches in the path are second values; and coding the numerical value based on the assignment corresponding to the branch of the path from the root node of the tree structure to the numerical value between the corresponding nodes in the tree structure.
In a second aspect, the present application provides a data decompression method, including: acquiring data to be decompressed; the data to be decompressed is represented by a coding value, and the data to be decompressed carries a corresponding relation between a numerical value and the coding value; the data to be decompressed is obtained by compressing numerical values in the data to be compressed based on the corresponding relation, and the length of the coded values corresponding to the numerical values is inversely related to the occurrence frequency of the numerical values; and decompressing the coding values in the data to be decompressed based on the corresponding relation.
Further, the numerical value is a weight parameter of the neural network model; the method further comprises the following steps: and operating the neural network model based on the weight parameters of the neural network model obtained after decompression.
In a third aspect, the present application provides a data compression apparatus comprising: the first acquisition module is used for acquiring data to be compressed; the encoding module is used for encoding the numerical values in the data to be compressed based on the numerical value types contained in the data to be compressed and the occurrence frequency of each numerical value to obtain the encoding values corresponding to each numerical value; wherein, the length of the code value corresponding to the numerical value is inversely related to the occurrence frequency of the numerical value; and the compression module is used for compressing the numerical values in the data to be compressed based on the corresponding relation between the numerical values and the coding values.
In a fourth aspect, the present application provides a data decompression apparatus, comprising: the second acquisition module is used for acquiring data to be decompressed; the data to be decompressed is represented by a coding value, and the data to be decompressed carries a corresponding relation between a numerical value and the coding value; the data to be decompressed is obtained by compressing numerical values in the data to be compressed based on the corresponding relation, and the length of the coded values corresponding to the numerical values is inversely related to the occurrence frequency of the numerical values; and the decompression module is used for decompressing the coding values in the data to be decompressed based on the corresponding relation.
In a fifth aspect, the present application provides an electronic device, comprising: a processor and a storage device; the storage means has stored thereon a computer program which, when executed by the processor, performs the method of any of the first aspects or performs the method of any of the second aspects.
The data compression method and the data compression device can encode the numerical values in the data to be compressed based on the numerical value types and the occurrence frequency of each numerical value contained in the data to be compressed, and further compress the numerical values in the data to be compressed based on the corresponding relation between the numerical values and the encoded values. Wherein the length of the code value corresponding to the numerical value is inversely related to the frequency of occurrence of the numerical value. The data compression method performs coding compression based on the corresponding relation between the numerical value and the coding value (obtained by coding based on the appearance frequency of the numerical value), on one hand, lossless compression is performed, on the other hand, the length of the coding value corresponding to the numerical value is in negative correlation with the appearance frequency of the numerical value, so that the data can be greatly compressed, and the data compression effect is comprehensively guaranteed. Through the data compression mode, the difficulty of data storage/operation of the electronic equipment and the hardware cost limit are reduced.
The data decompression method and the data decompression device can decompress the coded values in the data to be decompressed based on the corresponding relation between the numerical values and the coded values carried by the data to be decompressed; the data to be decompressed is obtained by compressing numerical values in the data to be compressed based on the corresponding relation, and the length of the coded value corresponding to the numerical value is inversely related to the occurrence frequency of the numerical value. When the electronic equipment runs data, only the lossless decompression is needed through the mode, so that the difficulty of storing/running the data of the electronic equipment and the hardware cost limit can be effectively reduced.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart illustrating a method of data compression according to an exemplary embodiment;
FIG. 2 is a schematic diagram illustrating a tree structure in accordance with an exemplary embodiment;
FIG. 3 is a flow diagram illustrating a method of data decompression according to an exemplary embodiment;
FIG. 4 is a schematic diagram illustrating information interaction between a server side and an edge side according to an example embodiment;
FIG. 5 is a block diagram illustrating the structure of a data compression apparatus according to an exemplary embodiment;
fig. 6 is a block diagram illustrating a data decompression apparatus according to an exemplary embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail below. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the examples given herein without making any creative effort, shall fall within the protection scope of the present application.
Taking a neural network model as an example of a deep neural network, the deep neural network is widely applied to the fields of artificial intelligence such as computer vision, speech recognition, natural language and the like, but the deep neural network generally needs huge calculation overhead and memory storage, which prevents the deep neural network from being popularized and used in the environment of limited hardware resources of electronic equipment. Although the depth neural network has great application potential in the field of the internet of things due to the remarkable high detection precision, the excellent performance of the depth neural network mainly depends on millions or even hundreds of millions of parameter quantities and high computing capacity of a GPU (Graphics Processing Unit), and the depth neural network is transplanted to a platform with limited hardware resources such as embedded equipment of the internet of things like a mobile phone and a tablet personal computer, and the like, and the following difficulties are mainly caused in that 1) the model is large, for example, the neural network model AlexNet is 240MB, and the VGG-16 model is 552MB, and the model occupies a larger storage space of the equipment; 2) the calculation amount is large, a large amount of storage access and dot product operation are needed when the deep neural network is operated, and the requirement on the hardware resource cost of equipment is high. Therefore, for embedded devices with limited resources, it is important to reduce the storage and computation costs of deep neural networks. Based on the above, the inventor firstly provides a data compression method and a data decompression method, the data compression method is used for reducing the storage/operation difficulty of the internet of things embedded equipment such as a mobile phone for data with large parameter quantity and large calculation quantity such as a deep neural network, and the corresponding data decompression method is executed when the internet of things embedded equipment such as the mobile phone runs the data.
Referring first to a flowchart of a data compression method provided by the present application shown in fig. 1, the method mainly includes steps S102 to S106:
step S102, obtaining data to be compressed. In practical applications, the data to be compressed may be any data, which is not limited in this embodiment. In one embodiment, the data to be compressed may be data related to the neural network model, such as weight parameters of the neural network model.
Step S104, coding the numerical values in the data to be compressed based on the numerical value types contained in the data to be compressed and the occurrence frequency of each numerical value to obtain a coded value corresponding to each numerical value; wherein the length of the code value corresponding to the numerical value is inversely related to the frequency of occurrence of the numerical value. That is, each value corresponds to an encoded value, such as a string of codes consisting of 0 and 1, the larger the frequency of occurrence of the value, the shorter the length of the corresponding encoded value, and the smaller the frequency of occurrence of the value, the longer the length of the corresponding encoded value. It can be understood that by using variable length coding, frequency statistics are performed on all data to be coded, values with high frequency are coded short, and values with low frequency are coded long.
And step S106, compressing the numerical values in the data to be compressed based on the corresponding relation between the numerical values and the encoding values. That is, each numerical value in the data to be compressed is correspondingly converted into a coding value occupying a smaller storage space, so that the data compression effect is achieved.
The data compression method performs coding compression based on the corresponding relation between the numerical value and the coding value (obtained by coding based on the appearance frequency of the numerical value), on one hand, lossless compression is performed, on the other hand, the length of the coding value corresponding to the numerical value is in negative correlation with the appearance frequency of the numerical value, so that the data can be greatly compressed, and the data compression effect is comprehensively guaranteed. Through the data compression mode, the difficulty of data storage/operation of the electronic equipment and the hardware cost limit are reduced.
In the prior art, although data compression methods such as low-rank approximation, parameter quantization, pruning and sparse constraint, binary network and the like are provided, the data compression methods are all lossy compression and easily affect the data accuracy, and particularly when the data is a neural network model, the accuracy of the operation result of the neural network is further affected.
The data compression method provided by the embodiment can be better applied to the compression processing of the neural network model, and when the data to be compressed is obtained, the trained neural network model can be obtained firstly; then extracting the weight parameters of the neural network model; and finally, taking the extracted weight parameters as data to be compressed. In practical applications, the neural network model may be, for example, an object detection model or an object recognition model, and the function of the neural network model is not limited herein. The compressed neural network model can effectively reduce the cost of storage media such as SDRAM (synchronous dynamic random-access memory) and flash memory of the Internet of things embedded equipment such as mobile phones.
In one embodiment, the step S104 (i.e., encoding the values in the data to be compressed based on the types of the values contained in the data to be compressed and the occurrence frequency of each value) can be implemented by referring to the following steps one and two:
the method comprises the following steps: and constructing a tree structure based on the numerical value types contained in the data to be compressed and the occurrence frequency of each numerical value.
For example, when a tree structure is constructed, a binary tree set may be constructed based on the kinds of numerical values included in the data to be compressed and the occurrence frequency of each numerical value; each binary tree corresponds to the occurrence frequency of a numerical value, each binary tree only comprises a root node represented by the occurrence frequency currently, and left and right subtrees of the root node are empty currently; the following operations are then repeated for the binary tree set until there is only one tree in the binary tree set.
In a specific embodiment, the operations specifically include: sequentially sequencing binary trees in the binary tree set according to the occurrence frequency of the root node of each binary tree (such as sequencing the occurrence frequency from small to large or sequencing the occurrence frequency from large to small), respectively taking two adjacent binary trees with the minimum occurrence frequency as a left sub-tree and a right sub-tree, and constructing a new binary tree; the appearance frequency corresponding to the new binary tree is the sum of the appearance frequency corresponding to the left sub-tree and the appearance frequency corresponding to the right sub-tree; and deleting the two adjacent binary trees with the minimum occurrence frequency from the binary tree set, and adding the new binary tree into the binary tree set.
Step two: for each value in the data to be compressed, the value is encoded based on its position in the tree structure.
In a simple and convenient implementation manner, the tree structure is a binary tree, and when the numerical value is encoded based on the position of the numerical value in the tree structure, the assignment of each branch in the path from the root node to the leaf node of the tree structure may be set first, and then the numerical value is encoded based on the assignment corresponding to the branch of the path from the root node of the tree structure to the node corresponding to the numerical value in the tree structure. The assignment of all the left branches in the path is a first value, and the assignment of all the right branches is a second value; such as a first value of 0 and a second value of 1; of course, the above is merely an example, and in practical applications, the first value may be 1, the second value may be 0, or other values may also be used, which is not limited herein. In encoding, assignments corresponding to each branch of the path may be encoded, such as in a direction from the root node to the leaf node.
The encoding method used by the data compression method implemented through the first step and the second step may also be referred to as a tree coding algorithm.
For convenience of understanding, the present embodiment provides a specific example of the first step and the second step, and is implemented with reference to the following steps a to e:
step a: the occurrence frequency of i numbers contained in the data to be compressed forms a set of i binary trees, wherein the initial value of each tree is a root node with only one frequency, and the left and right subtrees of each tree are empty.
For ease of understanding, a simple example is as follows: assuming that the weight file (including the weight parameters of the neural network model) of the neural network model to be compressed is composed of 120 bytes of data, for the sake of simplifying the description, it is assumed that these data have only 6 values: 0, 0x64, 0x78, 0x1D, 0x3C, 0x1E, the frequencies are 10, 11, 30, 15, 2, 53, and the corresponding relationship between the numerical value and the occurrence frequency shown in table 1 can be referred to:
TABLE 1
| Data of | 0 | 0x64 | 0x78 | 0x1D | 0x3C | 0x1E |
| Frequency of occurrence | 10 | 11 | 30 | 15 | 2 | 32 |
Step b: and (3) sorting the tree from small to large by using a quick sorting algorithm, selecting the tree with the minimum frequency of two root nodes as a left subtree and a right subtree in the tree to form a new binary tree, wherein the frequency of the root node of the new binary tree is the sum of the root node frequencies of the left subtree and the right subtree.
After the above 6 values are sorted according to the order of frequency from small to large, the sort order of the obtained values and the appearance frequency is shown in table 2:
TABLE 2
| Data of | |
0 | 0x64 | 0x1D | 0x78 | 0x1E |
| Frequency of occurrence | 2 | 10 | 11 | 15 | 30 | 32 |
As can be seen from table 2, the two trees with the smallest occurrence frequency are 0x3C and 0, which may form a new tree RC0, and the corresponding occurrence frequency is 12 ═ 2+ 10.
Step c: from which the two smaller trees are removed and to which the newly derived binary tree RC0 is added.
Deleting 0x3C and 0 from table 2, adding RC0 to table 2 and reordering the magnitude of the frequency of occurrence can be accomplished with reference to table 3 below:
TABLE 3
| Data of | 0x64 | RC0 | 0x1D | 0x78 | 0x1E |
| Frequency of occurrence | 11 | 12 | 15 | 30 | 32 |
Step d: repeating steps b and c until there is only one tree.
Similarly, the two trees with the minimum occurrence frequency, 0x64 and RC0, form a new tree RC1, and the corresponding occurrence frequency is 23-11 + 12. Then 0x64 and RC0 were deleted from O and RC1 was added to give the following Table 4:
TABLE 4
| Data of | 0x1D | RC1 | 0x78 | 0x1E |
| Frequency of occurrence | 15 | 23 | 30 | 32 |
The two trees with the lowest frequency, 0x1D and RC1, are then combined into a new one-tree RC2, with a corresponding frequency of occurrence of 38-15 + 23. Then 0x1D and RC1 were deleted from O and RC2 was added to give the following Table 5:
TABLE 5
| Data of | 0x78 | 0x1E | RC2 |
| Frequency of occurrence | 30 | 32 | 38 |
The two trees with the lowest frequency, 0x78 and 0x1E, are then formed into a new one-tree RC3, with a corresponding occurrence frequency of 62-30 + 32. Then 0x78 and 0x1E were deleted from O and RC3 was added to give the following Table 6:
TABLE 6
| Data of | RC2 | RC3 |
| Frequency of occurrence | 38 | 62 |
The two trees RC2 and RC3 with the lowest frequency are then formed into a new one RC4, and the corresponding occurrence frequency is 100-38 + 62. RC2 and RC3 were then deleted from O and RC4 was added to give the following Table 7:
TABLE 7
| Data of | RC4 |
| Frequency of occurrence | 100 |
Based on the foregoing steps, a tree structure can be constructed as shown in fig. 2.
Step e: based on the optimal tree, the left branch on the path from the root node to each leaf node is assigned with 0, the right branch is assigned with 1, and the encoding of the leaf node is formed from the root to the leaf direction, so that the corresponding relation between the numerical value and the encoding value is obtained. In practical applications, the longest and shortest code lengths may also be recorded.
As shown in fig. 2, in the tree structure, the root node is RC4, and the leaf node codes are formed in the direction from the root to the leaves, so that the code of 0 is 0111, and 4 bits in total; the code of 0x3c is 0110, and 4 bits in total; the code of 0x64 is 010, and the total number is 3 bits; the code of 0x1D is 00, for a total of 2 bits; the code of 0x1E is 11, for a total of 2 bits; 0x78 is coded as 10, for a total of 2 bits; wherein, the longest code is 4 bits, and the shortest code is 2 bits.
The foregoing is by way of example only for ease of understanding and is not intended as limiting.
It will be appreciated that the tree compression algorithm (which may also be referred to as a tree coding algorithm) has no relation to the specific values of the data to be compressed. For example, if one weight parameter is 10 bits, the total of 3 weight parameters is 30 bits, and the total of 4 byte spaces is occupied. The parameters of the neural network are represented by fixed point numbers, and between-511 and 512, assuming that the space occupied by the weight parameters to be compressed is 102400 bytes, the tree coding algorithm processes the weights to be compressed by taking bytes as units, i is 102400, and the data range of each byte is an integer of 0-255. The tree compression algorithm can also perform compression by preferably utilizing the characteristics of the weight parameters of the neural network, wherein the characteristics of the weight parameters are as follows: the parameters of the convolution kernel are normalized to a floating point number in the range-2.0 to 2.0 during the training phase and the weighted floating point number is converted to a fixed point decimal in the range-511 to 512 during the compiler conversion weighting phase. The byte data used is related to the weight parameters of the neural network model, and for ease of understanding, data of a convolution kernel of 3 x3 is assumed to be 2, 5,8,9, 14, 28. Because each convolution kernel data (weight) occupies 10 bits, the data only needs to occupy two bytes of space, and the data in the memory is: first byte data: 2 (5< <10) | (8< <20), the second byte data is: 9 (14< <10) | (28< < 20). And the required byte data can be obtained by analogy, and the frequency of the byte data is further calculated.
Corresponding to the foregoing data compression method, the present embodiment further provides a data decompression method, such as the flowchart of the data decompression method shown in fig. 3, which mainly includes steps S302 to S304:
step S302: acquiring data to be decompressed; the data to be decompressed is represented by the coding value, and the data to be decompressed carries the corresponding relation between the numerical value and the coding value; the data to be decompressed is obtained by compressing numerical values in the data to be compressed based on the corresponding relation, and the length of the coded value corresponding to the numerical value is inversely related to the occurrence frequency of the numerical value; in one embodiment, the numerical value may be a weight parameter of the neural network model.
Step S304: and decompressing the coding values in the data to be decompressed based on the corresponding relation.
When the electronic equipment runs data, only the lossless decompression is needed through the mode, so that the difficulty of storing/running the data of the electronic equipment and the hardware cost limit can be effectively reduced. The above-described data decompression process can be considered as the inverse process of the data compression process.
When the value is the weight parameter of the neural network model, the method further comprises operating the neural network model based on the weight parameter of the neural network model obtained after decompression. For example, when the neural network model is an object recognition model, the device hardware may read the neural network weight parameters compressed by the data compression method for each layer, and decompress the neural network weight parameters according to the corresponding relationship between the above values and the encoded values. Wherein the correspondence of the numerical value and the encoded value has been stored in the neural network model.
In practical applications, the device for performing the data compression method and the device for performing the data decompression method may be different devices, for example, the device for performing the data compression method is a server side, the device for performing the data decompression method is an internet of things embedded edge side (also referred to as an edge side or a decoding side) such as a mobile phone and a tablet computer, and the decoding side is used for acquiring compressed data from the server side for decoding and performing subsequent applications based on the decoded data. For example, when a neural network model is operated, decoding end hardware only decodes model data required by a current network layer and performs operator calculation by applying the decoded data, wherein the decoding process can be realized mainly by adopting hardware, and is equivalent to hard decoding, and software decoding is not required.
For convenience of understanding, in this embodiment, an information interaction schematic diagram of a server side and an edge side is given, and referring to fig. 4, the server side mainly performs step S402 and step S404, and the edge side mainly performs step S406, step S408 and step S410.
Step S402: and training the initial neural network model to obtain an object recognition network model. Of course, in practical applications, other neural network models may be used, such as an object detection model, and the like, which are not limited herein.
Step S404: and compressing the weight parameters in the object recognition network model by adopting a tree compression algorithm. The tree compression algorithm is realized by referring to the data compression method, and in practical application, the weight of each layer in the neural network model can be compressed by the method, so that the neural network model can be greatly and effectively compressed.
Step S406: and acquiring the object recognition network model compressed by the server.
Step S408: and decompressing the weight parameters in the object identification network model by adopting a tree decompression algorithm. The tree-shaped decompression algorithm is realized by referring to the data decompression method, and is the inverse process of the tree-shaped compression algorithm.
Step S410: and adopting the decompressed object recognition network model to perform object recognition.
Through the above manner, the weight parameters (also referred to as weights) of the Neural network model are greatly reduced, and can be directly calculated in the sram bank by an NPU (Neural network Processing Unit), without the need for a CPU (Central Processing Unit) to copy a large amount of data from a flash to a dynamic memory and calculate, thereby reducing the copy number from the sram bank to the memory and from the memory to the sram bank. The efficiency of calculating convolution by the NPU is far greater than that of the CPU, so that the running speed of the neural network model can be further improved.
The edge end stores the network model which is trained and compressed by the server end in the flash, and in the identification process, the NPU of the edge end does not need to completely decompress the network model data and store the network model data in the sram bank every time in the operator calculation of each layer according to the locality principle of a program, and only partial data needs to be decompressed for NPU calculation.
The data compression method provided by the embodiment can better realize data compression, shorten the calculation amount and reduce the hardware requirement on equipment, and compared with common compression algorithms such as low-rank approximation, parameter quantization, pruning, sparse constraint, binary network and the like, the method provided by the embodiment is lossless compression and can effectively guarantee data precision.
Corresponding to the foregoing data compression method, the present embodiment further provides a data compression apparatus, as shown in fig. 5, which mainly includes the following modules:
a first obtaining module 502, configured to obtain data to be compressed;
the encoding module 504 is configured to encode the numerical values in the data to be compressed based on the numerical value types included in the data to be compressed and the occurrence frequency of each numerical value, so as to obtain an encoded value corresponding to each numerical value; wherein, the length of the code value corresponding to the numerical value is inversely related to the occurrence frequency of the numerical value;
and a compressing module 506, configured to compress the numerical value in the data to be compressed based on the corresponding relationship between the numerical value and the encoded value.
The data compression device performs coding compression based on the corresponding relation between the numerical value and the coding value (obtained by coding based on the appearance frequency of the numerical value), on one hand, lossless compression is performed, on the other hand, the length of the coding value corresponding to the numerical value is in negative correlation with the appearance frequency of the numerical value, so that the data can be greatly compressed, and the data compression effect is comprehensively guaranteed. Through the data compression mode, the difficulty of data storage/operation of the electronic equipment and the hardware cost limit are reduced.
In one embodiment, the first obtaining module 502 is further configured to: acquiring a trained neural network model; extracting weight parameters of the neural network model; and taking the extracted weight parameters as data to be compressed.
In one embodiment, the encoding module 504 is further configured to: constructing a tree structure based on the numerical value types and the occurrence frequency of each numerical value contained in the data to be compressed; for each value in the data to be compressed, the value is encoded based on its position in the tree structure.
In a specific embodiment, the encoding module 504 is further configured to: forming a binary tree set based on the numerical value types contained in the data to be compressed and the occurrence frequency of each numerical value; each binary tree corresponds to the occurrence frequency of a numerical value, each binary tree only comprises a root node represented by the occurrence frequency currently, and left and right subtrees of the root node are empty currently; repeatedly executing the following operations on the binary tree set until only one tree exists in the binary tree set;
wherein the operations include: sequentially sequencing binary trees in the binary tree set according to the occurrence frequency of the root node of each binary tree, respectively taking two adjacent binary trees with the minimum occurrence frequency as a left sub-tree and a right sub-tree, and constructing a new binary tree; the appearance frequency corresponding to the new binary tree is the sum of the appearance frequency corresponding to the left sub-tree and the appearance frequency corresponding to the right sub-tree; and deleting the two adjacent binary trees with the minimum occurrence frequency from the binary tree set, and adding the new binary tree into the binary tree set.
In a specific embodiment, the tree structure is a binary tree; the encoding module 504 is further configured to: setting the assignment of each branch in a path of the tree structure from the root node to the leaf node, wherein the assignments of all left branches in the path are first values, and the assignments of all right branches are second values; and coding the numerical value based on the assignment corresponding to the branch of the path from the root node of the tree structure to the corresponding node of the numerical value in the tree structure.
Corresponding to the foregoing data decompression method, the present embodiment further provides a data decompression apparatus, as shown in fig. 6, which mainly includes the following modules:
a second obtaining module 602, configured to obtain data to be decompressed; the data to be decompressed is represented by the coding value, and the data to be decompressed carries the corresponding relation between the numerical value and the coding value; the data to be decompressed is obtained by compressing numerical values in the data to be compressed based on the corresponding relation, and the length of the coded value corresponding to the numerical value is inversely related to the occurrence frequency of the numerical value;
and the decompression module 604 is configured to decompress the encoded values in the data to be decompressed based on the corresponding relationship.
When the electronic equipment runs data, only the lossless decompression is needed through the device, so that the difficulty of storing/running the data of the electronic equipment and the hardware cost limit can be effectively reduced.
Further, the present embodiment provides an electronic device, including: a processor and a storage device; the storage device has stored thereon a computer program which, when executed by the processor, performs the aforementioned data compression method, or alternatively, performs the aforementioned data decompression method. In practical applications, the electronic device executing the data compression method may be a server, and the electronic device executing the data decompression method may be an internet of things embedded device (also referred to as an edge device) such as a mobile phone and a computer, which can effectively reduce the hardware cost of the edge device, accelerate the data processing speed of the edge device, and improve user experience.
It is understood that the same or similar parts in the above embodiments may be mutually referred to, and the same or similar parts in other embodiments may be referred to for the content which is not described in detail in some embodiments.
It should be noted that, in the description of the present application, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In addition, in the description of the present application, the meaning of "plurality" means at least two unless otherwise specified.
It will be understood that when an element is referred to as being "secured to" or "disposed on" another element, it can be directly on the other element or intervening elements may also be present; when an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present, and further, as used herein, connected may include wirelessly connected; the term "and/or" is used to include any and all combinations of one or more of the associated listed items.
Any process or method descriptions in flow charts or otherwise described herein may be understood as: represents modules, segments or portions of code which include one or more executable instructions for implementing specific logical functions or steps of a process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
Claims (10)
1. A method of data compression, comprising:
acquiring data to be compressed;
coding the numerical values in the data to be compressed based on the numerical value types contained in the data to be compressed and the occurrence frequency of each numerical value to obtain a coded value corresponding to each numerical value; wherein, the length of the code value corresponding to the numerical value is inversely related to the occurrence frequency of the numerical value;
and compressing the numerical value in the data to be compressed based on the corresponding relation between the numerical value and the encoding value.
2. The method of claim 1, wherein the step of obtaining data to be compressed comprises:
acquiring a trained neural network model;
extracting weight parameters of the neural network model;
and taking the extracted weight parameters as data to be compressed.
3. The method according to claim 1 or 2, wherein the step of encoding the values in the data to be compressed based on the kinds of the values contained in the data to be compressed and the occurrence frequency of each value comprises:
constructing a tree structure based on the numerical value types and the occurrence frequency of each numerical value contained in the data to be compressed;
for each value in the data to be compressed, encoding the value based on the position of the value in the tree structure.
4. The method according to claim 3, wherein the step of constructing a tree structure based on the kinds of values and the frequency of occurrence of each value contained in the data to be compressed comprises:
forming a binary tree set based on the numerical value types contained in the data to be compressed and the occurrence frequency of each numerical value; each binary tree corresponds to the occurrence frequency of a numerical value, each binary tree only comprises a root node represented by the occurrence frequency currently, and left and right subtrees of the root node are empty currently;
repeatedly executing the following operations on the binary tree set until only one tree exists in the binary tree set;
wherein the operations comprise: sequentially sequencing the binary trees in the binary tree set according to the occurrence frequency of the root node of each binary tree, respectively taking two adjacent binary trees with the minimum occurrence frequency as a left sub-tree and a right sub-tree, and constructing a new binary tree; wherein, the appearance frequency corresponding to the new binary tree is the sum of the appearance frequency corresponding to the left sub-tree and the appearance frequency corresponding to the right sub-tree; and deleting the two adjacent binary trees with the minimum occurrence frequency from the binary tree set, and adding the new binary tree into the binary tree set.
5. The method of claim 3, wherein the tree structure is a binary tree; the step of encoding the value based on its position in the tree structure comprises:
setting the assignment of each branch in a path of the tree structure from a root node to a leaf node, wherein the assignments of all left branches in the path are first values, and the assignments of all right branches in the path are second values;
and coding the numerical value based on the assignment corresponding to the branch of the path from the root node of the tree structure to the numerical value between the corresponding nodes in the tree structure.
6. A method of data decompression, comprising:
acquiring data to be decompressed; the data to be decompressed is represented by a coding value, and the data to be decompressed carries a corresponding relation between a numerical value and the coding value; the data to be decompressed is obtained by compressing numerical values in the data to be compressed based on the corresponding relation, and the length of the coded values corresponding to the numerical values is inversely related to the occurrence frequency of the numerical values;
and decompressing the coding values in the data to be decompressed based on the corresponding relation.
7. The method of claim 6, wherein the numerical value is a weight parameter of a neural network model;
the method further comprises the following steps:
and operating the neural network model based on the weight parameters of the neural network model obtained after decompression.
8. A data compression apparatus, comprising:
the first acquisition module is used for acquiring data to be compressed;
the encoding module is used for encoding the numerical values in the data to be compressed based on the numerical value types contained in the data to be compressed and the occurrence frequency of each numerical value to obtain the encoding values corresponding to each numerical value; wherein, the length of the code value corresponding to the numerical value is inversely related to the occurrence frequency of the numerical value;
and the compression module is used for compressing the numerical values in the data to be compressed based on the corresponding relation between the numerical values and the coding values.
9. A data decompression apparatus, comprising:
the second acquisition module is used for acquiring data to be decompressed; the data to be decompressed is represented by a coding value, and the data to be decompressed carries a corresponding relation between a numerical value and the coding value; the data to be decompressed is obtained by compressing numerical values in the data to be compressed based on the corresponding relation, and the length of the coded values corresponding to the numerical values is inversely related to the occurrence frequency of the numerical values;
and the decompression module is used for decompressing the coding values in the data to be decompressed based on the corresponding relation.
10. An electronic device, comprising: a processor and a storage device;
the storage device has stored thereon a computer program which, when executed by the processor, performs the method of any of claims 1 to 5, or the method of any of claims 6 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011000571.7A CN112101548A (en) | 2020-09-22 | 2020-09-22 | Data compression method and device, data decompression method and device, and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011000571.7A CN112101548A (en) | 2020-09-22 | 2020-09-22 | Data compression method and device, data decompression method and device, and electronic device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112101548A true CN112101548A (en) | 2020-12-18 |
Family
ID=73754850
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011000571.7A Withdrawn CN112101548A (en) | 2020-09-22 | 2020-09-22 | Data compression method and device, data decompression method and device, and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112101548A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114723033A (en) * | 2022-06-10 | 2022-07-08 | 成都登临科技有限公司 | Data processing method, data processing device, AI chip, electronic device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103379136A (en) * | 2012-04-17 | 2013-10-30 | 中国移动通信集团公司 | Compression method and decompression method of log acquisition data, compression apparatus and decompression apparatus of log acquisition data |
| CN104283567A (en) * | 2013-07-02 | 2015-01-14 | 北京四维图新科技股份有限公司 | Method for compressing or decompressing name data, and equipment thereof |
| CN107423397A (en) * | 2017-07-26 | 2017-12-01 | 北京时代民芯科技有限公司 | A kind of self-adapting compressing storage and decompression extracting method towards multitask micro-system |
| CN107565970A (en) * | 2017-08-17 | 2018-01-09 | 郑州云海信息技术有限公司 | A kind of the mixing lossless compression method and device of feature based identification |
| CN111130569A (en) * | 2019-12-17 | 2020-05-08 | 佛山科学技术学院 | Spatial information data self-adaptive fault-tolerant processing method and system |
-
2020
- 2020-09-22 CN CN202011000571.7A patent/CN112101548A/en not_active Withdrawn
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103379136A (en) * | 2012-04-17 | 2013-10-30 | 中国移动通信集团公司 | Compression method and decompression method of log acquisition data, compression apparatus and decompression apparatus of log acquisition data |
| CN104283567A (en) * | 2013-07-02 | 2015-01-14 | 北京四维图新科技股份有限公司 | Method for compressing or decompressing name data, and equipment thereof |
| CN107423397A (en) * | 2017-07-26 | 2017-12-01 | 北京时代民芯科技有限公司 | A kind of self-adapting compressing storage and decompression extracting method towards multitask micro-system |
| CN107565970A (en) * | 2017-08-17 | 2018-01-09 | 郑州云海信息技术有限公司 | A kind of the mixing lossless compression method and device of feature based identification |
| CN111130569A (en) * | 2019-12-17 | 2020-05-08 | 佛山科学技术学院 | Spatial information data self-adaptive fault-tolerant processing method and system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114723033A (en) * | 2022-06-10 | 2022-07-08 | 成都登临科技有限公司 | Data processing method, data processing device, AI chip, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101049699B1 (en) | Data Compression Method | |
| CN107565971B (en) | Data compression method and device | |
| US11722148B2 (en) | Systems and methods of data compression | |
| CN116681036B (en) | Industrial data storage method based on digital twinning | |
| CN110021369B (en) | Gene sequencing data compression and decompression method, system and computer readable medium | |
| CN110008192A (en) | A kind of data file compression method, apparatus, equipment and readable storage medium storing program for executing | |
| CN111510156A (en) | Method for dynamically compressing and decompressing large file based on segmentation | |
| CN112463784A (en) | Data deduplication method, device, equipment and computer readable storage medium | |
| CN112101548A (en) | Data compression method and device, data decompression method and device, and electronic device | |
| CN108880559B (en) | Data compression method, data decompression method, compression equipment and decompression equipment | |
| CN117095685B (en) | Concurrent department platform terminal equipment and control method thereof | |
| CN117040539B (en) | Petroleum logging data compression method and device based on M-ary tree and LZW algorithm | |
| CN116934487B (en) | Financial clearing data optimal storage method and system | |
| CN117254820A (en) | Data compression method, device, equipment and storage medium | |
| Malach et al. | Hardware-based real-time deep neural network lossless weights compression | |
| CN109698703B (en) | Gene sequencing data decompression method, system and computer readable medium | |
| CN113810058A (en) | Data compression method, data decompression method, device and electronic equipment | |
| CN111143641A (en) | Deep learning model training method and device and electronic equipment | |
| JP2005521324A (en) | Method and apparatus for lossless data compression and decompression | |
| US20240184763A1 (en) | Data compression method, data decompression method, and electronic device | |
| CN111274950A (en) | Feature vector data encoding and decoding method, server and terminal | |
| CN112200301B (en) | Convolution computing device and method | |
| US20180145701A1 (en) | Sonic Boom: System For Reducing The Digital Footprint Of Data Streams Through Lossless Scalable Binary Substitution | |
| CN109698704B (en) | Comparative gene sequencing data decompression method, system and computer readable medium | |
| Jianjun et al. | Research and Improvement of Huffman Compression Method Based on Linear Linked Forest |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20201218 |
|
| WW01 | Invention patent application withdrawn after publication |