CN112803950A - Data compression method, device and equipment and computer storage medium - Google Patents
Data compression method, device and equipment and computer storage medium Download PDFInfo
- Publication number
- CN112803950A CN112803950A CN202110006929.5A CN202110006929A CN112803950A CN 112803950 A CN112803950 A CN 112803950A CN 202110006929 A CN202110006929 A CN 202110006929A CN 112803950 A CN112803950 A CN 112803950A
- Authority
- CN
- China
- Prior art keywords
- data
- transmitted
- compression
- preset
- compressed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
- H03M7/70—Type of the data to be coded, other than image and sound
- H03M7/702—Software
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Information Transfer Between Computers (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention discloses a data compression method, a data compression device, data compression equipment and a computer storage medium. The data compression method comprises the following steps: acquiring first data to be transmitted, wherein the first data to be transmitted comprises a head field and a body field; performing first compression processing on first data to be transmitted according to a preset compression algorithm to obtain compressed data; encrypting the compressed data according to a first preset encryption algorithm to obtain encrypted compressed data; acquiring first verification data from the encrypted compressed data according to a preset data extraction rule; and generating second data to be transmitted according to the first verification data and the compressed data. According to the embodiment of the invention, the compression rate of the data to be transmitted can be improved, the occupation of transmission bandwidth is effectively reduced, and the cost is saved.
Description
Technical Field
The present invention belongs to the technical field of data processing, and in particular, to a data compression method, apparatus, device, and computer storage medium.
Background
With the rapid development of the internet technology, more and more data are transmitted among different devices, and if the original data is directly used as a transmission object, a large amount of bandwidth resources are occupied, and the operation and maintenance cost is increased.
At present, data compression refers to compressing data to be transmitted on the premise of reducing data volume without losing useful information, and in order to save cost and reduce bandwidth occupied during data transmission, for example, compressing a Body (Body) of a HyperText Transfer Protocol (HTTP) request by a compression method of a file compression program (gzip) of GNU free software. However, the compression rate of the transmission data is not high enough in the existing compression method, redundant data still exists, and transmission bandwidth is occupied.
Disclosure of Invention
Embodiments of the present invention provide a data compression method, apparatus, device, and computer storage medium, which can improve the compression rate of data to be transmitted, effectively reduce the occupation of transmission bandwidth, and save cost.
In a first aspect, an embodiment of the present invention provides a data compression method, where the method includes:
acquiring first data to be transmitted, wherein the first data to be transmitted comprises a head field and a body field;
performing first compression processing on first data to be transmitted according to a preset compression algorithm to obtain compressed data;
encrypting the compressed data according to a first preset encryption algorithm to obtain encrypted compressed data;
acquiring first verification data from the encrypted compressed data according to a preset data extraction rule;
and generating second data to be transmitted according to the first verification data and the compressed data.
In some implementations of the first aspect, performing a first compression process on the first data to be transmitted according to a preset compression algorithm to obtain compressed data, includes:
performing second compression processing on the first data to be transmitted according to a preset data dictionary to obtain third data to be transmitted;
and performing first compression processing on the third data to be transmitted according to a preset compression algorithm to obtain compressed data.
In some implementations of the first aspect, the preset compression algorithm is a Zstandard compression algorithm.
In some implementations of the first aspect, before generating the second data to be transmitted according to the first verification data and the compressed data, the method further includes:
acquiring second verification data of the preset data dictionary according to a second preset encryption algorithm;
generating second data to be transmitted according to the first verification data and the compressed data, wherein the second data to be transmitted comprises:
and generating second data to be transmitted according to the first verification data, the second verification data and the compressed data.
In some implementation manners of the first aspect, after performing first compression processing on the third to-be-transmitted data according to a preset compression algorithm to obtain compressed data, the method further includes:
and determining a compression rate according to the data quantity of the first data to be transmitted and the data quantity of the compressed data, wherein the compression rate is used for determining whether to update the preset data dictionary.
In some implementations of the first aspect, the first predetermined encryption algorithm is a message digest algorithm MD 5.
In a second aspect, an embodiment of the present invention provides a data compression apparatus, including:
the device comprises an acquisition module, a transmission module and a transmission module, wherein the acquisition module is used for acquiring first data to be transmitted, and the first data to be transmitted comprises a head field and a body field;
the compression module is used for carrying out first compression processing on first data to be transmitted according to a preset compression algorithm to obtain compressed data;
the encryption module is used for encrypting the compressed data according to a first preset encryption algorithm to obtain encrypted compressed data;
the processing module is used for acquiring first verification data from the encrypted compressed data according to a preset data extraction rule;
and the processing module is also used for generating second data to be transmitted according to the first verification data and the compressed data.
In some implementation manners of the second aspect, the compression module is further configured to perform second compression processing on the first data to be transmitted according to a preset data dictionary to obtain third data to be transmitted; and performing first compression processing on the third data to be transmitted according to a preset compression algorithm to obtain compressed data.
In a third aspect, the present invention provides a data compression apparatus comprising: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the data compression method described in the first aspect or any of the realizable forms of the first aspect.
In a fourth aspect, the present invention provides a computer-readable storage medium, on which computer program instructions are stored, which, when executed by a processor, implement the data compression method of the first aspect or any of the realizable manners of the first aspect.
The embodiment of the invention provides a data compression method, before sending first data to be transmitted to a server, the whole information of the first data to be transmitted is compressed according to a preset compression algorithm, namely, a head field and a body field of the first data to be transmitted are compressed to obtain compressed data, so that redundant data are reduced, the compression rate of the first data to be transmitted is improved, and then first verification data of the encrypted compressed data is obtained according to a first preset encryption algorithm and a preset data extraction rule, wherein the first verification data can be used for verifying the received compressed data by the server to ensure the correctness of the compressed data received by the server, and the compression rate of the data to be transmitted can be effectively improved by generating new second data to be transmitted to the server by the first verification data and the compressed data, the occupation of transmission bandwidth is reduced, and the cost is saved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a data transmission process according to an embodiment of the present invention;
FIG. 2 is a flow chart of a data compression method according to an embodiment of the present invention;
FIG. 3 is a block diagram illustrating an alternative data transmission process according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a data compression apparatus according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a data compression device according to an embodiment of the present invention.
Detailed Description
Features and exemplary embodiments of various aspects of the present invention will be described in detail below, and in order to make objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting. It will be apparent to one skilled in the art that the present invention may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the present invention.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone.
With the rapid development of internet technology, more and more data are transmitted among different devices, as a specific example, fig. 1 shows an architecture diagram of an architecture data transmission process of an existing data transmission process, an Application (APP) on an electronic device generates data to be transmitted in an HTTP format, directly sends out the data to be transmitted, and then the data to be transmitted is received by a gateway and then forwarded to a server, and the server performs corresponding service processing according to the received data to be transmitted.
If the original data is directly used as a transmission object, a large amount of bandwidth resources are occupied, and the operation and maintenance cost is increased.
At present, data compression refers to reducing data volume on the premise of not losing useful information, and often compressing data to be transmitted in order to save cost and reduce bandwidth occupied during data transmission, for example, compressing Body (Body) of a HyperText Transfer Protocol (HTTP) request by using a gzip compression method. However, the compression rate of the transmission data is not high enough in the existing compression method, redundant data still exists, and transmission bandwidth is occupied.
In view of the above, embodiments of the present invention provide a data compression method, apparatus, device, and computer readable storage medium, before sending first data to be transmitted to a server, the whole information of the first data to be transmitted is compressed according to a preset compression algorithm, that is, a header field and a body field of the first data to be transmitted are compressed to obtain compressed data, so as to reduce redundant data and improve a compression ratio of the first data to be transmitted, then, according to a first preset encryption algorithm and a preset data extraction rule, first verification data of the encrypted compressed data is obtained, and new second data to be transmitted is generated by the first verification data and the compressed data and sent to the server, so that the compression ratio of the transmitted data can be effectively improved, occupation of a transmission bandwidth is reduced, and cost is saved.
The data compression method provided by the embodiment of the invention is described below with reference to the accompanying drawings.
Fig. 2 is a flowchart illustrating a data compression method according to an embodiment of the present invention. The data compression method may be applied to an electronic device, and as shown in fig. 2, the method may include S210-S250.
S210, acquiring first data to be transmitted.
In this embodiment of the present invention, the electronic device may install multiple Applications (APPs), and the electronic device may directly obtain an original data packet that each APP needs to send, that is, first data to be transmitted, where the first data to be transmitted includes, for example, a header field (head) and a body field (body).
After the first data to be transmitted is obtained, S220 may be performed next.
S220, performing first compression processing on the first data to be transmitted according to a preset compression algorithm to obtain compressed data.
In some embodiments, the predetermined compression algorithm may be a predetermined compression algorithm that is a zstandard (zsttd) compression algorithm. The method comprises the steps of performing first compression processing on an entire original data packet through a ZSTD compression algorithm to obtain compressed data, wherein the compressed data can be used as a main body field of new data to be transmitted (second data to be transmitted). The first data to be transmitted is subjected to the first compression processing through the preset compression algorithm, so that the compression rate of the first data to be transmitted is improved.
After the compressed data is obtained, S230 may be performed next.
And S230, encrypting the compressed data according to a first preset encryption algorithm to obtain encrypted compressed data.
In some embodiments, the first predetermined encryption Algorithm may be a Message Digest Algorithm (MD 5).
After the encrypted compressed data is obtained, S240 may be performed.
S240, acquiring first verification data from the encrypted compressed data according to a preset data extraction rule.
The first verification data can be used for the server to verify the received compressed data so as to ensure the correctness of the compressed data received by the server. The preset data extraction rule may be preset to extract the 8 bits of data before the encrypted compressed data as data for the server to verify the received compressed data.
After the first authentication data and the compressed data are obtained, S250 may be performed next.
And S250, generating second data to be transmitted according to the first verification data and the compressed data.
In some embodiments, the first verification data may be a header field (head) of the second data to be transmitted, and the compressed data may be a body field (body) of the second data to be transmitted. The first data to be transmitted is subjected to first compression processing through a preset compression algorithm, so that the compression rate of the first data to be transmitted is improved, and the second data to be transmitted also comprises first verification data which can be used for verifying whether the compressed data is damaged or not, so that a server receiving the compressed data can verify the compressed data according to the first verification data, and the received compressed data is ensured to be damaged or tampered in the transmission process.
As a specific example, in combination with the schematic architecture diagram of the data transmission process shown in fig. 3, the electronic device may first perform compression processing on the first to-be-transmitted data of each application by using a zsttd compression algorithm to obtain compressed data, and then send the second to-be-transmitted data to the server through the HTTP field.
By the data compression method provided by the embodiment of the invention, before the first data to be transmitted is sent to the server, the whole information of the first data to be transmitted is compressed according to a preset compression algorithm, namely, the header field and the body field of the first data to be transmitted are compressed to obtain compressed data, so that redundant data are reduced, the compression rate of the first data to be transmitted is improved, and then the first verification data of the encrypted compressed data is obtained according to the first preset encryption algorithm and the preset data extraction rule, wherein the first verification data can be used for the server to verify the received compressed data to ensure the correctness of the compressed data received by the server, and new second data to be transmitted is generated by the first verification data and the compressed data and sent to the server, so that on one hand, the compression rate of the transmitted data can be effectively improved, the occupation of transmission bandwidth is reduced, and the cost is saved.
On the other hand, the data volume of the transmission data is reduced, so that the number of unpacked transmission data is reduced, or the data can be directly sent to the server without unpacking, the accuracy of data transmission can be improved, multiple transmissions caused by transmission errors are effectively avoided, the transmission bandwidth is occupied, and the operation and maintenance cost is further reduced.
In some embodiments, in order to improve the forwarding efficiency, as shown in fig. 3, the compression gateway service may receive the second to-be-transmitted data first, decompress the second to-be-transmitted data by the compression gateway service to obtain decompressed second to-be-transmitted data, then send the decompressed second to-be-transmitted data to the forwarding gateway, send the decompressed second to-be-transmitted data to the server by the forwarding gateway, and finally verify the compressed data according to the first verification information in the decompressed second to-be-transmitted data, where if the compressed data is error-free, the server may perform corresponding operations corresponding to the compressed data. If the compressed data has errors or does not provide services, the server can feed back the dislocation prompt or service-free prompt information to the electronic equipment, and the electronic equipment can resend the first data to be transmitted or stop sending the first data to be transmitted according to the received information.
In some embodiments, in order to further improve the compression rate of the first to-be-transmitted data, when S220 is executed, the first to-be-transmitted data may be subjected to second compression processing according to a preset data dictionary to obtain third to-be-transmitted data, and then the third to-be-transmitted data is subjected to first compression processing according to a preset compression algorithm to obtain compressed data.
The preset data dictionary is obtained by training data to be transmitted sent by the electronic equipment in advance, and some fields with high occurrence frequency in the data to be transmitted can be represented by using specific fields through the preset data dictionary. Therefore, the third to-be-transmitted data obtained after the second compression processing can prevent the information covered by the compressed first to-be-transmitted data from being lost, and further reduce the data volume of the to-be-transmitted data.
In some embodiments, in order to enable the server that receives the second to-be-transmitted data to decompress the received compressed data according to the same preset data dictionary as the electronic device, before S250 is executed, second verification data of the preset data dictionary may be further obtained according to a second preset encryption algorithm, and when S250 is executed, the second to-be-transmitted data may be generated according to the first verification data, the second verification data, and the compressed data.
As a specific example, the second preset encryption algorithm may be a message digest algorithm (MD 5).
For example, a header field (head) of the second data to be transmitted may include the first verification data and the second verification data, and a body field (body) of the second data to be transmitted may include the compressed data.
In some embodiments, the second data to be transmitted may further include an identifier of whether the preset data dictionary is used, so that the server may quickly recognize whether the preset data dictionary is used by the compressed data in the second data to be transmitted.
In some embodiments, each server may correspondingly receive second data to be transmitted of the multiple electronic devices, different digital dictionaries may be preset on different electronic devices, and multiple different digital dictionaries may also be preset on the same electronic device, so that the server may quickly determine and search which digital dictionary is used by compressed data in the received second data to be transmitted according to the received second verification data, thereby obtaining the original data to be transmitted (the first data to be transmitted).
In some embodiments, after the third data to be transmitted is subjected to the first compression processing according to the preset compression algorithm to obtain the compressed data, a compression rate may be further determined according to the data amount of the first data to be transmitted and the data amount of the compressed data, where the compression rate is used to determine whether to update the preset data dictionary. When the compression rate is too low, it is indicated that the accuracy of the preset data dictionary in the electronic device for searching the fields with higher frequency in the first data to be transmitted is not high, and therefore, the preset data dictionary needs to be retrained to improve the compression rate of the first data to be transmitted.
According to the data compression method provided by the embodiment of the invention, the first verification data, the second verification data and the compressed data are generated into new second data to be transmitted and sent to the server, so that the compression rate of the transmitted data is further improved, on one hand, the occupation of transmission bandwidth can be reduced, and the operation and maintenance cost is reduced, on the other hand, the data volume of the transmitted data is reduced, so that the unpacking quantity of the transmitted data is reduced, or the data can be directly sent to the server without unpacking, therefore, the accuracy of data transmission can be improved, multiple transmissions caused by transmission errors are effectively avoided, the transmission bandwidth is occupied, and the operation and maintenance cost is further reduced.
Fig. 4 is a schematic structural diagram of a data compression apparatus according to an embodiment of the present invention, and as shown in fig. 4, the data compression apparatus 400 may include: an acquisition module 410, a compression module 420, an encryption module 430, and a processing module 440.
The obtaining module 410 is configured to obtain first data to be transmitted, where the first data to be transmitted includes a header field and a body field.
The compression module 420 is configured to perform a first compression process on the first data to be transmitted according to a preset compression algorithm, so as to obtain compressed data.
The encryption module 430 is configured to encrypt the compressed data according to a first preset encryption algorithm to obtain encrypted compressed data.
The processing module 440 is configured to obtain the first verification data from the encrypted compressed data according to a preset data extraction rule.
The processing module 440 is further configured to generate second data to be transmitted according to the first verification data and the compressed data.
In some embodiments, the processing module 440 is further configured to perform a second compression process on the first data to be transmitted according to a preset data dictionary to obtain third data to be transmitted; and performing first compression processing on the third data to be transmitted according to a preset compression algorithm to obtain compressed data.
In some embodiments, the predetermined compression algorithm is a Zstandard compression algorithm.
In some embodiments, the processing module 440 is further configured to obtain second verification data of the preset data dictionary according to a second preset encryption algorithm; and the data transmission device is used for generating second data to be transmitted according to the first verification data, the second verification data and the compressed data.
In some embodiments, the processing module 440 is further configured to determine a compression rate according to the data amount of the first data to be transmitted and the data amount of the compressed data, wherein the compression rate is used for determining whether to update the preset data dictionary.
In some embodiments, the first predetermined encryption algorithm is message digest algorithm MD 5.
It is understood that the data compression apparatus 400 according to the embodiment of the present invention may correspond to an execution main body of the data compression method according to the embodiment of the present invention, and specific details of operations and/or functions of each module/unit of the data compression apparatus 400 may refer to the descriptions of corresponding parts in the data compression method according to the embodiment of the present invention, which are not described herein again for brevity.
According to the data compression device provided by the embodiment of the invention, the first verification data, the second verification data and the compressed data are generated into new second data to be transmitted and sent to the server, so that the compression rate of the transmitted data is further improved, on one hand, the occupation of transmission bandwidth can be reduced, and therefore, the operation and maintenance cost is reduced, on the other hand, the data volume of the transmitted data is reduced, so that the unpacking quantity of the transmitted data is reduced, or the data can be directly sent to the server without unpacking, therefore, the accuracy of data transmission can be improved, multiple transmissions caused by transmission errors can be effectively avoided, the occupation of the transmission bandwidth is effectively avoided, and the operation and maintenance cost is further reduced.
Fig. 5 is a schematic diagram of a hardware structure of a data compression device according to an embodiment of the present invention.
As shown in fig. 5, the data compression apparatus 500 in the present embodiment includes an input device 501, an input interface 502, a central processing unit 503, a memory 504, an output interface 505, and an output device 506. The input interface 502, the central processing unit 503, the memory 504, and the output interface 505 are connected to each other through a bus 510, and the input device 501 and the output device 506 are connected to the bus 510 through the input interface 502 and the output interface 505, respectively, and further connected to other components of the data compression device 500.
Specifically, the input device 501 receives input information from the outside and transmits the input information to the central processor 503 through the input interface 502; the central processor 503 processes input information based on computer-executable instructions stored in the memory 504 to generate output information, temporarily or permanently stores the output information in the memory 504, and then transmits the output information to the output device 506 through the output interface 505; the output device 506 outputs the output information to the outside of the data compression device 500 for use by the user.
That is, the data compression apparatus shown in fig. 5 may also be implemented to include: a memory storing computer-executable instructions; and a processor which, when executing computer executable instructions, may implement the data compression methods described in connection with the examples of the invention.
In one embodiment, the data compression apparatus 500 shown in fig. 5 includes: a memory 504 for storing programs; the processor 503 is configured to execute the program stored in the memory to perform the data compression method provided by the embodiment of the present invention.
An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium has computer program instructions stored thereon; the computer program instructions, when executed by a processor, implement a method of data compression provided by an embodiment of the invention.
It is to be understood that the invention is not limited to the specific arrangements and instrumentality described above and shown in the drawings. A detailed description of known methods is omitted herein for the sake of brevity. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present invention are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications and additions or change the order between the steps after comprehending the spirit of the present invention.
The functional blocks shown in the above-described structural block diagrams may be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic Circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, plug-in, function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine-readable medium or transmitted by a data signal carried in a carrier wave over a transmission medium or a communication link. A "machine-readable medium" may include any medium that can store or transfer information. Examples of a machine-readable medium include electronic circuits, semiconductor Memory devices, Read-Only memories (ROMs), flash memories, Erasable Read-Only memories (EROMs), floppy disks, Compact disk Read-Only memories (CD-ROMs), optical disks, hard disks, optical fiber media, Radio Frequency (RF) links, and so forth. The code segments may be downloaded via computer networks such as the internet, intranet, etc.
It should also be noted that the exemplary embodiments mentioned in this patent describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, may be performed in an order different from the order in the embodiments, or may be performed simultaneously.
Aspects of the present disclosure are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions/acts specified in the flowchart and/or block diagram block or blocks. Such a processor may be, but is not limited to, a general purpose processor, a special purpose processor, an application specific processor, or a field programmable logic circuit. It will also be understood that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware for performing the specified functions or acts, or combinations of special purpose hardware and computer instructions.
As described above, only the specific embodiments of the present invention are provided, and it can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system, the module and the unit described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. It should be understood that the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the present invention, and these modifications or substitutions should be covered within the scope of the present invention.
Claims (10)
1. A method of data compression, the method comprising:
acquiring first data to be transmitted, wherein the first data to be transmitted comprises a head field and a body field;
performing first compression processing on the first data to be transmitted according to a preset compression algorithm to obtain compressed data;
encrypting the compressed data according to a first preset encryption algorithm to obtain encrypted compressed data;
acquiring first verification data from the encrypted compressed data according to a preset data extraction rule;
and generating second data to be transmitted according to the first verification data and the compressed data.
2. The method according to claim 1, wherein the performing a first compression process on the first data to be transmitted according to a preset compression algorithm to obtain compressed data comprises:
performing second compression processing on the first data to be transmitted according to a preset data dictionary to obtain third data to be transmitted;
and performing the first compression processing on the third data to be transmitted according to a preset compression algorithm to obtain the compressed data.
3. Method according to claim 1 or 2, characterized in that the preset compression algorithm is a Zstandard compression algorithm.
4. The method of claim 2, wherein prior to the generating second data to be transmitted from the first verification data and the compressed data, the method further comprises:
acquiring second verification data of the preset data dictionary according to a second preset encryption algorithm;
generating second data to be transmitted according to the first verification data and the compressed data, wherein the generating of the second data to be transmitted comprises:
and generating the second data to be transmitted according to the first verification data, the second verification data and the compressed data.
5. The method according to claim 2, wherein after the performing the first compression processing on the third data to be transmitted according to a preset compression algorithm to obtain the compressed data, the method further comprises:
and determining a compression rate according to the data volume of the first data to be transmitted and the data volume of the compressed data, wherein the compression rate is used for determining whether to update the preset data dictionary.
6. Method according to claim 1 or 2, characterized in that said first predetermined encryption algorithm is the message digest algorithm MD 5.
7. An apparatus for compressing data, the apparatus comprising:
the device comprises an acquisition module, a transmission module and a transmission module, wherein the acquisition module is used for acquiring first data to be transmitted, and the first data to be transmitted comprises a head field and a body field;
the compression module is used for carrying out first compression processing on the first data to be transmitted according to a preset compression algorithm to obtain compressed data;
the encryption module is used for encrypting the compressed data according to a first preset encryption algorithm to obtain encrypted compressed data;
the processing module is used for acquiring first verification data from the encrypted compressed data according to a preset data extraction rule;
the processing module is further configured to generate second data to be transmitted according to the first verification data and the compressed data.
8. The apparatus according to claim 7, wherein the compression module is further configured to perform a second compression process on the first data to be transmitted according to a preset data dictionary to obtain third data to be transmitted; and performing the first compression processing on the third data to be transmitted according to a preset compression algorithm to obtain the compressed data.
9. An apparatus for data compression, the apparatus comprising: a processor, and a memory storing computer program instructions;
the processor reads and executes the computer program instructions to implement the data compression method of any one of claims 1-6.
10. A computer storage medium having computer program instructions stored thereon which, when executed by a processor, implement a method of data compression as claimed in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110006929.5A CN112803950A (en) | 2021-01-05 | 2021-01-05 | Data compression method, device and equipment and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110006929.5A CN112803950A (en) | 2021-01-05 | 2021-01-05 | Data compression method, device and equipment and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112803950A true CN112803950A (en) | 2021-05-14 |
Family
ID=75808192
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110006929.5A Pending CN112803950A (en) | 2021-01-05 | 2021-01-05 | Data compression method, device and equipment and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112803950A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114040030A (en) * | 2021-11-18 | 2022-02-11 | 深圳智慧林网络科技有限公司 | Data compression method, device, equipment and medium based on preset rules |
| CN116827488A (en) * | 2023-08-30 | 2023-09-29 | 广东电网有限责任公司东莞供电局 | Power data transmission control method and device based on block chain and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104202054A (en) * | 2014-09-16 | 2014-12-10 | 东南大学 | Hardware LZMA (Lempel-Ziv-Markov chain-Algorithm) compression system and method |
| CN107786562A (en) * | 2017-11-01 | 2018-03-09 | 北京知道创宇信息技术有限公司 | Data ciphering method, decryption method and device |
| CN109842653A (en) * | 2017-11-27 | 2019-06-04 | 电信科学技术研究院 | A kind of method and apparatus carried out data transmission |
| CN111782660A (en) * | 2020-07-17 | 2020-10-16 | 支付宝(杭州)信息技术有限公司 | Data compression method and system based on key value storage |
| CN112118073A (en) * | 2019-06-19 | 2020-12-22 | 华为技术有限公司 | Data processing method, optical transmission equipment and digital processing chip |
-
2021
- 2021-01-05 CN CN202110006929.5A patent/CN112803950A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104202054A (en) * | 2014-09-16 | 2014-12-10 | 东南大学 | Hardware LZMA (Lempel-Ziv-Markov chain-Algorithm) compression system and method |
| CN107786562A (en) * | 2017-11-01 | 2018-03-09 | 北京知道创宇信息技术有限公司 | Data ciphering method, decryption method and device |
| CN109842653A (en) * | 2017-11-27 | 2019-06-04 | 电信科学技术研究院 | A kind of method and apparatus carried out data transmission |
| CN112118073A (en) * | 2019-06-19 | 2020-12-22 | 华为技术有限公司 | Data processing method, optical transmission equipment and digital processing chip |
| CN111782660A (en) * | 2020-07-17 | 2020-10-16 | 支付宝(杭州)信息技术有限公司 | Data compression method and system based on key value storage |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114040030A (en) * | 2021-11-18 | 2022-02-11 | 深圳智慧林网络科技有限公司 | Data compression method, device, equipment and medium based on preset rules |
| CN114040030B (en) * | 2021-11-18 | 2023-11-24 | 深圳智慧林网络科技有限公司 | Data compression method, device, equipment and medium based on preset rules |
| CN116827488A (en) * | 2023-08-30 | 2023-09-29 | 广东电网有限责任公司东莞供电局 | Power data transmission control method and device based on block chain and storage medium |
| CN116827488B (en) * | 2023-08-30 | 2024-01-05 | 广东电网有限责任公司东莞供电局 | Power data transmission control method and device based on block chain and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108965484B (en) | Internet of things data transmission method, system and terminal | |
| CN111131231A (en) | Method and equipment for accessing data of vehicle-mounted terminal into Internet of vehicles monitoring platform | |
| CN109388417B (en) | Communication protocol updating method, system and terminal equipment | |
| CN105389177A (en) | Software version confirmation method, device and system | |
| CN112803950A (en) | Data compression method, device and equipment and computer storage medium | |
| CN109391673A (en) | A kind of method, system and the terminal device of management update file | |
| CN112822199B (en) | OTA (over the air) upgrading method and system based on protocol conversion | |
| CN112087475B (en) | Message pushing method and device for cloud platform component application and message server | |
| CN111225077B (en) | Network distribution method, device and system for Internet of things equipment | |
| CN112804284A (en) | Data chaining device, data verification device and data verification method | |
| US20120044954A1 (en) | Communication device | |
| JP6992309B2 (en) | Transmitter, receiver, and communication method | |
| CN112434231B (en) | Data processing method and device and electronic equipment | |
| CN115858320A (en) | Operation log recording method, apparatus, medium and product | |
| CN112559546B (en) | Database synchronization method, database synchronization device, computer equipment and readable storage medium | |
| WO2021036189A1 (en) | Rdma data sending and receiving methods, electronic device and readable storage medium | |
| CN115152180A (en) | Improved packet transmission | |
| CN113259108A (en) | Certificate updating method, Internet of things platform and Internet of things equipment | |
| CN102158818A (en) | Communication method of vehicle-borne information service system based on Socket protocol | |
| CN111740817A (en) | Code tampering detection method and system for concentrator in electric power data acquisition system | |
| CN113850988B (en) | Key value correction method, device, equipment and computer readable medium | |
| CN111181628A (en) | Method, terminal and storage medium for transmitting voice data through Beidou short message | |
| CN110929500B (en) | File comparison method and related device | |
| CN112165465B (en) | Data processing method and device for outgoing third-party platform and electronic device | |
| CN114745368B (en) | File downloading method, device, computer equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210514 |
|
| WD01 | Invention patent application deemed withdrawn after publication |