CN111835686A - Data compression method, data decompression method, data compression device, data decompression device, electronic equipment and storage medium - Google Patents

Data compression method, data decompression method, data compression device, data decompression device, electronic equipment and storage medium Download PDF

Info

Publication number
CN111835686A
CN111835686A CN201910320309.1A CN201910320309A CN111835686A CN 111835686 A CN111835686 A CN 111835686A CN 201910320309 A CN201910320309 A CN 201910320309A CN 111835686 A CN111835686 A CN 111835686A
Authority
CN
China
Prior art keywords
message body
current
current target
target message
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910320309.1A
Other languages
Chinese (zh)
Inventor
王希斌
刘涛
樊宇
尹飞云
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN201910320309.1A priority Critical patent/CN111835686A/en
Publication of CN111835686A publication Critical patent/CN111835686A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
    • H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/546—Message passing systems or structures, e.g. queues
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/565—Conversion or adaptation of application format or content
    • H04L67/5651—Reducing the amount or size of exchanged application data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/04—Protocols for data compression, e.g. ROHC
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/54—Indexing scheme relating to G06F9/54
    • G06F2209/548—Queue

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The embodiment of the invention discloses a data compression method, a data decompression method, a data compression device, a data decompression device, electronic equipment and a storage medium. The compression method comprises the following steps: the sending end equipment acquires a current original message body in a current data stream according to a predetermined message body size; generating a current transmission message body corresponding to the current original message body according to a compression algorithm corresponding to the current original message body; wherein the current transfer message body comprises: a header of the current transfer message body and a content of the current transfer message body; and sending the current transmission message body to receiving end equipment through a preset message queue. The data compression effect can be effectively improved, and the universality is stronger.

Description

Data compression method, data decompression method, data compression device, data decompression device, electronic equipment and storage medium
Technical Field
The embodiment of the invention relates to the technical field of computers, in particular to a data compression method, a data decompression method, a data compression device, a data decompression device, electronic equipment and a storage medium.
Background
In a system that needs to process a large amount of data, a large amount of data often needs to be stored and transmitted, and the larger the amount of data is, the more storage space, computing resources and operation time are occupied for storing and transmitting the data, and for a system with a large amount of data and high concurrency, storing and transmitting the large amount of data may reduce system performance. In order to reduce the influence of storing and transmitting large quantities of data on the system performance, it is common to reduce the amount of data stored and transmitted by compressing the data to improve the read-write performance of the database and the data transmission performance.
In the existing data compression method, sending end equipment firstly obtains a current original message body in a current data stream according to a predetermined message body size; then sending the current original message body to a message queue; the message queue compresses the current original message body into a current transmission message body and then sends the current transmission message body to the receiving end equipment; in the existing data decompression method, a receiving end device receives a current transmission message body compressed by a message queue first, and then decompresses the current transmission message body into a current original message body.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:
in the existing data compression method, a message queue usually compresses a current original message body according to a general compression algorithm; in the existing data decompression method, the receiving end device usually decompresses the current transmission message body according to a general decompression algorithm; neither compression nor decompression is ideal; moreover, most message queues do not support data compression, and the application range is relatively limited.
Disclosure of Invention
In view of this, embodiments of the present invention provide a data compression method, a data decompression method, an apparatus, an electronic device, and a storage medium, which can effectively improve a data compression effect and have stronger universality.
In a first aspect, an embodiment of the present invention provides a data compression method, which is applied to a sending end device, and the method includes:
acquiring a current original message body in a current data stream according to a predetermined message body size;
generating a current transmission message body corresponding to the current original message body according to a compression algorithm corresponding to the current original message body; wherein the current transfer message body comprises: a header of the current transfer message body and a content of the current transfer message body;
and sending the current transmission message body to receiving end equipment through a preset message queue.
In the above embodiment, the generating a current transmission message body corresponding to the current original message body according to the compression algorithm corresponding to the current original message body includes:
determining a compression algorithm corresponding to the current original message body according to the data type of the current original message body; and encapsulating the identity of the compression algorithm in the header of the current body of transmitted messages;
compressing the current original message body into a current target message body according to a compression algorithm corresponding to the current original message body; and encapsulating the current target message body in the content of the current transfer message body.
In the above embodiment, the method further comprises:
calculating the size of the current target message body and the signature of the current target message body according to a predetermined calculation method;
encapsulating the size of the current target message body and the signature of the current target message body into a header of the current transfer message body.
In a second aspect, an embodiment of the present invention further provides a data decompression method, which is applied to a receiving end device, where the method includes:
receiving a current transmission message body sent by sending end equipment through a preset message queue;
acquiring a current target message body and a decompression algorithm corresponding to the current target message body according to the current transmission message body;
and decompressing the current target message body into a current original message body according to a decompression algorithm corresponding to the current target message body.
In the above embodiment, the obtaining, according to the current transmission message body, a current target message body and a decompression algorithm corresponding to the current target message body includes:
decapsulating the content of the current transmission message body, and acquiring the current target message body from the decapsulated content of the current transmission message body;
decapsulating the header of the current transmission message body, and acquiring an identifier of a compression algorithm corresponding to the current target message body from the decapsulated header of the current transmission message body; and determining the decompression algorithm corresponding to the current target message body according to the identifier of the compression algorithm corresponding to the current target message body.
In the above embodiment, before the decompressing the current target message body into the current original message body according to the decompression algorithm corresponding to the current target message body, the method further includes:
acquiring the size of the current target message body and the signature of the current target message body in the head of the decapsulated current transmission message body;
calculating the size of the current target message body and the signature of the current target message body according to a predetermined calculation method;
and if the obtained size of the current target message body is the same as the calculated size of the current target message body and the obtained signature of the current target message body is the same as the calculated signature of the current target message body, executing the decompression algorithm corresponding to the current target message body, and decompressing the current target message body into a current original message body.
In a third aspect, an embodiment of the present invention provides a data compression apparatus, where the apparatus includes: the device comprises a first acquisition module, a compression module and a sending module; wherein,
the first obtaining module is used for obtaining a current original message body in a current data stream according to a predetermined message body size;
the compression module is used for generating a current transmission message body corresponding to the current original message body according to a compression algorithm corresponding to the current original message body; wherein the current transfer message body comprises: a header of the current transfer message body and a content of the current transfer message body;
and the sending module is used for sending the current transmission message body to receiving end equipment through a preset message queue.
In the foregoing embodiment, the compression module is specifically configured to determine, according to a data type of a current original message body, a compression algorithm corresponding to the current original message body; and encapsulating the identity of the compression algorithm in the header of the current body of transmitted messages; compressing the current original message body into a current target message body according to a compression algorithm corresponding to the current original message body; and encapsulating the current target message body in the content of the current transfer message body.
In the above embodiment, the compression module is further configured to calculate the size of the current target message body and the signature of the current target message body according to a predetermined calculation method; encapsulating the size of the current target message body and the signature of the current target message body into a header of the current transfer message body.
In a fourth aspect, an embodiment of the present invention provides a data decompression apparatus, where the apparatus includes: the device comprises a receiving module, a second obtaining module and a decompressing module; wherein,
the receiving module is used for receiving a current transmission message body sent by the sending terminal equipment through a preset message queue;
the second obtaining module is configured to obtain a current target message body and a decompression algorithm corresponding to the current target message body according to the current transmission message body;
and the decompression module is used for decompressing the current target message body into the current original message body according to the decompression algorithm corresponding to the current target message body.
In the foregoing embodiment, the second obtaining module is specifically configured to decapsulate the content of the current transmission message body, and obtain the current target message body from the decapsulated content of the current transmission message body; decapsulating the header of the current transmission message body, and acquiring an identifier of a compression algorithm corresponding to the current target message body from the decapsulated header of the current transmission message body; and determining the decompression algorithm corresponding to the current target message body according to the identifier of the compression algorithm corresponding to the current target message body.
In the foregoing embodiment, the decompression module is specifically configured to obtain, in a header of a decapsulated current transmission message body, a size of the current target message body and a signature of the current target message body; calculating the size of the current target message body and the signature of the current target message body according to a predetermined calculation method; and if the obtained size of the current target message body is the same as the calculated size of the current target message body and the obtained signature of the current target message body is the same as the calculated signature of the current target message body, executing the decompression algorithm corresponding to the current target message body, and decompressing the current target message body into a current original message body.
In a fifth aspect, an embodiment of the present invention provides an electronic device, including:
one or more processors;
a memory for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement the data compression method or the data decompression method according to any embodiment of the present invention.
In a sixth aspect, an embodiment of the present invention provides a storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements a data compression method or a data decompression method according to any embodiment of the present invention.
The embodiment of the invention provides a data compression method, a data decompression method, a data compression device, an electronic device and a storage medium, wherein a sending end device firstly obtains a current original message body in a current data stream according to a predetermined message body size; then generating a current transmission message body corresponding to the current original message body according to a compression algorithm corresponding to the current original message body; then sending the current transmission message body to the receiving end equipment through a preset message queue; the receiving end equipment receives a current transmission message body sent by the sending end equipment through a preset message queue; then, acquiring a current target message body and a decompression algorithm corresponding to the current target message body according to the current transmission message body; and decompressing the current target message body into the current original message body according to the decompression algorithm corresponding to the current target message body. That is to say, in the technical solution of the present invention, the sending end device generates the current transmission message body corresponding to the current original message body first; and then the current transmission message body is sent to the receiving end equipment through a preset message queue. In the existing data compression method, the message queue usually compresses the current original message body according to a general compression algorithm; in the existing data decompression method, the receiving end device usually decompresses the current transmission message body according to a general decompression algorithm; neither compression nor decompression is ideal; moreover, most message queues do not support data compression, and the application range is relatively limited. Therefore, compared with the prior art, the data compression and decompression method, the data compression and decompression device, the electronic equipment and the storage medium provided by the embodiment of the invention can effectively improve the data compression effect and have stronger universality; moreover, the technical scheme of the embodiment of the invention is simple and convenient to realize, convenient to popularize and wider in application range.
Drawings
Fig. 1 is a schematic flow chart of a data compression method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a data compression method according to a second embodiment of the present invention;
fig. 3 is a schematic flowchart of a data decompression method according to a third embodiment of the present invention;
fig. 4 is a flowchart illustrating a data decompression method according to a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of a data compression apparatus according to a fifth embodiment of the present invention;
fig. 6 is a schematic structural diagram of a data decompression apparatus according to a sixth embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to a seventh embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some but not all of the relevant aspects of the present invention are shown in the drawings.
Example one
Fig. 1 is a flowchart of a data compression method according to an embodiment of the present invention, where the method may be executed by a data compression apparatus or a sending end device, where the apparatus or the sending end device may be implemented by software and/or hardware, and the apparatus or the sending end device may be integrated in any intelligent device with a network communication function. As shown in fig. 1, the data compression method may include the steps of:
s101, acquiring a current original message body in a current data stream according to a predetermined message body size.
In a specific embodiment of the present invention, the sending end device may obtain the current original message body in the current data stream according to a predetermined message body size. Specifically, the sending end device may receive each data packet in the current data stream in real time, and calculate the size of the received data packet in real time; when the size of the received data packet is equal to the predetermined threshold, the received data packet is encapsulated as a current original message body.
S102, generating a current transmission message body corresponding to the current original message body according to a compression algorithm corresponding to the current original message body; wherein the current transmission message body comprises: the header of the currently transmitted message body and the content of the currently transmitted message body.
In a specific embodiment of the present invention, the sending end device may generate a current transmission message body corresponding to the current original message body according to a compression algorithm corresponding to the current original message body; wherein the current transmission message body comprises: the header of the currently transmitted message body and the content of the currently transmitted message body. Specifically, the sending end device may determine a compression algorithm corresponding to the current original message body according to the data type of the current original message body; and the identification of the compression algorithm is encapsulated in the header of the current transmission message body; compressing the current original message body into a current target message body according to a compression algorithm corresponding to the current original message body; and encapsulates the current target message body in the contents of the current transfer message body.
S103, sending the current transmission message body to the receiving end equipment through a preset message queue.
In a specific embodiment of the present invention, the sending-end device may send the current transmission message body to the receiving-end device through a preset message queue. A Message Queue (MQ) is a method of communicating applications to applications. Applications communicate by reading and writing messages to and from the queue without requiring a dedicated connection to link them. MQ is a typical representative of "producer-consumer" model, where a sending end device may continuously write a current transport message body in a message queue in the past, and a receiving end device may continuously read the current transport message body in the message queue; among the more mature MQ, there are IBM WEBSPHERE MQ, etc.
In the data compression method provided by the embodiment of the invention, sending end equipment firstly obtains a current original message body in a current data stream according to a predetermined message body size; then generating a current transmission message body corresponding to the current original message body according to a compression algorithm corresponding to the current original message body; and then the current transmission message body is sent to the receiving end equipment through a preset message queue. That is to say, in the technical solution of the present invention, the sending end device generates the current transmission message body corresponding to the current original message body first; and then the current transmission message body is sent to the receiving end equipment through a preset message queue. In the existing data compression method, the sending end equipment sends the current original message body to the message queue; the message queue compresses the current original message body into a current transmission message body and then sends the current transmission message body to the receiving end equipment. Moreover, the message queue usually compresses the current original message body according to a general compression algorithm; the compression effect is not ideal; moreover, most message queues do not support data compression, and the application range is relatively limited. Therefore, compared with the prior art, the data compression method provided by the embodiment of the invention can effectively improve the data compression effect and has stronger universality; moreover, the technical scheme of the embodiment of the invention is simple and convenient to realize, convenient to popularize and wider in application range. Therefore, compared with the prior art, the data compression method provided by the embodiment of the invention can effectively improve the data compression effect and has stronger universality; moreover, the technical scheme of the embodiment of the invention is simple and convenient to realize, convenient to popularize and wider in application range.
Example two
Fig. 2 is a schematic flow chart of a data compression method according to a second embodiment of the present invention. As shown in fig. 2, the data compression method may include the steps of:
s201, acquiring a current original message body in a current data stream according to a predetermined message body size.
In a specific embodiment of the present invention, the sending end device may obtain the current original message body in the current data stream according to a predetermined message body size. Specifically, the sending end device may receive each data packet in the current data stream in real time, and calculate the size of the received data packet in real time; when the size of the received data packet is equal to the predetermined threshold, the received data packet is encapsulated as a current original message body.
S202, determining a compression algorithm corresponding to the current original message body according to the data type of the current original message body; and encapsulates the identification of the compression algorithm in the header of the current transmitted message body.
In a specific embodiment of the present invention, the sending end device may determine, according to the data type of the current original message body, a compression algorithm corresponding to the current original message body; and encapsulates the identification of the compression algorithm in the header of the current transmitted message body. Specifically, the sending end device may store a correspondence between a data type of the current original message body and an identifier of the compression algorithm; if the data type of the current original message body is the first data type, the sending end device may determine that the compression algorithm corresponding to the current original message body is the first type of compression algorithm; if the data type of the current original message body is the second data type, the sending end device may determine that the compression algorithm corresponding to the current original message body is the second type of compression algorithm; and so on.
S203, compressing the current original message body into a current target message body according to the compression algorithm corresponding to the current original message body; and encapsulates the current target message body in the contents of the current transfer message body.
In a specific embodiment of the present invention, the sending end device may compress the current original message body into the current target message body according to a compression algorithm corresponding to the current original message body; and encapsulates the current target message body in the contents of the current transfer message body. Specifically, if the data type of the current original message body is the first data type, the sending end device may compress the current original message body into the current target message body according to the first type of compression algorithm; encapsulating the current target message body in the content of the current transmission message body; if the data type of the current original message body is the second data type, the sending end equipment can compress the current original message body into a current target message body according to a second type compression algorithm; encapsulating the current target message body in the content of the current transmission message body; and so on.
Preferably, in the specific embodiment of the present invention, the sending end device may further calculate the size of the current target message body and the signature of the current target message body according to a predetermined calculation method; the size of the current target message body and the signature of the current target message body are encapsulated into the header of the current transfer message body. Specifically, the sending end device may encapsulate the compression identifier in a first byte of a header of a current transmission message body; if the compression identifier is 0, the compression is not started by the sending terminal equipment; if the compression identifier is 1, the compression of the sending end equipment is started; the identifier of the compression algorithm corresponding to the current original message body can be encapsulated in the second byte of the header of the current transmission message body; the size of the current target message body and the signature of the current target message body may also be encapsulated in the third through sixth bytes of the header of the current transfer message body.
And S204, sending the current transmission message body to the receiving end equipment through a preset message queue.
In a specific embodiment of the present invention, the sending-end device may send the current transmission message body to the receiving-end device through a preset message queue. Specifically, the sending end device may continuously write the current transmission message body in the past message queue, and the receiving end device may continuously read the current transmission message body in the message queue.
In the data compression method provided by the embodiment of the invention, sending end equipment firstly obtains a current original message body in a current data stream according to a predetermined message body size; then generating a current transmission message body corresponding to the current original message body according to a compression algorithm corresponding to the current original message body; and then the current transmission message body is sent to the receiving end equipment through a preset message queue. That is to say, in the technical solution of the present invention, the sending end device generates the current transmission message body corresponding to the current original message body first; and then the current transmission message body is sent to the receiving end equipment through a preset message queue. In the existing data compression method, the sending end equipment sends the current original message body to the message queue; the message queue compresses the current original message body into a current transmission message body and then sends the current transmission message body to the receiving end equipment. Moreover, the message queue usually compresses the current original message body according to a general compression algorithm; the compression effect is not ideal; moreover, most message queues do not support data compression, and the application range is relatively limited. Therefore, compared with the prior art, the data compression method provided by the embodiment of the invention can effectively improve the data compression effect and has stronger universality; moreover, the technical scheme of the embodiment of the invention is simple and convenient to realize, convenient to popularize and wider in application range. Therefore, compared with the prior art, the data compression method provided by the embodiment of the invention can effectively improve the data compression effect and has stronger universality; moreover, the technical scheme of the embodiment of the invention is simple and convenient to realize, convenient to popularize and wider in application range.
EXAMPLE III
Fig. 3 is a flowchart illustrating a data decompression method according to a third embodiment of the present invention. As shown in fig. 3, the data decompression method may include the steps of:
s301, receiving a current transmission message body sent by the sending terminal equipment through a preset message queue.
In a specific embodiment of the present invention, the receiving end device may receive the current transmission message body sent by the sending end device through a preset message queue. Specifically, the sending end device may continuously write the current transmission message body in the past message queue, and the receiving end device may continuously read the current transmission message body in the message queue.
S302, acquiring a current target message body and a decompression algorithm corresponding to the current target message body according to the current transmission message body.
In a specific embodiment of the present invention, the receiving end device may obtain the current target message body and a decompression algorithm corresponding to the current target message body according to the current transmission message body. Specifically, the receiving end device may store a corresponding relationship between the data type of the current target message body and the identifier of the decompression algorithm; if the data type of the current target message body is the first data type, the receiving end device may determine that the decompression algorithm corresponding to the current target message body is the first type of decompression algorithm; if the data type of the current target message body is the second data type, the receiving end device may determine that the decompression algorithm corresponding to the current target message body is the second type of decompression algorithm; and so on.
S303, decompressing the current target message body into the current original message body according to the decompression algorithm corresponding to the current target message body.
In a specific embodiment of the present invention, the receiving end device may decompress the current target message body into the current original message body according to a decompression algorithm corresponding to the current target message body. Specifically, if the data type of the current target message body is the first data type, the receiving end device may decompress the current target message body into the current original message body according to the first type of decompression algorithm; if the data type of the current target message body is the second data type, the receiving end device can decompress the current target message body into the current original message body according to the second type decompression algorithm; and so on.
In the data decompression method provided by the embodiment of the invention, the receiving end equipment receives the current transmission message body sent by the sending end equipment through the preset message queue; then, acquiring a current target message body and a decompression algorithm corresponding to the current target message body according to the current transmission message body; and decompressing the current target message body into the current original message body according to the decompression algorithm corresponding to the current target message body. That is to say, in the technical solution of the present invention, the receiving end device receives the current transmission message body compressed by the sending end device through the message queue, instead of receiving the current transmission message body compressed by the message queue. In the existing data decompression method, the receiving end device receives the current transmission message body compressed by the message queue, and then decompresses the current transmission message body into the current original message body. Moreover, the receiving end device usually decompresses the current transmission message body according to a general decompression algorithm; the decompression effect is not ideal; moreover, most message queues do not support data compression, and the application range is relatively limited. Therefore, compared with the prior art, the data decompression method provided by the embodiment of the invention can effectively improve the data compression effect and has stronger universality; moreover, the technical scheme of the embodiment of the invention is simple and convenient to realize, convenient to popularize and wider in application range.
Example four
Fig. 4 is a flowchart illustrating a data decompression method according to a fourth embodiment of the present invention. As shown in fig. 4, the data decompression method may include the steps of:
s401, receiving a current transmission message body sent by the sending terminal equipment through a preset message queue.
In a specific embodiment of the present invention, the receiving end device may receive the current transmission message body sent by the sending end device through a preset message queue. Specifically, the sending end device may continuously write the current transmission message body in the past message queue, and the receiving end device may continuously read the current transmission message body in the message queue.
S402, decapsulating the content of the current transmission message body, and acquiring the current target message body from the decapsulated content of the current transmission message body.
In the specific embodiment of the present invention, the receiving end device may decapsulate the content of the current transmission message body, and obtain the current target message body from the decapsulated content of the current transmission message body. Specifically, the receiving end device may decapsulate the content of the current transmission message body according to a predetermined encapsulation format of the current transmission message body, and obtain the current target message body from the decapsulated content of the current transmission message body.
S403, decapsulating the header of the current transmission message body, and acquiring the identifier of the compression algorithm corresponding to the current target message body from the decapsulated header of the current transmission message body; and determining the decompression algorithm corresponding to the current target message body according to the identifier of the compression algorithm corresponding to the current target message body.
In a specific embodiment of the present invention, the receiving end device may decapsulate the header of the current transmission message body, and obtain, in the decapsulated header of the current transmission message body, an identifier of a compression algorithm corresponding to the current target message body; and determining the decompression algorithm corresponding to the current target message body according to the identifier of the compression algorithm corresponding to the current target message body. Specifically, the receiving end device may decapsulate the header of the current transmission message body according to a predetermined encapsulation format of the current transmission message body, and obtain the compression identifier in the first byte of the decapsulated header of the current transmission message body; if the compression identifier is 0, the compression is not started by the sending terminal equipment; if the compression identifier is 1, the compression of the sending end equipment is started; the receiving end equipment can also obtain the identifier of the compression algorithm corresponding to the current target message body in the second byte of the head of the decapsulated current transmission message body; determining a decompression algorithm corresponding to the current target message body according to the identifier of the compression algorithm corresponding to the current target message body; the receiving end equipment can also obtain the size of the current target message body and the signature of the current target message body in the third byte to the sixth byte of the head of the decapsulated current transmission message body; meanwhile, the size of the current target message body and the signature of the current target message body are calculated according to a predetermined calculation method; and if the obtained size of the current target message body is the same as the calculated size of the current target message body and the obtained signature of the current target message body is the same as the calculated signature of the current target message body, executing an operation of decompressing the current target message body into a current original message body according to a decompression algorithm corresponding to the current target message body.
S404, decompressing the current target message body into the current original message body according to the decompression algorithm corresponding to the current target message body.
In a specific embodiment of the present invention, the receiving end device may decompress the current target message body into the current original message body according to a decompression algorithm corresponding to the current target message body. Specifically, if the data type of the current target message body is the first data type, the receiving end device may decompress the current target message body into the current original message body according to the first type of decompression algorithm; if the data type of the current target message body is the second data type, the receiving end device can decompress the current target message body into the current original message body according to the second type decompression algorithm; and so on.
In the data decompression method provided by the embodiment of the invention, the receiving end equipment receives the current transmission message body sent by the sending end equipment through the preset message queue; then, acquiring a current target message body and a decompression algorithm corresponding to the current target message body according to the current transmission message body; and decompressing the current target message body into the current original message body according to the decompression algorithm corresponding to the current target message body. That is to say, in the technical solution of the present invention, the receiving end device receives the current transmission message body compressed by the sending end device through the message queue, instead of receiving the current transmission message body compressed by the message queue. In the existing data decompression method, the receiving end device receives the current transmission message body compressed by the message queue, and then decompresses the current transmission message body into the current original message body. Moreover, the receiving end device usually decompresses the current transmission message body according to a general decompression algorithm; the decompression effect is not ideal; moreover, most message queues do not support data compression, and the application range is relatively limited. Therefore, compared with the prior art, the data decompression method provided by the embodiment of the invention can effectively improve the data compression effect and has stronger universality; moreover, the technical scheme of the embodiment of the invention is simple and convenient to realize, convenient to popularize and wider in application range.
EXAMPLE five
Fig. 5 is a schematic structural diagram of a data compression apparatus according to a fifth embodiment of the present invention. As shown in fig. 5, the data compression apparatus according to the embodiment of the present invention may include: a first obtaining module 501, a compressing module 502 and a sending module 503; wherein,
the first obtaining module 501 is configured to obtain a current original message body in a current data stream according to a predetermined message body size;
the compression module 502 is configured to generate a current transmission message body corresponding to the current original message body according to a compression algorithm corresponding to the current original message body; wherein the current transfer message body comprises: a header of the current transfer message body and a content of the current transfer message body;
the sending module 503 is configured to send the current transmission message body to a receiving end device through a preset message queue.
Further, the compressing module 502 is specifically configured to determine a compression algorithm corresponding to the current original message body according to the data type of the current original message body; and encapsulating the identity of the compression algorithm in the header of the current body of transmitted messages; compressing the current original message body into a current target message body according to a compression algorithm corresponding to the current original message body; and encapsulating the current target message body in the content of the current transfer message body.
Further, the compressing module 502 is further configured to calculate the size of the current target message body and the signature of the current target message body according to a predetermined calculation method; encapsulating the size of the current target message body and the signature of the current target message body into a header of the current transfer message body.
The data compression device can execute the method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. For details of the technique not described in detail in this embodiment, reference may be made to the data compression method provided in any embodiment of the present invention.
EXAMPLE six
Fig. 6 is a schematic structural diagram of a data decompression device according to a sixth embodiment of the present invention. As shown in fig. 6, the data decompression apparatus according to the embodiment of the present invention may include: a receiving module 601, a second obtaining module 602 and a decompressing module 603; wherein,
the receiving module 601 is configured to receive a current transmission message body sent by sending end equipment through a preset message queue;
the second obtaining module 602 is configured to obtain a current target message body and a decompression algorithm corresponding to the current target message body according to the current transmission message body;
the decompressing module 603 is configured to decompress the current target message body into the current original message body according to the decompressing algorithm corresponding to the current target message body.
Further, the second obtaining module 602 is specifically configured to decapsulate the content of the current transmission message body, and obtain the current target message body from the decapsulated content of the current transmission message body; decapsulating the header of the current transmission message body, and acquiring an identifier of a compression algorithm corresponding to the current target message body from the decapsulated header of the current transmission message body; and determining the decompression algorithm corresponding to the current target message body according to the identifier of the compression algorithm corresponding to the current target message body.
Further, the decompression module 603 is specifically configured to obtain the size of the current target message body and the signature of the current target message body in the header of the decapsulated current transmission message body; calculating the size of the current target message body and the signature of the current target message body according to a predetermined calculation method; and if the obtained size of the current target message body is the same as the calculated size of the current target message body and the obtained signature of the current target message body is the same as the calculated signature of the current target message body, executing the decompression algorithm corresponding to the current target message body, and decompressing the current target message body into a current original message body.
The data decompression device can execute the method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. For details of the technique not described in detail in this embodiment, reference may be made to the data decompression method provided in any embodiment of the present invention.
EXAMPLE seven
Fig. 7 is a schematic structural diagram of an electronic device according to a seventh embodiment of the present invention. FIG. 7 illustrates a block diagram of an exemplary electronic device suitable for use in implementing embodiments of the present invention. The electronic device 12 shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiment of the present invention.
As shown in FIG. 7, electronic device 12 is embodied in the form of a general purpose computing device. The components of electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.
Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, micro-channel architecture (MAC) bus, enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.
Electronic device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by electronic device 12 and includes both volatile and nonvolatile media, removable and non-removable media.
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. The electronic device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 7, and commonly referred to as a "hard drive"). Although not shown in FIG. 7, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.
Electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with electronic device 12, and/or with any devices (e.g., network card, modem, etc.) that enable electronic device 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, the electronic device 12 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet) via the network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via the bus 18. It should be appreciated that although not shown in FIG. 7, other hardware and/or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processing unit 16 executes programs stored in the system memory 28 to perform various functional applications and data processing, such as implementing data compression or data decompression methods provided by embodiments of the present invention.
Example eight
The eighth embodiment of the invention provides a computer storage medium.
The computer-readable storage media of embodiments of the invention may take any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims (14)

1. A data compression method is applied to a sending terminal device, and the method comprises the following steps:
acquiring a current original message body in a current data stream according to a predetermined message body size;
generating a current transmission message body corresponding to the current original message body according to a compression algorithm corresponding to the current original message body; wherein the current transfer message body comprises: a header of the current transfer message body and a content of the current transfer message body;
and sending the current transmission message body to receiving end equipment through a preset message queue.
2. The method according to claim 1, wherein said generating a current transmission message body corresponding to the current original message body according to the compression algorithm corresponding to the current original message body comprises:
determining a compression algorithm corresponding to the current original message body according to the data type of the current original message body; and encapsulating the identity of the compression algorithm in the header of the current body of transmitted messages;
compressing the current original message body into a current target message body according to a compression algorithm corresponding to the current original message body; and encapsulating the current target message body in the content of the current transfer message body.
3. The method of claim 2, further comprising:
calculating the size of the current target message body and the signature of the current target message body according to a predetermined calculation method;
encapsulating the size of the current target message body and the signature of the current target message body into a header of the current transfer message body.
4. A data decompression method is applied to a receiving end device, and comprises the following steps:
receiving a current transmission message body sent by sending end equipment through a preset message queue;
acquiring a current target message body and a decompression algorithm corresponding to the current target message body according to the current transmission message body;
and decompressing the current target message body into a current original message body according to a decompression algorithm corresponding to the current target message body.
5. The method according to claim 4, wherein said obtaining a current target message body and a decompression algorithm corresponding to the current target message body according to the current transmission message body comprises:
decapsulating the content of the current transmission message body, and acquiring the current target message body from the decapsulated content of the current transmission message body;
decapsulating the header of the current transmission message body, and acquiring an identifier of a compression algorithm corresponding to the current target message body from the decapsulated header of the current transmission message body; and determining the decompression algorithm corresponding to the current target message body according to the identifier of the compression algorithm corresponding to the current target message body.
6. The method according to claim 5, wherein before said decompressing said current target message body into a current original message body according to a decompression algorithm corresponding to said current target message body, said method further comprises:
acquiring the size of the current target message body and the signature of the current target message body in the head of the decapsulated current transmission message body;
calculating the size of the current target message body and the signature of the current target message body according to a predetermined calculation method;
and if the obtained size of the current target message body is the same as the calculated size of the current target message body and the obtained signature of the current target message body is the same as the calculated signature of the current target message body, executing the decompression algorithm corresponding to the current target message body, and decompressing the current target message body into a current original message body.
7. An apparatus for compressing data, the apparatus comprising: the device comprises a first acquisition module, a compression module and a sending module; wherein,
the first obtaining module is used for obtaining a current original message body in a current data stream according to a predetermined message body size;
the compression module is used for generating a current transmission message body corresponding to the current original message body according to a compression algorithm corresponding to the current original message body; wherein the current transfer message body comprises: a header of the current transfer message body and a content of the current transfer message body;
and the sending module is used for sending the current transmission message body to receiving end equipment through a preset message queue.
8. The apparatus of claim 7, wherein:
the compression module is specifically configured to determine a compression algorithm corresponding to the current original message body according to the data type of the current original message body; and encapsulating the identity of the compression algorithm in the header of the current body of transmitted messages; compressing the current original message body into a current target message body according to a compression algorithm corresponding to the current original message body; and encapsulating the current target message body in the content of the current transfer message body.
9. The apparatus of claim 8, wherein the compression module is further configured to calculate a size of the current target message body and a signature of the current target message body according to a predetermined calculation method; encapsulating the size of the current target message body and the signature of the current target message body into a header of the current transfer message body.
10. An apparatus for decompressing data, the apparatus comprising: the device comprises a receiving module, a second obtaining module and a decompressing module; wherein,
the receiving module is used for receiving a current transmission message body sent by the sending terminal equipment through a preset message queue;
the second obtaining module is configured to obtain a current target message body and a decompression algorithm corresponding to the current target message body according to the current transmission message body;
and the decompression module is used for decompressing the current target message body into the current original message body according to the decompression algorithm corresponding to the current target message body.
11. The apparatus of claim 10, wherein:
the second obtaining module is specifically configured to decapsulate the content of the current transmission message body, and obtain the current target message body from the decapsulated content of the current transmission message body; decapsulating the header of the current transmission message body, and acquiring an identifier of a compression algorithm corresponding to the current target message body from the decapsulated header of the current transmission message body; and determining the decompression algorithm corresponding to the current target message body according to the identifier of the compression algorithm corresponding to the current target message body.
12. The apparatus according to claim 11, wherein the decompression module is specifically configured to obtain a size of the current target message body and a signature of the current target message body in a header of the decapsulated current transport message body; calculating the size of the current target message body and the signature of the current target message body according to a predetermined calculation method; and if the obtained size of the current target message body is the same as the calculated size of the current target message body and the obtained signature of the current target message body is the same as the calculated signature of the current target message body, executing the decompression algorithm corresponding to the current target message body, and decompressing the current target message body into a current original message body.
13. An electronic device, comprising:
one or more processors;
a memory for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the data compression method of any one of claims 1 to 3 or the data decompression method of any one of claims 4 to 6.
14. A storage medium on which a computer program is stored, which program, when executed by a processor, implements a data compression method as claimed in any one of claims 1 to 3 or a data decompression method as claimed in any one of claims 4 to 6.
CN201910320309.1A 2019-04-19 2019-04-19 Data compression method, data decompression method, data compression device, data decompression device, electronic equipment and storage medium Pending CN111835686A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910320309.1A CN111835686A (en) 2019-04-19 2019-04-19 Data compression method, data decompression method, data compression device, data decompression device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910320309.1A CN111835686A (en) 2019-04-19 2019-04-19 Data compression method, data decompression method, data compression device, data decompression device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN111835686A true CN111835686A (en) 2020-10-27

Family

ID=72912337

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910320309.1A Pending CN111835686A (en) 2019-04-19 2019-04-19 Data compression method, data decompression method, data compression device, data decompression device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN111835686A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113609429A (en) * 2021-08-10 2021-11-05 广州至真信息科技有限公司 Data processing method and device based on message queue, computer equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102075287A (en) * 2010-11-22 2011-05-25 浪潮(北京)电子信息产业有限公司 Data processing method and device
CN106936441A (en) * 2017-02-27 2017-07-07 深圳国泰安教育技术股份有限公司 A kind of data compression method and device
CN109194647A (en) * 2018-08-30 2019-01-11 北京金山安全软件有限公司 Data transmission method and device, electronic equipment and storage medium
CN109634529A (en) * 2018-12-12 2019-04-16 浪潮(北京)电子信息产业有限公司 A kind of data compression method and decompressing method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102075287A (en) * 2010-11-22 2011-05-25 浪潮(北京)电子信息产业有限公司 Data processing method and device
CN106936441A (en) * 2017-02-27 2017-07-07 深圳国泰安教育技术股份有限公司 A kind of data compression method and device
CN109194647A (en) * 2018-08-30 2019-01-11 北京金山安全软件有限公司 Data transmission method and device, electronic equipment and storage medium
CN109634529A (en) * 2018-12-12 2019-04-16 浪潮(北京)电子信息产业有限公司 A kind of data compression method and decompressing method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113609429A (en) * 2021-08-10 2021-11-05 广州至真信息科技有限公司 Data processing method and device based on message queue, computer equipment and storage medium
CN113609429B (en) * 2021-08-10 2024-04-05 广州至真信息科技有限公司 Data processing method and device based on message queue, computer equipment and storage medium

Similar Documents

Publication Publication Date Title
CN109194647B (en) Data transmission method and device, electronic equipment and storage medium
US20210089492A1 (en) Rdma data sending and receiving methods, electronic device, and readable storage medium
CN110888862A (en) Data storage method, data query method, data storage device, data query device, server and storage medium
CN109756568A (en) Processing method, equipment and the computer readable storage medium of file
CN112887429A (en) Data transmission method and device, electronic equipment and storage medium
US11671678B2 (en) Method and device, equipment, and storage medium for data processing
US20120166585A1 (en) Apparatus and method for accelerating virtual desktop
US20090182798A1 (en) Method and apparatus to improve the effectiveness of system logging
CN112491421A (en) Analysis method, device, equipment and medium for log file compression package
CN112671771A (en) Data transmission method, device, electronic equipment and medium
CN114020529A (en) Backup method and device of flow table data, network equipment and storage medium
CN112311720A (en) Data transmission method and device
CN111835686A (en) Data compression method, data decompression method, data compression device, data decompression device, electronic equipment and storage medium
WO2021036189A1 (en) Rdma data sending and receiving methods, electronic device and readable storage medium
CN110889880A (en) Map processing method, device, equipment and storage medium
CN108494700B (en) Cross-link data transmission method and device, computer equipment and storage medium
CN107992457B (en) Information conversion method, device, terminal equipment and storage medium
CN108259958B (en) Remote control key response method, device, equipment and storage medium
CN115499393A (en) TCP connection message processing method based on network interface chip
CN115348320A (en) Communication data conversion method and device and electronic equipment
CN113849449A (en) Communication system and information interaction method, device and medium
CN111639055B (en) Differential packet calculation method, differential packet calculation device, differential packet calculation equipment and storage medium
CN111080728A (en) Map processing method, device, equipment and storage medium
CN114338798B (en) Data transmission method, device, system, equipment and storage medium
CN109032724B (en) Method, device, terminal and storage medium for processing user-defined bullet screen message

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20201027