CN110611680B - Data packet compression control method and device - Google Patents
Data packet compression control method and device Download PDFInfo
- Publication number
- CN110611680B CN110611680B CN201910912810.7A CN201910912810A CN110611680B CN 110611680 B CN110611680 B CN 110611680B CN 201910912810 A CN201910912810 A CN 201910912810A CN 110611680 B CN110611680 B CN 110611680B
- Authority
- CN
- China
- Prior art keywords
- data
- compression
- server
- data packet
- terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/04—Protocols for data compression, e.g. ROHC
Abstract
The invention provides a data packet compression control method and a device, which are used for monitoring the output flow of a data packet sent to a terminal by a server, analyzing the output flow of the data packets in the monitoring process to obtain the ratio of the data group of each data packet sent by the server to the terminal, selecting a target data group from all the data groups based on the ratio of the data group of each data packet, obtaining a compression parameter based on the data quantity range of the target data group, so that the server and/or the terminal can compress the data packet based on the compression parameters, when the output flow of the data packet sent by the server to the terminal changes, the corresponding compression parameters based on the data size range of the target data set are also adjusted accordingly, thereby realizing real-time adjustment of compression parameters based on the output flow of the data packet sent by the server to the terminal, so that the data packets compressed by the server and/or the terminal can change along with the change of the output flow.
Description
Technical Field
The invention belongs to the technical field of data processing, and particularly relates to a data packet compression control method and device.
Background
At present, a storage space occupied by a data packet can be reduced by compressing the data packet through a compression algorithm, the data packet is a data unit in TCP/IP (Transmission Control Protocol/Internet Protocol ) communication Transmission, for example, when a terminal communicates with a server based on TCP/IP, the terminal can compress the data packet transmitted by the terminal through at least one algorithm of a zlib compression algorithm, an lz4 compression algorithm, a snapshot compression algorithm, and the like, but at present, when compressing the data packet, usually, a user manually compresses one or more data packets, or a compression threshold is set, and if the data amount of the data packet received by the terminal reaches the compression threshold, the data packet whose data amount reaches the compression threshold is compressed.
Disclosure of Invention
In view of the above, an object of the present invention is to provide a method and an apparatus for controlling packet compression, which are used to adjust compression parameters in real time based on the outgoing flow of a packet sent by a server to a terminal. The technical scheme is as follows:
the invention provides a data packet compression control method, which comprises the following steps:
monitoring the output flow of a data packet sent to a terminal by a server, wherein the output flow of the data packet represents the total data volume of the data packet sent to the terminal by the server in unit time;
analyzing the output flow of the data packets to obtain the ratio of data groups of the data packets sent to the terminal by the server, wherein any data group corresponds to a data quantity range, and the data quantity ranges of the data groups are different;
selecting a target data group from all data groups based on the ratio of the data group in which each data packet is positioned;
and obtaining a compression parameter based on the data volume range of the target data group, so that the server and/or the terminal compresses the data packet based on the compression parameter.
Preferably, the selecting a target data group from all data groups based on the ratio of the data group in which each data packet is located includes:
obtaining a data set with the largest proportion sum based on the proportion of the data set of each data packet and the data volume range of the data set of each data packet, wherein the data volume range of each data set in the data set is larger than the data volume range of the rest data sets;
and selecting the data group with the minimum data volume range from the data group set as the target data group.
Preferably, the obtaining of the compression parameter based on the data size range of the target data set includes:
and selecting a data volume from the data volume range of the target data set as a compression threshold value in the compression parameter.
Preferably, the method further comprises:
displaying a control interface for setting compression parameters in a display area;
and acquiring compression time in compression parameters set in the control interface by a user and/or a service partition to which a data packet to be compressed belongs, wherein the compression time is used for indicating a time range in which the server and/or the terminal performs compression processing on the data packet.
Preferably, the method further comprises:
receiving a data compression response sent by the server and/or the terminal;
and if the data compression response indicates that the service of the server and/or the terminal is abnormal, the data packet is prohibited from being compressed.
The invention also provides a data packet compression control method, which comprises the following steps:
receiving a compression configuration instruction, wherein the compression configuration instruction carries compression parameters, and the compression parameters are obtained by the data packet compression control method;
compressing and transmitting the data packet to be compressed matched with the compression parameters;
encrypting the compression parameters to obtain encrypted compression parameters;
and sending the encrypted compression parameters to a terminal.
The invention also provides a data packet compression control method, which comprises the following steps:
receiving a compression configuration instruction sent by a server, wherein the compression configuration instruction carries encrypted compression parameters;
decrypting the encrypted compression parameters to obtain compression parameters;
and after compressing the data packet to be compressed matched with the compression parameters, sending the compressed data packet to other terminals through the transfer equipment.
The present invention also provides a data packet compression control apparatus, comprising:
the monitoring unit is used for monitoring the output flow of a data packet sent to a terminal by a server, wherein the output flow of the data packet represents the total data volume of the data packet sent to the terminal by the server in unit time;
the analysis unit is used for analyzing the output flow of the data packets to obtain the proportion of data groups where the data packets are sent to the terminal by the server, wherein any data group corresponds to a data quantity range, and the data quantity ranges of the data groups are different;
the selecting unit is used for selecting a target data group from all the data groups based on the ratio of the data group in which each data packet is positioned;
and the compression parameter obtaining unit is used for obtaining compression parameters based on the data volume range of the target data group so as to enable the server and/or the terminal to compress the data packet based on the compression parameters.
The present invention also provides a management server, comprising: a processor and a memory;
the processor is configured to monitor an outgoing flow of a data packet sent by a server to a terminal, where the outgoing flow of the data packet represents a total data amount of the data packet sent by the server to the terminal in a unit time; analyzing the output flow of the data packets to obtain the ratio of data groups of the data packets sent to the terminal by the server, wherein any data group corresponds to a data quantity range, and the data quantity ranges of the data groups are different; based on the ratio of the data groups in which the data packets are located, selecting a target data group from all the data groups, and based on the data volume range of the target data group, obtaining a compression parameter so that the server and/or the terminal compresses the data packets based on the compression parameter;
the memory is used for storing the compression parameters.
The present invention also provides a terminal, including: a communication interface and a processor;
the communication interface is used for receiving a compression configuration instruction sent by the server, wherein the compression configuration instruction carries encrypted compression parameters;
the processor is used for decrypting the encrypted compression parameters to obtain compression parameters; and after compressing the data packet to be compressed matched with the compression parameters, sending the compressed data packet to other terminals through the transfer equipment.
By means of the technical scheme, the output flow of the data packets sent by the server to the terminal is monitored, the output flow of the data packets is analyzed in the monitoring process, the proportion of the data group where each data packet sent by the server to the terminal is located is obtained, the target data group is selected from all the data groups based on the proportion of the data group where each data packet is located, and the compression parameters are obtained based on the data quantity range of the target data group, so that the server and/or the terminal can compress the data packets based on the compression parameters. The output flow of the data packet sent by the server to the terminal can change along with at least one factor of a communication protocol based on the server and a service provided by the server, a target data group selected and taken out after the output flow of the data packet changes can also change, the data quantity ranges of different data groups are different, and the corresponding compression parameters obtained based on the data quantity ranges of the target data group can also be correspondingly adjusted, so that the compression parameters can be adjusted in real time based on the output flow of the data packet sent by the server to the terminal, and the data packet compressed by the server and/or the terminal can change along with the change of the output flow.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a method for controlling packet compression according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a data group ratio of a data packet according to an embodiment of the present invention;
fig. 3 is a flowchart of another method for controlling packet compression according to an embodiment of the present invention;
FIG. 4 is a schematic view of a control interface provided by an embodiment of the present invention;
FIG. 5 is a schematic diagram of a "big zone" option in a control interface provided by embodiments of the present invention;
fig. 6 is a signaling diagram of another method for controlling packet compression according to an embodiment of the present invention;
fig. 7 is a signaling diagram of another method for controlling packet compression according to an embodiment of the present invention;
fig. 8 is a schematic diagram illustrating comparison of average outgoing flow rates of a single transit device according to an embodiment of the present invention;
FIG. 9 is a schematic diagram illustrating the percentage of average out-flow reduction of a single transit apparatus according to an embodiment of the present invention;
fig. 10 and fig. 11 are schematic diagrams illustrating a comparison between the outgoing flow of a single transit device transmitting a compressed packet and an uncompressed packet during a peak period according to an embodiment of the present invention;
fig. 12 and fig. 13 are schematic diagrams illustrating a comparison of CPU utilization of a single forwarding device for transmitting compressed data packets and uncompressed data packets according to an embodiment of the present invention;
fig. 14 is a schematic structural diagram of a packet compression control apparatus according to an embodiment of the present invention;
fig. 15 is a schematic structural diagram of a management server according to an embodiment of the present invention;
fig. 16 is a schematic structural diagram of a terminal according to an embodiment of the present invention.
Detailed Description
Currently, when a server and/or a terminal compresses a data packet, a compression threshold is preset, so that the server and/or the terminal compresses the data packet when the data amount of the data packet is greater than the compression threshold, for example, the compression threshold may be 1024 bytes, and if the data amount of the data packet is greater than 1024 bytes, the data packet is compressed.
However, it is found through research that the data amount of the data packets exchanged between the server and the terminal may vary according to at least one of the communication protocol based on the two and the service provided by the server, for example, the variation is detected by the output flow of the data packets sent by the server to the terminal, and therefore, if the output flow of the data packets varies, the compression parameters for controlling the server and/or the terminal to compress also need to vary. The method for controlling packet compression provided in this embodiment may be applied to a management server, where the management server monitors the outgoing flow of a packet sent by the server to a terminal, or may be applied to a server of a packet sent to a terminal, or may be applied to a terminal, which is not limited in this embodiment.
The form of the management server may be a Manager Tool located on the cloud management platform, or software having a function of adjusting the compression parameters in real time, or a server capable of communicating with the server, and for this embodiment, the form of the management server is not described
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a flowchart of a method for controlling packet compression according to an embodiment of the present invention is shown, where the method includes the following steps:
101: the method includes monitoring an outgoing flow rate of a packet sent by a server to a terminal, where the outgoing flow rate of the packet indicates a total data amount of the packet sent by the server to the terminal in a unit time, that is, indicates how much data amount the server sends to the terminal in total in the unit time and a data amount of each packet sent by monitoring, and the setting of the unit time may be hourly, daily, or the like, which is not limited in this embodiment.
In this embodiment, the outgoing flow of the data packet sent by the server to the terminal may be continuously monitored, and the adjustment of the compression parameter may be triggered when the outgoing flow of the data packet changes to a certain extent, for example, when the outgoing flow of the data packet decreases the first data amount or the outgoing flow of the data packet increases the second data amount, the adjustment of the compression parameter may be triggered, where values of the first data amount and the second data amount may be adjusted according to actual application. Or the output flow of the data packet is obtained at every unit time in the process of continuously monitoring the output flow to trigger the adjustment of the compression parameters.
The way of monitoring the outgoing flow of the data packet may be: in the process that the server sends the data packet to the terminal, the data packet sent by the server is captured through a tcpdump (bump the traffic on a network) tool, and the data volume of the captured data packet each time is summed to obtain the output flow of the data packet, so that the output flow of the data packet not only can represent the total data volume of the data packet sent by the server to the terminal in unit time, but also can record the data volume of the data packet sent each time.
102: analyzing the outgoing flow of the data packets to obtain the ratio of the data group where each data packet is sent to the terminal by the server, for example, after capturing the data packets by the tcpdump tool, sequentially importing the data packets captured by the tcpdump tool into a wirereshik (a network packet analysis software), and obtaining the ratio of the data group where each data packet is located by the wirereshik.
Any one of the data sets corresponds to a data volume range, and the data volume ranges of the data sets are different. For any data group, the data amount range of the data group is represented by a maximum data amount and a minimum data amount, for example, the data amount range of one data group is represented as [ N, M ], N is the minimum data amount, and M is the maximum data amount, so that a data packet whose data amount is greater than or equal to the minimum data amount but less than or equal to the maximum data amount belongs to the data group whose data amount range is [ N, M ], and values of the maximum data amount and the minimum data amount of each data group are not limited in this embodiment.
Because the data volume ranges of different data groups are different (i.e. they are not overlapped with each other), after capturing any data packet, the data volume of the data packet is compared with the data volume range of each data group, so that the data group where the data packet is located can be determined, and the data group where each data packet is located captured in one compression parameter adjustment process can be obtained. And then obtaining the number of data packets and the total number of the data packets corresponding to each data group in the compression parameter adjustment process based on the data group where each data packet is located, and taking the ratio of the number of the data packets corresponding to each data group to the total number of the data packets as the ratio of the data group where each data packet is located.
As shown in fig. 2, 10 data sets are preset (Packet Lengths in fig. 2 indicate a data amount range corresponding to each data set), the number of data packets corresponding to each data set in one compression parameter adjustment process is represented by Count in fig. 2, and the duty ratio of each data set can be obtained based on the Count value of each data set, which is represented by Percent in fig. 2.
103: and selecting a target data group from all the data groups based on the ratio of the data groups in which the data packets are positioned, so that the compression parameters obtained based on the selected target data group can reduce the data volume of the transmitted data packets to the maximum extent.
In this embodiment, based on statistics of the ratio of the data group in which each data packet is located and the data volume range of each data group, feasibility of traffic optimization (i.e., data volume reduction by compressing the data packet) and expected optimization gains are analyzed, so that a target data group corresponding to a compression parameter that can maximize compression performance and compression gains is selected from each data group, and maximizing compression performance and compression gains can represent reducing the transmitted data volume without increasing resource consumption.
For example, one way to select a target data set is; and selecting the data group with the largest proportion as a target data group, taking the data group with the largest proportion as the data group with the data amount range of 40-79 by taking the data group with the largest proportion as an example in fig. 2, and taking the data group with the data amount range of 40-79 as the target data group.
Another way to select the target data set is: obtaining a data set with the largest proportion sum based on the proportion of the data set of each data packet and the data volume range of the data set of each data packet, selecting the data set with the smallest data volume range from the data set as a target data set, wherein the data volume range of each data set in the data set is larger than the data volume range of the rest data sets, and compared with the previous mode, the method can select the target data set from the proportion of the data sets and the data volume range of the data sets to improve the accuracy of selecting the target data set, so that compared with the previous mode, the accuracy can be improved because: if the target data set is selected only from the data set occupation ratio, there is a case where the data volume range is smaller although the data set occupation ratio is larger, and the compression effect and compression profit for the data packets belonging to such data set are not as good as those for the data packets in the data set larger than the data volume range, but the data volume range of the data set can be considered not only from the data set occupation ratio but also from the data volume range of the data set, thereby improving the accuracy of the target data set selection.
In the practical application process, the process of selecting the target data set from the proportion of the data set and the data quantity range of the data set is as follows:
and selecting each data group with the occupation ratio sum larger than the preset occupation ratio from all the data groups based on the maximum data amount or the minimum data amount of the data amount range from large to small, and forming a data group set by the selected data groups. Taking fig. 2 as an example, in a manner that the minimum data amount based on the data amount range is decreased from large to small, each data group with a ratio sum close to 50% (one value of a preset ratio) is selected from the 10 th data group (the data amount is 5120and grease), and, for example, the 9 th data group to the 5 th data group are selected from fig. 2 to form a data group set, as shown in a block in fig. 2 (the ratio of the 10 th data group is zero and therefore is ignored).
104: and obtaining a compression parameter based on the data volume range of the target data group, so that the server and/or the terminal compresses the data packet based on the compression parameter.
In this embodiment, the compression threshold in the compression parameter is mainly obtained based on the data amount range of the target data group, for example, one data amount is selected from the data amount range of the target data group as the compression threshold in the compression parameter, and specifically, the maximum data amount or the minimum data amount in the data amount range of the target data group may be used as the compression threshold. It is needless to say that the data amount range of the target data group may be used as the compression threshold to compress only the data packets belonging to the target data group, which may be the case when the data amount range of the target data group is small, and therefore, in order to solve this problem, the present embodiment preferably uses the maximum data amount or the minimum data amount as the compression threshold, for example, uses the minimum data amount of the target data group as the compression threshold to compress the data packets with the data amount larger than the compression threshold.
Still taking fig. 2 as an example, if the target data set is the data set with the data size range of 40 to 79 in fig. 2, the minimum data size 40 may be used as the compression threshold, and for example, if the target data set is the data set with the data size range of 160 to 319 in fig. 2, the minimum data size 160 may be used as the compression threshold to compress the data packets with the data size greater than the compression threshold.
In addition to adjusting the compression threshold in the compression parameter, the compression algorithm in the compression parameter may also be set, such as by applying the existing compression algorithm: the zlib compression algorithm, the lz4 compression algorithm and the snappy compression algorithm are used for carrying out experimental analysis on compression efficiency and resource consumption analysis, the compression algorithm with the highest compression rate is the zlib compression algorithm, and the actual resource overhead is within an acceptable range when the zlib compression algorithm is applied for compression, so that the zlib compression algorithm is used for compressing the data packet.
By means of the technical scheme, the output flow of the data packets sent by the server to the terminal is monitored, the output flow of the data packets is analyzed in the monitoring process, the proportion of the data group where each data packet sent by the server to the terminal is located is obtained, the target data group is selected from all the data groups based on the proportion of the data group where each data packet is located, and the compression parameters are obtained based on the data quantity range of the target data group, so that the server and/or the terminal can compress the data packets based on the compression parameters. The output flow of the data packet sent by the server to the terminal can change along with at least one factor of a communication protocol based on the server and a service provided by the server, a target data group selected and taken out after the output flow of the data packet changes can also change, the data quantity ranges of different data groups are different, and the corresponding compression parameters obtained based on the data quantity ranges of the target data group can also be correspondingly adjusted, so that the compression parameters can be adjusted in real time based on the output flow of the data packet sent by the server to the terminal, and the data packet compressed by the server and/or the terminal can change along with the change of the output flow.
In this embodiment, the compression time in the compression parameter and/or the service partition to which the data packet to be compressed belongs may also be set, as shown in fig. 3, based on the foregoing fig. 1, the method may further include the following steps:
105: and displaying a control interface for setting the compression parameters in the display area. In this embodiment, a feasible manner of the control interface is shown in fig. 4, and each activity capable of performing compression processing is displayed in the control interface shown in fig. 4, and if any activity is checked, it is considered that the compression switch is turned on for the data packet under the activity, so that the data packet under the activity can be compressed, and the specific compression depends on the compression threshold.
106: and acquiring compression time in compression parameters set in a control interface by a user and/or a service partition to which a data packet to be compressed belongs, wherein the compression time is used for indicating a time range of compression processing of the data packet by a server and/or a terminal.
In this embodiment, the compression time may be determined according to a communication protocol or a service change based on the server and the terminal, for example, when it is monitored that the communication protocol based on the server and the terminal is changed from the first communication protocol to the second communication protocol, the time changed to the second communication protocol is a start time of the compression time, and if the communication protocol is changed from the second communication protocol to another communication protocol, the end time of the compression time is a time at which the second communication protocol ends. Of course, the method may also be determined based on other ways, for example, if any service (sometimes also referred to as project activity) in fig. 4 checks that the compression switch is turned on for the data packet under the service, the start date of the service is the start time of the compression time, and the end date of the service is the end time of the compression time.
The service partition to which the data packet to be compressed belongs may be set by the option of "large area" in fig. 4, and the setting of whether to compress the data packets of all partitions under any service or compress the data packets of some partitions under any service is set by the option of "large area", and the setting diagram is shown in fig. 5.
For example, for a game service, it is assumed that a game may have 100 partitions, each partition carries a certain number of users to play the game at the same time, and when compressing the game, at least one of the 100 partitions may be selected to be compressed, in this way, whether compression has an effect on the game service may be verified in a small range, if not, data packets of all partitions under the game service may be compressed, so as to reduce the effect caused by implementation of compression parameters, and if the effect is small because the number of partitions for verifying compression is small, the effect is controllable, and the game experience of a large number of users is not disturbed.
According to the technical scheme, the compression time in the compression parameters and/or the service partition to which the data packet to be compressed belongs can be set through the control interface, so that the compression object can be flexibly set and the compression time can be flexibly adjusted through the control interface, for example, the compression time in the compression parameters and/or the service partition to which the data packet to be compressed belongs can be adjusted at any time according to actual needs after the compression parameters are set.
In addition, the embodiment may also monitor a condition of performing compression based on the compression parameter, for example, receive a data compression response sent by the server and/or the terminal, and prohibit compressing the data packet if the data compression response indicates that the service of the server and/or the terminal is abnormal.
One way to prohibit compressing the data packet in this embodiment is: deleting the compression parameters, for example, setting values of various parameters in the compression parameters to be null, or setting the end time of the compression time in the compression parameters to be time for receiving the indication that the service of the server and/or the terminal is abnormal, so as to close the use of the compression parameters, for example, closing a compression switch for starting the compression parameters, so that the service on the server and/or the terminal can be ensured to be normal by the method.
If the above-mentioned data packet compression control method is applied to a management server except for a server and a terminal, the management server obtains the above-mentioned compression parameter and then sends the compression parameter to the server, and a signaling diagram thereof is shown in fig. 6, and may include the following steps:
601: the management server monitors the output flow of the data packets sent to the terminal by the server, wherein the output flow of the data packets represents the total data volume of the data packets sent to the terminal by the server in unit time.
602: the management server analyzes the output flow of the data packets to obtain the proportion of the data group of each data packet sent to the terminal by the server.
603: the management server selects a target data group from all the data groups based on the ratio of the data group in which each data packet is located, so that the compression parameters obtained based on the selected target data group can reduce the data volume of the transmitted data packet to the maximum extent.
604: the management server obtains the compression parameters based on the data volume range of the target data set.
605: and the management server sends a compression configuration instruction to the server, wherein the compression configuration instruction carries compression parameters. The management server may send the compressed configuration instruction based on a communication protocol used for communication with the server, which is not limited in this embodiment.
606: and the server compresses the data packet to be compressed matched with the compression parameters and then transmits the compressed data packet. The data packet to be compressed matched with the compression parameters may be: data packets with data volume matching the compression threshold of the compression parameter, such as data packets with data volume greater than the compression threshold; of course, the compression may also be performed with reference to other parameters in the compression parameters, such as the compression time and the type of the data packet to be compressed, and the specific embodiment is not described in detail.
607: and the server encrypts the compression parameters to obtain the encrypted compression parameters.
608: the server sends the encrypted compression parameters to the terminal, so that the management server can send the compression parameters to the terminal by means of the server, and encrypts the compression parameters in the process of sending the compression parameters, so that the safety factor of the compression parameters is improved, and the safety risk is reduced. For example, the server can forward the compression parameters through an encryption proprietary protocol so that the terminal can compress with reference to the compression parameters. When the encryption private protocol is adopted to forward the compression parameters, the encrypted compression parameters can be carried in the compression configuration instruction to be sent.
The process of compressing by the terminal is shown in fig. 7, and on the basis of fig. 6, the method may further include the following steps:
609: and the terminal decrypts the encrypted compression parameters in the compression configuration instruction after receiving the compression configuration instruction sent by the server to obtain the compression parameters.
610: and after compressing the data packet to be compressed matched with the compression parameters by the terminal, sending the compressed data packet to other terminals through the transfer equipment. The description of the data packet to be compressed matched with the compression parameter can refer to the above description, and the compressed data packet is sent to other terminals through the transfer device to solve the problem that no connection exists between the terminals.
In this embodiment, the resource consumption of the terminal increases with the increase of the compression parameter, but its FPS (Frames Per Second) is stable within an acceptable range, as shown in table 1, starting from four terminals (uesr), compression is not enabled and compression is enabled for illustration, so that even if the terminal enables compression, the performance of the terminal does not change greatly, and the FPS is stable within the acceptable range, which means that the FPS is not reduced significantly before and after the terminal starts compression, so that the terminal does not generate a frame capture phenomenon after starting compression.
TABLE 1 NOT COMPRESSION-ACTIVATED AND COMPRESSED-ACTIVATED COMPARATIONS
The points to be explained here are: when the compressed data packet is forwarded by the forwarding device, the forwarding device is only responsible for transmission, and decompression of the data packet is executed by the terminal receiving the data packet, so that compression transmission of the data packet on a forwarding path between the terminal and other terminals is realized, and the data volume transmitted by the forwarding device is reduced.
As shown in fig. 8 and 9, the average output flow rate of the single transit apparatus is reduced from 4.08KB to 2.86KB, and the percentage of the average output flow rate of the single transit apparatus is reduced to 29.7%. And during peak periods of outgoing traffic, such as 20:00 to 22: 00 comparing the data packet after being compressed and the data packet without being compressed by a single transit device, as shown in fig. 10 and 11, the peak value of the outgoing flow of the transit device is obviously reduced when the transit device transmits the compressed data packet. However, as can be seen from the CPU (central Processing unit) usage rates shown in fig. 12 and 13 for transmitting the compressed packet and the uncompressed packet, the CPU usage rate is reduced when the compressed packet is transmitted.
While, for purposes of simplicity of explanation, the foregoing method embodiments have been described as a series of acts or combination of acts, it will be appreciated by those skilled in the art that the present invention is not limited by the illustrated ordering of acts, as some steps may occur in other orders or concurrently with other steps in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
Corresponding to the foregoing method embodiment, an embodiment of the present invention further provides a data packet compression control apparatus, where the structure of the data packet compression control apparatus is shown in fig. 14, and the data packet compression control apparatus may include: a monitoring unit 10, an analyzing unit 20, a selecting unit 30 and a compression parameter obtaining unit 40.
The monitoring unit 10 is configured to monitor an outgoing flow rate of a data packet sent by the server to the terminal, where the outgoing flow rate of the data packet represents a total data amount of the data packet sent by the server to the terminal in a unit time. In this embodiment, the outgoing flow of the data packet sent by the server to the terminal may be continuously monitored, and the adjustment of the compression parameter may be triggered when the outgoing flow of the data packet changes to a certain extent, for example, when the outgoing flow of the data packet decreases the first data amount or the outgoing flow of the data packet increases the second data amount, the adjustment of the compression parameter may be triggered, where values of the first data amount and the second data amount may be adjusted according to actual application. Or the output flow of the data packet is obtained at every unit time in the process of continuously monitoring the output flow to trigger the adjustment of the compression parameters.
The way of monitoring the outgoing flow of the data packet by the monitoring unit 10 may be: in the process that the server sends the data packet to the terminal, the data packet sent by the server is captured through a tcpdump tool, and the data volume of the captured data packet at each time is summed to obtain the output flow of the data packet, so that the output flow of the data packet not only can represent the total data volume of the data packet sent by the server to the terminal in unit time, but also can record the data volume of the data packet sent at each time.
The analyzing unit 20 is configured to analyze the outgoing flow of the data packet to obtain a ratio of data groups where the data packets sent by the server to the terminal are located, for example, after the data packets are captured by the tcpdump tool, the data packets captured by the tcpdump tool are sequentially introduced into a wirereshark (a network packet analyzing software), and the ratio of the data groups where the data packets are located is obtained by the wirereshark.
Any one of the data sets corresponds to a data volume range, and the data volume ranges of the data sets are different. Because the data volume ranges of different data groups are different (i.e. they are not overlapped with each other), after capturing any data packet, the data volume of the data packet is compared with the data volume range of each data group, so that the data group where the data packet is located can be determined, and the data group where each data packet is located captured in one compression parameter adjustment process can be obtained. And then obtaining the number of data packets and the total number of the data packets corresponding to each data group in the compression parameter adjustment process based on the data group where each data packet is located, and taking the ratio of the number of the data packets corresponding to each data group to the total number of the data packets as the ratio of the data group where each data packet is located.
And the selecting unit 30 is configured to select a target data group from all the data groups based on the ratio of the data groups in which the data packets are located. In this embodiment, based on statistics of the ratio of the data group in which each data packet is located and the data volume range of each data group, feasibility of traffic optimization (i.e., data volume reduction by compressing the data packet) and expected optimization gains are analyzed, so that a target data group corresponding to a compression parameter that can maximize compression performance and compression gains is selected from each data group, and maximizing compression performance and compression gains can represent reducing the transmitted data volume without increasing resource consumption.
For example, one way to select a target data set is; selecting the data group with the largest proportion as a target data group; another way to select the target data set is: obtaining a data set with the largest proportion sum based on the proportion of the data set of each data packet and the data volume range of the data set of each data packet, and selecting the data set with the smallest data volume range from the data set as a target data set, wherein the data volume range of each data set in the data set is larger than the data volume range of the rest data sets; for a detailed description of these two ways, reference is made to the above method embodiments, which are not described again.
And the compression parameter obtaining unit 40 is configured to obtain a compression parameter based on the data volume range of the target data set, so that the server and/or the terminal compresses the data packet based on the compression parameter. In this embodiment, the compression threshold in the compression parameter is mainly obtained based on the data amount range of the target data group, for example, one data amount is selected from the data amount range of the target data group as the compression threshold in the compression parameter, and specifically, the maximum data amount or the minimum data amount in the data amount range of the target data group may be used as the compression threshold. It is needless to say that the data amount range of the target data group may be used as the compression threshold to compress only the data packets belonging to the target data group, which may be the case when the data amount range of the target data group is small, and therefore, in order to solve this problem, the present embodiment preferably uses the maximum data amount or the minimum data amount as the compression threshold, for example, uses the minimum data amount of the target data group as the compression threshold to compress the data packets with the data amount larger than the compression threshold.
In addition to adjusting the compression threshold in the compression parameter, the compression algorithm in the compression parameter may also be set, such as by applying the existing compression algorithm: the zlib compression algorithm, the lz4 compression algorithm and the snappy compression algorithm are used for carrying out experimental analysis on compression efficiency and resource consumption analysis, the compression algorithm with the highest compression rate is the zlib compression algorithm, and the actual resource overhead is within an acceptable range when the zlib compression algorithm is applied for compression, so that the zlib compression algorithm is used for compressing the data packet.
By means of the technical scheme, the output flow of the data packets sent by the server to the terminal is monitored, the output flow of the data packets is analyzed in the monitoring process, the proportion of the data group where each data packet sent by the server to the terminal is located is obtained, the target data group is selected from all the data groups based on the proportion of the data group where each data packet is located, and the compression parameters are obtained based on the data quantity range of the target data group, so that the server and/or the terminal can compress the data packets based on the compression parameters. The output flow of the data packet sent by the server to the terminal can change along with at least one factor of a communication protocol based on the server and a service provided by the server, a target data group selected and taken out after the output flow of the data packet changes can also change, the data quantity ranges of different data groups are different, and the corresponding compression parameters obtained based on the data quantity ranges of the target data group can also be correspondingly adjusted, so that the compression parameters can be adjusted in real time based on the output flow of the data packet sent by the server to the terminal, and the data packet compressed by the server and/or the terminal can change along with the change of the output flow.
In this embodiment, the compression time in the compression parameter and/or the service partition to which the packet to be compressed belongs may also be set, and the packet compression control apparatus of this embodiment may further include: the display unit is used for displaying a control interface for setting compression parameters in a display area; the compression parameter obtaining unit is further configured to obtain compression time in compression parameters set in the control interface by the user and/or a service partition to which the data packet to be compressed belongs, where the compression time is used to indicate a time range in which the server and/or the terminal performs compression processing on the data packet, and refer to the relevant description in the foregoing method embodiment.
In addition, the packet compression control apparatus provided in this embodiment may also monitor the compression condition based on the compression parameter, and for example, may further include: the device comprises a receiving unit used for receiving a data compression response sent by the server and/or the terminal, and a control unit used for forbidding to compress the data packet if the data compression response indicates that the server and/or the terminal are abnormal in service.
One way for the control unit to prohibit compression of the data packet in this embodiment is: deleting the compression parameters, for example, setting values of various parameters in the compression parameters to be null, or setting the end time of the compression time in the compression parameters to be time for receiving the indication that the service of the server and/or the terminal is abnormal, so as to close the use of the compression parameters, for example, closing a compression switch for starting the compression parameters, so that the service on the server and/or the terminal can be ensured to be normal by the method.
An embodiment of the present invention further provides a management server, a structure of which is shown in fig. 15, and the management server may include: a processor 110 and a memory 120.
The processor 110 is configured to monitor an outgoing flow rate of a data packet sent by the server to the terminal, where the outgoing flow rate of the data packet represents a total data amount of the data packet sent by the server to the terminal in a unit time; analyzing the output flow of the data packets to obtain the ratio of data groups of the data packets sent to the terminal by the server, wherein any data group corresponds to a data quantity range, and the data quantity ranges of the data groups are different; selecting a target data group from all data groups based on the ratio of the data group in which each data packet is positioned, and obtaining a compression parameter based on the data volume range of the target data group so as to enable the server and/or the terminal to compress the data packet based on the compression parameter; a memory 120 for storing the compression parameters.
Optionally, the management server may further include a display screen 130, an input unit 140, a communication interface 150, and a communication bus 160.
The processor 110, the memory 120, the display 130, the input unit 140 and the communication interface 150 are all communicated with each other through a communication bus 160.
In this embodiment, the processor 110 may be a CPU, an asic, a digital signal processor, an off-the-shelf programmable gate array or other programmable logic device, etc.
The processor may call a program stored in the memory 120, the program stored in the memory being at least for implementing the above-described packet compression control method.
In one possible implementation, the memory 120 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a compression function, etc.), and the like; the storage data area may store data, such as compressed data, etc., which are involved in the use of the computer.
Further, the memory 120 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device or other volatile solid state storage device.
The communication interface 150 may be an interface of a communication module, such as an interface of a GSM module.
Of course, the structure of the management server shown in fig. 15 does not limit the management server in the present embodiment, and the management server may include more or less components than those shown in fig. 15 or some components in combination in practical applications.
An embodiment of the present invention further provides a terminal, a structure of which is shown in fig. 16, and the terminal may include: a communication interface 1100 and a processor 1200.
A communication interface 1100, configured to receive a compression configuration instruction sent by a server, where the compression configuration instruction carries encrypted compression parameters; a processor 1200, configured to decrypt the encrypted compression parameter to obtain a compression parameter; and after compressing the data packet to be compressed matched with the compression parameters, sending the compressed data packet to other terminals through the transfer equipment. And the compression parameters in the compression configuration instruction sent by the server are acquired from the management server side, encrypted and sent to the terminal.
Optionally, the terminal may further include a display screen 1300, an input unit 1400, a memory 1500, and a communication bus 1600.
The communication interface 1100, the processor 1200, the display 1300, the input unit 1400 and the memory 1500 are all communicated with each other through the communication bus 1600.
In this embodiment, the processor 1200 may be a CPU, an asic, a digital signal processor, an off-the-shelf programmable gate array or other programmable logic device, etc.
The processor may call a program stored in the memory 1500, the program stored in the memory being at least for implementing the above-described packet compression control method.
In one possible implementation, the memory 1500 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function (such as a compression function, etc.), and the like; the storage data area may store data, such as compressed data, etc., which are involved in the use of the computer.
Further, the memory 1500 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device or other volatile solid state storage device.
The communication interface 1100 may be an interface of a communication module, such as an interface of a GSM module.
Of course, the structure of the terminal shown in fig. 16 does not constitute a limitation of the terminal in the present embodiment, and the terminal may include more or less components than those shown in fig. 16 or some components in combination in practical applications.
The embodiment of the invention also provides a storage medium, wherein the storage medium is stored with computer program codes, and when the computer program codes are executed, the data packet compression control method is realized.
It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (11)
1. A method for controlling compression of a data packet, the method comprising:
monitoring the output flow of a data packet sent to a terminal by a server, wherein the output flow of the data packet represents the total data volume of the data packet sent to the terminal by the server in unit time;
analyzing the output flow of the data packets to obtain the ratio of the data groups of the data packets sent to the terminal by the server, wherein the ratio of the data groups of the data packets is the ratio of the number of the data packets corresponding to each data group to the total number of the data packets, any data group corresponds to a data amount range, and the data amount ranges of the data groups are different;
selecting a target data group from all data groups based on the ratio of the data group in which each data packet is positioned;
and obtaining a compression parameter based on the data volume range of the target data group, so that the server and/or the terminal compresses the data packet based on the compression parameter.
2. The method of claim 1, wherein the selecting the target data group from all the data groups based on the ratio of the data groups in which the respective data packets are located comprises:
obtaining a data set with the largest proportion sum based on the proportion of the data set of each data packet and the data volume range of the data set of each data packet, wherein the data volume range of each data set in the data set is larger than the data volume range of the rest data sets;
and selecting the data group with the minimum data volume range from the data group set as the target data group.
3. The method of claim 1 or 2, wherein deriving compression parameters based on the range of data volumes for the target data set comprises:
and selecting a data volume from the data volume range of the target data set as a compression threshold value in the compression parameter.
4. The method according to claim 1 or 2, characterized in that the method further comprises:
displaying a control interface for setting compression parameters in a display area;
and acquiring compression time in compression parameters set in the control interface by a user and/or a service partition to which a data packet to be compressed belongs, wherein the compression time is used for indicating a time range in which the server and/or the terminal performs compression processing on the data packet.
5. The method according to claim 1 or 2, characterized in that the method further comprises:
receiving a data compression response sent by the server and/or the terminal;
and if the data compression response indicates that the service of the server and/or the terminal is abnormal, the data packet is prohibited from being compressed.
6. A method for controlling compression of a data packet, the method comprising:
receiving a compression configuration instruction, wherein the compression configuration instruction carries compression parameters, and the compression parameters are obtained by the data packet compression control method according to any one of claims 1 to 5;
compressing and transmitting the data packet to be compressed matched with the compression parameters;
encrypting the compression parameters to obtain encrypted compression parameters;
and sending the encrypted compression parameters to a terminal.
7. A method for controlling compression of a data packet, the method comprising:
receiving a compression configuration instruction sent by a server, wherein the compression configuration instruction carries encrypted compression parameters;
decrypting the encrypted compression parameters to obtain compression parameters, wherein the compression parameters are obtained by the data packet compression control method according to any one of claims 1 to 5;
and after compressing the data packet to be compressed matched with the compression parameters, sending the compressed data packet to other terminals through the transfer equipment.
8. An apparatus for controlling compression of a data packet, the apparatus comprising:
the monitoring unit is used for monitoring the output flow of a data packet sent to a terminal by a server, wherein the output flow of the data packet represents the total data volume of the data packet sent to the terminal by the server in unit time;
the analysis unit is used for analyzing the output flow of the data packets to obtain the proportion of the data group where each data packet is sent to the terminal by the server, the proportion of the data group where each data packet is located is the proportion of the number of the data packets corresponding to each data group to the total number of the data packets, any data group corresponds to a data amount range, and the data amount ranges of the data groups are different;
the selecting unit is used for selecting a target data group from all the data groups based on the ratio of the data group in which each data packet is positioned;
and the compression parameter obtaining unit is used for obtaining compression parameters based on the data volume range of the target data group so as to enable the server and/or the terminal to compress the data packet based on the compression parameters.
9. A management server, characterized in that the management server comprises: a processor and a memory;
the processor is configured to monitor an outgoing flow of a data packet sent by a server to a terminal, where the outgoing flow of the data packet represents a total data amount of the data packet sent by the server to the terminal in a unit time; analyzing the output flow of the data packets to obtain the ratio of the data groups of the data packets sent to the terminal by the server, wherein the ratio of the data groups of the data packets is the ratio of the number of the data packets corresponding to each data group to the total number of the data packets, any data group corresponds to a data amount range, and the data amount ranges of the data groups are different; based on the ratio of the data groups in which the data packets are located, selecting a target data group from all the data groups, and based on the data volume range of the target data group, obtaining a compression parameter so that the server and/or the terminal compresses the data packets based on the compression parameter;
the memory is used for storing the compression parameters.
10. A terminal, characterized in that the terminal comprises: a communication interface and a processor;
the communication interface is used for receiving a compression configuration instruction sent by the server, wherein the compression configuration instruction carries encrypted compression parameters;
the processor is configured to decrypt the encrypted compression parameter to obtain a compression parameter, where the compression parameter is obtained by performing packet compression control on the management server according to claim 9; and after compressing the data packet to be compressed matched with the compression parameters, sending the compressed data packet to other terminals through the transfer equipment.
11. A computer-readable storage medium having stored thereon computer-executable instructions which, when loaded and executed by a processor, implement a method for controlling packet compression as claimed in any one of claims 1 to 5, or implement a method for controlling packet compression as claimed in claim 6, or implement a method for controlling packet compression as claimed in claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910912810.7A CN110611680B (en) | 2019-09-25 | 2019-09-25 | Data packet compression control method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910912810.7A CN110611680B (en) | 2019-09-25 | 2019-09-25 | Data packet compression control method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110611680A CN110611680A (en) | 2019-12-24 |
| CN110611680B true CN110611680B (en) | 2021-06-25 |
Family
ID=68893513
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910912810.7A Active CN110611680B (en) | 2019-09-25 | 2019-09-25 | Data packet compression control method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110611680B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114125071B (en) * | 2021-11-23 | 2024-02-09 | 北京人大金仓信息技术股份有限公司 | Data compression transmission method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104184819A (en) * | 2014-08-29 | 2014-12-03 | 城云科技（杭州）有限公司 | Multi-hierarchy load balancing cloud resource monitoring method |
| US9008621B2 (en) * | 2006-06-23 | 2015-04-14 | Fujitsu Limited | Communication server, mobile communication terminal, communication method, and computer product |
| CN105723678A (en) * | 2013-10-29 | 2016-06-29 | 瑞典爱立信有限公司 | Dynamic compression coverage |
| CN110099092A (en) * | 2018-01-31 | 2019-08-06 | 慧与发展有限责任合伙企业 | Dynamic data compression |
| CN110175185A (en) * | 2019-04-17 | 2019-08-27 | 南京天数智芯科技有限公司 | A kind of self-adaptive non-loss compression based on time series data distribution characteristics |
-
2019
- 2019-09-25 CN CN201910912810.7A patent/CN110611680B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9008621B2 (en) * | 2006-06-23 | 2015-04-14 | Fujitsu Limited | Communication server, mobile communication terminal, communication method, and computer product |
| CN105723678A (en) * | 2013-10-29 | 2016-06-29 | 瑞典爱立信有限公司 | Dynamic compression coverage |
| CN104184819A (en) * | 2014-08-29 | 2014-12-03 | 城云科技（杭州）有限公司 | Multi-hierarchy load balancing cloud resource monitoring method |
| CN110099092A (en) * | 2018-01-31 | 2019-08-06 | 慧与发展有限责任合伙企业 | Dynamic data compression |
| CN110175185A (en) * | 2019-04-17 | 2019-08-27 | 南京天数智芯科技有限公司 | A kind of self-adaptive non-loss compression based on time series data distribution characteristics |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110611680A (en) | 2019-12-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3780523B1 (en) | Network traffic identification method and related device | |
| KR101578473B1 (en) | Real-time network monitoring and subscriber identification with an on-demand appliance | |
| CN104394486B (en) | A kind of video stream processing method and device based on cloud desktop | |
| Kua et al. | Using active queue management to assist IoT application flows in home broadband networks | |
| US10924577B2 (en) | Fractional pre-delivery of content to user devices for uninterrupted playback | |
| US9178929B2 (en) | Client-side class-of-service-based bandwidth management in over-the-top video delivery | |
| US20120198020A1 (en) | Content distribution within a service provider network | |
| CN107181743B (en) | Network live broadcast data reporting method and device | |
| US10499214B2 (en) | Data usage analytics application for dynamic control of data usage on a client device | |
| CN109218216B (en) | Link aggregation flow distribution method, device, equipment and storage medium | |
| US8886615B2 (en) | Web service performance optimization by adaptively using compression | |
| Seufert et al. | Features that matter: Feature selection for on-line stalling prediction in encrypted video streaming | |
| CN113891175B (en) | Live broadcast push flow method, device and system | |
| CN109257335B (en) | Method for maintaining back source link, back source method, related device and storage medium | |
| CN110611680B (en) | Data packet compression control method and device | |
| CN113194134A (en) | Node determination method and device | |
| EP3417571A1 (en) | Method and system for compression and optimization of in-line and in-transit information security data | |
| US20120213133A1 (en) | Method and system for identifying media type transmitted over an atm network | |
| EP3930287A1 (en) | System and method for managing adaptive bitrate video streaming | |
| CN112335203B (en) | Processing local area network diagnostic data | |
| Szabo et al. | Potential Gains of Reactive Video QoE Enhancement by App Agnostic QoE Deduction | |
| CN112422525A (en) | Fault data transmission method, device, equipment and storage medium | |
| CN105939208B (en) | The transmission method and device of sampled data | |
| US20150085656A1 (en) | Technique for Projecting Network Load in a Communication Network | |
| CN114222153A (en) | Online user number counting method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |