CN115276662B - Method for efficiently compressing and transmitting micro-service user information - Google Patents
Method for efficiently compressing and transmitting micro-service user information Download PDFInfo
- Publication number
- CN115276662B CN115276662B CN202210891833.6A CN202210891833A CN115276662B CN 115276662 B CN115276662 B CN 115276662B CN 202210891833 A CN202210891833 A CN 202210891833A CN 115276662 B CN115276662 B CN 115276662B
- Authority
- CN
- China
- Prior art keywords
- coding
- interval
- segment
- code
- bit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
- H03M7/3084—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction using adaptive string matching, e.g. the Lempel-Ziv method
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
- H03M13/03—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words
- H03M13/05—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits
- H03M13/09—Error detection only, e.g. using cyclic redundancy check [CRC] codes or single parity bit
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Probability & Statistics with Applications (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention relates to the technical field of communication, in particular to a method for efficiently compressing and transmitting user information of micro-services, which comprises the steps of obtaining binary codes of the user information in the micro-services, taking each four-digit binary digit as a code segment, dividing the binary codes of the user information into a plurality of target code segments, taking a code segment characteristic code segment with the probability in each code segment larger than the sum of other probabilities, taking the four-digit binary digit corresponding to the code segment in each target code segment as a characteristic data segment, taking the binary data corresponding to the other code segments as non-characteristic data segments, taking the non-characteristic data between each characteristic data segment and the previous characteristic data segment as error detection data, carrying out modular two-way division operation on the error detection data to judge whether the bit of the characteristic data segment is a compression bit, replacing each characteristic data segment on the compression bit in each code segment by a vacant bit, carrying out compression transmission, and improving the data compression and transmission efficiency.
Description
Technical Field
The application relates to the technical field of communication, in particular to a method for efficiently compressing and transmitting micro-service user information.
Background
With the development of science and technology, the development of the software industry is gradually improved, the architecture of the single application is gradually abandoned due to the defects of high deployment cost, large modification influence, low deployment frequency and the like, and the micro service architecture is replaced by the new micro service architecture, so that the micro service architecture is more and more concerned by people due to the advantages of the micro service architecture in the aspects of development, operation, maintenance, service cutting and the like. The processing of the user information in the micro service is always subject to the problem of complicated operation procedures, so that the user information in the micro service needs to be compressed and then transmitted to improve the processing efficiency.
The conventional compression transmission method for user information in micro-service is to compress user information once, add a check code after the compressed information, transmit the compressed information and the check code as whole data to an information receiving party, the information receiving party checks the compressed information according to the check code in the whole data, and determine whether the information is correct, for example, the conventional error detection method is CRC error detection code, i.e. cyclic redundancy check code, which includes an information field and a check field, usually, a certain number of extra check codes are added after 128-bit binary coding, and the receiving party detects errors in the transmitted information according to the check code.
Disclosure of Invention
The invention provides a method for efficiently compressing and transmitting information of a micro-service user, which solves the problem of low error detection efficiency after information compression and adopts the following technical scheme:
acquiring binary codes of user information in the micro service, and taking every four binary codes as a code segment;
setting an initial coding interval for binary coding of user information, and obtaining the probability of each coding segment according to the occurrence frequency of each coding segment in the initial coding interval;
if the code segment with the probability larger than the sum of other probabilities occurs in the initial code interval, the code segment is a characteristic code segment, the initial code interval is a target code interval, otherwise, the length of the initial code interval is increased until the characteristic code segment occurs in the interval, and the initial code interval with the characteristic code segment as the target code interval;
taking four-bit binary digits corresponding to the feature coding section of each target coding interval as a feature data section, and taking four-bit binary digits corresponding to other digits as a non-feature data section;
taking a non-characteristic data segment between each characteristic data segment and the previous adjacent characteristic data segment as error detection data, performing modular division operation on the error detection data, and judging whether the position of the characteristic data segment is a compression position or not according to the sum of a quotient and a remainder obtained by the modular division operation;
and replacing the characteristic data segment on the compression bit in each target coding interval by using a vacant bit, and compressing and transmitting the replaced target code.
The specific method for acquiring the binary code of the user information in the microservice comprises the following steps:
and primarily coding the user information in the micro service by using the GZIP to obtain the binary code of the user information in the micro service.
Another method for obtaining the probability of each code segment is as follows:
and converting each code segment in the initial code interval into a decimal number, and counting the times of the decimal number appearing in the initial code interval to obtain the probability of each code segment.
The method for acquiring the target coding interval comprises the following steps:
setting an initial coding interval length A;
calculating the occurrence frequency of each coding segment in the initial coding interval to obtain the probability of each coding segment;
judging whether the initial coding interval is a target coding interval according to the probability of each coding segment:
if the probability of the code segment X is greater than the sum of the probabilities of other code segments, the initial segment is a target code segment, the code segment X is a characteristic code segment of the target code segment, and otherwise, the initial code segment is not the target code segment;
and if the initial coding interval is not the target coding interval, increasing the length of the initial coding interval, wherein the length of the initial coding interval is increased to be alpha each time until the characteristic coding segment appears in the coding interval after the length is increased, and taking the coding interval when the characteristic coding segment appears as the target coding interval.
And when the four-bit binary digits corresponding to the characteristic coding segment of each target coding interval are used as the characteristic data segment, if the last bit of one characteristic data segment is adjacent to the first bit of another characteristic data segment, combining the last bit and the first bit into the same characteristic data segment.
The method for judging whether the bit of the characteristic data segment is a compressed bit according to the sum of the quotient and the remainder obtained by the modulo two division operation comprises the following steps:
taking the error detection data as dividend and the characteristic data segment as divisor, and performing modulo two division operation on the error detection data to obtain quotient and remainder;
and adding the quotient and the remainder, wherein if the last digit of the number obtained after the addition is 0, the position of the characteristic data segment is an uncompressed digit, and if the last digit of the number obtained after the addition is 1, the position of the characteristic data segment is a compressed digit.
The method for compressing and transmitting the replaced target code comprises the following steps:
and replacing the characteristic data segment on the compression bit in each coding interval by using a vacant bit, adding the characteristic data in the coding interval as an identifier on the first vacant bit, keeping the characteristic data segment on the non-compression bit and other non-characteristic data segments unchanged, and integrally transmitting all data in the coding interval.
The null needs to be delayed by one clock signal transmission.
The beneficial effects of the invention are: preprocessing user information in the micro-service to enable the user information to be structured and convenient to compress; the method comprises the steps that the structured user information is subjected to primary coding in a conventional GZIP mode to obtain binary codes of the user information, binary data of every 4 bytes are converted into decimal, and then self-adaptive partition is carried out according to the probability of the decimal data, 4-bit binary is converted into decimal and self-adaptive partition is carried out according to the probability, under the condition that the interval is shortest, the repeatability of a certain 4-bit binary code in each interval is ensured to be higher, and therefore the compression efficiency is higher; on the basis of dividing coding intervals, the binary code of each interval contains characteristic data to detect errors of the current interval, the compression bit is determined according to the detection result, the characteristic data in the compression bit is compressed by the characteristic data, the errors are detected on the basis of compressing the characteristic data of each interval, no extra space is used for adding any redundant error detection code, and the compression efficiency is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a method for efficiently compressing and transmitting information of a microserver according to the present invention;
FIG. 2 is a schematic diagram of feature data and non-feature data in a method for efficiently compressing and transmitting user information of a micro service according to the present invention;
FIG. 3 is a schematic diagram of error detection data selected in a method for efficiently compressing and transmitting information of a microserver according to the present invention;
FIG. 4 is a schematic diagram of a compression bit followed by non-feature data in a method for efficiently compressing and transmitting information of a micro service user according to the present invention;
fig. 5 is a schematic diagram of a non-compressed bit following the feature data in the method for efficiently compressing and transmitting the information of the microserver user according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
An embodiment of a method for efficiently compressing and transmitting microservice user information according to the present invention, as shown in fig. 1, includes:
the method comprises the following steps: acquiring binary codes of user information in the micro-service, and taking every four binary codes as a code segment;
the step aims to preprocess the user information of the micro service to be transmitted, wherein the preprocessing comprises structuralization processing and primary coding compression to obtain the binary coding of the user information.
The method for acquiring the binary code of the user information in the microservice comprises the following steps:
firstly, carrying out structured processing on user information in the micro service, wherein the structured processing mode is a time plus user ID plus operation mode, namely all user information of the micro service is stored in the structure, then, carrying out primary coding compression on the micro service user information after structured processing by utilizing a GZIP compression mode to obtain binary codes of the user information in the micro service, and obtaining the binary codes D after primary compression of micro service user information data I I is the length of the binary code;
the GZIP coding compression mode mainly comprises the steps of firstly carrying out lz77 coding compression on structured micro-service user information to obtain lz77 coded and compressed structured data compression codes, then carrying out secondary compression on the codes compressed by the lz77 codes by utilizing a Hofmann coding technology, and taking every four binary codes in the binary codes of the user information in the micro-service as a code segment.
Step two: setting an initial coding interval for binary coding of user information, and obtaining the probability of each coding segment according to the occurrence frequency of each coding segment in the initial coding interval; if the code segment with the probability larger than the sum of other probabilities occurs in the initial code interval, the code segment is a characteristic code segment, the initial code interval is a target code interval, otherwise, the length of the initial code interval is increased until the characteristic code segment occurs in the interval, and the initial code interval with the characteristic code segment as the target code interval;
the step aims to divide coding intervals of binary data of the micro-service user information, acquire the probability of occurrence probability of each coding segment in each coding interval, and select the data characteristic quantity of the interval according to the probability.
The method for acquiring the probability of each coding segment comprises the following steps:
acquiring binary coding information of user information, wherein the length of the binary coding information is I, setting an initial coding interval for the binary coding of the user information, the length of the initial coding interval is A, and the interval starts from the first bit of the binary coding information of the user information to the end of the A bit;
calculating the occurrence frequency of each coding segment in the initial coding interval in all the coding segments in the interval to obtain the probability of each coding segment;
another method for acquiring the probability of each code segment is as follows:
for binary coding, each coded segment, namely each 4-bit binary (setting the subsequent error detection code to be 4 bits), is converted into a decimal number by using the prior art, and the binary coding of the user information corresponds to a decimal coded sequence D' I′ Wherein I' is the whole length of the decimal coding sequence,
and D' i′ ∈[0,15]) The ith decimal code is denoted as D' i′ Because the 4-bit binary minimum value is 0000 and the maximum value is 1111, the maximum decimal number converted into decimal is 15 and the minimum value is 0;
setting an initial coding section with the length of A for decimal coding of user information, calculating the probability of each decimal digit in the initial coding section as the probability of the coding section corresponding to the decimal digit, wherein A is an empirical value, in the embodiment, if the probability is calculated by adopting decimal digits, the value of A is 128, 128 decimal digits, because the section length of the check code specified by the conventional check mode is 128 bits, and if D' I The length is less than 128, and the length is the initial interval length, if D' I If the length is more than or equal to 128, taking 128 as the length of an initial interval, wherein the initial coding interval is from the first decimal digit to the A-th decimal digit of the decimal code of the user information; if the probability is calculated using binary coded segments, the value of A is 512.
It should be noted that the conventional partition compression coding mode is to perform partition of a fixed bit number based on a bit number, and this partition compression mode performs fixed-bit compression on data, and the bit number selected during partition is too long, and retrieval efficiency during compression is too slow; the bit number selected during partitioning is too small, and the same uncompressed data cannot be compressed when not in the same interval.
In view of the above problems, the present embodiment utilizes the structural features of encoding to convert every 4 bits of binary data into decimal numbers, and then adaptively divides the regions according to the probability of the decimal data, and the method has the beneficial effects that: the 4-bit binary digit is decimal and self-adaptive divided according to the probability, so that the repeatability of a certain 4-bit binary code in each interval is high under the condition that the coding interval is shortest, the compression efficiency is higher, the calculated amount is reduced after decimal conversion, and the probability is calculated more simply and quickly.
The method for acquiring the target coding interval comprises the following steps: judging whether the initial coding interval is a target coding interval according to the probability of each coding segment: if the probability of the code segment X is greater than the sum of the probabilities of other code segments, the initial segment is a target code segment, the code segment X is a characteristic code segment of the target code segment, and otherwise, the initial code segment is not the target code segment; and if the initial coding interval is not the target coding interval, increasing the length of the initial coding interval, wherein the length of the initial coding interval is increased to be alpha each time until the characteristic coding segment appears in the coding interval after the length is increased, and taking the coding interval when the characteristic coding segment appears as the target coding interval.
The present embodiment selects to obtain the target code interval and the characteristic code segment by converting each code segment into decimal number:
(1) Obtaining the probability of each decimal number in the initial coding interval;
(2) Judging whether the initial coding interval is the target coding interval according to the probability of each decimal number:
(3) If the probability of the decimal number M is greater than the sum of the probabilities of other decimal numbers, the initial interval is a target coding interval, the decimal number M is a characteristic number of the target coding interval, a binary coding section corresponding to the decimal number M is a characteristic coding section, and otherwise, the initial coding interval is not the target coding interval;
(4) If the initial coding interval is not the target coding interval, increasing the length of the initial coding interval, wherein the length of the initial coding interval is increased to be alpha each time until the characteristic number appears in the coding interval after the length is increased, taking the coding interval in which the characteristic number appears as the target coding interval, and taking the binary coding segment corresponding to the characteristic number of the target coding interval as the characteristic coding segment.
According to the method, the adaptive interval length is divided on the basis of the initial interval length:
taking the 1 st interval as an example, the data in the interval is: [ D' 1′ ,D′ 128′ ]Respectively calculating the occurrence probability of each decimal digit, namely the number of times each decimal digit appears in the interval as the probability, wherein the probability is calculated by the ratio of the number of times each data appears to the total number of digits in the interval, and each decimal digit D 'in the interval can be obtained' i′ Probability of occurrence, a maximum of 16 probabilities can be obtained (since each decimal number ranges from 0,15](ii) a And (4) carrying out probability judgment, and judging whether the maximum probability in the probability of each decimal number in the interval is greater than the sum of all the rest probabilities: if the probability exists in the interval, the decimal number corresponding to the maximum probability is a characteristic number, and the length of the current interval is the length of the target interval; if the probability does not exist in the interval, the interval is subjected to lengthening self-adaptation, the length of the interval is increased to be alpha each time, and an empirical value of alpha is
Recalculating the probability of each decimal digit in the interval with increased length, performing probability judgment, repeating the process until the first interval meets the probability judgment condition, completing the length adaptation of the first interval, and completing all intervals by using the methodThe adaptation of the length may divide the decimal code into J decimal code intervals.
Step three: taking four-bit binary digits corresponding to the feature coding section of each target coding interval as a feature data section, and taking four-bit binary digits corresponding to other digits as a non-feature data section; taking a non-characteristic data segment between each characteristic data segment and the previous adjacent characteristic data segment as error detection data, performing modular division operation on the error detection data, and judging whether the position of the characteristic data segment is a compression position or not according to the sum of a quotient and a remainder obtained by the modular division operation;
the purpose of the step is to segment each coding interval by using the characteristic data piece in each coding interval and judge the compression bit in each interval according to the non-characteristic data and the characteristic data;
it should be noted that, the J decimal coding sections obtained in step two record the feature numbers corresponding to the maximum probability of each section, and perform binary conversion operation on each data in all the sections, so as to obtain J binary value coding sections, and obtain the feature coding segments corresponding to the decimal feature numbers in each section.
The method for dividing each target coding interval into a plurality of characteristic data segments and non-characteristic data segments according to the starting and ending position of each characteristic data in each target coding interval as an end point comprises the following steps:
(1) Acquiring all four-bit binary digits corresponding to the characteristic coding segments in each target coding interval;
(2) Taking each four-bit binary digit corresponding to each feature coding segment as a feature data segment, and if the last bit of one feature data segment is adjacent to the first bit of another feature data segment, combining the feature data segments into the same feature data segment;
(3) Taking other binary digits except the characteristic data segment as non-characteristic data segments, the method comprises the steps of obtaining the characteristic coding segment of each code in a segment, dividing each target coding segment into a plurality of segments, and as shown in figure 2, taking the data corresponding to the characteristic coding segment as the characteristic data, taking the data corresponding to the non-characteristic coding segment as the non-characteristic data, and combining the characteristic data and the non-characteristic data in each target coding segment into the same characteristic data segment if the last bit of one characteristic data segment is adjacent to the first bit of the other characteristic data segment in order to ensure that the characteristic coding segment and the non-characteristic coding segment appear alternately.
It should be noted that, when the conventional data compression technology compresses the information of the micro service user, an additional error detection code needs to be added for error detection, resulting in coding redundancy, the present invention calculates the compression bit (the feature data in the compression bit is compressed, and the feature data in the non-compression bit is not compressed) by using the feature data on the basis of using the feature data in the binary coding of the information of the micro service user, and performs secondary compression on the information of the micro service user by using the error detection result and the compression bit under the condition of sacrificing less secondary compression efficiency. The specific implementation process is as follows:
the step takes a binary coding interval of jth microservice user information as an example (j belongs to [1, J ]), and the specific mode of compressing and detecting errors by utilizing characteristic data in the interval is as follows:
the characteristic data of the jth binary coding interval is obtained, the selection mode is that the characteristic code (4-bit binary code, namely the 4-bit binary code corresponding to the decimal data with the maximum occurrence probability) corresponding to the characteristic number in the jth coding interval is subjected to area division by utilizing the occurrence probability of the decimal code corresponding to the characteristic code, so that the data is selected as the characteristic value to compress the data of the whole interval to ensure the highest compression efficiency, and the binary code and the decimal code of each interval have unique correspondence, namely each decimal code corresponds to the corresponding binary code, and disorder is avoided.
The method for judging whether the position of each characteristic data segment is a compression bit or not according to the sum of a quotient and a remainder obtained by the modulo two division operation comprises the following steps:
(1) Obtaining non-characteristic data between each characteristic data segment and the previous characteristic data segment, taking the non-characteristic data as error detection data, taking the error detection data as dividends, taking the group of characteristic data as divisors, and performing modulo two division operation on the non-characteristic data to obtain a quotient and a remainder;
taking the nth set of feature data as an example, non-feature data after the (n-1) th set of feature data and before the nth set of feature data is first selected as error detection data, and as shown in fig. 3, non-feature data between the (n-1) th set of feature data and the nth set of feature data is selected data (error detection data).
(2) And adding the quotient and the remainder, wherein if the last digit of the number obtained after the addition is 0, the position of the characteristic data segment is an uncompressed digit, and if the last digit of the number obtained after the addition is 1, the position of the characteristic data segment is a compressed digit.
The method comprises performing modulo two division on non-characteristic data (error detection data) in each coding interval, wherein the dividend in the modulo two division is selected non-characteristic data (error detection data), the divisor is characteristic data, and the selected non-characteristic data is calculated by modulo two division to obtain quotient and remainder R by taking the nth group of characteristic data as an example n And Q n Then to R n And Q n Adding and taking the last bit for judgment, and if the added last bit is '0', considering the nth group of characteristic data as a non-compression bit; if the last bit after the addition is "1", the nth set of feature data is considered to be a compressed bit.
Although the non-feature data (error detection data) between the nth group of feature data and the nth-1 feature data has the structural feature of the non-feature data, specifically, the corresponding length and value are both binary structures, the non-feature data (error detection data) can be subjected to the modulo two division operation with the nth group of feature data because the modulo two division operation is essentially a binary and or operation, and the data with the length of 4 bytes of all the data structures same as the divisor are named as the feature data before the modulo two division operation, so the non-feature data before the nth group of feature data and the nth group of feature data are calculated by the modulo two division method, and the corresponding quotient and remainder R can be obtained certainly n And Q n Used as the calculation of the compression bit.
(3) And calculating the whole coded data in a compression bit calculation mode of the nth group data of the jth interval to obtain a compression bit and a non-compression bit in each coding interval of the binary coded data of the whole micro service user information.
Step four: and replacing the characteristic data segment on the compression bit in each target coding interval by using a vacant bit, and compressing and transmitting the replaced target code.
The purpose of this step is to utilize the characteristic data to realize the compression transmission of the code, has promoted the transmission efficiency.
Wherein, each characteristic data segment on the compression bit in each coding interval is replaced by a vacancy, and the specific method for compressing and transmitting the replaced code comprises the following steps:
compressing the whole data according to the compression bit and the non-compression, specifically taking the jth interval as an example, replacing all the characteristic data segments on the compression bit in the jth interval, taking a vacant bit as a replacement (a vacant bit encountered in the subsequent transmission process can delay a clock signal for transmission as a distinction), and adding the characteristic data of the interval at the first vacant bit in the jth interval as an identifier; the characteristic data segment on the non-compression bit is not compressed, and because the division of each interval is based on the interval containing a large number of characteristic data segments in the data partitioning process, the compression of the whole interval by the characteristic data segments has a relatively high compression rate;
therefore, the compression and transmission of the binary codes of all the microserver user information are completed.
The error detection process of the binary coding whole data of all the microservice user information is as follows:
after receiving the compressed data of the microservice user information, the receiving end searches the characteristic data of the jth interval, and then positions the compressed bit and the uncompressed bit according to the characteristic data, so that two conditions can be obtained:
in the first case: a group of NOTThe characteristic data is followed by a compression bit, as shown in fig. 4, in this case, a group of non-characteristic data is followed by a compression bit, the error detection mode takes the nth group of characteristic data as an example (the nth group of characteristic data is on the compression bit), and the non-characteristic data before the compression bit and the characteristic data in the jth interval are used for performing modulo two division to obtain a quotient and a remainder R n And Q n Then, summing the data, judging the last bit of the summing result, if the last bit is '1', the influence of the channel or other factors which are not received in the transmission process of the non-characteristic data between the previous compression bit and the non-compression bit is not wrong; if the last bit is "0", the error is caused by the channel or other factors during the transmission of the non-specific data between the previous compressed bit and the non-compressed bit.
In the second case: as shown in fig. 5, the error detection mode takes the nth set of feature data (the nth set of feature data is in the uncompressed bit), and performs modulo-two division on the non-feature data before the uncompressed bit and the feature data in the jth interval to obtain a quotient and a remainder R n And Q n Then, summing the data, judging the last bit of the summing result, if the last bit is '0', the non-compressed bit is not affected by a channel or other factors in the transmission process of non-characteristic data between the last compressed bit and the non-compressed bit, and no error occurs; if the last bit is "1", the non-compressed bit is affected by channel or other factors during the transmission of non-specific data between the last compressed bit and the non-compressed bit.
So far, the error detection compression of the whole micro-service user information is completed, the steps can be utilized to obtain the compressed error detection codes of all the micro-service user information, and the codes are transmitted, wherein the transmission mode is as follows: and respectively transmitting the binary compression error detection coding data of each interval, wherein the transmission format of the data is that the characteristic data of the interval is added to the characteristic data identification bit reserved in the step two to be used as an identifier, and then the compression error detection data in the interval is transmitted integrally.
In this embodiment, the binary code of the entire microservice user information is partitioned into intervals, an error detection code is obtained for each interval, error detection is performed on data in the interval by using the error detection code, and detection characteristics are quantized, the detection mode of the conventional code is to perform error detection by using a mode of adding a redundant code, but the error detection length is conventionally selected to be 128 bits, that is, each 128 bits of code needs to be added with a corresponding error detection code for error detection, so that certain data redundancy is caused.
The embodiment firstly partitions the GZIP compressed code of the microservice user data, acquires the code characteristic data in each interval on the basis of binary code partitioning, detects the error of the current interval by using the characteristic data contained in the binary code of each interval, then determines the compression bit according to the detection result, and then compresses the characteristic data in the compression bit by using the characteristic data. Compared with the conventional error detection mode, the error detection is carried out on the basis of compressing by utilizing the data characteristics of each interval, any redundant code is not required to be added, and the compression efficiency is improved.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (8)
1. A method for efficiently compressing and transmitting information of micro-service users is characterized by comprising the following steps:
acquiring binary codes of user information in the micro service, and taking every four binary codes as a code segment;
setting an initial coding interval for binary coding of user information, and obtaining the probability of each coding segment according to the occurrence frequency of each coding segment in the initial coding interval;
if the code segment with the probability larger than the sum of other probabilities occurs in the initial code interval, the code segment is a characteristic code segment, the initial code interval is a target code interval, otherwise, the length of the initial code interval is increased until the characteristic code segment occurs in the interval, and the initial code interval with the characteristic code segment as the target code interval;
taking four-bit binary digits corresponding to the feature coding section of each target coding interval as a feature data section, and taking four-bit binary digits corresponding to other digits as a non-feature data section;
taking a non-characteristic data segment between each characteristic data segment and the previous adjacent characteristic data segment as error detection data, performing modular division operation on the error detection data, and judging whether the position of the characteristic data segment is a compression position or not according to the sum of a quotient and a remainder obtained by the modular division operation;
and replacing the characteristic data segment on the compression bit in each target coding interval by using a vacant bit, and compressing and transmitting the replaced target code.
2. The method for efficiently compressing and transmitting the user information of the microservice according to claim 1, wherein the specific method for obtaining the binary code of the user information in the microservice comprises the following steps:
and primarily coding the user information in the micro service by using the GZIP to obtain the binary code of the user information in the micro service.
3. The method as claimed in claim 1, wherein the another method for obtaining the probability of each code segment is:
and converting each coding section in the initial coding interval into a decimal number, and counting the times of the decimal number appearing in the initial coding interval to obtain the probability of each coding section.
4. The method for efficiently compressing and transmitting the information of the microservice user according to claim 1, wherein the method for acquiring the target coding interval comprises the following steps:
setting an initial coding interval length A;
calculating the occurrence frequency of each coding segment in the initial coding interval to obtain the probability of each coding segment;
judging whether the initial coding interval is a target coding interval according to the probability of each coding segment:
if the probability of the coding section X is greater than the sum of the probabilities of other coding sections, the initial coding section is a target coding section, the coding section X is a characteristic coding section of the target coding section, and otherwise, the initial coding section is not the target coding section;
and if the initial coding interval is not the target coding interval, increasing the length of the initial coding interval, wherein the length of the initial coding interval is increased to be alpha each time until the characteristic coding segment appears in the coding interval after the length is increased, and taking the coding interval when the characteristic coding segment appears as the target coding interval.
5. The method as claimed in claim 1, wherein the four-bit binary digits corresponding to the signature segment of each target coding interval are used as the signature data segment, and if the last bit of one signature data segment is adjacent to the first bit of another signature data segment, the two are merged into the same signature data segment.
6. The method for efficiently compressing and transmitting the information of the microservice user according to the claim 1, wherein the method for judging whether the bit of the characteristic data segment is the compression bit according to the sum of the quotient and the remainder obtained by the modular two division operation comprises the following steps:
taking the error detection data as a dividend and the characteristic data segment as a divisor, and performing modulo two division operation on the error detection data to obtain a quotient and a remainder;
and adding the quotient and the remainder, wherein if the last digit of the number obtained after the addition is 0, the position of the characteristic data segment is an uncompressed digit, and if the last digit of the number obtained after the addition is 1, the position of the characteristic data segment is a compressed digit.
7. The method for efficiently compressing and transmitting microservice user information according to claim 1, wherein said method for compressing and transmitting the replaced target code comprises:
and replacing the characteristic data segment on the compression bit in each coding interval by using a vacant bit, adding the characteristic data in the coding interval as an identifier on the first vacant bit, keeping the characteristic data segment on the non-compression bit and other non-characteristic data segments unchanged, and integrally transmitting all data in the coding interval.
8. The method as claimed in claim 7, wherein the transmission of the vacancy is delayed by one clock signal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210891833.6A CN115276662B (en) | 2022-07-27 | 2022-07-27 | Method for efficiently compressing and transmitting micro-service user information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210891833.6A CN115276662B (en) | 2022-07-27 | 2022-07-27 | Method for efficiently compressing and transmitting micro-service user information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115276662A CN115276662A (en) | 2022-11-01 |
| CN115276662B true CN115276662B (en) | 2023-01-31 |
Family
ID=83771788
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210891833.6A Active CN115276662B (en) | 2022-07-27 | 2022-07-27 | Method for efficiently compressing and transmitting micro-service user information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115276662B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115955513B (en) * | 2023-03-10 | 2023-05-05 | 湖南洛普洛科技有限公司 | Data optimization transmission method for Internet of things |
| CN117579079B (en) * | 2024-01-15 | 2024-03-29 | 每日互动股份有限公司 | Data compression processing method, device, equipment and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2002054602A1 (en) * | 2000-12-27 | 2002-07-11 | Mitsubishi Denki Kabushiki Kaisha | Multiple quality data creation encoder, multiple quality data creation decoder, multiple quantity data encoding decoding system, multiple quality data creation encoding method, multiple quality data creation decoding method, and multiple quality data creation encoding/decoding method |
| CN104038233A (en) * | 2014-05-09 | 2014-09-10 | 安庆师范学院 | Testing data compression and decompression method based on ortho-position exclusive or operation |
| CN110502599A (en) * | 2019-08-23 | 2019-11-26 | 腾讯科技（深圳）有限公司 | Querying method, device and the computer readable storage medium of map datum |
| CN110932736A (en) * | 2019-11-09 | 2020-03-27 | 天津大学 | DNA information storage method based on Raptor code and quaternary RS code |
| CN114567681A (en) * | 2022-01-25 | 2022-05-31 | 浙江数秦科技有限公司 | High-efficiency data transmission method for block chain network |
-
2022
- 2022-07-27 CN CN202210891833.6A patent/CN115276662B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2002054602A1 (en) * | 2000-12-27 | 2002-07-11 | Mitsubishi Denki Kabushiki Kaisha | Multiple quality data creation encoder, multiple quality data creation decoder, multiple quantity data encoding decoding system, multiple quality data creation encoding method, multiple quality data creation decoding method, and multiple quality data creation encoding/decoding method |
| CN104038233A (en) * | 2014-05-09 | 2014-09-10 | 安庆师范学院 | Testing data compression and decompression method based on ortho-position exclusive or operation |
| CN110502599A (en) * | 2019-08-23 | 2019-11-26 | 腾讯科技（深圳）有限公司 | Querying method, device and the computer readable storage medium of map datum |
| CN110932736A (en) * | 2019-11-09 | 2020-03-27 | 天津大学 | DNA information storage method based on Raptor code and quaternary RS code |
| CN114567681A (en) * | 2022-01-25 | 2022-05-31 | 浙江数秦科技有限公司 | High-efficiency data transmission method for block chain network |
Non-Patent Citations (1)
| Title |
|---|
| 基于分区的倒排索引压缩算法研究;李俊廷;《中国优秀硕士学位论文全文数据库 (信息科技辑)》;20170615;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115276662A (en) | 2022-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115276662B (en) | Method for efficiently compressing and transmitting micro-service user information | |
| EP0216813B1 (en) | Data transmission | |
| CN108347302B (en) | Coding and decoding method and terminal | |
| Takishima et al. | Reversible variable length codes | |
| CN1101997C (en) | Method and apparatus for rate determination in communication system | |
| US7770091B2 (en) | Data compression for use in communication systems | |
| CN108631942A (en) | Coding method, interpretation method, device and equipment | |
| US11342945B2 (en) | Method and apparatus for processing rate matching of polar codes | |
| CN1426629A (en) | Method and apparatus for optimized lossless compression using plurality of coders | |
| CN114614833A (en) | Test data compression and decompression method for self-adaptive run-length coding | |
| CN116033034B (en) | Data processing system for wireless receiving and transmitting platform | |
| CN1168218C (en) | Device and method for generating data flow and device and method for reading data flow | |
| US6298165B1 (en) | Method for improving data encoding and decoding efficiency | |
| JP3639184B2 (en) | Method for encoding control information in communication system | |
| CN111030704B (en) | Polarization code-based synchronization-free communication method, device and system | |
| CN110401458B (en) | Data check coding method and system | |
| WO1999062181A1 (en) | Error protecting method and error protective device | |
| US10931303B1 (en) | Data processing system | |
| US7272131B2 (en) | TFCI decoding circuit and decoding method | |
| US20240056097A1 (en) | Compressing probability tables for entropy coding | |
| CN117240409B (en) | Data processing method for smart phone and smart wearable device | |
| CN1258888C (en) | Lossy compression of stakes for turbo decoder | |
| RU2510941C1 (en) | Information transmission and reception system | |
| JPS6276931A (en) | Data compressor | |
| CN115589229A (en) | Efficient storage method for e-commerce platform transaction data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |