CN118018147A - Data processing method, device and storage medium - Google Patents
Data processing method, device and storage medium Download PDFInfo
- Publication number
- CN118018147A CN118018147A CN202410100535.XA CN202410100535A CN118018147A CN 118018147 A CN118018147 A CN 118018147A CN 202410100535 A CN202410100535 A CN 202410100535A CN 118018147 A CN118018147 A CN 118018147A
- Authority
- CN
- China
- Prior art keywords
- data
- binary
- compression
- transmitted
- compressed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 31
- 238000007906 compression Methods 0.000 claims abstract description 219
- 230000006835 compression Effects 0.000 claims abstract description 218
- 230000005540 biological transmission Effects 0.000 claims abstract description 148
- 238000000034 method Methods 0.000 claims abstract description 25
- 230000006837 decompression Effects 0.000 claims description 55
- 238000007667 floating Methods 0.000 claims description 51
- 238000012545 processing Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000002829 reductive effect Effects 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 9
- 238000004422 calculation algorithm Methods 0.000 description 7
- 229910002056 binary alloy Inorganic materials 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000036961 partial effect Effects 0.000 description 4
- 238000000354 decomposition reaction Methods 0.000 description 3
- 239000003550 marker Substances 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Landscapes
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The application relates to the technical field of computers, and provides a data processing method, a device and a storage medium, wherein the method comprises the following steps: determining a compression mode of data to be transmitted based on the data type and transmission demand information of the data to be transmitted in the computing power network; based on the compression mode, binary numbers of the data to be transmitted or successive bits of the binary numbers of the data to be transmitted are erased and the compressed data are transmitted to the receiving side. According to the method, different compression modes are selected according to different data types and transmission requirement information, the binary bit number or the compression continuous bit is erased on the data to be transmitted, on the premise that the data integrity and the safety are ensured, the overhead of data transmission is reduced, the network performance and the binary compression transmission efficiency are improved, the resource utilization is optimized, and higher efficiency and competitiveness are brought to the fields of high-performance computing, distributed systems, cloud computing and the like.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method, apparatus, and storage medium.
Background
In a computing power network, a large number of computing tasks, distributed systems and cloud computing applications are involved, which tasks require the transfer of large amounts of data, including inputs, intermediate results and outputs, between different nodes. Transmitting large amounts of data can lead to a number of problems such as bandwidth bottlenecks, transmission delays, waste of resources, and data security. Currently, large amounts of digital information are transmitted in binary form, and effective binary compression algorithms can solve or alleviate the above-mentioned problems by reducing the volume of data. However, existing digital binary compression transmission of the power network is inefficient.
Disclosure of Invention
The application provides a data processing method, a device and a storage medium, which are used for solving the problem of low digital binary compression transmission efficiency of an algorithm network in the prior art, and erasing binary bit numbers or compressing continuous bits of data to be transmitted by selecting different compression modes according to different data types and transmission requirement information, so that the cost of data transmission is reduced, the network performance and binary compression transmission efficiency are improved, the resource utilization is optimized on the premise of ensuring the data integrity and the safety, and higher efficiency and competitiveness are brought to the fields of high-performance computing, distributed systems, cloud computing and the like.
The application provides a data processing method, which is applied to a sender and comprises the following steps:
Determining a compression mode of data to be transmitted based on the data type and transmission demand information of the data to be transmitted in the computing power network;
And based on the compression mode, erasing binary numbers of the data to be transmitted or compressing continuous bits of the binary numbers of the data to be transmitted, and transmitting the compressed data to a receiving party.
According to the data processing method provided by the application, if the data type is floating point number, the compression mode is lossy compression; based on the compression mode, erasing binary numbers of the data to be transmitted and sending the compressed data to a receiver, wherein the method comprises the following steps:
determining a first significant fraction of a binary number of the data to be transmitted based on a first transmission error;
Based on the first effective decimal place, erasing binary numbers of the data to be transmitted to obtain first transmission data;
Dividing the binary number of the first transmission data into first integer data and first decimal data based on the position of the decimal point in the binary number of the first transmission data;
And sequentially transmitting the first integer data and the first decimal data to the receiver.
According to the data processing method provided by the application, if the data type is a floating point number, the compression mode is lossless compression; based on the compression mode, erasing binary numbers of the data to be transmitted and sending the compressed data to a receiver, wherein the method comprises the following steps:
determining a second significant fraction of the binary number of the data to be transmitted based on a second transmission error; the second transmission error is determined based on the significant decimal place of the decimal number of the data to be transmitted;
based on the second effective decimal place, erasing binary numbers of the data to be transmitted to obtain second transmission data;
dividing the binary number of the second transmission data into second integer data and second decimal data based on the position of the decimal point in the binary number of the second transmission data;
and sequentially transmitting the second integer data and the second decimal data to the receiver.
According to the data processing method provided by the application, if the data type is an integer, the compression mode is lossy compression; based on the compression mode, compressing the continuous bits of the binary number of the data to be transmitted and transmitting the compressed data to a receiving party, including:
Determining a first compressed bit number of binary numbers of the data to be transmitted based on a third transmission error;
determining third transmission data based on the first number of compression bits;
dividing binary numbers of the third transmission data into prefix data and suffix data based on the first compression bit number;
compressing the suffix data to obtain binary representation of the first compression bit number, and sequentially transmitting the prefix data and the binary representation of the first compression bit number to the receiver.
According to the data processing method provided by the application, if the data type is an integer, the compression mode is lossless compression; based on the compression mode, compressing the continuous bits of the binary number of the data to be transmitted and transmitting the compressed data to a receiving party, including:
determining binary numbers to be compressed based on the number of continuous occurrence of each numerical value in the binary numbers of the data to be transmitted; the binary number to be compressed represents the binary number with the largest continuous occurrence number;
compressing the binary number to be compressed to obtain a binary representation of a second compression bit number of the binary number to be compressed;
And sequentially sending the binary number of the second compression bit, the compression identifier of the binary number to be compressed and the discontinuous binary number in the data to be transmitted to the receiver.
The application provides a data processing method, which is applied to a receiving party and comprises the following steps:
receiving compressed data sent by a sender;
Determining a decompression mode of compressed data based on a data type and a compression mode of data to be transmitted;
and decompressing the compressed data based on the decompression mode.
According to the data processing method provided by the application, if the data type is floating point number, the compression mode is lossy compression; the decompressing the compressed data based on the decompression mode includes:
Converting first integer data and first decimal data in the compressed data into decimal numbers to obtain first decompressed data, wherein the error between the first decompressed data and the data to be transmitted is smaller than a first transmission error;
if the data type is a floating point number, the compression mode is lossless compression; the decompressing the compressed data based on the decompression mode includes:
and converting the second integer data and the second decimal data in the compressed data into decimal numbers to obtain second decompressed data, wherein the sum of the second decompressed data and the second transmission error is the data to be transmitted.
According to the data processing method provided by the application, if the data type is an integer, the compression mode is lossy compression; the decompressing the compressed data based on the decompression mode includes:
Decompressing suffix data in the compressed data to obtain decimal numbers of a first compressed digit; the decimal number of the first compressed bit number characterizes the number of binary numbers compressed in the suffix data;
recovering the suffix data based on the decimal number of the first compressed bit number;
converting prefix data and suffix data in the compressed data into decimal numbers to obtain third decompressed data, wherein the error between the third decompressed data and the data to be transmitted is smaller than a third transmission error;
if the data type is an integer, the compression mode is lossless compression; the decompressing the compressed data based on the decompression mode includes:
Decompressing binary numbers to be compressed in the compressed data based on the compression identifier to obtain decimal numbers of second compression digits; the decimal number of the second compression bit number characterizes the number of compressed binary numbers in the binary numbers to be compressed;
Restoring the binary number to be compressed based on the decimal number of the second compression bit number;
and converting the discontinuous binary numbers in the data to be transmitted and the binary numbers to be compressed into decimal numbers to obtain the data to be transmitted.
The application provides a data processing device, which comprises a converter, an eraser, a compressor and a decompressor;
the converter is used for carrying out the binary conversion on the data of the sender or the receiver;
the eraser is used for erasing data to be transmitted;
The compressor is used for compressing data to be transmitted;
the decompressor is used for decompressing the compressed data.
The present application also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a data processing method as described in any of the above.
The data processing method, the data processing device and the storage medium provided by the application determine the compression mode of the data to be transmitted based on the data type and the transmission demand information of the data to be transmitted in the computing power network; based on the compression mode, binary numbers of the data to be transmitted or successive bits of the binary numbers of the data to be transmitted are erased and the compressed data are transmitted to the receiving side. According to the method, different compression modes are selected according to different data types and transmission requirement information, the binary bit number or the compression continuous bit is erased on the data to be transmitted, on the premise that the data integrity and the safety are ensured, the overhead of data transmission is reduced, the network performance and the binary compression transmission efficiency are improved, the resource utilization is optimized, and higher efficiency and competitiveness are brought to the fields of high-performance computing, distributed systems, cloud computing and the like.
Drawings
In order to more clearly illustrate the application or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the application, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a data processing method according to the present application;
FIG. 2 is one of the floating point number decomposition schematics provided by the present application;
FIG. 3 is a second exemplary floating point number decomposition diagram provided by the present application;
FIG. 4 is one of the integer exploded schematic diagrams provided by the present application;
FIG. 5 is a second diagram of integer decomposition provided by the present application;
FIG. 6 is a schematic diagram of integer compression provided by the present application;
FIG. 7 is one of the integer decompression schematics provided by the present application;
FIG. 8 is a second diagram of integer decompression provided by the present application;
FIG. 9 is a second flow chart of the data processing method according to the present application;
FIG. 10 is a schematic diagram of a data processing apparatus according to the present application;
fig. 11 is a schematic structural diagram of an electronic device provided by the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The data processing method, apparatus and storage medium of the present application are described below in conjunction with fig. 1-11.
Specifically, the present application provides a data processing method, and referring to fig. 1, fig. 1 is one of flow charts of the data processing method provided by the present application.
The data processing method provided by the embodiment of the application comprises the following steps:
step 100, determining a compression mode of data to be transmitted based on the data type and transmission demand information of the data to be transmitted in a computing power network;
In the computing power network, the data to be transmitted comprises task data, calculation result data, control information and the like. Wherein the data types include floating point numbers and integers. The compression mode comprises lossy compression and lossless compression, and specifically comprises lossy compression of floating point numbers, lossless compression of floating point numbers, lossy compression of integers and lossless compression of integers.
The transmission requirement information at least comprises: data size: refers to the size of the data file or data stream to be transmitted; bandwidth limitation: refers to the upper limit of bandwidth available during transmission, i.e., how fast data can be transmitted; transmission delay: refers to the time required from the start of transmission to the completion of reception, including transmission delay, processing delay, and the like; transmission reliability requirements: the requirement degree of ensuring the integrity and reliability of data in the data transmission process is referred to as a mechanism such as repeated data transmission, error correction codes and the like; target device and network environment: the device and network environment of the receiving party refer to data, and the device performance, network connection quality, stability and other factors are included; safety requirements: refers to the degree of security measures such as encryption, authentication and the like required for data transmission.
For example, if the integrity of the binary data is important to the recipient, no loss or distortion can occur, then a lossless compression mode is selected. The lossless compression can ensure that the original data is completely restored after decompression, and is suitable for scenes in which each detail and precision of the data need to be reserved. For another example, if the space for transmission or storage is limited and the data can withstand some degree of loss and distortion, a lossy compression scheme is selected to achieve a higher compression rate. Wherein lossy compression may reduce the amount of data by sacrificing some of its details and accuracy. As another example, different types of binary data may have different characteristics and application scenarios, such as audio data may use lossy compression (e.g., MP3 format) to reduce file size, because human ears have a weaker perceptibility of some high frequency details; for text data, execution code, etc., lossless compression may be selected to ensure accuracy and integrity of the data.
Step 200, based on the compression mode, erasing the binary number of the data to be transmitted or compressing the continuous bits of the binary number of the data to be transmitted, and sending the compressed data to the receiving party.
After the compression mode is determined, binary numbers of data to be transmitted (such as floating point numbers) are erased based on the compression mode, or successive bits of binary numbers of the data to be transmitted (such as integers) are compressed, and the compressed data are transmitted to a receiving side. The number of erasure bits plays a compressing role, so the number of erasure bits can be understood as a compressing way.
The embodiment of the application mainly compresses the secondary system number in the following two modes:
(1) Number of erasure binary bits: the amount of data is reduced by discarding some bits in the binary data. For example, the number of bits and the position of erasure may be selected based on the characteristics of the data, the requirements for the quality of the data, and the transmission error.
(2) Compressing successive bits: by compressing the consecutive bit sequences, a smaller number of bits is used for representation.
According to the data processing method provided by the embodiment of the application, the compression mode of the data to be transmitted is determined based on the data type and the transmission demand information of the data to be transmitted in the computing power network; based on the compression mode, binary numbers of the data to be transmitted or successive bits of the binary numbers of the data to be transmitted are erased and the compressed data are transmitted to the receiving side. According to the embodiment of the application, different compression modes are selected according to different data types and transmission requirement information, the binary bit number or the compression continuous bit is erased on the data to be transmitted, the overhead of data transmission is reduced, the network performance and binary compression transmission efficiency are improved, the resource utilization is optimized on the premise of ensuring the data integrity and the safety, and higher efficiency and competitiveness are brought to the fields of high-performance computing, distributed systems, cloud computing and the like.
Based on the above embodiment, if the data type is a floating point number, the compression mode is lossy compression; based on the compression mode, erasing binary numbers of the data to be transmitted and sending the compressed data to a receiver, wherein the method comprises the following steps:
step 211, determining a first significant fraction of binary numbers of the data to be transmitted based on the first transmission error;
Step 212, erasing the binary number of the data to be transmitted based on the first effective decimal place to obtain first transmission data;
step 213, dividing the binary number of the first transmission data into first integer data and first decimal data based on the position of the decimal point in the binary number of the first transmission data;
step 214, sequentially sending the first integer data and the first fractional data to the receiving party.
Assuming that the data to be transmitted is a decimal floating point number a, the compression mode is lossy compression. Let a=a '+δ, where a' is a number with a greater number of zero suffixes in binary representation, and a 'is replaced by a' in the subsequent transmission process; 0.ltoreq.delta < delta is a very small number determined by the error accuracy delta (i.e., the first transmission error) acceptable to the receiving party.
As shown in fig. 2, assuming 0.ltoreq.δ < 0.01, the decimal number of the data to be transmitted is 3.17, and the binary number of the first transmission error 0.01 is:
0.01 (decimal) = 0.0000001010001111 (binary), where the binary result retains sixteen decimal places.
The binary numbers of the data 3.17 to be transmitted are:
3.17 (decimal) = 11.0010101110000101 (binary), where the binary result retains sixteen decimal places.
According to the binary number with the first transmission error of 0.01, the binary numbers with six bits after the decimal point are all 0, and based on the binary number, the decimal of the binary number with the data to be transmitted of 3.17 needs to be reserved for at least 6 bits, so that the transmission error can be ensured to be smaller than 0.01.
Fig. 2 retains 7 significant decimal places (i.e., the first significant decimal place) of the binary number of 3.17 of the data to be transmitted, and then erases the data after the 7 th digit to obtain first transmission data a ', i.e., a ' = 11.0010101, where the decimal number of a ' is 3.1640625. Further based on the position of the a 'decimal point, a' will be divided into 11 (i.e. first integer data) and 0010101 (i.e. first decimal data), and finally 11 and 0010101 will be sent to the receiving party in turn. That is, the portion before the a 'decimal point is transmitted as first partial data, and the portion after the a' decimal point is transmitted as second partial data.
Alternatively, δ=0 is allowed, meaning that a is a fractional number with a finite number of binary digits, or that a's binary has a lot of suffix zeros.
The lossy compression of the floating point number provided by the embodiment of the application can reduce the data volume by sacrificing a certain precision, thereby saving space and bandwidth during storage and transmission and improving the data transmission efficiency.
Based on the above embodiment, if the data type is a floating point number, the compression mode is lossless compression; based on the compression mode, erasing binary numbers of the data to be transmitted and sending the compressed data to a receiver, wherein the method comprises the following steps:
step 221, determining a second significant fraction of the binary number of the data to be transmitted based on the second transmission error; the second transmission error is determined based on the significant decimal place of the decimal number of the data to be transmitted;
step 222, erasing the binary number of the data to be transmitted based on the second effective decimal place to obtain second transmission data;
Step 223, dividing the binary number of the second transmission data into second integer data and second decimal data based on the position of the decimal point in the binary number of the second transmission data;
And 224, sequentially transmitting the second integer data and the second decimal data to the receiving party.
Assuming that the data to be transmitted is a decimal floating point number a, the compression mode is lossless compression, the effective decimal number of the decimal floating point number a is DP (a) =α, and the sender and the receiver communicate in advance, and both parties know α.
Let 0 < δ < 10 -α be a number meeting the transmission error, and obtain a ' =a- δ meeting the requirement, where a ' is a number with more suffix zero in binary representation, and a ' is substituted for a in the subsequent transmission process. Meanwhile, a=a ' +δ=d α(a')+10-α,Dα (a ') at the same time means that only the α -bit fraction of a ' is retained; 10 -α denotes a second transmission error, i.e. a determination based on the significant fraction (i.e. α) of the decimal number of the data to be transmitted.
As shown in fig. 3, assuming 0.ltoreq.δ < 0.01, the decimal number of the data to be transmitted is 3.17, and the binary number of the second transmission error 0.01 is:
0.01 (decimal) = 0.0000001010001111 (binary), where the binary result retains sixteen decimal places.
The binary numbers of the data 3.17 to be transmitted are:
3.17 (decimal) = 11.0010101110000101 (binary), where the binary result retains sixteen decimal places.
According to the binary number with the second transmission error of 0.01, the binary numbers with six bits after the decimal point are all 0, and based on the binary number, the decimal of the binary number with the data to be transmitted of 3.17 needs to be reserved for at least 6 bits, so that the transmission error can be ensured to be smaller than 0.01.
Fig. 3 retains 7 significant decimal places (i.e., the second significant decimal place) of the binary number of data 3.17 to be transmitted, and then erases the data after the 7 th digit to obtain second transmission data a ', i.e., a ' = 11.0010101, where the decimal number of a ' is 3.1640625. Further based on the position of the a 'decimal point, a' will be divided into 11 (i.e. the second integer data) and 0010101 (i.e. the second decimal data), and finally 11 and 0010101 will be sent to the receiving party in turn. That is, the portion before the a 'decimal point is transmitted as first partial data, and the portion after the a' decimal point is transmitted as second partial data.
Optionally, δ is not equal to 0, because the embodiment of the present application needs to carry a 'after decompressing to obtain a', if the binary of a is limited or suffixed with zero, it is not suitable for lossless compression of floating point numbers. For example, the binary representation 11.101 of the floating point number 3.625 is only 5 bits, and cannot be broken into a binary shorter approximation and a sum of a small number, so that the floating point number 3.625 is not required to be erased and is directly sent in the two parts before and after the small number.
The lossless compression of the floating point number provided by the embodiment of the application can reduce the data quantity while maintaining the data precision, thereby improving the storage and transmission efficiency. Meanwhile, the data integrity can be ensured, and no extra error or distortion is introduced.
Based on the above embodiment, if the data type is an integer, the compression mode is lossy compression; based on the compression mode, compressing the continuous bits of the binary number of the data to be transmitted and transmitting the compressed data to a receiving party, including:
step 231, determining a first compression bit number of the binary number of the data to be transmitted based on the third transmission error;
Step 232, determining third transmission data based on the first compression bit number;
Step 233, dividing the binary number of the third transmission data into prefix data and suffix data based on the first compression bit number;
and step 234, compressing the suffix data to obtain a binary representation of the first compression bit number, and sequentially transmitting the prefix data and the binary representation of the first compression bit number to the receiver.
Assuming that the data to be transmitted is a decimal integer a, the compression mode is lossy compression, so that the decimal integer a=a ' +delta, wherein a ' is an integer with more suffix zero after binary coding, and a ' is used for replacing a in the subsequent transmission process; 0.ltoreq.delta < delta is a very small integer determined by the error accuracy delta (i.e., the third transmission error) acceptable to the receiver.
As shown in fig. 4, assuming 0.ltoreq.δ < 10, the decimal number of the data to be transmitted is 317, and the binary number of the third transmission error 10 is:
10 (decimal) =1010 (binary).
The binary numbers of the data 317 to be transmitted are:
317 (decimal) = 100111101 (binary).
As can be seen from the binary number of the third transmission error 10, if the binary number is greater than four, the transmission error is greater than 10, based on which it can be determined that the first compressed number of bits of the binary number of the data 317 to be transmitted is 3 bits or less.
In fig. 4, the decimal number of the data 317 to be transmitted is replaced by 0 in 3 bits, so as to obtain third transmission data a '= 100111000, i.e. 101 is replaced by 0, wherein 101 (binary) corresponds to 5 (decimal), and therefore, the transmission error between a and a' is smaller than 10. Further, based on the first number of compression bits, the binary number of the third transmission data is divided into prefix data and suffix data, for example, the binary representation of a' is divided into two parts: prefix data (i.e., non-suffix) and suffix data, wherein the suffix data is defined as all zeros at the end of the a 'binary, the prefix data is defined as the remainder, and as shown in fig. 4, the prefix data of the third transmission data a' is 100111, and the suffix data is 000.
The suffix data of the third transmission data a' is compressed, that is, the consecutive bits 0 are compressed. In transmitting the third transmission data, a' is divided into two parts for transmission, the prefix data is directly transmitted, the suffix data is transmitted as binary representation of the number of bits of zero, as shown in fig. 4, the prefix data is 10011 and the suffix data 000, the suffix has 30 s, and the transmission becomes binary representation 11 of 3, that is, the transmission time is divided into two parts 10011 and 11.
The integer lossy compression provided by the embodiment of the application can reduce the data volume at the expense of certain precision, thereby improving the storage and transmission efficiency.
Based on the above embodiment, if the data type is an integer, the compression mode is lossless compression; based on the compression mode, compressing the continuous bits of the binary number of the data to be transmitted and transmitting the compressed data to a receiving party, including:
step 241, determining a binary number to be compressed based on the number of continuous occurrences of each value in the binary number of the data to be transmitted; the binary number to be compressed represents the binary number with the largest continuous occurrence number;
step 242, compressing the binary number to be compressed to obtain a binary representation of a second compression bit number of the binary number to be compressed;
step 243, sequentially sending the binary number of the second compression bit, the compression identifier of the binary number to be compressed, and the discontinuous binary number in the data to be transmitted to the receiving party.
The data to be transmitted is assumed to be decimal integer a, and the compression mode is lossless compression. Firstly converting a into binary representation, then counting the number of continuous occurrences of 0 and 1 in the binary representation of a, and determining binary numbers to be compressed according to the duty ratio of continuous 1 or continuous 0 occurrences. If the number of continuous 1 in the binary system of a is greater than the number of continuous 0, compressing 1, wherein the continuous bit 1 is the binary system to be compressed; if the number of consecutive 1 s < the number of consecutive 0 s in the binary of a, then 0 s are compressed, at which time the 0 s of consecutive bits are the binary number to be compressed. If the number of consecutive 1 s is the same as the number of consecutive 0 s, compression 0 or 1 may be selected. Wherein the second number of compression bits of the binary number to be compressed characterizes the number of bits compressed by 0 or 1, e.g. assuming that 1 of 4 consecutive bits is compressed, the second number of compression bits is 4.
When consecutive 1 s are compressed, the binary representation is partitioned according to whether or not it is consecutive 1 s. The non-continuous 1 is not compressed and is directly transmitted, and continuous 1 only transmits the binary representation of continuous number. As shown in fig. 5 to 6, the decimal number of the data to be transmitted is 317, its binary number is 100111101, the number of consecutive 1 s is 4 in the binary representation of 317, and there is no consecutive 0s, so it is divided into three parts, the first part is 100, the second part is 1111, and the third part is 01, wherein the first part and the third part represent discontinuous 1 s, the second part represent continuous 1 s, the first part and the third part do not compress, the second part compresses into the binary representation of the number of continuous 1 s, that is, the binary representation of 4s is 100, and thus the whole binary information is finally compressed into three blocks 100, 01. Further, in order to facilitate distinguishing compression 0 from compression 1, a marker block is added at the forefront of the transmitted information to represent the compressed number, namely, a compression identifier is added, for example, if compression 1 is carried out, 1 is taken as the compression identifier; if 0 is compressed, 0 is used as the compression identifier, that is, the 317 last transmission data is 110010001.
Alternatively, if there are more consecutive 0s in the binary, compression 0s may be selected, decompression of the information by compression 0s is simpler, because the binary does not start with 0s, and therefore the first segment should be a non-consecutive block, the second segment is a consecutive block of 0s, and so on, except for the marker block, the transmission number may be resumed.
Alternatively, if there are neither consecutive 1 nor consecutive 0 in the binary representation, then no more marker blocks are added nor blocked, and sent directly. If the receiver only receives one binary block, the corresponding integer is not compressed, and the corresponding integer can be directly converted into decimal and output.
The lossless compression of the integer provided by the embodiment of the application can reduce the data volume while maintaining the data integrity, thereby improving the storage and transmission efficiency.
Referring to fig. 9, the present application provides a data processing method, which is applied to a receiving side, including:
step 300, receiving compressed data sent by a sender;
step 400, determining a decompression mode of compressed data based on a data type and a compression mode of the data to be transmitted;
The compression method includes lossy decompression of floating point numbers, lossless decompression of floating point numbers, lossy decompression of integers, and lossless decompression of integers. If the data type is floating point number, the compression mode is the lossy compression of the floating point number, and the decompression mode is the lossy decompression of the floating point number; if the data type is the floating point number, the compression mode is lossless compression of the floating point number, and the decompression mode is lossless decompression of the floating point number; if the data type is an integer, the compression mode is the lossy compression of the integer, and the decompression mode is the lossy decompression of the integer; if the data type is an integer, the compression mode is lossless compression of the integer, and the decompression mode is lossless decompression of the integer.
And 500, decompressing the compressed data based on the decompression mode.
And decompressing the compressed data based on the determined decompression mode.
(1) If the data type is floating point number, the compression mode is lossy compression; based on decompression mode, decompress compressed data, include: and converting the first integer data and the first decimal data in the compressed data into decimal numbers to obtain first decompressed data, wherein the error between the first decompressed data and the data to be transmitted is smaller than the first transmission error.
Specifically, after receiving the two binary representations, the receiving side recovers the integer part and the decimal part in sequence, and obtains a value a' which is different from the true value by not more than the error precision delta.
For example, referring to fig. 2, after receiving the first integer data 11 and the first fractional data 0010101, the receiving party combines the first integer data 11 and the first fractional data 0010101 to obtain a binary representation 11.0010101 of a', and converts 11.0010101 to decimal to obtain a received number 3.1640625, wherein an error between 3.1640625 and 3.17 is less than 0.01, and a requirement of less than 0.01 of the first transmission error is satisfied.
The lossy decompression of the floating point number provided by the embodiment of the application can realize a faster decompression speed while maintaining a certain compression ratio and data precision.
(2) If the data type is floating point number, the compression mode is lossless compression; based on decompression mode, decompress compressed data, include: and converting the second integer data and the second decimal data in the compressed data into decimal numbers to obtain second decompressed data, wherein the sum of the second decompressed data and the second transmission error is the data to be transmitted.
Specifically, after receiving the binary representation of a ' and converting the binary representation into decimal, the receiver only retains the alpha decimal of a ', deletes the decimal after alpha, and then obtains D α (a '), and then, the actual transmission data a can be recovered for the result + -α.
For example, referring to fig. 3, after receiving the second integer data 11 and the second fractional data 0010101, the receiving side combines the second integer data 11 and the second fractional data 0010101 to obtain 11.0010101, and converts 11.0010101 into decimal to obtain the receiving number 3.1640625. Since a=a' +δ=d α(a')+10-α, 0 < δ < 0.01, where DP (3.17) =α=2, 10 -2＝0.01;Dα (3.1640625) means that only the 2-bit fraction of 3.1640625 is retained, i.e. D α (3.1640625) =3.16, i.e. a=3.16+0.01=3.17, resulting in the number actually transmitted.
The lossless decompression of the floating point number provided by the embodiment of the application can restore the precision and detail of the original data while maintaining the data integrity.
(3) If the data type is an integer, the compression mode is lossy compression; based on decompression mode, decompress compressed data, include: decompressing suffix data in the compressed data to obtain decimal numbers of a first compressed digit; the decimal number of the first compressed bit number characterizes the number of binary numbers compressed in the suffix data; recovering suffix data based on the decimal number of the first compressed bit number; and converting the prefix data and the suffix data in the compressed data into decimal numbers to obtain third decompressed data, wherein the error between the third decompressed data and the data to be transmitted is smaller than the third transmission error.
Specifically, after receiving the two-section binary representation, the receiver recovers the digits of the suffix 0, spells the two-section prefix and the suffix into a 'correct binary representation, and finally converts the binary representation into decimal to obtain a'. Wherein a 'is less than or equal to a, and a-a' is less than or equal to 0 and less than delta.
For example, referring to fig. 4, the prefix data is 100111, the binary number of the first compression bit of the suffix data is 11, and the corresponding decimal number is 3, that is, it means that the suffix data is compressed by 30 s, and at this time, the suffix data may be obtained as 000. Prefix data 100111 and suffix data 000 are then combined to yield 100111000 and 100111000 is converted to decimal to yield 312. The error between 312 and 317 is 5, which satisfies the requirement of less than the third transmission error 10.
The integer lossy decompression provided by the embodiment of the application can realize faster decompression speed while maintaining a certain compression ratio and data precision.
(4) If the data type is an integer, the compression mode is lossless compression; based on decompression mode, decompress compressed data, include: decompressing binary numbers to be compressed in the compressed data based on the compression identifier to obtain decimal numbers of second compression digits; the decimal number of the second compression digit characterizes the number of binary numbers compressed in the binary numbers to be compressed; restoring the binary number to be compressed based on the decimal number of the second compression bit number; and converting the discontinuous binary numbers and the binary numbers to be compressed in the data to be transmitted into decimal numbers to obtain the data to be transmitted.
Specifically, when the receiving party receives compressed binary information, compressed numbers are obtained from the marking blocks, and then the decoding is performed on the parts, the principle is that continuous parts and discontinuous parts (also called compressed parts and non-compressed parts) are staggered, two continuous blocks or two discontinuous blocks cannot occur, and therefore the decoding is unique.
Referring to fig. 6-7, taking the integer 317 compressed information as an example for decompression, if the first portion 100 is the number of consecutive 1 s, 4 consecutive 1111 s, then the second portion 100 should be a non-consecutive block, and the non-consecutive blocks except for the non-consecutive block located at the first stage when compressing 1 s should start with 0 s, otherwise the 1 s at the beginning of the non-consecutive block should be written into the consecutive block of the previous 1 s when compressing. Therefore, this contradicts the second portion 100, and therefore, it is known that the first portion 100 is a discontinuous block, the second portion 100 is 4 continuous 1111, and the third portion 01 is a discontinuous block. Based on this, the first, second and third parts are concatenated to give 100111101, and 100111101 is then converted to decimal 317, the number actually transmitted.
Similarly, referring to fig. 8, in the decompression mode of compressing the continuous 0, the first portion 101 is a non-continuous block, the second portion 101 is 5 continuous 00000, the third portion 111 is a non-continuous block, the first portion, the second portion and the third portion are spliced to obtain 10100000111, and then 10100000111 is converted into decimal to obtain 1287, that is, the number actually transmitted.
The lossless decompression of the integer provided by the embodiment of the application can restore the precision and detail of the original data while maintaining the data integrity.
For further analytical description of the data processing method proposed by the present application, reference is made to the following examples.
In order to solve the related problems of the existing digital binary compression transmission of the power network, the embodiment of the application particularly provides a compression decompression method of the data binary transmission of the power network, which provides two compression transmission modes of lossy compression and lossless compression according to the type (floating point number or integer) and transmission requirement of the data to be transmitted, and replaces the compression effect by erasing the binary bit number or compression continuous bit with acceptable transmission error or simpler calculation process, thereby saving bandwidth, accelerating transmission speed, reducing delay and transmission consumption and improving the safety of the data.
The compression and decompression method provided by the embodiment of the application is a binary transmission compression and decompression method based on the prior interaction of necessary information of a transmitting party and a receiving party, and specifically comprises the following contents:
The key information of the transmission number is transmitted interactively in advance by the sending and receiving parties, including whether the transmission data type is a floating point number or an integer, whether the compression mode adopts lossy compression or lossless compression, the acceptable error precision delta for lossy compression or the effective decimal number DP (a) of the lossless compression of the floating point number (the effective decimal number of the original transmission number a), which is the precondition of the further compression of the embodiment of the application.
When the floating point number is compressed, a method of erasing binary bits smaller than the acceptable error precision delta in the binary representation of the original floating point number a is adopted, so that a '=a-delta meeting the requirement is obtained, and the longer binary is compressed into the binary representation of a number a' with an acceptable length.
When floating point number is in lossy compression, delta is required to be more than or equal to 0 and less than delta; in the case of floating point lossless compression, 0 < delta is required. If the binary system of the transmission floating point number is limited, the method is not suitable for erasure compression, and the binary system is selected to be directly transmitted according to the blocks before and after the binary system decimal point.
When decompressing the floating point number without lossless compression, converting the received information into decimal representation, and judging whether the floating point number is compressed or not. If the effective decimal number of the decimal floating point number converted directly from the received information is consistent with DP (a), the number is not compressed to be the original transmission number; if the effective decimal number of the decimal floating point number converted directly from the received information is inconsistent with the DP (a), the number is a number a' obtained by erasing and compressing the original transmission number. And then decompressing, only retaining the decimal number of the DP (a) bit of the received information decimal number a', deleting the decimal number after the DP (a) bit, and adding the negative DP (a) power of 10, namely + -α, to recover the original number.
When integer lossy compression is performed, the binary representation of the number a' within an acceptable error range is divided into two parts, and the non-suffix and the suffix are transmitted respectively. The non-suffix is transmitted directly, and the suffix is transmitted with a binary representation of the number of bits of zero.
When the integer lossless compression is carried out, the continuous 0 or 1 with larger proportion in the digital binary system to be transmitted is directly compressed, wherein the continuous is more than or equal to 3. The compression is performed by replacing the original consecutive digits with a binary representation of the number of consecutive digits to compress the number of binary digits to be transmitted.
The obtained transmitting information after the integer lossless compression is crossed with the continuous blocks and the discontinuous blocks except the marked blocks, so that the obtained compression information can obtain a unique decompression result. If the binary representation has neither continuous 1 nor continuous 0, the binary representation is not suitable for an integer compression algorithm for compressing continuous 0 or 1, a mark block is not added any more, and the binary representation is not segmented and directly transmitted.
The receiving party firstly judges whether the original data is compressed or not according to whether the information is segmented after receiving the information, if the binary information is not segmented, the receiving party indicates that the original data is not compressed and can be directly restored into a decimal integer, and if the binary information is segmented, decompression is carried out according to a segmented decompression algorithm.
The embodiment of the application realizes the compression transmission of the data on the power network, realizes the lossy or lossless compression of different types of binary data based on the advanced interactive transmission of the necessary information of the sender and the receiver, and improves the transmission efficiency. The concrete steps are as follows:
1. Aiming at the transmission of different data types, the embodiment of the application provides different compression algorithms according to the characteristics of binary representation of different data, thereby greatly reducing the bandwidth required by data transmission, improving the transmission speed, reducing the delay and the cost of data transmission and optimizing the resource utilization.
2. Each compression algorithm has the applicable scene and advantages and disadvantages, and the selection of the proper compression algorithm is dependent on the information loss degree, network condition, compression ratio requirement, decompression speed, cost, performance and the like which can be accepted by a receiver according to different transmission scenes and transmission requirements.
3. A large amount of data is divided into smaller batches for transmission, the data volume of each transmission is reduced, the data smoothness can be improved, the low-bandwidth environment is adapted, the management and the control are easy, meanwhile, the batch transmission can be better adapted to the dynamic network environment, and the stability and the flexibility of the data transmission are improved.
4. The compression process also realizes the encryption of the data to a certain extent, enhances the privacy and the security of the data, makes the data more difficult to read and steal in the transmission process, reduces the loss of the data when directly exposed in an unsafe environment and is attacked, and thus protects the confidentiality of the data.
FIG. 10 is a schematic diagram of a data processing apparatus according to an embodiment of the present application, and referring to FIG. 10, there is provided a data processing apparatus including a converter, an eraser, a compressor and a decompressor; the converter is used for carrying out binary conversion on the data of the sender or the receiver; the eraser is used for erasing the data to be transmitted; the compressor is used for compressing the data to be transmitted; and the decompressor is used for decompressing the compressed data. The specific functions of each component are as follows:
A converter: the sender's converter is responsible for converting floating point number a 1 or integer a 2 from decimal to binary representation; the receiver's converter not only converts the received or decompressed binary into a decimal representation, but also computes the decompressed floating point number a ' 1 as a floating point number a 1 based on whether it is a lossless transmission or not.
An eraser: erasure of binary representation of a 1(2)(a1) a few bits later, ensuring that the erased portion 0 < delta or 0 < delta;
A compressor: compressing the suffix zero or consecutive 0 or1 of a ' 2 (2) or a 2 (2);
A decompressor: decompression of a ' 2 (2) or a 2 (2) with zero or consecutive 0 or 1 compression suffixes, recovering a ' 2 (2) or a 2 (2).
According to the embodiment of the application, different compression modes are selected according to different data types and transmission requirement information, the binary bit number or the compression continuous bit is erased on the data to be transmitted, the overhead of data transmission is reduced, the network performance and binary compression transmission efficiency are improved, the resource utilization is optimized on the premise of ensuring the data integrity and the safety, and higher efficiency and competitiveness are brought to the fields of high-performance computing, distributed systems, cloud computing and the like.
Fig. 11 illustrates a physical structure diagram of an electronic device, as shown in fig. 11, which may include: processor 1110, communication interface CommunicationsInterface 1120, memory 1130, and communication bus 1140, wherein processor 1110, communication interface 1120, memory 1130 perform communication with each other through communication bus 1140. Processor 1110 may call logic instructions in memory 1130 to perform a data processing method comprising:
Determining a compression mode of data to be transmitted based on the data type and transmission demand information of the data to be transmitted in the computing power network;
And based on the compression mode, erasing binary numbers of the data to be transmitted or compressing continuous bits of the binary numbers of the data to be transmitted, and transmitting the compressed data to a receiving party.
Or receiving compressed data sent by a sender;
Determining a decompression mode of compressed data based on a data type and a compression mode of data to be transmitted;
and decompressing the compressed data based on the decompression mode.
Further, the logic instructions in the memory 1130 described above may be implemented in the form of software functional units and sold or used as a stand-alone product, stored on a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read-only memory (ROM), a random access memory (RAM, randomAccessMemory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present application also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the data processing method provided by the above methods, the method comprising:
Determining a compression mode of data to be transmitted based on the data type and transmission demand information of the data to be transmitted in the computing power network;
And based on the compression mode, erasing binary numbers of the data to be transmitted or compressing continuous bits of the binary numbers of the data to be transmitted, and transmitting the compressed data to a receiving party.
Or receiving compressed data sent by a sender;
Determining a decompression mode of compressed data based on a data type and a compression mode of data to be transmitted;
and decompressing the compressed data based on the decompression mode.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. A data processing method, applied to a sender, comprising:
Determining a compression mode of data to be transmitted based on the data type and transmission demand information of the data to be transmitted in the computing power network;
And based on the compression mode, erasing binary numbers of the data to be transmitted or compressing continuous bits of the binary numbers of the data to be transmitted, and transmitting the compressed data to a receiving party.
2. The method according to claim 1, wherein if the data type is floating point number, the compression mode is lossy compression; based on the compression mode, erasing binary numbers of the data to be transmitted and sending the compressed data to a receiver, wherein the method comprises the following steps:
determining a first significant fraction of a binary number of the data to be transmitted based on a first transmission error;
Based on the first effective decimal place, erasing binary numbers of the data to be transmitted to obtain first transmission data;
Dividing the binary number of the first transmission data into first integer data and first decimal data based on the position of the decimal point in the binary number of the first transmission data;
And sequentially transmitting the first integer data and the first decimal data to the receiver.
3. The method according to claim 1, wherein if the data type is floating point number, the compression mode is lossless compression; based on the compression mode, erasing binary numbers of the data to be transmitted and sending the compressed data to a receiver, wherein the method comprises the following steps:
determining a second significant fraction of the binary number of the data to be transmitted based on a second transmission error; the second transmission error is determined based on the significant decimal place of the decimal number of the data to be transmitted;
based on the second effective decimal place, erasing binary numbers of the data to be transmitted to obtain second transmission data;
dividing the binary number of the second transmission data into second integer data and second decimal data based on the position of the decimal point in the binary number of the second transmission data;
and sequentially transmitting the second integer data and the second decimal data to the receiver.
4. The data processing method according to claim 1, wherein if the data type is an integer, the compression mode is lossy compression; based on the compression mode, compressing the continuous bits of the binary number of the data to be transmitted and transmitting the compressed data to a receiving party, including:
Determining a first compressed bit number of binary numbers of the data to be transmitted based on a third transmission error;
determining third transmission data based on the first number of compression bits;
dividing binary numbers of the third transmission data into prefix data and suffix data based on the first compression bit number;
compressing the suffix data to obtain binary representation of the first compression bit number, and sequentially transmitting the prefix data and the binary representation of the first compression bit number to the receiver.
5. The method according to claim 1, wherein if the data type is an integer, the compression mode is lossless compression; based on the compression mode, compressing the continuous bits of the binary number of the data to be transmitted and transmitting the compressed data to a receiving party, including:
determining binary numbers to be compressed based on the number of continuous occurrence of each numerical value in the binary numbers of the data to be transmitted; the binary number to be compressed represents the binary number with the largest continuous occurrence number;
compressing the binary number to be compressed to obtain a binary representation of a second compression bit number of the binary number to be compressed;
And sequentially sending the binary number of the second compression bit, the compression identifier of the binary number to be compressed and the discontinuous binary number in the data to be transmitted to the receiver.
6. A data processing method, for use by a receiver, comprising:
receiving compressed data sent by a sender;
Determining a decompression mode of compressed data based on a data type and a compression mode of data to be transmitted;
and decompressing the compressed data based on the decompression mode.
7. The method of claim 6, wherein if the data type is floating point number, the compression mode is lossy compression; the decompressing the compressed data based on the decompression mode includes:
Converting first integer data and first decimal data in the compressed data into decimal numbers to obtain first decompressed data, wherein the error between the first decompressed data and the data to be transmitted is smaller than a first transmission error;
if the data type is a floating point number, the compression mode is lossless compression; the decompressing the compressed data based on the decompression mode includes:
and converting the second integer data and the second decimal data in the compressed data into decimal numbers to obtain second decompressed data, wherein the sum of the second decompressed data and the second transmission error is the data to be transmitted.
8. The method according to claim 6, wherein if the data type is an integer, the compression mode is lossy compression; the decompressing the compressed data based on the decompression mode includes:
Decompressing suffix data in the compressed data to obtain decimal numbers of a first compressed digit; the decimal number of the first compressed bit number characterizes the number of binary numbers compressed in the suffix data;
recovering the suffix data based on the decimal number of the first compressed bit number;
converting prefix data and suffix data in the compressed data into decimal numbers to obtain third decompressed data, wherein the error between the third decompressed data and the data to be transmitted is smaller than a third transmission error;
if the data type is an integer, the compression mode is lossless compression; the decompressing the compressed data based on the decompression mode includes:
Decompressing binary numbers to be compressed in the compressed data based on the compression identifier to obtain decimal numbers of second compression digits; the decimal number of the second compression bit number characterizes the number of compressed binary numbers in the binary numbers to be compressed;
Restoring the binary number to be compressed based on the decimal number of the second compression bit number;
and converting the discontinuous binary numbers in the data to be transmitted and the binary numbers to be compressed into decimal numbers to obtain the data to be transmitted.
9. A data processing apparatus, characterized by being applied to the data processing method of any one of claims 1 to 8, comprising a converter, an eraser, a compressor and a decompressor;
the converter is used for carrying out the binary conversion on the data of the sender or the receiver;
the eraser is used for erasing data to be transmitted;
The compressor is used for compressing data to be transmitted;
the decompressor is used for decompressing the compressed data.
10. A non-transitory computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the data processing method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410100535.XA CN118018147A (en) | 2024-01-24 | 2024-01-24 | Data processing method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410100535.XA CN118018147A (en) | 2024-01-24 | 2024-01-24 | Data processing method, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118018147A true CN118018147A (en) | 2024-05-10 |
Family
ID=90953732
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410100535.XA Pending CN118018147A (en) | 2024-01-24 | 2024-01-24 | Data processing method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118018147A (en) |
-
2024
- 2024-01-24 CN CN202410100535.XA patent/CN118018147A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107395209B (en) | Data compression method, data decompression method and equipment thereof | |
| US8902090B2 (en) | Compressing and decompressing signal data | |
| US8144037B2 (en) | Blocking for combinatorial coding/decoding for electrical computers and digital data processing systems | |
| US7623047B2 (en) | Data sequence compression | |
| US20070279261A1 (en) | Method and apparatus for lossless run-length data encoding | |
| US20090019071A1 (en) | Blocking for combinatorial coding/decoding for electrical computers and digital data processing systems | |
| US7671767B2 (en) | LIFO radix coder for electrical computers and digital data processing systems | |
| US20090016453A1 (en) | Combinatorial coding/decoding for electrical computers and digital data processing systems | |
| US20100085224A1 (en) | Adaptive combinatorial coding/decoding with specified occurrences for electrical computers and digital data processing systems | |
| JP6045123B2 (en) | Encoder, decoder and method | |
| US20090019069A1 (en) | Data coding/decoding for electrical computers and digital data processing systems | |
| US20090015445A1 (en) | Fifo radix coder for electrical computers and digital data processing systems | |
| WO2015124323A1 (en) | Devices and methods of source-encoding and decoding of data | |
| US7518538B1 (en) | Adaptive entropy coding compression with multi-level context escapes | |
| CN114337678A (en) | Data compression method, device, equipment and storage medium | |
| US20220005229A1 (en) | Point cloud attribute encoding method and device, and point cloud attribute decoding method and devcie | |
| WO2010044099A1 (en) | Lossless content encoding | |
| US10565182B2 (en) | Hardware LZMA compressor | |
| US20140015698A1 (en) | System and method for fixed rate entropy coded scalar quantization | |
| US5010344A (en) | Method of decoding compressed data | |
| EP3224957A1 (en) | Improved compression and encryption of a file | |
| CN118018147A (en) | Data processing method, device and storage medium | |
| KR100636370B1 (en) | Apparatus and method for coding using bit-precision, and apparatus and method for decoding according to the same | |
| US6101281A (en) | Method for improving data encoding and decoding efficiency | |
| CN1656688B (en) | Processing digital data prior to compression |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |