CN102710263A - Entropy decision optimal differential coding-based Lempel-Ziv-Welch (LZW) compression method - Google Patents

Abstract

The invention discloses an entropy decision optimal differential coding-based Lempel-Ziv-Welch (LZW) compression method, and mainly aims to solve the problem that a slowly variable data compression ratio is low and cannot reach an optimal compression ratio in the conventional method. The method is implemented by the following steps of: performing differential coding on raw data for many times, and calculating the entropy of data obtained by the differential coding of each time; comparing the entropies of the data obtained by adjacent differential coding, and performing LZW compression on the differentially coded data with the smallest entropy; storing the maximum and minimum of the differentially coded data, the differential coding times and codewords into a compressed file; and during decompression, first performing LZW decompression on the compressed file, then performing differential decoding for many times the same as the differential coding times, and finally obtaining the raw data. Slowly variable data in a wireless sensor network can be effectively compressed, and the optimal compression ratio can be obtained; and the method can be used for the compression of the slowly variable data in the wireless sensor network.

Description

LZW compression method based on the optimum differential coding of entropy judgement

Technical field

The invention belongs to technical field of data processing, particularly a kind of LZW (Lempel-Ziv-Welch) compression method is suitable in the wireless sensor network slow parameter according to compression.

Background technology

One of important process of wireless sensor network is an image data, and the data of gathering are handled accordingly, sends to receiving terminal then.A plurality of sensor nodes produce lot of data, directly do not transmit if do not handle accordingly, can consume a large amount of node energies like this.Data compression can reduce the transmission quantity of data, has great importance for saving sensor node energy and bandwidth.

Israel professor Jcacob Ziv in 1977 and Abraham Lempel have proposed famous LZ77 algorithm, and make improvements in 1978, and Here it is LZ78 has opened the gate based on the dictionary compression.Terry A.Welch improved this algorithm in 1984, had just formed present LZW (Lempel-Ziv-Welch) compressed encoding.Lzw algorithm has adopted a kind of lossless data compression algorithms of advanced string list; Each string that occurs for the first time is placed in the string list; Represent string with a numeral, if when new data splitting occurs once more, promptly available its numeral of expression replaces; And this numeral deposited in the file in the characteristic of data and few initial data after the compressed file store compressed.Earlier set up initial dictionary table during decompress(ion), set up dictionary gradually based on the data that receive then, and the string of corresponding data is extracted decompress(ion) come out based on the initial data characteristic.Its feature be can both be correct in the process of compression and decompression this dictionary of setting up, compression or decompress to accomplish after, this dictionary is dropped again.It is a kind of stable effectively with compression method fast, also use this algorithm at compression softwares such as PKZIP commonly used, this algorithm mainly is used in during view data, text application compress.It mainly is the improvement that improves storage class, the optimization of output code word length and the compressing data memory length of dictionary that existing LZW improves algorithm; Yet directly carry out the LZW compression for the data that become slowly; Though compression ratio is to a certain degree arranged, compression ratio improves and is not obvious.Therefore how to improve LZW compression algorithm in the wireless sensor network, propose fast and effectively compression algorithm and the improved compression algorithm of the LZW particularly necessity that just seems.

Summary of the invention

The objective of the invention is to deficiency, propose a kind of LZW compression method,, realize optimal compression ratio with the slow delta data in effective compression wireless sensor network based on the optimum differential coding of entropy judgement to above-mentioned prior art.

Realize that technical scheme of the present invention is: utilize the data that become slowly in the wireless sensor network, adopt repeatedly differential coding, confirm optimum differential coding number of times, carry out LZW compression, LZW decompression and m differential decoding at last, obtain initial data through calculating entropy.Concrete steps are following:

(1) establishing initial data is x ₀(i), i=1,2 ..., N, N are the length of initial data, then data x behind the m time differential coding _m(i) be:

x _m(i)=x _m-1(i)-x _m-1(i-1)

Wherein set x _M-1(0)=0, i=1,2 ... N, m=1,2,3 ..., x _M-1(i) be the m-1 time data behind the differential coding;

(2) data x behind the m time differential coding of calculating _m(i) entropy H _m(x _m):

$H_m\left(x_m\right)=-\underset{k_m=0}{\overset{N_m}{\mathrm{\Σ}}}P\left(x_m^m\left(k_m\right)\right){\log }_{2}P\left(x_m^m\left(k_m\right)\right),$

Wherein

Be data x behind the m time differential coding _m(i) all do not repeat the data that occur in,

For

Data x behind the m time differential coding _m(i) probability that occurs in, N _mFor

Number, k _m=1,2 ..., N _m

(3) establish data x behind the m+1 time differential coding _M+1(i) be:

x _m+1(i)=x _m(i)-x _m(i-1)，

Wherein set x _m(0)=0, i=1,2 ... N, x _m(i) be the m time data behind the differential coding;

(4) data x behind the m+1 time differential coding of calculating _M+1(i) entropy H _M+1(x _M+1):

$H_{m+1}\left(x_{m+1}\right)=-\underset{k_{m+1}=0}{\overset{N_{m+1}}{\mathrm{\Σ}}}P\left(x_{m+1}^{m+1}\left(k_{m+1}\right)\right){\log }_{2}P\left(x_{m+1}^{m+1}\left(k_{m+1}\right)\right),$

Wherein

Be data x behind the m+1 time differential coding _M+1(i) all do not repeat the data that occur in, For

Data x behind the m time differential coding _M+1(i) probability that occurs in, N _M+1For

Number, k _M+1=1,2 ..., N _M+1

(5) with data x behind the m time differential coding of trying to achieve in the step (2) _m(i) entropy H _m(x _m) with step (4) in data x behind the m+1 time differential coding of trying to achieve _M+1(i) entropy H _M+1(x _M+1) compare, when satisfying H _m(x _m)<h _M+1(x _M+1) time carry out next step, otherwise, continue the data behind the last time differential coding are carried out differential coding, up to the entropy of the entropy that satisfies data behind this differential coding, carry out next step greater than data behind the last time differential coding;

(6) adopt the LZW method to compress the data x behind the differential coding the m time _m(i), obtain compressed file;

(7) adopt LZW method decompressing compressed file, obtain the data x behind the differential coding the m time _m(i);

(8) to the data x behind the m time differential coding _m(i) carry out differential decoding m time, thereby obtain initial data x ₀(i).

The present invention is owing to utilize the correlation between the data to carry out differential coding earlier; Reduced the bit of storage initial data on the one hand; Increased the number of the identical string of continuous appearance on the other hand; Be fit to the dictionary coding, reduce the entropy of data, thereby can effectively compress the slow delta data in the wireless sensor network; Adopt the size of entropy to measure the result of differential coding owing to the present invention simultaneously, thereby can obtain optimum differential coding number of times, and then obtain optimal compression ratio.

Description of drawings

Fig. 1 is a realization general flow chart of the present invention;

Fig. 2 is the sub-process figure of optimum differential coding among the present invention;

Fig. 3 is LZW compression sub-process figure among the present invention;

Fig. 4 is LZW decompression sub-process figure among the present invention;

Fig. 5 is m differential decoding sub-process figure among the present invention;

Fig. 6 carries out 6 compression ratios behind the differential coding with differential coding number of times variation diagram with the present invention;

Fig. 7 is that the present invention carries out 6 entropys behind the differential coding with differential coding number of times variation diagram.

Embodiment

With reference to Fig. 1, the present invention includes following steps:

Step 1, the entropy of calculating initial data carries out differential coding to initial data, and asks the entropy behind the differential coding, with differential coding entropy comparison last time, confirms optimum differential coding number of times.

With reference to Fig. 2, the concrete realization of this step is following:

(1a) establishing initial data is x ₀(i), i=1,2 ..., N, N are the length of initial data, calculate initial data x ₀(i) entropy H ₀(x ₀):

$H_0\left(x_0\right)=-\underset{{\mathrm{<figure-callout\; id="0"\; label="k"\; filenames="HDA00001713581400062.png"\; state="\{\{state\}\}">k</figure-callout>}}_0=1}{\overset{N_0}{\mathrm{\&Sigma;}}}P\left(x_0^0\left(k_0\right)\right){\log }_{2}P\left(x_0^0\left(k_0\right)\right),$

Wherein

Be initial data x ₀(i) all do not repeat the data that occur in, For

At initial data x ₀(i) probability that occurs in, N ₀For

Number, k ₀=1,2 ..., N ₀

(1b) to initial data x ₀(i) carry out the differential coding x first time ₁(i):

x ₁(i)=x ₀(i)-x ₀(i-1)，

Wherein set x ₀(0)=0, i=1,2 ... N;

(1c) data x behind the calculating differential coding first time ₁(i) entropy H ₁(x ₁):

$H_1\left(x_1\right)=-\underset{k_1=1}{\overset{N_1}{\mathrm{\&Sigma;}}}P\left(x_1^1\left(k_1\right)\right){\log }_{2}P\left(x_1^1\left(k_1\right)\right),$

Wherein

Be data x behind the first time differential coding ₁(i) all do not repeat the data that occur in,

For

Data x behind first time differential coding ₁(i) probability that occurs in, N ₁For

Number, k ₁=1,2 ..., N ₁

(1d) with the initial data x that tries to achieve in the above-mentioned steps (1a) ₀(i) entropy H ₀(x ₀) with step (1c) in first time of trying to achieve data x behind the differential coding ₁(i) entropy H ₁(x ₁) compare, if H ₀(x ₀)<h ₁(x ₁), then execution in step 2, otherwise, carry out next step;

(1e) to the data x behind the first time differential coding ₁(i) carry out the differential coding x second time ₂(i):

x ₂(i)=x ₁(i)-x ₁(i-1)，

Wherein, set x ₁(0)=0, i=1,2 ... N;

(1f) data x behind the calculating differential coding second time ₂(i) entropy H ₂(x ₂):

$H_2\left(x_2\right)=-\underset{{\mathrm{<figure-callout\; id="2"\; label="k"\; filenames="HDA00001713581400062.png"\; state="\{\{state\}\}">k</figure-callout>}}_2=1}{\overset{N_2}{\mathrm{\&Sigma;}}}P\left(x_2^2\left(k_2\right)\right){\log }_{2}P\left(x_2^2\left(k_2\right)\right),$

Wherein

Be data x behind the second time differential coding ₂(i) all do not repeat the data that occur in,

For

Data x behind second time differential coding ₂(i) probability that occurs in, N ₂For

Number, k ₂=1,2 ..., N ₂

(1g) with the first time of trying to achieve in above-mentioned (1c) of data x behind the differential coding ₁(i) entropy H ₁(x ₁) with step (1f) in second time of trying to achieve data H behind the differential coding ₂(x ₂) entropy H ₂(x ₂) compare, if H ₁(x ₁)<h ₂(x ₂), then execution in step 2, otherwise, to the data x behind the second time differential coding ₂(i) carry out for the third time data x behind the differential coding ₃(i), then ask for the third time data x behind the differential coding ₃(i) entropy H ₃(x ₃), will be for the third time data x behind the differential coding ₃(i) entropy H ₃(x ₃) with differential coding for the second time after the entropy H of data ₂(x ₂) compare, subsequent processes is promptly worked as H by that analogy _m(x _m)<h _M+1(x _M+1) time, execution in step 2, m=0,1,2 ..., M, M satisfy H _M(x _M)<h _M+1(x _M+1) the differential coding number of times, otherwise, continue the data behind the last time differential coding are carried out differential coding, up to the entropy of the entropy that satisfies data behind this differential coding greater than data behind the last time differential coding, execution in step 2;

Step 2 is to the data x behind the differential coding _m(i) carry out the LZW compression, m=0,1,2 ..., M.

The LZW compression algorithm is the mutation about the LZ78 compression algorithm that proposes in 1984 paper of on " Computer ", being delivered by T.A.Welch " A Technique for High-Performance Data Compression "; Set up initial dictionary according to data characteristics, encoder is exported its index value as encoded radio according to the position of data in dictionary, continuous extended lexicon in the compression process; With location index representative data string; Do not preserve dictionary during compression, can set up initial dictionary according to data characteristics during decompress(ion), find data value corresponding in the dictionary according to encoded radio; Thereby decompress(ion) obtains initial data; The dictionary that extended lexicon in the process of decompress(ion), the dictionary of setting up when this decompresses are set up during with compression is identical, thereby guarantees that the data that decompress(ion) obtains are identical with initial data.

With reference to Fig. 3, the realization of this step is summarized as follows:

(2a) through the data x behind the differential coding _m(i) x that obtains _m(i) maximum in

And minimum value

By

With

Initialization dictionary D, D comprises from minimum value To maximum

All data;

(2b) initialization P is empty, and P is current prefix;

(2c) with x _m(i) the next data in are composed to C, and C is a currency, judge that [P C] is whether in dictionary D, if [P C] composed to P; If not corresponding code word W is in compressed file in dictionary D to preserve current prefix P, W is P position encoded in dictionary D, and [P C] added among the dictionary D, and C is composed to P;

(2d) inspection data x _mWhether coding (i) arrives the end, if, search current prefix P corresponding code word W in dictionary D, be saved in code word W in the compressed file; If not, return step (2c);

(2e)

m is saved in the compressed file.

Step 3, LZW decompression compressed file.

With reference to Fig. 4, the realization of this step is summarized as follows:

(3a) by x in the compressed file _m(i) maximum in

And minimum value

Initialization dictionary D, D comprises from minimum value

To maximum

All data;

(3b) first code word in the compressed file is composed to current code word cW;

(3c) that current code word cW is corresponding data are saved in the decompressing files;

(3d) current code word cW is composed to code word pW, pW is prior to the code word of current code word in the encoded data stream;

(3e) the next code word in the compressed file is composed to current code word cW;

(3f) judge that the corresponding data of code word cW are whether in dictionary D; If; The data corresponding code word cW output in the decompressing files; Compose the data that code word pW is corresponding to current prefix P, compose first data of current code word cW corresponding data string to current data C, add to [P C] among the dictionary D; If not then that code word pW is corresponding data are composed to current prefix P, and first data of current code word cW corresponding data string are composed to current number C, preserve [P C] in decompressing files, and add to [P C] among the dictionary D;

(3g) judge whether also have code word to translate in the compressed file, if turn back to (3d); If not, finish, thereby obtain the data x behind the differential coding the m time _m(i).

Step 4 is to the data x behind the m time differential coding _m(i) carry out differential decoding m time.

With reference to Fig. 5, the concrete realization of this step is following:

(4a) data of the m-1 time differential coding of calculating, x _M-1(i)=x _M-1(i-1)+x _m(i);

(4b) data of the m-2 time differential coding of calculating, x _M-2(i)=x _M-2(i-1)+x _M-1(i);

(4c) by that analogy, x _K-1(i)=x _K-1(i-1)+x _k(i), k=0,1,2 ..., m;

(4d) data of the 1st differential coding of calculating, x ₁(i)=x ₁(i-1)+x ₂(i);

(4e) calculate original data, x ₀(i)=x ₀(i-1)+x ₁(i);

Wherein, i=1,2 ..., N, x _M-1(0)=0, thus obtain initial data x ₀(i).

Effect of the present invention can be explained through following simulation example:

1. simulated conditions: adopt the 16QAM modulation system in the wireless transmitting system, chip rate 1Mbit, signal bandwidth 1M, sample rate is 160MHZ, intermediate frequency is 10.7MHZ;

2. emulation platform: MATLAB;

3. emulated data: data after the down-conversion that wireless channel is gathered down;

4. emulation content and result:

Data are compressed after the down-conversion of a) wireless channel being gathered down with the inventive method, and the emulation compression ratio is with the variation of differential coding number of times, result such as Fig. 6.

Data are carried out differential coding and are asked entropy after the down-conversion of b) wireless channel being gathered down with the inventive method, and the entropy behind the emulation differential coding is with the variation of differential coding number of times, result such as Fig. 7.

C) concrete numerical value such as the table 1 among Fig. 6 and Fig. 7.

Concrete numerical value among table 1 Fig. 6 and Fig. 7

The differential coding number of times	Compression ratio	Entropy
				0	0.98%	13.61
1	61.69%	7.99
			2	81.21%	2.87
3	83.13%	2.46
			4	80.22%	3.35
5	76.39%	4.19
			6	71.45%	5.19

5. emulation conclusion:

As can beappreciated from fig. 6 do not have the direct LZW of differential coding compression, compression ratio is not high, and it is higher to carry out LZW compression compression ratio behind the differential coding, can calculate compression ratio from table 1 and exceed 82.15% at most, explain that the present invention can realize that valid data compress;

Can find out that from Fig. 6 and Fig. 7 compression algorithm according to the present invention is that data are carried out the LZW compression behind the 3rd differential coding, this moment, compression ratio was the highest, explain that the present invention can realize the compression ratio of optimum.

To sum up, the LZW compression method based on the optimum differential coding of entropy judgement proposed by the invention can be realized bigger compression ratio to the slow delta data in the wireless sensor network, is the compression method of the slow parameter certificate of a kind of effective compression.

Claims (4)

1. the LZW compression method based on the optimum differential coding of entropy judgement comprises the steps:

(1) establishing initial data is x ₀(i), i=1,2 ..., N, N are the length of initial data, then data x behind the m time differential coding _m(i) be:

x _m(i)=x _m-1(i)-x _m-1(i-1)

Wherein set x _M-1(0)=0, i=1,2 ... N, m=1,2,3 ..., x _M-1(i) be the m-1 time data behind the differential coding;

(2) data x behind the m time differential coding of calculating _m(i) entropy H _m(x _m):

$H_m\left(x_m\right)=-\underset{k_m=0}{\overset{N_m}{\mathrm{\&Sigma;}}}P\left(x_m^m\left(k_m\right)\right){\log }_{2}P\left(x_m^m\left(k_m\right)\right),$

Wherein

Be data x behind the m time differential coding _m(i) all do not repeat the data that occur in,

For

Data x behind the m time differential coding _m(i) probability that occurs in, N _mFor

Number, k _m=1,2 ..., N _m

(3) establish data x behind the m+1 time differential coding _M+1(i) be:

x _m+1(i)＝x _m(i)-x _m(i-1)，

Wherein set x _m(0)=0, i=1,2 ... N, x _m(i) be the m time data behind the differential coding;

(4) data x behind the m+1 time differential coding of calculating _M+1(i) entropy H _M+1(x _M+1):

$H_{m+1}\left(x_{m+1}\right)=-\underset{k_{m+1}=0}{\overset{N_{m+1}}{\mathrm{\&Sigma;}}}P\left(x_{m+1}^{m+1}\left(k_{m+1}\right)\right){\log }_{2}P\left(x_{m+1}^{m+1}\left(k_{m+1}\right)\right),$

Wherein

Be data x behind the m+1 time differential coding _M+1(i) all do not repeat the data that occur in,

For

Data x behind the m time differential coding _M+1(i) probability that occurs in, N _M+1For Number, k _M+1=1,2 ..., N _M+1

(5) with data x behind the m time differential coding of trying to achieve in the step (2) _m(i) entropy H _m(x _m) with step (4) in data x behind the m+1 time differential coding of trying to achieve _M+1(i) entropy H _M+1x _(m+1) compare, when satisfying H _m(x _m)<h _M+1(x _M+1) time carry out next step, otherwise, continue the data behind the last time differential coding are carried out differential coding, up to the entropy of the entropy that satisfies data behind this differential coding, carry out next step greater than data behind the last time differential coding;

(6) adopt the LZW method to compress the data x behind the differential coding the m time _m(i), obtain compressed file;

(7) adopt LZW method decompressing compressed file, obtain the data x behind the differential coding the m time _m(i);

(8) to the data x behind the m time differential coding _m(i) carry out differential decoding m time, thereby obtain initial data x ₀(i).

2. the LZW compression method based on the optimum differential coding of entropy judgement according to claim 1, wherein the described employing of step (6) LZW method is compressed the data x behind the differential coding the m time _m(i), carry out as follows:

(6a) through the data x behind the differential coding _m(i) x that obtains _m(i) maximum in

And minimum value

By

With

Initialization dictionary D, D comprises from minimum value

To maximum All data;

(6b) initialization P is empty, and P is current prefix;

(6c) with x _m(i) the next data in are composed to C, and C is a currency, judge that [P C] is whether in dictionary D, if [P C] composed to P; If not, search current prefix P corresponding code word W in dictionary D, be saved in code word W in the compressed file, [P C] added among the dictionary D, C is composed to P;

(6d) inspection data x _mWhether coding (i) arrives the end, if, search current prefix P corresponding code word W in dictionary D, be saved in code word W in the compressed file; If not, return step (6c);

(6e) with x _m(i) maximum in

x _m(i) minimum value in

Differential coding number of times m is saved in the compressed file.

3. the LZW compression method based on the optimum differential coding of entropy judgement according to claim 1, the described employing of step (7) LZW method decompressing compressed file wherein, carry out as follows:

(7a) by x in the compressed file _m(i) maximum in

And minimum value

Initialization dictionary D, the D bag

Including from the minimum

to a maximum

all the data;

(7b) first code word in the compressed file is composed to current code word cW;

(7c) that current code word cW is corresponding data are saved in the decompressing files;

(7d) current code word cW is composed to code word pW, pW is prior to the code word of current code word in the encoded data stream;

(7e) the next code word in the compressed file is composed to current code word cW;

(7f) judge that the corresponding data of code word pW are whether in dictionary D; If; The data corresponding code word pW are saved in decompressing files; Compose the data that code word pW is corresponding to current prefix P, compose first data of current code word cW corresponding data string to current data C, add to [P C] among the dictionary D; If not then that code word pW is corresponding data are composed to current prefix P, and first data of current code word cW corresponding data string are composed to current number C, preserve [P C] in decompressing files, and add to [P C] among the dictionary D;

(7g) judge whether also have code word to translate in the compressed file, if turn back to (7d); If not, finish, thereby obtain the data x behind the differential coding the m time _m(i).

4. the LZW compression method based on the optimum differential coding of entropy judgement according to claim 1, wherein step (8) is said to the data x behind the m time differential coding _m(i) carry out differential decoding m time, carry out as follows:

(8a) data of the m-1 time differential coding of calculating, x _M-1(i)=x _M-1(i-1)+x _m(i);

(8b) data of the m-2 time differential coding of calculating, x _M-2(i)=x _M-2(i-1)+x _M-1(i);

(8c) by that analogy, x _K-1(i)=x _K-1(i-1)+x _k(i), k=0,1,2 ..., m;

(8d) data of the 1st differential coding of calculating, x ₁(i)=x ₁(i-1)+x ₂(i);

(8e) calculate original data, x ₀(i)=x ₀(i-1)+x ₁(i);

Wherein, i=1,2 ..., N, x _M-1(0)=0, thus obtain initial data x ₀(i).

Images