CN102710263B - Entropy decision optimal differential coding-based Lempel-Ziv-Welch (LZW) compression method - Google Patents

Abstract

The invention discloses an entropy decision optimal differential coding-based Lempel-Ziv-Welch (LZW) compression method, and mainly aims to solve the problem that a slowly variable data compression ratio is low and cannot reach an optimal compression ratio in the conventional method. The method is implemented by the following steps of: performing differential coding on raw data for many times, and calculating the entropy of data obtained by the differential coding of each time; comparing the entropies of the data obtained by adjacent differential coding, and performing LZW compression on the differentially coded data with the smallest entropy; storing the maximum and minimum of the differentially coded data, the differential coding times and codewords into a compressed file; and during decompression, first performing LZW decompression on the compressed file, then performing differential decoding for many times the same as the differential coding times, and finally obtaining the raw data. Slowly variable data in a wireless sensor network can be effectively compressed, and the optimal compression ratio can be obtained; and the method can be used for the compression of the slowly variable data in the wireless sensor network.

Description

Based on the LZW compression method of the optimum differential coding of entropy judgement

Technical field

The invention belongs to technical field of data processing, particularly a kind of LZW (Lempel-Ziv-Welch) compression method, be suitable for becoming data compression slowly in wireless sensor network.

Background technology

One of important process of wireless sensor network is image data, and processes accordingly the data gathered, and is then sent to receiving terminal.Multiple sensor node produces a large amount of data, if do not process accordingly and directly transmit, can consume a large amount of node energies like this.Data compression can reduce the transmission quantity of data, has great importance for saving sensor node energy and bandwidth.

Within 1977, Israel professor Jcacob Ziv and Abraham Lempel proposes famous LZ77 algorithm, and makes improvements in 1978, Here it is LZ78, opens the gate based on dictionary compression.Terry A.Welch improved this algorithm in 1984, just defined present LZW (Lempel-Ziv-Welch) compressed encoding.Lzw algorithm have employed a kind of lossless data compression algorithms of advanced string list, the string that each first time occurs is placed in a string list, string is represented by a numeral, if when new data splitting occurs again, namely can with representing that its numeral replaces, and by this numeral stored in file, the characteristic of data and few initial data after a compressed file store compressed.First set up initial dictionary table according to initial data feature during decompress(ion), then set up dictionary gradually according to the data received, and the string of corresponding data extraction solution is pressed out.Its feature be can be correct in the process of compression and decompression set up this dictionary, compression or after having decompressed, this dictionary is dropped again.It is a kind of stable effectively and fast compression method, and also use this algorithm in compressed softwares such as conventional PKZIP, this algorithm is mainly used in view data, text application compression.Existing LZW innovatory algorithm mainly improves the improvement of the storage class of dictionary, output codons length optimization and compressing data memory length, but directly LZW compression is carried out for the slow data become, although there is compression ratio to a certain degree, compression ratio raising is also not obvious.Therefore how to improve LZW compression algorithm in wireless sensor network, propose the compression algorithm that compression algorithm and LZW improve fast and effectively and just seem particularly necessary.

Summary of the invention

The object of the invention is to the deficiency for above-mentioned prior art, propose a kind of LZW compression method based on the optimum differential coding of entropy judgement, effectively to compress the slow delta data in wireless sensor network, realize optimal compression ratio.

Realizing technical scheme of the present invention is: utilize the slow data become in wireless sensor network, adopt multi-difference coding, determining optimum differential coding number of times, finally carrying out LZW compression, LZW decompresses and m differential decoding, obtaining initial data by calculating entropy.Concrete steps are as follows:

(1) set initial data as x ₀(i), i=1,2 ..., N, N are the length of initial data, then data x after the m time differential coding _m(i) be:

x _m(i)=x _m-1(i)-x _m-1(i-1)

Wherein set x _m-1(0)=0, i=1,2 ... N, m=1,2,3 ..., x _m-1i () is the data after the m-1 time differential coding;

(2) data x after the m time differential coding is calculated _mthe entropy H of (i) _m(x _m):

$H_m\left(x_m\right)=-\underset{k_m=0}{\overset{N_m}{\mathrm{\Σ}}}P\left(x_m^m\left(k_m\right)\right){\log }_{2}P\left(x_m^m\left(k_m\right)\right),$

Wherein be data x after the m time differential coding _mall data do not repeated in (i), for data x after the m time differential coding _mthe probability occurred in (i), N _mfor number, k _m=1,2 ..., N _m;

(3) data x after the m+1 time differential coding is established _m+1(i) be:

x _m+1(i)=x _m(i)-x _m(i-1)，

Wherein set x _m(0)=0, i=1,2 ... N, x _mi () is the data after the m time differential coding;

(4) data x after the m+1 time differential coding is calculated _m+1the entropy H of (i) _m+1(x _m+1):

$H_{m+1}\left(x_{m+1}\right)=-\underset{k_{m+1}=0}{\overset{N_{m+1}}{\mathrm{\Σ}}}P\left(x_{m+1}^{m+1}\left(k_{m+1}\right)\right){\log }_{2}P\left(x_{m+1}^{m+1}\left(k_{m+1}\right)\right),$

Wherein be data x after the m+1 time differential coding _m+1all data do not repeated in (i), for data x after the m time differential coding _m+1the probability occurred in (i), N _m+1for number, k _m+1=1,2 ..., N _m+1;

(5) by data x after the m time differential coding of trying to achieve in step (2) _mthe entropy H of (i) _m(x _m) with the m+1 time differential coding of trying to achieve in step (4) after data x _m+1the entropy H of (i) _m+1(x _m+1) compare, when meeting H _m(x _m) <H _m+1(x _m+1) time perform next step, otherwise, continue to carry out differential coding to the data after last time differential coding, until the entropy of data is greater than the entropy of data after differential coding last time after meeting this differential coding, perform next step;

(6) the data x after adopting LZW method to compress the m time differential coding _mi (), obtains compressed file;

(7) adopt LZW method decompressing compressed file, obtain the data x after the m time differential coding _m(i);

(8) to the data x after the m time differential coding _mi () carries out m differential decoding, thus obtain initial data x ₀(i).

The present invention first carries out differential coding owing to utilizing the correlation between data, reduce the bit storing initial data on the one hand, add the number of the identical string occurred continuously on the other hand, be applicable to dictionary coding, reduce the entropy of data, thus effectively can compress the slow delta data in wireless sensor network; Simultaneously because the present invention adopts the size of entropy to carry out the result of metric difference Coded, thus can obtain optimum differential coding number of times, and then obtain optimal compression ratio.

Accompanying drawing explanation

Fig. 1 of the present inventionly realizes general flow chart;

Fig. 2 is the sub-process figure of optimum differential coding in the present invention;

Fig. 3 is that in the present invention, LZW compresses sub-process figure;

Fig. 4 is LZW decompression flow chart in the present invention;

Fig. 5 is m differential decoding sub-process figure in the present invention;

Fig. 6 carries out the compression ratio after 6 differential codings with differential coding number of times variation diagram with the present invention;

Fig. 7 is that the present invention carries out the entropy after 6 differential codings with differential coding number of times variation diagram.

Embodiment

With reference to Fig. 1, the present invention includes following steps:

Step 1, calculates the entropy of initial data, carries out differential coding, and ask the entropy after differential coding, compare with differential coding entropy last time, determine optimum differential coding number of times to initial data.

With reference to Fig. 2, being implemented as follows of this step:

(1a) set initial data as x ₀(i), i=1,2 ..., N, N are the length of initial data, calculate initial data x ₀the entropy H of (i) ₀(x ₀):

$H_0\left(x_0\right)=-\underset{k_0=1}{\overset{N_0}{\mathrm{\&Sigma;}}}P\left(x_0^0\left(k_0\right)\right){\log }_{2}P\left(x_0^0\left(k_0\right)\right),$

Wherein for initial data x ₀all data do not repeated in (i), for at initial data x ₀the probability occurred in (i), N ₀for number, k ₀=1,2 ..., N ₀;

(1b) to initial data x ₀i () carries out first time differential coding x ₁(i):

x ₁(i)=x ₀(i)-x ₀(i-1)，

Wherein set x ₀(0)=0, i=1,2 ... N;

(1c) data x after calculating first time differential coding ₁the entropy H of (i) ₁(x ₁):

$H_1\left(x_1\right)=-\underset{k_1=1}{\overset{N_1}{\mathrm{\&Sigma;}}}P\left(x_1^1\left(k_1\right)\right){\log }_{2}P\left(x_1^1\left(k_1\right)\right),$

Wherein for data x after first time differential coding ₁all data do not repeated in (i), for data x after first time differential coding ₁the probability occurred in (i), N ₁for number, k ₁=1,2 ..., N ₁;

(1d) the initial data x will tried to achieve in above-mentioned steps (1a) ₀the entropy H of (i) ₀(x ₀) with the data x after differential coding of first time of trying to achieve in step (1c) ₁the entropy H of (i) ₁(x ₁) compare, if H ₀(x ₀) <H ₁(x ₁), then perform step 2, otherwise, perform next step;

(1e) to the data x after first time differential coding ₁i () carries out second time differential coding x ₂(i):

x ₂(i)=x ₁(i)-x ₁(i-1)，

Wherein, x is set ₁(0)=0, i=1,2 ... N;

(1f) data x after calculating second time differential coding ₂the entropy H of (i) ₂(x ₂):

$H_2\left(x_2\right)=-\underset{k_2=1}{\overset{N_2}{\mathrm{\&Sigma;}}}P\left(x_2^2\left(k_2\right)\right){\log }_{2}P\left(x_2^2\left(k_2\right)\right),$

Wherein for data x after second time differential coding ₂all data do not repeated in (i), for data x after second time differential coding ₂the probability occurred in (i), N ₂for number, k ₂=1,2 ..., N ₂;

(1g) by first time of trying to achieve in above-mentioned (1c) data x after differential coding ₁the entropy H of (i) ₁(x ₁) with the second time differential coding of trying to achieve in step (1f) after data H ₂(x ₂) entropy H ₂(x ₂) compare, if H ₁(x ₁) <H ₂(x ₂), then perform step 2, otherwise, to the data x after second time differential coding ₂i () carries out data x after third time differential coding ₃(i), then ask third time differential coding after data x ₃the entropy H of (i) ₃(x ₃), data x after general's third time differential coding ₃the entropy H of (i) ₃(x ₃) with second time differential coding after the entropy H of data ₂(x ₂) compare, subsequent processes by that analogy, namely works as H _m(x _m) <H _m+1(x _m+1) time, perform step 2, m=0,1,2 ..., M, M meet H _m(x _m) <H _m+1(x _m+1) differential coding number of times, otherwise, continue to carry out differential coding to the data after last time differential coding, until the entropy of data is greater than the entropy of data after differential coding last time after meeting this differential coding, perform step 2;

Step 2, to the data x after differential coding _mi () carries out LZW compression, m=0, and 1,2 ..., M.

LZW compression algorithm is the paper delivered on " Computer " by T.A.Welch " A Techniquefor High-Performance Data Compression " the middle mutation about LZ78 compression algorithm proposed in 1984, initial dictionary is set up according to data characteristics, encoder exports its index value as encoded radio according to the position of data in dictionary, continuous extended lexicon in compression process, with location index representative data string, dictionary is not preserved during compression, initial dictionary can be set up according to data characteristics during decompress(ion), data value corresponding in dictionary is found according to encoded radio, thus decompress(ion) obtains initial data, extended lexicon in the process of decompress(ion), the dictionary set up when the dictionary set up when this decompresses and compression is identical, thus ensure that the data that decompress(ion) obtains are identical with initial data.

With reference to Fig. 3, the realization of this step is summarized as follows:

(2a) by the data x after differential coding _mi x that () obtains _mmaximum in (i) and minimum value by with initialization dictionary D, D comprises from minimum value to maximum all data;

(2b) initialization P is empty, and P is current prefix;

(2c) by x _mi the next data in () are assigned to C, C is currency, judge that [P C] is whether in dictionary D, is if so, assigned to P by [P C]; If not preserve current prefix P code word W corresponding in dictionary D in compressed file, W is position encoded in dictionary D of P, adds in dictionary D, C is assigned to P by [P C];

(2d) data x is checked _mwhether the coding of (i) arrives end, if so, searches the code word W that current prefix P is corresponding in dictionary D, and code word W is saved in compressed file; If not, return step (2c);

(2e) will m is saved in compressed file.

Step 3, LZW decompression compressed file.

With reference to Fig. 4, the realization of this step is summarized as follows:

(3a) by x in compressed file _mmaximum in (i) and minimum value initialization dictionary D, D comprises from minimum value to maximum all data;

(3b) code word of first in compressed file is assigned to current code word cW;

(3c) data corresponding for current code word cW are saved in decompressing files;

(3d) current code word cW is assigned to code word pW, pW is the code word prior to current code word in encoded data stream;

(3e) the next code word in compressed file is assigned to current code word cW;

(3f) data that code word cW is corresponding are judged whether in dictionary D, if, data corresponding for code word cW are outputted in decompressing files, data corresponding for code word pW are assigned to current prefix P, first data of current code word cW corresponding data string are assigned to current data C, [P C] is added in dictionary D; If not, then data corresponding for code word pW are assigned to current prefix P, first data of current code word cW corresponding data string are assigned to current number C, preserve [P C] in decompressing files, and [P C] is added in dictionary D;

(3g) judge whether also have code word to translate in compressed file, if so, to turn back to (3d); If not, terminate, thus obtain the data x after the m time differential coding _m(i).

Step 4, to the data x after the m time differential coding _mi () carries out m differential decoding.

With reference to Fig. 5, being implemented as follows of this step:

(4a) data of the m-1 time differential coding are calculated, x _m-1(i)=x _m-1(i-1)+x _m(i);

(4b) data of the m-2 time differential coding are calculated, x _m-2(i)=x _m-2(i-1)+x _m-1(i);

(4c) by that analogy, x _k-1(i)=x _k-1(i-1)+x _k(i), k=0,1,2 ..., m;

(4d) data of the 1st differential coding are calculated, x ₁(i)=x ₁(i-1)+x ₂(i);

(4e) original data are calculated, x ₀(i)=x ₀(i-1)+x ₁(i);

Wherein, i=1,2 ..., N, x _m-1(0)=0, thus obtain initial data x ₀(i).

Effect of the present invention can be illustrated by simulation example below:

1. simulated conditions: adopt 16QAM modulation system in wireless transmitting system, chip rate 1Mbit, signal bandwidth 1M, sample rate is 160MHZ, and intermediate frequency is 10.7MHZ;

2. emulation platform: MATLAB;

3. emulated data: data after the down-conversion gathered under wireless channel;

4. emulate content and result:

A) compress data after the down-conversion gathered under wireless channel by the inventive method, stimulation compress is than the change with differential coding number of times, and result is as Fig. 6.

B) by the inventive method, data after the down-conversion gathered under wireless channel are carried out to differential coding and asked entropy, the entropy after emulation differential coding is with the change of differential coding number of times, and result is as Fig. 7.

C) the concrete numerical value in Fig. 6 and Fig. 7 is as table 1.

Concrete numerical value in table 1 Fig. 6 and Fig. 7

Differential coding number of times	Compression ratio	Entropy
			0	0.98%	13.61
1	61.69%	7.99
			2	81.21%	2.87
3	83.13%	2.46
			4	80.22%	3.35
5	76.39%	4.19
			6	71.45%	5.19

5. emulate conclusion:

Do not have differential coding direct LZW compression as can be seen from Figure 6, compression ratio is not high, and carrying out LZW after differential coding, to compress compression ratio higher, and can calculate compression ratio from table 1 exceeds at most 82.15%, illustrates that the present invention can realize valid data and compress;

Compression algorithm according to the present invention is that after the 3rd differential coding, data carry out LZW compression as can be seen from Figures 6 and 7, and now compression ratio is the highest, illustrates that the present invention can realize optimum compression ratio.

To sum up, the LZW compression method based on the optimum differential coding of entropy judgement proposed by the invention can realize larger compression ratio to the slow delta data in wireless sensor network, is the compression method of the slow parameter certificate of a kind of effective compression.

Claims (4)

1., based on a LZW compression method for the optimum differential coding of entropy judgement, comprise the steps:

(1) set initial data as x ₀(i), i=1,2 ..., N, N are the length of initial data, then data x after the m time differential coding _m(i) be:

x _m(i)＝x _m-1(i)-x _m-1(i-1)，

Wherein set x _m-1(0)=0, i=1,2 ... N, m=1,2,3 ..., x _m-1i () is the data after the m-1 time differential coding;

(2) data x after the m time differential coding is calculated _mthe entropy H of (i) _m(x _m):

$H_m\left(x_m\right)=-\underset{k_m=1}{\overset{N_m}{\mathrm{\&Sigma;}}}P\left(x_m^m\left(k_m\right)\right){\log }_{2}P\left(x_m^m\left(k_m\right)\right),$

Wherein be data x after the m time differential coding _mall data do not repeated in (i), for data x after the m time differential coding _mthe probability occurred in (i), N _mfor number, k _m=1,2 ..., N _m;

(3) data x after the m+1 time differential coding is established _m+1(i) be:

x _m+1(i)＝x _m(i)-x _m(i-1)，

Wherein set x _m(0)=0, i=1,2 ... N, x _mi () is the data after the m time differential coding;

(4) data x after the m+1 time differential coding is calculated _m+1the entropy H of (i) _m+1(x _m+1):

$H_{m+1}\left(x_{m+1}\right)=-\underset{k_{m+1}=1}{\overset{N_{m+1}}{\mathrm{\&Sigma;}}}P\left(x_{m+1}^{m+1}\left(k_{m+1}\right)\right){\log }_{2}P\left(x_{m+1}^{m+1}\left(k_{m+1}\right)\right),$

Wherein be data x after the m+1 time differential coding _m+1all data do not repeated in (i), for data x after the m+1 time differential coding _m+1the probability occurred in (i), N _m+1for number, k _m+1=1,2 ..., N _m+1;

(5) by data x after the m time differential coding of trying to achieve in step (2) _mthe entropy H of (i) _m(x _m) with the m+1 time differential coding of trying to achieve in step (4) after data x _m+1the entropy H of (i) _m+1(x _m+1) compare, when meeting H _m(x _m) <H _m+1(x _m+1) time perform next step, otherwise, continue to carry out differential coding to the data after last time differential coding, until the entropy of data is greater than the entropy of data after differential coding last time after meeting this differential coding, perform next step;

(6) the data x after adopting LZW method to compress the m time differential coding _mi (), obtains compressed file;

(7) adopt LZW method decompressing compressed file, obtain the data x after the m time differential coding _m(i);

(8) to the data x after the m time differential coding _mi () carries out m differential decoding, thus obtain initial data x ₀(i).

2. the LZW compression method based on the optimum differential coding of entropy judgement according to claim 1, the employing LZW method wherein described in step (6) compresses the data x after the m time differential coding _m(i), carry out as follows:

(6a) by the data x after differential coding _mi x that () obtains _mmaximum in (i) and minimum value by with initialization dictionary D, D comprises from minimum value to maximum all data;

(6b) initialization P is empty, and P is current prefix;

(6c) by x _mi the next data in () are assigned to C, C is currency, judge that [P C] is whether in dictionary D, is if so, assigned to P by [P C]; If not, search the code word W that current prefix P is corresponding in dictionary D, code word W is saved in compressed file, [P C] is added in dictionary D, C is assigned to P; _

(6d) data x is checked _mwhether the coding of (i) arrives end, if so, searches the code word W that current prefix P is corresponding in dictionary D, and code word W is saved in compressed file; If not, return step (6c);

(6e) by x _mmaximum in (i) x _mminimum value in (i) differential coding number of times m is saved in compressed file.

3. the LZW compression method based on the optimum differential coding of entropy judgement according to claim 1, the employing LZW method decompressing compressed file wherein described in step (7), carries out as follows:

(7a) by x in compressed file _mmaximum in (i) and minimum value initialization dictionary D, D comprises from minimum value to maximum all data;

(7b) code word of first in compressed file is assigned to current code word cW;

(7c) data corresponding for current code word cW are saved in decompressing files;

(7d) current code word cW is assigned to code word pW, pW is the code word prior to current code word in encoded data stream;

(7e) the next code word in compressed file is assigned to current code word cW;

(7f) data that code word pW is corresponding are judged whether in dictionary D, if, data corresponding for code word pW are saved in decompressing files, data corresponding for code word pW are assigned to current prefix P, first data of current code word cW corresponding data string are assigned to current data C, [P C] is added in dictionary D; If not, then data corresponding for code word pW are assigned to current prefix P, first data of current code word cW corresponding data string are assigned to current number C, preserve [P C] in decompressing files, and [P C] is added in dictionary D;

(7g) judge whether also have code word to translate in compressed file, if so, to turn back to (7d); If not, terminate, thus obtain the data x after the m time differential coding _m(i).

4. the LZW compression method based on the optimum differential coding of entropy judgement according to claim 1, wherein step (8) is described to the data x after the m time differential coding _mi () carries out m differential decoding, carry out as follows:

(8a) data of the m-1 time differential coding are calculated, x _m-1(i)=x _m-1(i-1)+x _m(i);

(8b) data of the m-2 time differential coding are calculated, x _m-2(i)=x _m-2(i-1)+x _m-1(i);

(8c) by that analogy, x _k-1(i)=x _k-1(i-1)+x _k(i), k=0,1,2 ..., m;

(8d) data of the 1st differential coding are calculated, x ₁(i)=x ₁(i-1)+x ₂(i);

(8e) original data are calculated, x ₀(i)=x ₀(i-1)+x ₁(i);

Wherein, i=1,2 ..., N, x _m-1(0)=0, thus obtain initial data x ₀(i).