CN102510287A - Method for rapidly compressing industrial real-time data - Google Patents

Abstract

The invention discloses a kind of Fast Compression methods of industrial real-time data, two slope ks up, klow up and down are generated according to compression limit value first and constitute a window, after reading in next data point, judge the point whether in this window, if, then former point can be given up, and then generate new slope up and down The two slopes are obtained compared with two original slopes, so that crucial window be made to reduce. Revolving door algorithm, if there is n point in window, it is (n+1) n/2 that the n point in window, which adds up number of comparisons, and calculation amount is relatively large, can be significantly lower than crucial window tendency method on judging speed. The each data point of the compression method of industrial real-time data of the present invention only judges once only to judge the size that each data point saves slope and bound slope that data point is formed with upper one. Industrial real-time data compression is carried out in this way, and calculation amount is small when being realized with code, judges that speed is fast.

Description

A kind of Fast Compression method of industrial real-time data

Technical field

The invention belongs to the data compression technique field, more specifically, relate to a kind of Fast Compression method of industrial real-time data.

Background technology

1, data compression general introduction

Data compression technique is at image, and fields such as Audio Processing are existing uses very widely, and technology is advanced and mature day by day and formed international standard, like the JEPG compress technique of image processing field, and the MP3 compress technique in the Audio Processing etc.But because the continuous increase of modern capacity of memory device; That then uses in the industrial automation field is less; Can produce magnanimity in real time and in the automated system of historical data at electric power system, fault detection and diagnosis system, process control, process monitoring, multichannel data acquisition system etc., data compression is not also paid attention to widely and is used.Data are generally obtained by data collecting module collected in the multichannel Auto-Test System, and the signal of collection is generally sensor signal, and present data acquisition module sample frequency is all higher; For example total sampling rate is 100KHz; If then system is 16 passages, single passage per second can be gathered 62 times, and the existing higher acquisition module that does not wait from tens MHz to tens GHz is than taller many of this number of times; Per second just can produce a large amount of high-precision floating datas like this; In the face of the storage data of magnanimity, the way that people solve is simple increase memory device, and seldom the application data compress technique is compressed wherein a large amount of redundant datas; To reach the minimizing data volume, the purpose of conserve memory equipment.

2. existing industrial real-time data compression method

Data compression to the difference loss effect that original file data produces, can be divided into lossless compress and lossy compression method to data compression technique according to different coding.Lossless compress generally is the basis with universal compressed theory, takes classical compression algorithm such as the graceful algorithm of Harvard, has the character of undistorted, zero defect or noiseless coding.Lossy compression method is in compression process, to lose certain information to obtain higher compression ratio.Though lossy compression method can not be recovered initial data fully, the data of loss are little to the informational influence of understanding initial data, and obtain bigger compression ratio thus, thereby practice thrift a large amount of memory spaces.

At present relatively effectively and use more industrial real-time data compression method and mainly contain the stable state threshold method, i.e. dead band algorithm, the revolving door algorithm, the linear extrapolation algorithm, these three kinds of methods all belong to lossy compression method.

2.1 stable state threshold method

Stable state threshold value ratio juris is to be qualification with general patient distortion range; Through judging whether current data value and next data value determine whether giving up or write down this data greater than the compression limit value; The limit value setting is big more, and data compression rate is high more, but the distortion factor is also big more.As shown in Figure 1, if the compression limit value is set to 0.5, the current data value is 10.0, if then next data value is more than 10.5 or all will be by record below 9.5; And be starting point with the data recorded point, the value of establishing this point is y, 0.5 is judgement threshold, judges that next data value is whether between y ± 0.5; If, then give up this data point, if do not exist, then write down this data point; Be starting point with the data recorded point again, judge, data are compressed.In Fig. 1, the data point of looping goes on record.

2.2 revolving door algorithm

The revolving door algorithm is a kind of linear trend compression algorithm, with the factor that the slope variation situation of linear trendization is considered as emphasis, stresses to seek the linearity " trigger point " that changes slope, mainly contains two kinds of processing modes of parallelogram and triangle.The main thought of algorithm is whether the compression limit value area of coverage that utilizes current data point and previous memory point to contract to constitute comes judgment data should keep.If the compression area of coverage of 2 formations can cover all data points between 2; Then give up current data point; If instead there is data point to drop on beyond the area of coverage; Just preserve the previous data point of current point, and with this point for new starting point with after the point that reads in constitute the choice point that the new area of coverage continues to judge compression.Concrete compression judges that the flow process introduction is following:

If the compression limit value of revolving door is made as 0.1, time data memory is spaced apart 1s.Begin from first data point of reading in; Is axis with it to the line between the current data point; Cross at these 2 and do the parallelogram that a width is 2 multiplication of voltage shrinkage limit values, judge the parallelogram region covered whether can cover all from last memory point to all data points the current point, along with reading in of data point; Make new parallelogram with same method, as shown in Figure 2.

When the parallelogram that produces can not hold all data points between current point of last memory point, when promptly having data point to drop on outside the current parallelogram area coverage, then to current point through this section compression, with data points preservation, other points are given up.In Fig. 2, there is data point to drop on outside the parallelogram coverage in the time of the 10th second, so with starting point and more preceding, promptly the 9th second data point is preserved, remainder data is given up.Data point with new preservation is that starting point continues the repetition said process, judges whether subsequent data point satisfies the differentiation requirement.

2.3 linear extrapolation algorithm

The linear extrapolation algorithm also is a kind of method of utilizing linearisation thought to carry out processed compressed, and its main processing mode is to read in two data points, makes straight line with these 2, and linear equation is y=ax+b, and the abscissa value of establishing subsequent point is x _i, bring the value of abscissa into linear equation, calculate the functional value of the correspondence of this point

y _iBe the actual data value that reads in a little, δ is a threshold value, judges whether subsequent point satisfies y '-δ＜y＜y '+δ, then gives up this data point if satisfy, and does not satisfy the more preceding value that then writes down this data point and this data point.And serve as the starting point of judging straight line next time with the data point that does not satisfy threshold value, make straight line with a follow-up data points and judge that the main thought of algorithm is as shown in Figure 3.

Repeat above-mentioned discriminating step, through judging, have only the point of looping among Fig. 3 to be preserved, all the other satisfy the point of judgement threshold and have all been pressed.

In the said method, the stable state threshold method more is applicable to the delta data of relative stable state, then is not fine to the bigger effect data of real-time change; Swinging door compression algorithm mainly utilizes the compression restriction area of coverage that current data point and previous memory point constituted to come whether this reservation of judgment data, in this algorithm, possibly repeat to judge a plurality of data points, thereby make compression time long; The linear extrapolation algorithm is better to the less data compression effect of compression limit value, and effect is then relatively poor when big to the compression limit value.

Summary of the invention

The objective of the invention is to overcome the long deficiency of swinging door compression algorithm compression time, the Fast Compression method that a kind of amount of calculation is little, judge fireballing industrial real-time data is provided.

For realizing the foregoing invention purpose, the Fast Compression method of industrial real-time data of the present invention is characterized in that, may further comprise the steps:

(1), with data origination (x _i, y _i) preserve, read in next data point (x _j, y _j), at this moment, j=i+1;

(2), with next data point (x _j, y _j) and last savepoint (x _i, y _i) generate two slope value k up and down _Up, k _Low:

k _up＝(y _j+d-y _i)/(x _j-x _i)

k _low＝(y _j-d-y _i)/(x _j-x _i) ①

Wherein d is compression limit value, j=i+1;

Wait for subsequent point (x _J+1, y _J+1) arrive, and as current point;

(3), calculate current point (x _J+1, y _J+1) and previous savepoint (x _i, y _i) slope value k:

k＝(y _j+1-y _i)/(x _j+1-x _i) ②；

(4) if k _Up≤k≤k _LowSet up, then carry out step (5),, then carry out step (6) if be false

(5), will go up a data point (x _j, y _j) give up, if current point (x _J+1, y _J+1) be last point of sampled data, then preserve this data point, compression finishes;

Otherwise, with current point (x _J+1, y _J+1) calculate two slope value up and down make new advances

$k_{\mathrm{up}}^{\′}=(y_{j+1}+d-y_i)/(x_{j+1}-x_i)$

$k_{\mathrm{low}}^{\′}=(y_{j+1}-d-y_i)/(x_{j+1}-x_i)---\left(3\right);$

And relatively, if

Then

Otherwise k _UpRemain unchanged; If

Then

Otherwise k _LowRemain unchanged;

J=j+1 continues to wait for next data point (x _J+1, y _J+1) arrival, and, return step (3) as current point;

(6), will go up a data point (x _j, y _j) preserve, and as savepoint (x _i, y _i); If current point (x _J+1, y _J+1) be last point of sampled data, then preserve this data point, compression finishes; Otherwise, return step (2).

Goal of the invention of the present invention is achieved in that

The Fast Compression method of industrial real-time data of the present invention abbreviates crucial window tendency method as, in the thinking of revolving door algorithm, the determinating area that generates has been added the notion of slope, is used for finding the slope trigger point of the signal of variation.The improvement that crucial window tendency method is carried out on the basis of revolving door algorithm, purpose is to make algorithm simpler and more direct, thus amount of calculation is little when code is realized, judgement speed is fast.At first generate two slope k up and down according to the compression limit value _Up, k _LowConstitute a window, read in next data point after, judge this whether in this window, if, a bit can give up before then, generate new slope up and down then

These two slopes are to draw with two original slope ratios, thereby the pass key window is dwindled.

Crucial each data point of window tendency method is only judged once, promptly only judges each data point and the slope of last preservation data point formation and the size of bound slope.And the revolving door algorithm, if the n point is arranged in window, then first in the window needs to judge n time; Be that same point has passed through n time relatively; Second is also repeated to have compared n-1 time, and the subsequent point number of comparisons is successively decreased successively, and the n in window some accumulative total number of comparisons is (n+1) n/2; Amount of calculation is relatively large, on judgement speed, can be starkly lower than crucial window tendency method.

Description of drawings

Fig. 1 is prior art stable state threshold method one example schematic;

Fig. 2 is prior art revolving door algorithm one example schematic;

Fig. 3 is prior art linear extrapolation algorithm one example schematic;

Fig. 4 is the Fast Compression method one embodiment sketch map of industrial real-time data of the present invention;

Fig. 5 is the industrial real-time data that need compression;

Fig. 6 is the compression ratio curve chart of four kinds of compression methods of industrial real-time data under difference compression limit value of Fig. 5.

Embodiment

Describe below in conjunction with the accompanying drawing specific embodiments of the invention, so that those skilled in the art understands the present invention better.What need point out especially is that in the following description, when perhaps the detailed description of known function and design can desalinate main contents of the present invention, these were described in here and will be left in the basket.

Embodiment

Fig. 4 is the Fast Compression method one embodiment sketch map of industrial real-time data of the present invention.

In the present embodiment, as shown in Figure 4, at Fig. 4 (a) with data origination (x _i, y _i) preserve savepoint (x _i, y _i) read in data point (x with the next one _j, y _j) generate the bound slope value k of crucial window _Up, k _Low, wherein, j=i+1.

Next data point (x in Fig. 4 (b) _J+1, y _J+1) as current point, calculate and savepoint (x _i, y _i) the slope value k that generates, and relatively this slope value k and the middle bound slope value k of Fig. 4 (a) _Up, k _LowSize, if satisfy k _Up≤k≤k _LowCondition is then given up current point (x _J+1, y _J+1) more preceding, i.e. data point (x _j, y _j).Can know that by Fig. 4 (b) this slope value k satisfies above-mentioned condition, so cast out current point (x _J+1, y _J+1) more preceding, i.e. data point (x _j, y _j).

The starting point of crucial window is constant, promptly still is starting point (x _i, y _i), regenerate two slope value up and down of crucial window And relatively, if $k_{\mathrm{Up}}>k_{\mathrm{Up}}^{\&prime;},$ Then $k_{\mathrm{Up}}=k_{\mathrm{Up}}^{\&prime;},$ Otherwise k _UpRemain unchanged; If $k_{\mathrm{Low}}<k_{\mathrm{Low}}^{\&prime;},$ Then $k_{\mathrm{Low}}=k_{\mathrm{Low}}^{\&prime;},$ Otherwise k _LowRemain unchanged, can know by Fig. 4 (b), $k_{\mathrm{Up}}>k_{\mathrm{Up}}^{\&prime;},$ $k_{\mathrm{Low}}>k_{\mathrm{Low}}^{\&prime;},$ So $k_{\mathrm{Up}}=k_{\mathrm{Up}}^{\&prime;},$ k _LowRemain unchanged, shown in Fig. 4 (c).

J=j+1 continues to wait for next data point (x _J+1, y _J+1) arrival, and, return step (3) as current point, recomputate slope value k.Can know that by Fig. 4 (d) slope value k still satisfies k _Up≤k≤k _Low, then give up current point (x _J+1, y _J+1) more preceding, i.e. data point (x _j, y _j).Calculate new two slope value up and down

Because

So

Remain unchanged new crucial window upper lower limit value k _Up, k _Low, the crucial window upper lower limit value of newly obtaining is shown in Fig. 4 (e).

J=j+1 continues to wait for next data point (x _J+1, y _J+1) arrival, and, return step (3) as current point, recomputate slope value k.In Fig. 4 (f), slope value k does not satisfy k _Up≤k≤k _Low, preserve this data point (x _J+1, y _J+1) last data point (x _j, y _j), return step (2), be new savepoint (x with this point _i, y _i) and next data (x _j, y _j) k of slope value up and down of the crucial window of dot generation _Up, k _Low, like figure (g).

In like manner, next data point (x _J+1, y _J+1) as current point, shown in Fig. 4 (h), calculate and savepoint (x _i, y _i) the slope value k that generates, and relatively this slope value k and the middle bound slope value k of Fig. 4 (a) _Up, k _LowSize, satisfy k _Up≤k≤k _LowCondition is given up current point (x _J+1, y _J+1) more preceding, i.e. data point (x _j, y _j); Regenerate two slope value up and down of crucial window

Because

So k _UpRemain unchanged, Crucial window is slope value k up and down _Up, k _LowShown in 4 figure (i), in this segment data, the data point of filing is the solid dot among Fig. 4 (j) at last.

1, the comparison of four all compression method compression ratios

Compression verification is the test to compression effectiveness, and this test industrial real-time data are as shown in Figure 5, and counting is 6000 points, draws the compression verification result of each compression method when difference compression limit value.Num representes to count after the compression, and ratio representes compression ratio, and the result is as shown in table 1.

Table 1

Fig. 6 is the compression ratio curve chart of four kinds of compression methods of industrial real-time data under difference compression limit value of Fig. 5.Like table 1, shown in Figure 6, the crucial window tendency method of the present invention is identical with the compression ratio of revolving door compression algorithm under different compression limit values of prior art, has the high characteristics of revolving door algorithm compression ratio.

2, each compression algorithm testing time relatively

Table 2

Table 2 is to be 0.0 at noise, and the compression limit value is the compression time of testing to obtain in 0.5 o'clock.Can find out from table 2, invent the time much less of crucial window tendency method than the revolving door compression algorithm needs of prior art.

The invention reside in to the industrial real-time data characteristic, analyze and study the characteristics and the structure of its image data, explore and design and be applicable to the industrial real-time data; Practical, the data compression method of reliable and effective makes a large amount of image data obtain better compression effectiveness; Improve compression ratio, reduce compression time, conserve storage; Reduce the commercial production cost, improve the speed of system handles data.

The application data compress technique has very important significance in the industrial automation field.At first, the real-time and historical data of the magnanimity that produces in the industrial processes of existing industrial automation system intractable is said intractable here, comprises processing speed and disk size.Disk size is an aspect of problem; On the other hand, the high compression rate of data means that the data processing speed of whole system is faster, and this is embodied in: the data of high compression rate; It is little to take disk space; Data are fast from the speed that disk reads in internal memory, and the speed of Network Transmission is fast, and data occupation space in internal memory is little.A favorable industrial automated system must be resolved the real-time handling problem of data, utilizes data compression technique, can not only conserve memory equipment, can also improve system speed, and make the overall performance of system reach certain availability index.The crucial window tendency method of the present invention not only have good data compression rate, and judgement speed is fast to the industrial real-time data characteristic, has the excellent real-time property handled, the handling problem that can well solve industrial data.

Although above the illustrative embodiment of the present invention is described; So that the technical staff of present technique neck understands the present invention, but should be clear, the invention is not restricted to the scope of embodiment; To those skilled in the art; As long as various variations appended claim limit and the spirit and scope of the present invention confirmed in, these variations are conspicuous, all utilize innovation and creation that the present invention conceives all at the row of protection.

Claims (1)

1. the Fast Compression method of industrial real-time data is characterized in that, may further comprise the steps:

(1), with image data starting point (x _i, y _i) preserve, read in next data point (x _j, y _j);

(2), with next data point (x _j, y _j) and last savepoint (x _i, y _i) generate two slope value k up and down _Up, k _Low:

k _up＝(y _j+d-y _i)/(x _j-x _i)

k _low＝(y _j-d-y _i)/(x _j-x _i) ①

Wherein d is compression limit value, j=i+1;

Wait for subsequent point (x _J+1, y _J+1) arrive, and as current point;

(3), calculate current point (x _J+1, y _J+1) and previous savepoint (x _i, y _i) slope value k:

k＝(y _j+1-y _i)/(x _j+1-x _i) ②；

(4) if k _Up≤k≤k _LowSet up, then carry out step (5),, then carry out step (6) if be false

(5), will go up a data point (x _j, y _j) give up, if current point (x _J+1, y _J+1) be last point of sampled data, then preserve this data point, compression finishes;

Otherwise, with current point (x _J+1, y _J+1) calculate two slope value up and down make new advances

$k_{\mathrm{up}}^{\&prime;}=(y_{j+1}+d-y_i)/(x_{j+1}-x_i)$

$k_{\mathrm{low}}^{\&prime;}=(y_{j+1}-d-y_i)/(x_{j+1}-x_i)---\left(3\right);$

And relatively, if

Then Otherwise k _UpRemain unchanged; If Then Otherwise k _LowRemain unchanged;

J=j+1 continues to wait for next data point (x _J+1, y _J+1) arrival, and, return step (3) as current point;

(6), will go up a data point (x _j, y _j) preserve, and as savepoint (x _i, y _i); If current point (x _J+1, y _J+1) be last point of sampled data, then preserve this data point, compression finishes; Otherwise, return step (2).

Images