CN101692251B - Function parameter estimation based on-line compression and decompression method for general process data - Google Patents

Abstract

The invention discloses function parameter estimation based on-line compression and decompression method for general process data. The compression method comprises the following steps: 1) estimating the time-dependent distribution condition of the data; 2) establishing a model for the data of a data source variable along with time by using a curve function; 3) reducing the number of parameters inthe curve function; and 4) performing on-line compression for the real-time database. The decompression method comprises the following steps: presetting a curve function which is the same as that in compression according to the preserved curve function parameter values, and calculating the decompressed data by using the compressed data and the curve function. The method can be applied to the real-time database or some situations need to store the process data. The method can acquire larger compression rate compared with the conventional linear process data compression algorithm. Meanwhile, the method has small calculation amount and little data to be cached, and is quite suitable for serving as an on-line process data compression method.

Description

The compression of universal process online data, decompressing method based on the function parameter estimation

Technical field

The present invention relates to a kind of line compression, decompressing method of process data.

Background technology

Real-time dataBase system be a kind of real-time to processing transactions require than the strictness of general data storehouse the Database Systems of Duoing.Even in general the application, the real-time dataBase system per second all needs to store several ten thousand even the record of hundreds of thousands bar, under so large-scale data rate memory occasion, if do not adopt the proper compression method to be difficult to satisfy the demand of real-time.

Data compression technique has the important meaning of three aspects for real-time dataBase system.The first, because the data volume that real-time data base need be gathered is very big, data compression can reduce the load of network data transmission effectively, saves bandwidth, improves the concurrent ability of network of system.Secondly, the data after compression handled deposit hard disk in can reduce pressure to the hard-disc storage space effectively, and after overcompression, the data of same capacity can be preserved the process data of longer time.The 3rd, in real-time data base, owing to adopt compress technique can reduce data volume, so just reduced the burden that follow-up data is handled and stored, effectively the processing power of Hoisting System integral body.

The compression algorithm of real-time data base needs to satisfy simultaneously three conditions, namely the compressibility height, EMS memory occupation is few, calculated amount is little.In the real-time data base field, the revolving door algorithm of U.S. OSI company invention, owing to calculate simply, EMS memory occupation is few, has obtained using widely for general data compressibility height.Swinging door compression algorithm comes down to a kind of linear compression algorithm of data, namely adopts the method for segmentation straight line to go approximate raw data, makes error simultaneously in default range of control.Facts have proved that the revolving door algorithm is very high for the process data compressibility of linear change in time, but when data in time during nonlinearities change, such as sine wave, then compressibility is very low.And general industry control occasion, general and time of various data source data is not linear relationship.

And some inventions attempt to adopt the compression of the method implementation procedure data of linear fit, but these class methods need a large amount of memory cache historical process data, data source owing to the needs compression in real-time data base is a lot, a large amount of internal memories will be taken, so these class methods not too are fit to carry out the process data compression.

The disclosed application number of China Intellectual Property Office is: CN02120383.0, name is called: the patent of invention of adaptive historical data compression method, it comprises step: compression time is judged: to current measured value, judge that Measuring Time is whether in given compression time interval, time difference of current measured value too hour, do not carry out the step of back, continue to investigate next measurement data; When too big in the time difference of current measured value, the last value of storage currency is as the starting point of next round data compressing and testing and a new last memory point; The calculating of slope: calculate currency slope and current maximum slope and minimum slope; To difference measurement data constantly, according to current maximum/minimum slope, the straggling parameter of dynamic calculation compression adaptively; With the compression straggling parameter of new calculating, calculate the upper bound and the lower bound of currency slope; Compression verification is judged: currency is by compression verification, and the previous value of storage currency as the starting point of next round data compressing and testing and a new last memory point, otherwise continues the next new data point of test.This patent is the same with the revolving door algorithm, has adopted the linear compression algorithm, has the low problem of compressibility for nonlinear data.

Disclosed another application number of China Intellectual Property Office is: CN200610052068.X, name is called: a kind of real-time data compression method based on least square linear fit, this method is huge and exist much noise to disturb and situations such as redundant data at the real time data data volume of industry spot collection, at first by the real time data of picking up from industry spot is carried out pre-service, then data processed is deposited in internal memory historical data buffer zone, and be sample dynamic construction least square linear fit straight line with the data that are kept in the buffer zone, need judge whether the encumbrance strong point by the distance of measuring this fitting a straight line of having a few in the buffer zone and the maximum compression deviation that contrasts user's appointment.This method has adopted the off-line compression algorithm, needs to consume a large amount of memory cache process datas, and exists for the low problem of nonlinear data compressibility equally.

Summary of the invention

The objective of the invention is to, a kind of universal process online data compression method of estimating based on function parameter is provided, to compress the process data that real-time data base collects.

The present invention adopts following technical scheme:

A kind of universal process online data compression method of estimating based on function parameter before describing the concrete technical scheme of the present invention, provides the definition that the present invention adopts the part term earlier:

Process data: refer to the in chronological sequence data of acquisition order, have timestamp information in the data.The data of most of industry spot collections all are process datas.

Curvilinear function: the curve negotiating quantic is shown the function express, such as the function that employing adds, subtracts, multiplication and division, index, logarithm, trigonometric function are formed by combining.The curve here is general concept, has both comprised crooked curve, also comprises straight line.

The inventive method specifically may further comprise the steps:

A kind of universal process online data compression method of estimating based on function parameter may further comprise the steps:

1) to data in time distribution situation estimate;

2) utilize curvilinear function that the data source data is changed modeling in time;

3) number of parameter in the minimizing curvilinear function;

4) carry out the line compression of real-time data base.

Further, described step 2) in, adopt the form of higher order polynomial to the Changing Pattern modeling of process data.

Further, the method that reduces the number of parameter in the curvilinear function in the described step 3) is, in process data, take the more representational point of a part, bring in the curvilinear function, the value of the partial parameters in the calculated curve function, perhaps utilize the experience that process data is distributed, in advance the value of determining section parameter.

Further, described step 4) specifically may further comprise the steps:

41) parameter of initialization higher order polynomial curvilinear function: according to the possible span of parameter, set span for unascertainable all parameters of higher order polynomial curvilinear function institute;

42) read in a real-time process data recording, substitution higher order polynomial curvilinear function obtains an inequality according to default error; Separate this inequality and obtain new higher order polynomial curvilinear function parameter range;

43) parameter value scope and the former span asked for are asked a common factor, obtain a new span, judge whether this new span is empty, if not, then upgrades former span with this new span, then changes step 42); If then enter step 44);

44) parameter of last process data of preservation and higher order polynomial curvilinear function as basic point, forwards step 41 with current process data to).

Further, in the described step 1), the process data that data source collects is Sine distribution in time, described step 2) in be modeled as y=asin ω (t-t ₀)+b, a wherein, ω, t ₀, b is sinusoidal parameter.

Further, in the described step 3), according to the process data by Sine distribution, determine amplitude a, fluctuation center b and vibration frequency ω in advance.

Further, described step 4) is specially:

41) parameter of initialization curvilinear function: according to the possible span of parameter, give the unascertainable parameter phase t of curvilinear function ₀Set initial range;

42) read in a real-time process data recording, the following formula of substitution obtains t ₀Span:

$t-\frac{1}{\mathrm{\&omega;}}\arcsin \frac{y-b+e}{a}<{\mathrm{<figure-callout\; id="0"\; label="t"\; filenames="H2009101948852E00021.png,H2009101948852E00031.png"\; state="\{\{state\}\}">t</figure-callout>}}_0<t-\frac{1}{\mathrm{\&omega;}}\arcsin \frac{y-b-e}{a}$

Wherein, e is default permissible error;

43) span and the former span that obtain are asked a common factor, obtain a new span, judge whether this new span is empty, if not, then upgrades former span with this new span, then changes step 42); If then enter step 44);

44) parameter of preservation active procedure data and curvilinear function as basic point, forwards step 41 with next process data to).

Further, preserve a record in the one-period of described sine function at least, described step 4) is specially:

41) parameter of initialization curvilinear function: according to possibility span in the one-period, give the unascertainable parameter phase t of curvilinear function ₀Set two initial range;

42) read in a real-time process data recording, the following formula of substitution obtains t ₀Two new spans:

$t-\frac{1}{\mathrm{\&omega;}}\arcsin \frac{y-b+e}{a}<{\mathrm{<figure-callout\; id="0"\; label="t"\; filenames="H2009101948852E00021.png,H2009101948852E00031.png"\; state="\{\{state\}\}">t</figure-callout>}}_0<t-\frac{1}{\mathrm{\&omega;}}\arcsin \frac{y-b-e}{a}$

$t-\frac{1}{\mathrm{\&omega;}}(\mathrm{\&pi;}-\arcsin \frac{y-b-e}{a})<{\mathrm{<figure-callout\; id="0"\; label="t"\; filenames="H2009101948852E00021.png,H2009101948852E00031.png"\; state="\{\{state\}\}">t</figure-callout>}}_0<t-\frac{1}{\mathrm{\&omega;}}(\mathrm{\&pi;}-\arcsin \frac{y-b+e}{a})$

Wherein, e is default permissible error;

Two spans of two spans that 43) will obtain and former correspondence seek common ground, obtain the span of two new correspondences, judge that whether these two new spans are entirely for empty, if not, then upgrade former span with new span, change step 42); If then enter step 44);

44) parameter of preservation active procedure data and curvilinear function as basic point, forwards step 41 with Next process data to).

The present invention also provides a kind of decompressing method of the universal process data of estimating based on function parameter, and the data after the compression that is used for the above-mentioned universal process online data compression method of estimating based on function parameter is obtained are carried out decompress(ion), comprise with step:

According to the curvilinear function parameter value of preserving, preestablish a curvilinear function the same during with compression, utilize the data after data after the described compression and this curvilinear function calculate acquisition decompress(ion).

The present invention can be applicable to real-time data base or some need the occasion of storing process data.The present invention can obtain than conventional linear process data compression algorithm, such as revolving door, obtains bigger compressibility.Simultaneously calculated amount of the present invention little, need data in buffer few, be suitable as very much the on-line process data compression method.

Further specify the present invention below in conjunction with drawings and Examples.

Description of drawings

Fig. 1 the present invention is based on the universal process online data compression method process flow diagram that function parameter is estimated;

Fig. 2 is the process flow diagram of the line compression step of real-time data base among the present invention;

Fig. 3 is the process data changing trend diagram in time of input;

Fig. 4 is for adopting the compression result synoptic diagram of the universal process online data compression method that the present invention is based on the function parameter estimation;

Fig. 5 is for adopting the compression result synoptic diagram of existing swinging door compression algorithm.

Embodiment

Embodiment one

As shown in Figure 1, a kind of universal process online data compression method of estimating based on function parameter may further comprise the steps:

1) to data in time distribution situation estimate;

2) utilize curvilinear function that the data source data is changed modeling in time;

3) number of parameter in the minimizing curvilinear function;

4) carry out the line compression of real-time data base.

Wherein, described step 2) in, adopt the form of higher order polynomial to the Changing Pattern modeling of process data.

Wherein, the method that reduces the number of parameter in the curvilinear function in the described step 3) is, in process data, take the more representational point of a part, bring in the curvilinear function, the value of the partial parameters in the calculated curve function, perhaps utilize the experience that process data is distributed, in advance the value of determining section parameter.

Wherein, described step 4) specifically may further comprise the steps:

41) parameter of initialization higher order polynomial curvilinear function: according to the possible span of parameter, set span for unascertainable all parameters of higher order polynomial curvilinear function institute;

42) read in a real-time process data recording, substitution higher order polynomial curvilinear function obtains an inequality according to default error; Separate this inequality and obtain new higher order polynomial curvilinear function parameter range;

43) parameter value scope and the former span asked for are asked a common factor, obtain a new span, judge whether this new span is empty, if not, then upgrades former span with this new span, then changes step 42); If then enter step 44);

44) parameter of last process data of preservation and higher order polynomial curvilinear function as basic point, forwards step 41 with current process data to).

Embodiment two

A kind of universal process online data compression method of estimating based on function parameter may further comprise the steps:

1) to data in time distribution situation estimate: evenly increase in time such as the data that collect from certain data source, then these data are linear distribution.If the data that collect from a data source are Sine distribution in time, then these data are the sinusoidal curve distribution;

2) utilize curvilinear function that the data source data is changed modeling in time: it is the function of variable with time that modeling is namely set up one, the regularity of distribution that this function can the data of description curve.Be linear distribution such as data, then the Changing Pattern of these data is y=at+b, a wherein, and b is function parameters, and t is the time, as long as determine parameter a and b, the regularity of distribution that just can the deterministic process data.If data are sinusoidal curve and distribute, then these data over time rule can be modeled as y=asin ω (t-t ₀)+b, a wherein, ω, t ₀, b is sinusoidal parameter, as long as determined these parameters, and just can unique distribution situation of determining these process datas.If the Changing Pattern to the data source data is difficult to obtain in advance, just can adopt the form of higher order polynomial to the Changing Pattern modeling of process data;

3) number of parameter in the minimizing curvilinear function: the curvilinear function that obtains according to modeling contains a plurality of parameters (being that function has a plurality of degree of freedom), reduces the number of parameter in the curvilinear function, to reduce calculated amount, improves arithmetic speed.The number that reduces parameter in the curvilinear function has following two kinds of methods:

One, utilizes the experience that process data is distributed, in advance the value of determining section parameter.Such as the process data by Sine distribution, if known it amplitude (be in the analytic expression a), fluctuation center (being the b in the analytic expression) and vibration frequency (being the ω in the analytic expression), just can in advance these three parameters be decided, this curvilinear function only has been left parametric t like this ₀, greatly reduce the number of parameter in the curvilinear function.

Two, in process data, take the more representational point of a part, bring in the curvilinear function, also can reduce the number of variable element in the curvilinear function effectively.

4) carry out the line compression of real-time data base, this is an iterative process, as shown in Figure 2, specifically may further comprise the steps:

41) parameter of initialization curvilinear function: according to the possible span of parameter, set an initial range for unascertainable all parameters of curvilinear function institute;

42) read in a real-time process data recording, the substitution curvilinear function obtains an inequality according to default error.Separate this inequality and can obtain new curvilinear function parameter range;

43) parameter value scope and the former scope asked for are asked a common factor, obtain a new span, judge whether this new span is empty, if, then upgrade former span with new span, then change step 42); Otherwise, enter step 44);

44) new span is empty, the current process data recording of this explanation does not satisfy the condition of the curvilinear function of parameter current, at this moment, need to preserve the parameter of last process data and corresponding curvilinear function, simultaneously with current process data as basic point, restart the compression process of a new round, and forward step 41 to).If parameter of curve has only a unknown parameter, as described in follow-up sinusoidal curve, because only need a point can try to achieve parameter of curve when decompressing, so when the parameter value scope is sky, only need to preserve current process data and get final product, and need not to preserve last process data.

At first this process data is carried out modeling.The process data of input distributes as shown in Figure 3 in time, and wherein transverse axis is time t, as seen, and this process data distribution compounded sine curvilinear function in time.

To the process data modeling.Because this process data distribution compounded sine function, therefore can obtain analytical expression and be:

y＝asinω(t-t ₀)+b

Wherein, a is sinusoidal amplitude, and ω is vibration frequency, t ₀Be phase place, b is the vibration off center.

Simplify the parameter of curvilinear function.Suppose the historical data according to engineering experience or accumulation, known this sinusoidal parameter a, ω and b, sine function only remains next parameter, i.e. phase place t so ₀

A process data for any input if its time t and numerical value y can be similar to by this curvilinear function, just must meet following inequality:

y-e≤asinω(t-t ₀)+b≤y+e

Wherein, e is default permissible error.If must preserve a record at least in the one-period of regulation sine function, then can further try to achieve t ₀Must meet following at least two inequality:

$t-\frac{1}{\mathrm{\&omega;}}\arcsin \frac{y-b+e}{a}<{\mathrm{<figure-callout\; id="0"\; label="t"\; filenames="H2009101948852E00021.png,H2009101948852E00031.png"\; state="\{\{state\}\}">t</figure-callout>}}_0<t-\frac{1}{\mathrm{\&omega;}}\arcsin \frac{y-b-e}{a}$ Or

$t-\frac{1}{\mathrm{\&omega;}}(\mathrm{\&pi;}-\arcsin \frac{y-b-e}{a})<{\mathrm{<figure-callout\; id="0"\; label="t"\; filenames="H2009101948852E00021.png,H2009101948852E00031.png"\; state="\{\{state\}\}">t</figure-callout>}}_0<t-\frac{1}{\mathrm{\&omega;}}(\mathrm{\&pi;}-\arcsin \frac{y-b+e}{a})$

Arcsin function can obtain the result fast by look-up table.In compression process, when obtaining article one record, can obtain t according to following formula ₀Two spans, and preserve this two spans.When collecting trailer record, can be in the hope of two new spans, and after asking friendship with preceding two spans, upgrade former span.Continue this process, if these two spans are not empty entirely, then Shu Ru process data can be predicted by curvilinear function, so can be with this input process rejection of data.If two spans all are empty set, then preserve the parameter value of this process data and corresponding curvilinear function, then start the compression process of a new round.

After overcompression, need the point of preservation as shown in Figure 4, adopt " X " mark.And the point that the employing swinging door compression algorithm need be preserved as shown in Figure 5.For the ease of comparison, two kinds of compression algorithms have adopted same permission compressed error.As can be seen from the figure, adopt same compressed error, the revolving door algorithm need be preserved 12 records, and adopts the universal process online data compression method of estimating based on function parameter only need preserve 5 records, has improved compressibility greatly.

Embodiment three

A kind of universal process online data decompressing method of estimating based on function parameter, according to the curvilinear function parameter value of preserving, preestablish a curvilinear function the same with when compression, utilize data after the described compression and this curvilinear function to calculate to obtain the data behind decompress(ion).After curvilinear function obtains parameter, need the data of decompress(ion) only to need the input time value just can obtain corresponding decompress(ion) result.

Above-described embodiment only is used for explanation technological thought of the present invention and characteristics, its purpose is to make those skilled in the art can understand content of the present invention and implements according to this, can not only limit claim of the present invention with present embodiment, be all equal variation or modifications of doing according to disclosed spirit, still drop in the claim of the present invention.

Claims (7)

1. universal process online data compression method of estimating based on function parameter is characterized in that may further comprise the steps:

1) to data in time distribution situation estimate;

2) utilize curvilinear function that the data source data is changed modeling in time;

3) number of parameter in the minimizing curvilinear function, the method that wherein reduces the number of parameter in the curvilinear function is, in process data, take the more representational point of a part, bring in the curvilinear function, the value of the partial parameters in the calculated curve function, perhaps utilize the experience that process data is distributed, in advance the value of determining section parameter;

4) carry out the line compression of real-time data base, specifically may further comprise the steps:

41) parameter of initialization curvilinear function: according to the possible span of parameter, set span for unascertainable all parameters of curvilinear function institute;

42) read in a real-time process data recording, the substitution curvilinear function obtains an inequality according to default error; Separate this inequality and obtain new curvilinear function parameter range;

43) parameter value scope and the former span asked for are asked a common factor, obtain a new span, judge whether this new span is empty, if not, then upgrades former span with this new span, then changes step 42); If then enter step 44);

44) parameter of last process data of preservation and curvilinear function as basic point, forwards step 41 with current process data to).

2. the universal process online data compression method of estimating based on function parameter according to claim 1 is characterized in that: described step 2), adopt the form of higher order polynomial to the Changing Pattern modeling of process data.

3. the universal process online data compression method of estimating based on function parameter according to claim 1, it is characterized in that: in the described step 1), the process data that data source collects is Sine distribution in time, described step 2) in be modeled as y=asin ω (t-t ₀)+b, wherein amplitude a, fluctuation center b, vibration frequency ω and phase place t ₀Be sinusoidal parameter.

4. the universal process online data compression method of estimating based on function parameter according to claim 3 is characterized in that: in the described step 3), according to the process data by Sine distribution, determine amplitude a, fluctuation center b and vibration frequency ω in advance.

5. the universal process online data compression method of estimating based on function parameter according to claim 4, it is characterized in that: described step 4) is specially:

41) parameter of initialization curvilinear function: according to the possible span of parameter, give the unascertainable parameter phase t of curvilinear function ₀Set initial range;

42) read in a real-time process data recording, the following formula of substitution obtains t ₀Span:

$t-\frac{1}{\mathrm{\&omega;}}\arcsin \frac{y-b+e}{a}<t_0<t-\frac{1}{\mathrm{\&omega;}}\arcsin \frac{y-b-e}{a}$

Wherein, e is default permissible error;

43) span and the former span that obtain are asked a common factor, obtain a new span, judge whether this new span is empty, if not, then upgrades former span with this new span, then changes step 42); If then enter step 44);

44) parameter of preservation active procedure data and curvilinear function as basic point, forwards step 41 with next process data to).

6. the universal process online data compression method of estimating based on function parameter according to claim 4, it is characterized in that: preserve a record in the one-period of described sine function at least, described step 4) is specially:

41) parameter of initialization curvilinear function: according to possibility span in the one-period, give the unascertainable parameter phase t of curvilinear function ₀Set two initial range;

42) read in a real-time process data recording, the following formula of substitution obtains t ₀Two new spans:

$t-\frac{1}{\mathrm{\&omega;}}\arcsin \frac{y-b+e}{a}<t_0<t-\frac{1}{\mathrm{\&omega;}}\arcsin \frac{y-b-e}{a}$

$t-\frac{1}{\mathrm{\&omega;}}(\mathrm{\&pi;}-\arcsin \frac{y-b-e}{a})<t_0<t-\frac{1}{\mathrm{\&omega;}}(\mathrm{\&pi;}-\arcsin \frac{y-b+e}{a})$

Wherein, e is default permissible error;

Two spans of two spans that 43) will obtain and former correspondence seek common ground, obtain the span of two new correspondences, judge that whether these two new spans are entirely for empty, if not, then upgrade former span with new span, change step 42); If then enter step 44);

44) parameter of preservation active procedure data and curvilinear function as basic point, forwards step 41 with Next process data to).

7. the decompressing method of universal process data of estimating based on function parameter, data after the compression that is used for the described universal process online data compression method of estimating based on function parameter of the arbitrary claim of claim 1 to 6 is obtained are carried out decompress(ion), it is characterized in that:

According to the curvilinear function parameter value of preserving, preestablish a curvilinear function the same during with compression, utilize the data after data after the described compression and this curvilinear function calculate acquisition decompress(ion).

Images