CN103795419B - A kind of piecewise linearity compression method of Wave data Real Time Compression - Google Patents
A kind of piecewise linearity compression method of Wave data Real Time Compression Download PDFInfo
- Publication number
- CN103795419B CN103795419B CN201410039805.7A CN201410039805A CN103795419B CN 103795419 B CN103795419 B CN 103795419B CN 201410039805 A CN201410039805 A CN 201410039805A CN 103795419 B CN103795419 B CN 103795419B
- Authority
- CN
- China
- Prior art keywords
- data
- compression
- siding
- maximum
- partition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The present invention relates to a kind of piecewise linearity compression method of Wave data Real Time Compression, for given data, specify the limits of error, maximum siding-to-siding block length and record limit value, carry out data partition, data are carried out with noise reduction and compression differentiates, the high-fidelity Real Time Compression of Wave data is realized, provides, with interval maximum length limitation and to the record limit value of dependent variable, different grades of data compression ratio to can be achieved by adjusting the error of compression method.Beneficial effects of the present invention are：Data after compression can meet the requirement of real-time radio transmission, it is easy to the mining analysis and data storage of big data, retained with high compression ratio, wave character, calculate it is simple, perform that speed is fast, noise data adjusts superior properties to be had important practical significance for the real-time Transmission of Wave data and compression storage, in the case where ensureing wave character, by adjustment algorithm parameter, the real compression of high-fidelity that carried out to Wave data of quick and high compression ratio is handled.
Description
Technical field
The present invention relates to a kind of piecewise linearity compression method of Wave data Real Time Compression.
Background technology
As automation and intelligentized high speed development is monitored and controlled in the industries such as traffic, oil, chemical industry, metallurgy, electric power,
The requirement of processing and storage capacity to real time data is also being gradually stepped up, industry monitoring system more have collection point it is many, be distributed
Extensively, how therefore a large amount of real time datas, are carried out data processing and profit by the characteristic such as high, continuous uninterrupted and dynamic realtime of frequency
Analyzed, excavated, predicted with these mass datas, be the severe challenge currently encountered.In order to alleviate in data processing and deposit
Pressure in terms of storage, data compression technique just progressively turns into the necessary link that industrial real-time data is handled.
By taking the safety monitoring of transportation industry Bridge as an example, for reach realize structural safety monitoring differentiate purpose, it is necessary to
A large amount of sensor devices are laid in the significant points of monitoring object, are realized to a large amount of medium and small bridges, tunnel, side slope collection in region
Into big quantity sensor monitor in real time, it is necessary to dynamic sensor data is acquired, handled, transmit with storage, its data
Amount is extremely huge.In addition, the structure position of bridge, tunnel, side slope is influenceed substantially by environment and load, key parameter is as added
The data such as velocity sensor, strain transducer, strain gauge, which have, to be changed relatively frequently, with waveform numbers such as very strong randomnesss
According to feature.Therefore, how research carries out real-time high-efficiency and Hi-Fi data compression to Sensor monitoring data just becomes
One very important problem of industrial real-time data processing.
The process data compression method proposed at present can be divided mainly into three classes, i.e. subsection linearity inser value method, vector
Quantization method and signal transformation method.Wherein vector quantization method and signal transformation method compression ratio are high, and wave character is retained,
But it is slow to perform speed, it is impossible to realize line compression, thus be more used in radar, seismic survey etc. without requirement of real-time
Complex process data processing.What is be most widely used in terms of industrial process data compression is then subsection-linear method, point
Section linear approach compression ratio when being compressed to waveform signal is high, speed is fast, but, Yi Zao bad to Dynamic Signal and noise data processing
Into Wave data distortion and peak value loss situation, therefore, a kind of piecewise linearity side suitable for Wave data Real Time Compression is found
Method, real-time Transmission and compression storage for Wave data have important practical significance.
The content of the invention
It is existing at present to overcome it is an object of the invention to provide a kind of piecewise linearity compression method of Wave data Real Time Compression
There are technology above shortcomings.
The purpose of the present invention is to be achieved through the following technical solutions：
A kind of piecewise linearity compression method of Wave data Real Time Compression, it is characterised in that comprise the following steps：
1）Set limits of error value E, maximum siding-to-siding block length L, the waveform of Wave data set of Wave data set
The data variation record limit value RangeLimit of data acquisition system；
2）According to step 1）The limits of error value E of setting, calculates subregion and judges fulcrum, using time quantum as X-axis,
Numerical quantities constitute coordinate system as Y-axis, will be first data point A0 in L data interval along Y-axis with maximum siding-to-siding block length
Two points that direction distance is E, fulcrum is judged as two subregions of data partition；
3）Maximum siding-to-siding block length is calculated respectively judges fulcrum line for the subsequent point in L data interval and the two subregions
Slope K 1 and K2, K1 be that subsequent point judges the slope of fulcrum line with the subregion more than A0, K1 only records maximum, and K2 only remembers
Record minimum value；
4）In the maximum siding-to-siding block length set as in L data interval, as long as K1<K2, then continue to calculate K1 and K2, until
Find K1>K2 point completes one and takes turns data partition as partition end, when in the data field that defined maximum siding-to-siding block length is L
In, K1 is not found>K2 point, then choose the terminal for the data interval that maximum siding-to-siding block length is L as partition end, complete one
Take turns data partition；
5）First data point using the partition end of previous round data partition as next round data partition, according to step
2）, step 3）, step 4）, epicycle data partition is carried out, often the operation of wheel data partition is similar afterwards, wheel partition end is above
Lower whorl subregion starting point starts division operation；
6）The region minimum in region maximum and data partition in difference record data subregion；
7）Limit value RangeLimit is recorded according to the data variation of setting, data are carried out with noise reduction and compression differentiates, respectively
Data variation the amount Rmax and Rmin of zoning maximum and region minimum and data partition starting point, when variable quantity is less than
Limit value RangeLimit is recorded, then is considered as that noise is compressed and given up, when variable quantity is more than record limit value RangeLimit, then
It is considered as that key waveforms point is recorded, the interval is replaced with the straight line between data area starting point and key waveforms point
Interior total data point.
The step 3）In when maximum siding-to-siding block length is only the first data point in L data interval, K1 be it is negative infinite,
K2 is just infinite, and when maximum siding-to-siding block length is has multiple data points in L data interval, more and more with counting, K1 is more next
Bigger, K2 is less and less.
The piecewise linearity compression method of described Wave data Real Time Compression, by adjusting limits of error value E, maximum
Siding-to-siding block length L, data variation record limit value RangeLimit, can be achieved different grades of data compression ratio.
Beneficial effects of the present invention are：Data after compression can meet the requirement of real-time radio transmission, be easy to big data
Mining analysis and data storage, with being retained with high compression ratio, wave character, calculate it is simple, perform that speed is fast, noise number
According to adjusting superior properties to be had important practical significance for the real-time Transmission of Wave data and compression storage, ensureing wave character
In the case of, by adjustment algorithm parameter, the real compression of high-fidelity that carried out to Wave data of quick and high compression ratio is handled.
Brief description of the drawings
The present invention is described in further detail below according to accompanying drawing.
Fig. 1 is the schematic diagram of the piecewise linearity compression method of the Wave data Real Time Compression described in the embodiment of the present invention.
Embodiment
As shown in figure 1, a kind of piecewise linearity compression method of Wave data Real Time Compression described in the embodiment of the present invention, bag
Include following steps：
The first step：Set the example limits of error as E, the maximum siding-to-siding block length of example be L=10 and Wave data set
Data variation records limit value RangeLimit, and it is that E, the maximum siding-to-siding block length of example are L=10 to be first depending on the example limits of error
This two big characteristic parameter carries out subregion for data：
First data point A0 respectively has a bit up and down, is E with the distance between A0 points, what the two points judged as subregion
Two fulcrums, the slope K 1 and K2, wherein K1 of difference calculated for subsequent point and this 2 lines only record maximum, and K2 is only recorded most
Small value, when only 1st, K1 is negative infinite, and K2 is just infinite；More and more with counting, K1 is increasing, and K2 is more next
It is smaller, inside defined maximum siding-to-siding block length is 10 data break, as long as K1<K2, this operation is that can proceed with, and is looked for
Point A7 when being more than K2 to K1 completes first round data partition [A0, A7] as partition end；
Then, using first round partition end A7 as the starting point B0 of next round subregion, determine its up and down distance be two of E
Fulcrum, continues above-mentioned slope and judges operation, until in the interval of maximum siding-to-siding block length 10, the point that K1 is more than K2 not being found, then is selected
Take the terminal B10 of maximum siding-to-siding block length as partition end, complete second and take turns subregion [B0, B10], division operation is often taken turns afterwards
Similar, wheel partition end is that lower whorl subregion starting point starts division operation above, is repeated no more；
Second step：Region maximum and region minimum are found in subregion, the capture of crest and trough is realized；
For the differentiation of waveform, catching for crest and trough is realized with the method for maximum and minimum in region is obtained
Obtain, as shown in figure 1, in first round subregion [A0, A7], region maximum Max=A4, region minimum Min=A1;Second wheel point
In area [B0, B10], region maximum Max=B10, region minimum Min=B3;So as to ensure the crest and trough of Wave data
It is successfully captured；
3rd step：Limit value RangeLimit is recorded according to the data variation of setting, data are carried out with noise reduction and compression differentiates,
In first round subregion [A0, A7], the data variation amount of difference zoning maximum and region minimum and subregion starting point
Rmax and Rmin, if variable quantity is less than record limit value RangeLimit, such as region minimum A1, then be considered as noise and by
Compression is given up, and is then considered as key waveforms point if variable quantity is more than record limit value RangeLimit, such as region maximum A4
And be recorded.The total data in the interval is thus replaced with the straight line between area starting point and key waveforms point
Point, if substituted after [A0, A7] eight point compressions by (A1, A4, A7) three points.Equally, region maximum in the second wheel subregion
B10 is also simultaneously partition end, so [B0, B10] 11 points are substituted by (B0, B3, B10) three points after compression, in high pressure
In the case of contracting ratio, the crest and trough of Wave data are maintained, the high-fidelity compression of Wave data is realized.
The present invention combines the integrated dynamic monitoring system in advanced bridge tunnel slope in intelligent transportation, by the present invention to tunnel,
The waveform sensor data that bridge, side slope significant points are laid carry out one-level Real Time Compression, and the data after compression can be met in real time
The requirement being wirelessly transferred；In addition, carrying out real-time data compression again to the magnanimity structured data of long-term acquisition, it is easy to big data
Mining analysis and data storage, meet the integrated real-time dynamic monitoring requirement in bridge tunnel slope, so as to ensure operation to greatest extent
Safety.With being retained with high compression ratio, wave character, calculate it is simple, perform that speed is fast, noise data adjusts superior properties.
The real-time collection that can be widely applied to the industry process data such as traffic, oil, chemical industry, metallurgy, electric power of the present invention, at a high speed biography
In defeated, compression storing process, real-time Transmission and compression storage for Wave data have important practical significance, and are ensureing ripple
In the case of shape feature, by adjustment algorithm parameter, quick and high compression ratio is carried out to Wave data at high-fidelity reality compression
Reason.Technical problem to be solved is the Dynamic Signal feature for Wave data, in the case of high compression ratio and noise reduction, is protected
The wave character of Wave data is held, compression algorithm calculates simple, performs speed soon, realizes that the high-fidelity of Wave data is pressed in real time
Contracting.
For given Wave data set, the limits of error, maximum siding-to-siding block length and the note to dependent variable are first provided
Limit value is recorded, is selected after starting point, provide according to error and interval maximum length limits and finds out most long trends of straight line conduct as far as possible
One interval, and interval terminal and region maximum and minimum are recorded, if the difference of local extremum and beginning and end data is equal
More than defined record limit, then the local extremum data and endpoint data are sequentially recorded, otherwise not posting field extreme value data,
Only record endpoint data.So as to accurately catch the crest and trough of Wave data, realize that the high-fidelity of Wave data is real
When compress.Carried out for data in interval selection differentiation, choose the limits of error and join with maximum two features of siding-to-siding block length
Data, as long as meeting any one condition, can all be carried out being divided into an interval by number.For the differentiation of waveform, with acquisition area
The method of maximum and minimum realizes the capture of crest and trough in domain, so as to ensure the fidelity of waveform.Enter to data
During row noise reduction and compression differentiate, by the way that the difference of local extremum and the beginning and end data in interval is compared with record limit value
Differentiate, can be realized to Noise reducing of data by adjusting record limit value.In addition, being provided and interval by adjusting the error of compression method
Maximum length is limited and to the record limit value of dependent variable, and different grades of data compression ratio can be achieved.
The present invention is not limited to above-mentioned preferred forms, and anyone can show that other are various under the enlightenment of the present invention
The product of form, however, make any change in its shape or structure, it is every that there is skill identical or similar to the present application
Art scheme, is within the scope of the present invention.
Claims (4)
1. a kind of piecewise linearity compression method of Wave data Real Time Compression, it is characterised in that comprise the following steps：
1）Set limits of error value E, maximum siding-to-siding block length L, the waveform number of Wave data set of Wave data set
According to the data variation record limit value RangeLimit of set；
2）According to step 1）The limits of error value E of setting, calculates subregion and judges fulcrum, using time quantum as X axles, number
Value amount constitutes coordinate system as Y axles, will be first data point A0 in L data interval with maximum siding-to-siding block length
Two points for being E along Y direction distance, fulcrum is judged as two subregions that subregion is carried out to data interval；
3）Maximum siding-to-siding block length is calculated respectively judges fulcrum line for the subsequent point in L data interval and the two subregions
Slope K 1 and K2, K1 are the slope that subsequent point judges fulcrum line with the subregion more than A0, and K1 only records maximum,
K2 only records minimum value；
4）In the maximum siding-to-siding block length set as in L data interval, as long as K1<K2, then continue to calculate K1 and K2, directly
To finding K1>K2 point completes one and takes turns data partition as partition end, when in the data that defined maximum siding-to-siding block length is L
In interval, K1 is not found>K2 point, then choose maximum siding-to-siding block length and be used as partition end for the terminal of L data interval
First round subregion is carried out to data interval, data partition is formed；
5）Using the partition end of previous round data partition as the starting point of next round data partition, according to step 2）, step 3）、
Step 4）, epicycle subregion is carried out to data interval, often wheel division operation is similar afterwards, the terminal of wheel data partition is made above
Division operation is carried out for the starting point of lower whorl data partition；
6）The region maximum and region minimum in each data partition are recorded respectively；
7）Limit value RangeLimit is recorded according to the data variation of setting, data are carried out with noise reduction and compression differentiates, is calculated respectively
Data variation the amount Rmax and Rmin of region maximum and region minimum and the starting point of the data partition, work as variable quantity
Then it is considered as that noise is compressed and given up less than record limit value RangeLimit, when variable quantity is more than record limit value
RangeLimit, then be considered as that key waveforms point is recorded, with starting point and the key waveforms point of the data partition it
Between straight line replace the interval in total data point.
2. the piecewise linearity compression method of the Wave data Real Time Compression according to claim 1, it is characterised in that：Institute
State step 3）In, when maximum siding-to-siding block length is only the first data point AO in L data interval, K1 is to bear infinite, K2
To be just infinite.
3. the piecewise linearity compression method of the Wave data Real Time Compression according to claim 2, it is characterised in that:When
Maximum siding-to-siding block length is has multiple data points in L data interval, and more and more with counting, K1 is increasing, and K2 is more next
It is smaller.
4. the piecewise linearity compression method of the Wave data Real Time Compression according to claim 3, it is characterised in that：It is logical
Adjustment limits of error value E, maximum siding-to-siding block length L, data variation record limit value RangeLimit are crossed, difference can be achieved
The data compression ratio of grade.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410039805.7A CN103795419B (en) | 2014-01-27 | 2014-01-27 | A kind of piecewise linearity compression method of Wave data Real Time Compression |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410039805.7A CN103795419B (en) | 2014-01-27 | 2014-01-27 | A kind of piecewise linearity compression method of Wave data Real Time Compression |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103795419A CN103795419A (en) | 2014-05-14 |
| CN103795419B true CN103795419B (en) | 2017-08-04 |
Family
ID=50670782
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410039805.7A Active CN103795419B (en) | 2014-01-27 | 2014-01-27 | A kind of piecewise linearity compression method of Wave data Real Time Compression |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103795419B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105375930A (en) * | 2015-04-09 | 2016-03-02 | 国家电网公司 | Energy storage power station massive data compression method and system |
| CN104917534B (en) * | 2015-06-30 | 2018-09-21 | 京东方科技集团股份有限公司 | Compression, the method and apparatus of decompression data information, the method and apparatus and display device of compensation driving |
| CN105955673A (en) * | 2016-05-26 | 2016-09-21 | 广东雅达电子股份有限公司 | Filtering method of energy consumption monitoring data |
| CN109143017B (en) * | 2018-07-31 | 2021-03-30 | 成都天衡智造科技有限公司 | Production test data processing method for semiconductor industry |
| CN109257047B (en) * | 2018-09-12 | 2019-12-27 | 中科驭数（北京）科技有限公司 | Data compression method and device |
| CN111447284A (en) * | 2020-04-10 | 2020-07-24 | 四川中鼎智能技术有限公司 | Dynamic data compression, communication and restoration method and device based on Fourier transform algorithm, terminal equipment and readable storage medium |
| CN111517195B (en) * | 2020-04-28 | 2021-11-19 | 广州广日电梯工业有限公司 | Data processing method for elevator acceleration sensor |
| CN113381766A (en) * | 2021-06-25 | 2021-09-10 | 河北省科学院应用数学研究所 | Method, terminal and storage medium for electrocardiogram data compression |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101069353A (en) * | 2004-09-28 | 2007-11-07 | 通用电气公司 | Industrial data compression systems and methods |
| CN102098058A (en) * | 2010-11-12 | 2011-06-15 | 中南大学 | Method for efficiently and linearly compressing and decompressing time-series data in real time |
| CN102393855A (en) * | 2011-10-18 | 2012-03-28 | 国电南瑞科技股份有限公司 | Dynamic control method for loss compression ratio of process data |
| CN102437856A (en) * | 2011-11-04 | 2012-05-02 | 国电南京自动化股份有限公司 | Characteristic point extraction-based three state data compression method of underground integrated monitoring system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7143046B2 (en) * | 2001-12-28 | 2006-11-28 | Lucent Technologies Inc. | System and method for compressing a data table using models |
-
2014
- 2014-01-27 CN CN201410039805.7A patent/CN103795419B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101069353A (en) * | 2004-09-28 | 2007-11-07 | 通用电气公司 | Industrial data compression systems and methods |
| CN102098058A (en) * | 2010-11-12 | 2011-06-15 | 中南大学 | Method for efficiently and linearly compressing and decompressing time-series data in real time |
| CN102393855A (en) * | 2011-10-18 | 2012-03-28 | 国电南瑞科技股份有限公司 | Dynamic control method for loss compression ratio of process data |
| CN102437856A (en) * | 2011-11-04 | 2012-05-02 | 国电南京自动化股份有限公司 | Characteristic point extraction-based three state data compression method of underground integrated monitoring system |
Non-Patent Citations (3)
| Title |
|---|
| An optimized algorithm for lossy compression of real-time data;Gang Chen et al.;《Intelligent Computing and Intelligent Systems (ICIS), 2010 IEEE International Conference on》;20101031;187-191 * |
| Process trending with piecewise linear smoothing;R.S.H. Mah et al.;《Computers & Chemical Engineering》;19950228;第19卷(第2期);129-137 * |
| 过程数据压缩技术综述;汤同奎 等;《计算机与应用化学》;20100331;第17卷(第03期);193-197 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103795419A (en) | 2014-05-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103795419B (en) | A kind of piecewise linearity compression method of Wave data Real Time Compression | |
| CN106507311A (en) | A kind of processing method of positioning and system | |
| CN103237097B (en) | Fall protection devices, control method, controller and electronic equipment | |
| CN104200620A (en) | Building healthy remote monitoring system and building healthy remote monitoring method | |
| CN106649026A (en) | Monitoring data compression method applicable to operation and maintenance automation system | |
| CN101038317A (en) | Voltage transformer wireless monitoring system and method | |
| CN102098058A (en) | Method for efficiently and linearly compressing and decompressing time-series data in real time | |
| CN104866636B (en) | A kind of well logging Real-time Data Processing Method | |
| CN103809087A (en) | PLC (programmable logic controller) based online monitoring device for partial discharge signals of transformer | |
| CN105608231A (en) | Cloud-based flow data storage system and query method thereof | |
| CN105701349B (en) | Non-uniform granular discrete unit fast linear contact detecting method | |
| CN202334922U (en) | Large-scale wireless monitoring and sensing device | |
| CN114001703B (en) | Landslide deformation data real-time filtering method | |
| CN115343366A (en) | Bridge structure monitoring system based on acoustic emission technology | |
| CN114310907A (en) | Multi-working-condition self-adaptive industrial robot tail end vibration suppression method | |
| CN204463412U (en) | A kind of supervising device based on internet | |
| CN113359644A (en) | Synchronous industrial internet acquisition control system and control method thereof | |
| CN107584488A (en) | A kind of robotic vision bootstrap technique | |
| CN102545909B (en) | Greedy type intelligent fitting lossy data compression method | |
| CN110795678B (en) | Weak information detection method based on small data volume chaotic technology | |
| CN111814256A (en) | RK4Henon method-based wing aeroelastic system chaotic response analysis method | |
| CN220516866U (en) | Pavement detection system for service robot | |
| CN104997146B (en) | The axle of MCU driving formula three at the uniform velocity moving method | |
| CN201477091U (en) | Powder percentage detector for flux-cored wire | |
| CN101692251A (en) | Function parameter estimation based on-line compression and decompression method for general process data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |