CN112234993A - Strong transient signal data compression method - Google Patents
Strong transient signal data compression method Download PDFInfo
- Publication number
- CN112234993A CN112234993A CN201910633185.2A CN201910633185A CN112234993A CN 112234993 A CN112234993 A CN 112234993A CN 201910633185 A CN201910633185 A CN 201910633185A CN 112234993 A CN112234993 A CN 112234993A
- Authority
- CN
- China
- Prior art keywords
- compression
- data
- lossy
- lossless
- strong transient
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
- H03M7/3084—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction using adaptive string matching, e.g. the Lempel-Ziv method
- H03M7/3088—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction using adaptive string matching, e.g. the Lempel-Ziv method employing the use of a dictionary, e.g. LZ78
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
- H03M7/70—Type of the data to be coded, other than image and sound
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention discloses a strong transient signal data compression method, which comprises the steps of (1) analyzing lossy compression and lossless compression algorithms; (2) analyzing the lossy compression and lossless compression effects; (3) and combining the lossy compression method and the lossless compression method to realize the self-adaptive data compression. The method aims to solve the problem that sampling frequency and measurement parameters aggravate data storage pressure. And simultaneously, the multi-parameter and high-density composite signal is optimally stored. The method has the advantage that a data compression scheme of strong transient signals is explored by using several mainstream data compression methods. By identifying strong transient signals, data are divided into stable data and dynamic transient data, and lossy and lossless compression effect research is carried out on the data on the basis. And finally, performing dynamic adaptive data compression on the process data. Thereby solving the data storage problem.
Description
Technical Field
The invention relates to a data compression method, in particular to a strong transient signal data compression method.
Background
In the field of electronic information technology today, a digital revolution is taking place that has a long-term impact. Since the amount of data in digitized multimedia information, especially digital video and audio signals, is particularly enormous, it is difficult to put it into practical use without efficient compression. Data compression techniques have therefore become a key common technology in digital communications, broadcasting, storage and multimedia entertainment today.
Strong transients are a non-uniformly distributed signal relative to stationary signals, the generation of which is often accompanied by strong energy release, high frequency vibration and high background noise, while inducing a series of changes in the momentum and attitude of the object. The non-uniformity, non-stability and transient change are the most obvious characteristics of strong transient signals, the measurement requirements of the strong transient signals are in the fields of petroleum exploration, mine exploitation, automobile safe collision test, weapon development, mechanical fault diagnosis and the like, and meanwhile, the changes of process pressure, temperature, object line motion state and angular motion state before and after the strong transient signals occur are also measurement parameters with high reference value. In a storage test system of multi-parameter and multi-mode composite signals including strong transients, due to lack of real-time identification of the strong transient signals, a large amount of low-value data are recorded in the whole process of high-density sampling. On the other hand, the increase in sampling frequency and measurement parameters also stresses the storage of test system data. The optimized storage of the multi-parameter high-density composite signal is an urgent practical requirement and is an inevitable development process in the field. By identifying strong transient signals, data are divided into stable data and dynamic transient data, and lossy and lossless compression effect research is carried out on the data on the basis. The revolving door compression algorithm is used as a lossy method, has great advantages under the condition of low requirement on signal reconstruction accuracy, can provide higher compression ratio, higher compression speed and lower resource occupation compared with lossless compression, has extremely strong compression performance on steady signals due to the compression property of STD, can reach the maximum compression ratio of 80.65 on steady data by utilizing the STD algorithm, and has reduction error not exceeding 0.95FS through a three-spline reconstruction algorithm. The LZW compression algorithm is used as a lossless method and has a good compression effect. When data is restored, the algorithm can establish a dictionary which is the same as that in the compression process through the compressed data stream, and finally the purpose of distortion-free compression is achieved.
In the aspect of data compression, the sensor data is processed by adopting a hybrid compression and self-adaptive compression method in an embedded system abroad, and the application in a storage test system is less. Meanwhile, the domestic unit also performs related optimization work: the LZSS lossless compression algorithm is applied to a missile-borne black box system by Nanjing university of science and technology in 2006, and millimeter wave signals are compressed and stored. The 2014 university in north and middle has studied the compression encryption technology of the missile-borne recorder, and the data security is ensured on the basis of realizing higher compression ratio. In 2017, in order to solve the storage problem of multi-channel transient signals, the university of catharanthus roseus adopts a self-adaptive frequency conversion algorithm, and redundant data is reduced on the premise of completely retaining data compression of transient information.
The invention aims at the characteristics of long-term stability and transient mutation of strong transient signals, analyzes the defects of the conventional general data compression method, combines the high compression ratio of lossy compression and the distortion-free advantage of lossless compression, and provides a strong transient signal data compression method.
Disclosure of Invention
The invention discloses a strong transient signal data compression method, which aims to solve the problem that sampling frequency and measurement parameters increase data storage pressure. And simultaneously, the multi-parameter and high-density composite signal is optimally stored. The invention explores the data compression scheme of strong transient signals by using several mainstream data compression methods. By identifying strong transient signals, data are divided into stable data and dynamic transient data, and lossy and lossless compression effect research is carried out on the data on the basis. Compared with lossless compression, lossy compression can provide higher compression ratio, faster compression speed and lower resource occupation, and the compression ratio of the maximum 80.65 can be achieved for smooth data by using the STD algorithm. And then, discussing a lossless compression method, analyzing to know that the LZW compression algorithm has a better compression effect, and finally, performing dynamic adaptive data compression on the process data. Thereby solving the data storage problem.
In order to solve the technical problems, the invention adopts the technical scheme that:
aiming at the characteristics of long-term stability and instantaneous mutation of strong transient signals, the defects of the conventional general data compression method are firstly analyzed, and the advantages of high compression ratio of lossy compression and no distortion of lossless compression are combined to provide the strong transient signal data compression method. Thereby solving the problem of data storage.
Compared with the prior art, the invention has the beneficial effects that:
1) the invention divides the data into stable data and dynamic transient data by identifying strong transient signals, and performs lossy and lossless compression effect research on the data on the basis. Lossy compression can provide higher compression ratios, faster compression speeds, and lower resource usage than lossless compression. For the lossless compression method, the LZW compression algorithm has a better compression effect. The problem of data storage difficulty caused by limited storage capacity is solved.
2) The invention performs dynamic adaptive data compression on process data. And (3) by utilizing the identification of the strong transient signal, isolating the steady signal and the non-steady signal in the measurement process of the strong transient signal, performing STD lossy compression on the steady signal, and performing lossless compression on the transient strong transient signal. The problem of data storage difficulty caused by limited storage capacity is solved.
Drawings
FIG. 1 is a flow chart of the steps of the present invention.
Fig. 2 is a schematic diagram of STD lossy compression.
FIG. 3 is a flow chart of the LZW lossless compression principle.
Detailed Description
The invention discloses a strong transient signal data compression method, which aims to solve the problem that sampling frequency and measurement parameters increase data storage pressure. And simultaneously, the multi-parameter and high-density composite signal is optimally stored. The invention explores the data compression scheme of strong transient signals by using several mainstream data compression methods. By identifying strong transient signals, data are divided into stable data and dynamic transient data, and lossy and lossless compression effect research is carried out on the data on the basis. Compared with lossless compression, lossy compression can provide higher compression ratio, faster compression speed and lower resource occupation, and the compression ratio of the maximum 80.65 can be achieved for smooth data by using the STD algorithm. And then, discussing a lossless compression method, analyzing to know that the LZW compression algorithm has a better compression effect, and finally, performing dynamic adaptive data compression on the process data. Thereby solving the data storage problem.
The invention provides a strong transient signal data compression method, which comprises the following steps:
step 1: lossy compression and lossless compression algorithms are analyzed.
In a missile-borne flight test, strong transient signals only occupy a small amount of time, stable non-transient signals occupy a large amount of storage space, and data compression on overload signals in the flight process is an effective means for reducing the storage space. The invention classifies data compression into lossy compression and lossless compression according to reconstruction effect classification. Lossy compression is applied in the field of sound and images, and it can reduce the file size without losing important information. Compression distortion occurs simultaneously with lossy compression, and when the lossy compression removes excessive information, key information is lost, so that the lossy compression cannot pursue high compression ratio at a glance, and the lossy compression method is suitable for occasions where some degree of information loss can be accepted in order to improve compression efficiency. The difference between lossless compression and lossy compression is that the compression process is reversible, and original data can be completely restored through a decompression algorithm without losing any information. Such algorithms remove redundant data during compression and add it during decompression, typically at a relatively low compression rate, typically 1/2-1/5. When there is no tolerance for loss of any data, lossless compression methods are typically used.
Step 2: and analyzing the lossy compression and lossless compression effects.
The revolving gate compression algorithm (STD) is a commonly used lossy compression algorithm in real-time data processing, and the idea is to classify data by a virtual parallelogram, and the enclosed area is considered as similar data, and the processing can be omitted for this part of data. All data are included by continuously constructing a parallelogram, and all similar data are ignored, so that the method is characterized by small operand and real-time tracking of the change trend. The realization process is as follows:
the first data A is taken as a starting point when compression starts, E is selected as compression deviation (the size of E determines the compression effect, the larger the E is, the higher the compression ratio is, the larger the deviation is), M, N two points are abstracted into two axes of a revolving door, the connecting line of a data point and M, N is taken as a door which is not opened continuously, the angle 1 and the angle 2 are the opening angles of the revolving door, the opening angle of the door is only increased but not decreased, the angle 1 and the angle 2 are increased along with the increase of the data quantity, a virtual parallelogram upper and lower boundary is constructed by the point B and the point C, the angle 1 plus 2 is less than 180 degrees before the point C, the angle 1 plus 2 is more than 180 degrees at the point D, so the original parallelogram cannot include the point D, compression is stopped, the data point C and the starting point before the point D are stored, and all data of the parallelogram are replaced by A, C two points. And after the completion, replacing the point A with the point C to serve as a new starting point, using the point C + E, C-E as a new M, N, and continuing to construct a new round of parallelogram, as shown in the STD lossy compression schematic diagram of FIG. 2.
The lossy compression algorithm has great advantages under the condition of low requirement on signal reconstruction accuracy, and compared with lossless compression, the lossy compression can provide higher compression ratio, higher compression speed and lower resource occupation. The LZW compression algorithm is used as a lossless method and has a good compression effect. The dictionary-based compression algorithm uses 8-bit characters as a basic unit, and uses fixed-length data to replace variable-length character strings, thereby achieving the purpose of data compression. The self-adaptive compression algorithm has the basic idea that a real-time dictionary is dynamically established according to input data, and data output is determined according to the existence of subsequent data in the dictionary. When data is restored, the algorithm can establish a dictionary which is the same as that in the compression process through the compressed data stream, and finally the purpose of distortion-free compression is achieved. As shown in the flow chart of the LZW lossless compression principle in fig. 3.
And step 3: and combining the lossy compression method and the lossless compression method to realize the self-adaptive data compression.
The invention takes adaptive data compression as an example, utilizes the proposed compression method to isolate stationary signals and non-stationary signals in the measurement process of strong transient signals, performs STD lossy compression on the stationary signals, and performs lossless compression on the strong transient signals. The test data reaches the global compression ratio of 50.48, strong transient signals can be completely reserved, the maximum reduction error of non-transient signals does not exceed 0.72% FS, and the transient signals can be reduced without distortion.
To further verify the effectiveness of the proposed compression method, a ground drop test of some object was performed. In the experimental process, signals can be reserved without distortion, the system successfully identifies the falling signals, and from the data reconstruction result, the maximum error of the steady-state signals is 0.2 and the maximum error is not more than 0.87% FS. The original size of the test data is 410kB, the compressed data is 6.95kB, and the comprehensive compression ratio reaches 59. The experimental results demonstrate the feasibility of the compression method proposed by the present invention.
Claims (2)
1. A strong transient signal data compression method is characterized in that: (1) analyzing lossy compression and lossless compression algorithms; (2) analyzing the lossy compression and lossless compression effects; (3) and combining the lossy compression method and the lossless compression method to realize the self-adaptive data compression.
2. A method of strong transient signal data compression as claimed in claim 1, characterized by: and (3) by utilizing the identification of the strong transient signal, isolating the steady signal and the non-steady signal in the measurement process of the strong transient signal, performing STD lossy compression on the steady signal, and performing lossless compression on the transient strong transient signal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910633185.2A CN112234993A (en) | 2019-07-15 | 2019-07-15 | Strong transient signal data compression method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910633185.2A CN112234993A (en) | 2019-07-15 | 2019-07-15 | Strong transient signal data compression method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112234993A true CN112234993A (en) | 2021-01-15 |
Family
ID=74111441
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910633185.2A Pending CN112234993A (en) | 2019-07-15 | 2019-07-15 | Strong transient signal data compression method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112234993A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113242041A (en) * | 2021-03-10 | 2021-08-10 | 湖南大学 | Data hybrid compression method and system thereof |
| CN114629501A (en) * | 2022-03-16 | 2022-06-14 | 重庆邮电大学 | Edge data classification compression method of machining process state information |
| CN116934431A (en) * | 2023-09-19 | 2023-10-24 | 贵昌集团有限公司 | Electronic commerce data intelligent management system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102035200A (en) * | 2009-09-29 | 2011-04-27 | 西门子公司 | Method and device for processing signals |
| CN102394658A (en) * | 2011-10-16 | 2012-03-28 | 西南科技大学 | Composite compression method oriented to mechanical vibration signal |
| CN102611823A (en) * | 2012-01-13 | 2012-07-25 | 百度在线网络技术（北京）有限公司 | Method and equipment capable of selecting compression algorithm based on picture content |
| CN107196660A (en) * | 2017-04-24 | 2017-09-22 | 南京数维康信息科技有限公司 | Low power consumption data compression algorithm |
-
2019
- 2019-07-15 CN CN201910633185.2A patent/CN112234993A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102035200A (en) * | 2009-09-29 | 2011-04-27 | 西门子公司 | Method and device for processing signals |
| CN102394658A (en) * | 2011-10-16 | 2012-03-28 | 西南科技大学 | Composite compression method oriented to mechanical vibration signal |
| CN102611823A (en) * | 2012-01-13 | 2012-07-25 | 百度在线网络技术（北京）有限公司 | Method and equipment capable of selecting compression algorithm based on picture content |
| CN107196660A (en) * | 2017-04-24 | 2017-09-22 | 南京数维康信息科技有限公司 | Low power consumption data compression algorithm |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113242041A (en) * | 2021-03-10 | 2021-08-10 | 湖南大学 | Data hybrid compression method and system thereof |
| CN114629501A (en) * | 2022-03-16 | 2022-06-14 | 重庆邮电大学 | Edge data classification compression method of machining process state information |
| CN116934431A (en) * | 2023-09-19 | 2023-10-24 | 贵昌集团有限公司 | Electronic commerce data intelligent management system |
| CN116934431B (en) * | 2023-09-19 | 2023-12-05 | 贵昌集团有限公司 | Electronic commerce data intelligent management system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112234993A (en) | Strong transient signal data compression method | |
| Sun et al. | Sound based fault diagnosis for RPMs based on multi-scale fractional permutation entropy and two-scale algorithm | |
| CN102510287B (en) | Method for rapidly compressing industrial real-time data | |
| DE102019113534A1 (en) | Time domain feature extraction method and system for automatic speech recognition | |
| AU559775B2 (en) | Voice digitized compander | |
| CN105512120A (en) | Earthquake data compression method, compression storage structure and random access method | |
| Yang et al. | Expanding trace buffer observation window for in-system silicon debug through selective capture | |
| CN105530013A (en) | Waveform data compression method and system | |
| CN108873062A (en) | A kind of Multi-encoder high-speed seismic data parallel lossless compression method based on FPGA | |
| CN102437854A (en) | Industrial real-time data compression method with high compression ratio | |
| CN103018514A (en) | Virtual oscilloscope device based on sound card | |
| CN113899863A (en) | High-voltage cable hidden defect detection method and system based on gas characteristics | |
| CN103078647A (en) | Hardware decoding implementation system and method of LZ77 compression algorithm | |
| CN202794336U (en) | Radio astronomical narrow-band signal digital analysis terminal based on undersampling technology | |
| CN117895950A (en) | Data compression method based on radar algorithm | |
| CN107292067B (en) | Gear fault diagnosis method based on compressed sensing and bispectrum analysis | |
| CN103440870A (en) | Method and device for voice frequency noise reduction | |
| US8730068B2 (en) | Method and apparatus for compression and de-compression of spectral data | |
| CN109194667A (en) | The device of realization I/Q data signal data compression and transfer function based on frequency domain detection | |
| CN110191345A (en) | A kind of increment type lossless compression method of prospect based on huffman coding, background separation | |
| CA3234711A1 (en) | Systems, methods and apparatus for producing sustainable aviation fuel | |
| Zhang et al. | A New Rockburst Experiment Data Compression Storage Algorithm Based on Big Data Technology. | |
| CN101064844A (en) | Method for performing matching compression to image using rotary compressed codebook | |
| JPH11505679A (en) | Improved data processing method and apparatus | |
| CN113217229A (en) | Method and system for testing instantaneous nozzle throat diameter |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210115 |
|
| WD01 | Invention patent application deemed withdrawn after publication |