CN108234464A - For the gathered data high-efficiency compression method of centralized signal supervision system - Google Patents
For the gathered data high-efficiency compression method of centralized signal supervision system Download PDFInfo
- Publication number
- CN108234464A CN108234464A CN201711422134.2A CN201711422134A CN108234464A CN 108234464 A CN108234464 A CN 108234464A CN 201711422134 A CN201711422134 A CN 201711422134A CN 108234464 A CN108234464 A CN 108234464A
- Authority
- CN
- China
- Prior art keywords
- data
- compression
- gathered
- gathered data
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
- H04L1/0056—Systems characterized by the type of code used
- H04L1/0057—Block codes
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The present invention relates to a kind of gathered data high-efficiency compression method for centralized signal supervision system, including：(1) compression pretreatment is carried out according to the type of gathered data and feature, forms the gathered data serialization unit of specification；(2) compression arbitration process is carried out to the data in gathered data serialization unit, to meet expected compression ratio as target, to select data compression scheme；(3) data in gathered data serialization unit are carried out with compression processing, and data after persistence compression key data and compression, data transmission and data is supplied to store and use；(4) processing is unziped it to the data after serializing, according to the corresponding decompression mode of the compression key data decimation of persistence in step (3), data lossless after compression is reduced to compress preceding data.Compared with prior art, the present invention can effectively reduce acquired data storage space and network transmission bandwidth in centralized signal supervision system, so as to improve the availability of centralized signal supervision system entirety.
Description
Technical field
The present invention relates to railway signal systems, high more particularly, to a kind of gathered data for centralized signal supervision system
Imitate compression method.
Background technology
With the development of centralized signal supervision system, gathered data amount is increasing.Traditional centralized signal supervision system
Data handling procedure of uniting does not carry out differentiation compression processing to gathered data, and in practical applications, caused direct result is：
First, data space consumption is big, can not long-time store historical data；Second, volume of transmitted data is big, occupies network bandwidth
Excessively, the real-time of the network transmission of other significant datas is affected.So that the availability of whole system lowers.
Invention content
It is an object of the present invention to overcome the above-mentioned drawbacks of the prior art and provide one kind for signal concentration
The gathered data high-efficiency compression method of monitoring system, this method can effectively reduce acquired data storage in centralized signal supervision system
Space and network transmission bandwidth, so as to improve the availability of centralized signal supervision system entirety.
The purpose of the present invention can be achieved through the following technical solutions：
A kind of gathered data high-efficiency compression method for centralized signal supervision system includes the following steps：
(1) compression pretreatment is carried out according to the type of gathered data and feature, the gathered data serializing for forming specification is single
Member；
(2) compression arbitration process is carried out to the data in gathered data serialization unit, to meet expected compression ratio as mesh
Mark, to select data compression scheme；
(3) compression processing is carried out to the data in gathered data serialization unit, is chosen according to the result of calculation of step (2)
Corresponding compress mode simultaneously carries out compressometer calculation, and data after persistence compression key data and compression, is supplied to data transmission
It stores and uses with data；
(4) processing is unziped it to the data after serializing, is selected according to the compression key data of persistence in step (3)
Corresponding decompression mode is taken, data lossless after compression is reduced to compress preceding data.
Preferably, the type and feature according to gathered data carries out compression pretreatment and specifically includes following steps：
101) lossless conversion and permutation and combination are carried out to the gathered data received；
102) for the real-time data collection of continuous type, the gathered data in time slice is created into sequence by Data Identification
Change unit, and temporally relationship is added in serialization unit by gathered data；It, will be single for the real-time data collection of discrete type
One moment discrete data creates corresponding serialization unit by concrete type；
103) for the real-time data collection of continuous type, serialization unit is submitted as unit of time slice data；For
The real-time data collection of discrete type submits serialization unit as unit of single point in time data.
Preferably, the data in the serialization unit to gathered data carry out compression arbitration process specifically include it is following
Step：
201) different arbitration calculation process is used to continuous type gathered data and discrete type gathered data；
202) arbitration calculating is according to priority carried out to various compress modes by flow, when the compression ratio being calculated meet it is pre-
The compress mode is then selected during the phase and terminates arbitration and is calculated；Compare choosing if when each compress mode fails to reach expected compression ratio
Take wherein optimal one.
Preferably, the data in the serialization unit to gathered data carry out compression processing and specifically include following step
Suddenly：
301) binary system based on continuously record data is byte-by-byte asks differential pressure to contract, for continuous type gathered data；
302) binary system based on continuously record data integrally asks differential pressure to contract, for continuous type gathered data；
303) numerical value based on continuously record data asks differential pressure to contract, for continuous type gathered data；
304) it is compressed based on the merger of same type gathered data, for discrete type gathered data；
305) it is encoded and compressed based on repeated data section, for continuous type and discrete type gathered data.
Preferably, class Huffman codings are encoded in the step 305).
Preferably, the data to after serializing unzip it processing and specifically include following steps：
401) the byte-by-byte decompression for seeking difference of the binary system based on continuously record data；
402) binary system based on continuously record data integrally seeks the decompression of difference；
403) numerical value based on continuously record data seeks the decompression of difference；
404) decompression based on same type gathered data merger；
405) decompression based on repeated data section coding.
Preferably, class Huffman codings are encoded in the step 405).
Compared with prior art, the present invention can effectively reduce acquired data storage space and net in centralized signal supervision system
Network transmission bandwidth, so as to improve the availability of centralized signal supervision system entirety.
Description of the drawings
Fig. 1 is for the flow diagram of gathered data compression method in the present invention.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present invention, the technical solution in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is the part of the embodiment rather than whole embodiments of the present invention.Based on this hair
Embodiment in bright, the every other reality that those of ordinary skill in the art are obtained under the premise of creative work is not made
Example is applied, should all belong to the scope of protection of the invention.
As shown in Figure 1, the gathered data of centralized signal supervision system has following features：For continuous type gathered data
(such as switching value, analog quantity), varied number is few between frequency acquisition height but continuous data and amplitude of variation is small；For from
Dissipate type gathered data (such as track switch operation curve), frequency acquisition it is indefinite but in the same time in data same or similar data compared with
It is more.The gathered data high-efficiency compression method of above-mentioned characteristic invention based on gathered data, embodiments thereof are as follows：
1st, compression pretreatment is carried out according to the type of gathered data and feature
(1) the acquisition initial data received is subjected to lossless conversion and rearranges group by acquisition source and type of service
It closes, converts data to the pattern convenient for compression processing.
(2) serialization unit is created according to the grab type of data.The serialization unit of continuous type gathered data is temporally
Segmented tissue data, the record service identification of data and surrounding time relationship, convert the data finished on time in serialization unit
Between relationship be added in serialization unit successively；The serialization unit of discrete type gathered data corresponds to the acquisition number of single point in time
According to.
(3) for the real-time data collection of continuous type, serialization unit is submitted as unit of time slice data, i.e., currently
Segment data time span is more than after preset value, serialization unit is submitted to subsequent processing and creates new serialization unit；It is right
In the real-time data collection of discrete type, serialization unit is submitted as unit of single point in time data.
2nd, compression arbitration process is carried out to the data in serialization unit
(1) according to the corresponding compression arbitration process flow of the grab type of data selection.
(2) arbitration calculating is according to priority carried out to various compress modes according to flow, when the compression ratio being calculated meets
It is expected that when then select the compress mode and terminate arbitration calculate；Compare if when each compress mode fails to reach expected compression ratio
Choose wherein optimal one.
For continuous type gathered data, arbitration calculating (prioritized) is carried out to following compress mode：It is based on
The binary system of continuously record data is byte-by-byte to ask differential pressure contracting, the binary system based on continuously record data integrally to ask differential pressure contracting, be based on
The numerical value of continuously record data asks differential pressure contracting, based on repeated data section coding (class Huffman codings) compression.
For discrete type gathered data, arbitration calculating (prioritized) is carried out to following compress mode：It is based on
Same type gathered data merger compression is compressed based on repeated data section coding (class Huffman codings).
3rd, compression processing is carried out to the data in serialization unit.The foundation of various compress modes, algorithm key point, compression
Key message is described as follows afterwards：
(1) binary system based on continuously record data is byte-by-byte asks differential pressure to contract
This method foundation：After pretreatment, same type data are organized in together continuous type gathered data, adjacent
The serialized binary content change of time data is little.
Algorithm key point：On the basis of the previous record data of current data, to the binary content of current data one by one
Byte is done differentiation and is compared, and the byte position information of difference and byte value are added in index.
Key message after compression：Binary data difference-byte index information.
(2) binary system based on continuously record data integrally asks differential pressure to contract
This method foundation：After pretreatment, identical services type data are organized in together continuous type gathered data,
The serialized binary content change of adjacent time data is little.
Algorithm key point：It is whole to the binary content of current data on the basis of the previous record data of current data
It does differentiation to compare, index information is established to binary system byte, by each byte data difference condition of bit stored records, pressure
Data also need to preserve difference-byte value after contracting.
Key message after compression：Binary data entirety difference index information and difference-byte value.
(3) numerical value based on continuously record data asks differential pressure to contract
This method foundation：After pretreatment, identical services type data are organized in together continuous type gathered data, special
It is not for analog quantity class gathered data, the numerical value change of adjacent time data is little, and only recording quantity difference can occupy
Smaller space.
Algorithm key point：On the basis of the previous record data of current data, to each gathered data point of current data
Do the difference that numerical value asks poor processing, record data point difference index and specific data.
Key message after compression：The difference index and data difference of all data points.
(4) it is compressed based on the merger of same type gathered data
This method foundation：Certain discrete type gathered datas, particularly track switch operation curve class data, synchronization are deposited
In a plurality of data, there are many same or similar data of wherein same type, if same type data carry out merger processing, record base
Quasi- data and different information can save significantly on data space.
Algorithm key point：Synchronization gathered data is grouped by type, reference data is chosen in every group of data into line number
According to merger, different information of other data relative to reference data is recorded.
Key message after compression：Merger index information, reference data and different information.
(5) based on repeated data section coding (class Huffman codings) compression
This method foundation：Continuous type and discrete type gathered data are after pretreatment, same or similar type of service data
It is organized in together.For the gathered data of single point in time or the same or similar type of service in the period, different acquisition
It is identical that the data of point have a very maximum probability, therefore has a large amount of repeated data section inside entire data record.By repeated data
Section coding is replaced, and can effectively reduce data space.
Algorithm key point：The repeated data section frequency of occurrences in all data of statistics, creates priority weights grade queue, and right
Repeated data section carries out coding replacement.
Key message after compression：Repeated data section weight information and coding information
4th, processing is unziped it to the data after serializing.According to crucial letter after serialized data compress mode and compression
Breath carries out inverse operation to the compress mode described in step 3, data lossless after compression is restored.Specially：
(1) the byte-by-byte decompression for seeking difference of the binary system based on continuously record data
Algorithm key point：Binary data difference-byte index information is obtained, on the basis of previous record data, according to rope
The difference-byte position recorded in fuse breath covers the position data with difference-byte value.
(2) binary system based on continuously record data integrally seeks the decompression of difference
Algorithm key point：Binary data entirety difference index information and difference-byte value, using previous record data as base
Standard traverses index information by bit, the position data is covered with difference-byte value according to difference-byte position.
(3) numerical value based on continuously record data seeks the decompression of difference
Algorithm key point：The difference index and data difference of all data points are obtained, on the basis of previous record data, is taken
Each data point difference does summation operation, restores data before compression.
(4) decompression based on same type gathered data merger
Algorithm key point：Obtain merger index information, reference data and different information.Restore data by merger index information
Grouping does summation operation according to its reference data and different information respectively in each grouping, restores data before compression.
(5) decompression based on repeated data section coding (class Huffman codings)
Algorithm key point：Repeated data section weight information and coding information are obtained, creates repeated data section priority weights grade
Queue, and reversely replaced according in coding information upon compression data, restore data before compression.
The above description is merely a specific embodiment, but protection scope of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can readily occur in various equivalent modifications or replace
It changes, these modifications or substitutions should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be with right
It is required that protection domain subject to.
Claims (7)
1. a kind of gathered data high-efficiency compression method for centralized signal supervision system, which is characterized in that include the following steps：
(1) compression pretreatment is carried out according to the type of gathered data and feature, forms the gathered data serialization unit of specification；
(2) compression arbitration process is carried out to the data in gathered data serialization unit, to meet expected compression ratio as target, come
Select data compression scheme；
(3) compression processing is carried out to the data in gathered data serialization unit, is chosen and corresponded to according to the result of calculation of step (2)
Compress mode and carry out compressometer calculation, and persistence compression key data and compression after data, be supplied to data transmission sum number
It is used according to storage；
(4) processing is unziped it to the data after serializing, according to the compression key data decimation pair of persistence in step (3)
Data lossless after compression is reduced to compress preceding data by the decompression mode answered.
2. according to the method described in claim 1, it is characterized in that, the type and feature according to gathered data is pressed
Contracting pretreatment specifically includes following steps：
101) lossless conversion and permutation and combination are carried out to the gathered data received；
102) for the real-time data collection of continuous type, the gathered data in time slice is created into serializing list by Data Identification
Member, and temporally relationship is added in serialization unit by gathered data；For the real-time data collection of discrete type, when will be single
It carves discrete data and creates corresponding serialization unit by concrete type；
103) for the real-time data collection of continuous type, serialization unit is submitted as unit of time slice data；For discrete
The real-time data collection of type submits serialization unit as unit of single point in time data.
3. according to the method described in claim 1, it is characterized in that, data in the serialization unit to gathered data into
Row compression arbitration process specifically includes following steps：
201) different arbitration calculation process is used to continuous type gathered data and discrete type gathered data；
202) arbitration calculating is according to priority carried out to various compress modes by flow, when the compression ratio being calculated meets expected
It then selectes the compress mode and terminates arbitration and calculate；Compare if when each compress mode fails to reach expected compression ratio and choose it
In optimal one.
4. according to the method described in claim 1, it is characterized in that, data in the serialization unit to gathered data into
Row compression processing specifically includes following steps：
301) binary system based on continuously record data is byte-by-byte asks differential pressure to contract, for continuous type gathered data；
302) binary system based on continuously record data integrally asks differential pressure to contract, for continuous type gathered data；
303) numerical value based on continuously record data asks differential pressure to contract, for continuous type gathered data；
304) it is compressed based on the merger of same type gathered data, for discrete type gathered data；
305) it is encoded and compressed based on repeated data section, for continuous type and discrete type gathered data.
5. according to the method described in claim 4, it is characterized in that, it is encoded to class Huffman volumes in the step 305)
Code.
6. according to the method described in claim 1, it is characterized in that, the data to after serializing unzip it processing
Specifically include following steps：
401) the byte-by-byte decompression for seeking difference of the binary system based on continuously record data；
402) binary system based on continuously record data integrally seeks the decompression of difference；
403) numerical value based on continuously record data seeks the decompression of difference；
404) decompression based on same type gathered data merger；
405) decompression based on repeated data section coding.
7. according to the method described in claim 6, it is characterized in that, it is encoded to class Huffman volumes in the step 405)
Code.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711422134.2A CN108234464B (en) | 2017-12-25 | 2017-12-25 | High-efficiency compression method for collected data of signal centralized monitoring system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711422134.2A CN108234464B (en) | 2017-12-25 | 2017-12-25 | High-efficiency compression method for collected data of signal centralized monitoring system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108234464A true CN108234464A (en) | 2018-06-29 |
| CN108234464B CN108234464B (en) | 2021-02-09 |
Family
ID=62648817
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711422134.2A Active CN108234464B (en) | 2017-12-25 | 2017-12-25 | High-efficiency compression method for collected data of signal centralized monitoring system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108234464B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109802683A (en) * | 2018-12-20 | 2019-05-24 | 深圳科安达电子科技股份有限公司 | A kind of track circuit monitoring data compression method and processing system based on parameter identification |
| CN110046159A (en) * | 2019-03-13 | 2019-07-23 | 平安科技（深圳）有限公司 | Bank Account Number storage method, device, computer equipment and storage medium |
| CN111551244A (en) * | 2019-02-12 | 2020-08-18 | 计算系统有限公司 | Compressing scalar vibration data |
| CN114401322A (en) * | 2022-01-14 | 2022-04-26 | 上海金融期货信息技术有限公司 | Financial data bandwidth compression method and device |
| CN117082153A (en) * | 2023-10-13 | 2023-11-17 | 深圳龙电华鑫控股集团股份有限公司 | Carrier communication data acquisition and data optimization method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101825890A (en) * | 2010-03-25 | 2010-09-08 | 刘文祥 | Object network |
| CN102006081A (en) * | 2010-12-01 | 2011-04-06 | 南京工业大学 | Compression method for mass energy consumption information in intelligent building |
| CN102497377A (en) * | 2011-12-15 | 2012-06-13 | 武汉理工大学 | Method for remote-monitoring data safety transmission of fuel cell standby power supply of communication base station |
| CN103049561A (en) * | 2012-12-31 | 2013-04-17 | 华为技术有限公司 | Data compressing method, storage engine and storage system |
-
2017
- 2017-12-25 CN CN201711422134.2A patent/CN108234464B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101825890A (en) * | 2010-03-25 | 2010-09-08 | 刘文祥 | Object network |
| CN102006081A (en) * | 2010-12-01 | 2011-04-06 | 南京工业大学 | Compression method for mass energy consumption information in intelligent building |
| CN102497377A (en) * | 2011-12-15 | 2012-06-13 | 武汉理工大学 | Method for remote-monitoring data safety transmission of fuel cell standby power supply of communication base station |
| CN103049561A (en) * | 2012-12-31 | 2013-04-17 | 华为技术有限公司 | Data compressing method, storage engine and storage system |
Non-Patent Citations (1)
| Title |
|---|
| 林瑜筠: "《高速铁路信号技术》", 31 December 2012 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109802683A (en) * | 2018-12-20 | 2019-05-24 | 深圳科安达电子科技股份有限公司 | A kind of track circuit monitoring data compression method and processing system based on parameter identification |
| CN111551244A (en) * | 2019-02-12 | 2020-08-18 | 计算系统有限公司 | Compressing scalar vibration data |
| CN111551244B (en) * | 2019-02-12 | 2022-07-05 | 计算系统有限公司 | Compressing scalar vibration data |
| CN110046159A (en) * | 2019-03-13 | 2019-07-23 | 平安科技（深圳）有限公司 | Bank Account Number storage method, device, computer equipment and storage medium |
| CN110046159B (en) * | 2019-03-13 | 2023-04-18 | 平安科技（深圳）有限公司 | Bank account storage method and device, computer equipment and storage medium |
| CN114401322A (en) * | 2022-01-14 | 2022-04-26 | 上海金融期货信息技术有限公司 | Financial data bandwidth compression method and device |
| CN117082153A (en) * | 2023-10-13 | 2023-11-17 | 深圳龙电华鑫控股集团股份有限公司 | Carrier communication data acquisition and data optimization method |
| CN117082153B (en) * | 2023-10-13 | 2024-02-13 | 深圳龙电华鑫控股集团股份有限公司 | Carrier communication data acquisition and data optimization method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108234464B (en) | 2021-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108234464A (en) | For the gathered data high-efficiency compression method of centralized signal supervision system | |
| CN100392981C (en) | Efficient coding of side information in lossless encoder | |
| CN116192971A (en) | Intelligent cloud energy operation and maintenance service platform data management method | |
| CN109255429A (en) | Parameter decompression method for sparse neural network model | |
| CN102422540B (en) | Coding method, code device, coding/decoding method, decoding device, program and recording medium | |
| CN116016606B (en) | Sewage treatment operation and maintenance data efficient management system based on intelligent cloud | |
| CN103886732B (en) | High-reliability ecological environmental parameter wireless-sensing system | |
| JPH08242177A (en) | Binary arithmetic coding of input symbol and its device | |
| CN105120276B (en) | Adaptive M otion JPEG coding methods and system | |
| CN107565973A (en) | The implementation method and circuit structure of a kind of expansible Huffman encoding of node | |
| CN115658628B (en) | Production data intelligent management method for MES system | |
| CN115529357B (en) | Updating abnormity matching method based on MES intercommunication interconnection production data | |
| CN107590157A (en) | A kind of date storage method, data query method and relevant device | |
| CN115882866A (en) | Data compression method based on data difference characteristic | |
| CN118075472A (en) | Spectrum compression method based on LOCO-I algorithm and Huffman coding | |
| CN117040542B (en) | Intelligent comprehensive distribution box energy consumption data processing method | |
| CN106790550A (en) | A kind of system suitable for the compression of power distribution network Monitoring Data | |
| CN107064621B (en) | Intelligent electric meter data compression method and system | |
| CN115269940A (en) | Data compression method of ERP management system | |
| CN101046961B (en) | Speech compression, decompression method and circuit based on wave shape | |
| CN108347474A (en) | State data compression method and device for mechanical equipment and mechanical equipment | |
| US6681049B1 (en) | Image encoding method and apparatus | |
| CN118523779B (en) | Intelligent monitoring method for running state of equipment based on sensing technology | |
| CN117155408B (en) | Efficient transmission method for production data | |
| CN114314709B (en) | Intelligent monitoring method for rural domestic sewage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1254102 Country of ref document: HK |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |