CN106330995A - Three-level data compression device used for vehicle networking and method thereof - Google Patents
Three-level data compression device used for vehicle networking and method thereof Download PDFInfo
- Publication number
- CN106330995A CN106330995A CN201510344462.XA CN201510344462A CN106330995A CN 106330995 A CN106330995 A CN 106330995A CN 201510344462 A CN201510344462 A CN 201510344462A CN 106330995 A CN106330995 A CN 106330995A
- Authority
- CN
- China
- Prior art keywords
- data
- compression
- time
- difference
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/565—Conversion or adaptation of application format or content
- H04L67/5651—Reducing the amount or size of exchanged application data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
Abstract
The present invention provides a three-level data compression device used for vehicle networking and a method thereof. According to the present invention, by the three-level compression of the data filtering, the data storage optimization and the universal lossless compression, and when a vehicle-mounted terminal sends the whole vehicle network acquired vehicle, road and environment data, the data compression can be carried out efficiently and rapidly, and after receiving a data packet, a management website can decompress rapidly to obtain the needed acquisition data. The method comprises the steps of firstly dividing the signals into the switch quantity signals/the enumeration quantity signals and the analog quantity signals according to the signal types, and carrying out the data filtering separately; secondly, carrying out the time storage optimization and the analog quantity numerical value storage optimization; then using a universal lossless data compression algorithm to compress the optimized data; caching the optimized data before compression, and when a set cycle arrives, compressing together. According to the present invention, the data transmission quantity between the vehicle-mounted terminal and the management website can be reduced, the channel occupation time and the flow cost are reduced, and the interaction efficiency of the terminal and the management website is improved.
Description
Technical field
The present invention relates to a kind of data compression device for car networking and method thereof, belong to technical field of data processing.
Background technology
Car networking is typically made up of car-mounted terminal and managing web two parts, and car-mounted terminal passes through CAN with hard
Lines etc. carry out information interconnection with electromotor, body electrical circuit etc., can be with collection vehicle, road and environmental correclation number
According to, by Radio Transmission Technology, the information collected is uploaded to managing web by car-mounted terminal.Managing web is to adopting
Collect to information be processed, calculate, share and safety is issued, according to different functional requirements, vehicle can be entered
Row is effective to be guided and supervision, and provides multimedia and the mobile Internet application service of specialty.
Owing to the networking of current car is in the application stage at initial stage, car-mounted terminal periodically reports the signal kind of managing web
Class is few, and data volume is smaller, so the less consideration of car-mounted terminal processes for the compression gathering data, and " road transport
Vehicle satellite positioning system terminal communications protocol and data form " message in (JT/T 808, hereinafter 808 agreement)
The definition of form also confirms that this point, and its message format is all to format, and each field has independent implication,
And the length of each field defines according to the greatest length that may use.Although 808 agreements are prospective fixed
Justice data compression reporting message (message id: 0x0901), i.e. uses GZIP compression algorithm to compress longer message,
But temporarily also there is no terminal support at present.
Along with going deep into of car working application, car-mounted terminal periodically reports the signal kinds of managing web will increasingly
Many, data volume also will sharply increase, and the speed of wireless network transmissions is limited, and the receipts of current operator
Expense is that the data traffic with transmission is associated, and therefore car-mounted terminal strengthens the process of data compression is then inexorable trend.
Summary of the invention
The technical problem to be solved is to provide a kind of three DBMS compressor and sides thereof for car networking
Method, periodically reports the car load network collection data of managing web, uses this three DBMSs pressure for car-mounted terminal
After compression method compression, then being uploaded to managing web, after receiving compression data, managing web uses corresponding step
Decompression recovers initial data, is then stored in data base.Use this three DBMSs compression method, car can be reduced
The data volume being wirelessly transferred between mounted terminal and managing web, reduces Channel holding time, improves terminal and management net
The interactive efficiency stood, also reduces campus network.
Three DBMS compressor and the methods thereof for car networking that the present invention provides, technical scheme is as follows:
A kind of three DBMS compressor for car networking, it is characterised in that data compression device receives from whole
The network collection data of car, are sent to managing web after treatment, and data compression device includes the data being sequentially connected with
Filtering module, data store optimization module, general lossless compression modules, wherein:
Data filtering module, is divided into the switching value/amount of enumerating class by signal, and analog quantity is another kind of, counts respectively
According to filtration；
Sorted data are carried out the storage optimization of time and analog quantity numerical value by data store optimization module respectively
Storage optimization；
General lossless compression modules, uses general lossless data compression algorithms to the data through data store optimization
It is compressed.
Further, the described three DBMS compressor for car networking are loaded in car-mounted terminal；Described
Data store optimization module includes cache module, and the data optimized first are stored by cache module, when reaching to set
During fixed press cycles, transmit together and be compressed to general lossless compression modules.
Further, managing web is provided with data decompression device, data decompression device include being sequentially connected with
Lower module: general lossless decompression module, data recovery module, wherein:
General lossless decompression module, enters message body with the general lossless compression algorithm identical with data compression device
Row decompresses；
Data recovery module, recovers the information through storage optimization.
Further, described data filtering device includes that signal type sort module, switching value and the amount of enumerating filter
Module, analog quantity filtering module:
Signal type sort module splits data into switching value and the amount of enumerating, analog quantity two kinds according to data type, and
Connect different filtering modules respectively to filter；
Switching value and the amount of enumerating filtering module only just process when changing at data value and preserve, and preserve data
Numerical value after the time of change and change, otherwise, data will be dropped；
Analog quantity filtering module, for every kind of signal, sets its cycle reported to managing web, such a
The data gathered in cycle, one data value of a Sampling hold.
Further, described data store optimization module, including time storage optimization module, switching value with enumerate
Amount storage optimization module, memorized in analogue optimize module, connect different modules according to data type difference,
Time storage optimization module is for the storage optimization of time, in each three DBMS press cycles, the most complete
Time on the basis of the initial time of record period, other time the most only records the time difference with this fiducial time, uses
One byte is deposited；
Switching value and the amount of enumerating storage optimization module, for the time of switching value He the amount of enumerating, each numerical value is corresponding
Record a time difference；
Memorized in analogue optimizes module, and the time for analog quantity only records first time difference corresponding to sampled data；
For the storage optimization of analog data value, carrying out by the way of reference value and difference, the reference standard of difference is
Minima in all numerical value, is defined as reference value minima, and other numerical value only record the difference with this reference value,
Shared by difference, storage size is determined by maximum difference.
A kind of three DBMS compression methods for car networking, it is characterised in that car-mounted terminal one three progression of definition
According to press cycles, data to car load network collection within each cycle, all use following three step that data are entered
Row three stage compression encapsulates after processing again and reports managing web:
Compression step one: data filtering, is divided into switching value, the amount of enumerating and analog quantity three according to signal type by signal
Class, carries out data filtering respectively；
Compression step two: data store optimization, including storage optimization and the storage optimization of analog quantity numerical value of time；
Compression step three: general lossless compress, uses general lossless data compression algorithms to through data store optimization
Data be compressed.
Further, data filtering method in compression step one is particularly as follows: for switching value and the amount of enumerating, data
Filter method is: data value only just processes when changing and preserves, after preserving time and the change of data variation
Numerical value, otherwise, data will be dropped；And for analog quantity, for every kind of signal, set it to managing web
The cycle reported, the data gathered within such a cycle, one data value of a Sampling hold.
Further, the data store optimization method in compression step two particularly as follows:
For the storage optimization of time, in each three DBMS press cycles, the initial time in a complete documentation cycle
On the basis of the time, other time the most only records and the time difference of this fiducial time, deposits by a byte；
For switching value and the amount of enumerating, one time difference of each numerical value corresponding record, and for analog quantity, due to
Sampling period is fixed, the time difference that only first sampled data of record is corresponding；
For the storage optimization of analog data value, carry out by the way of reference value and difference, the shared storage of difference
Size is determined by maximum difference；
Compression step one and compression step two are carried out in real time, and in described compression step three, data are accumulation
Carry out first compression to certain time, pass in compression step two or in the data of compression step two with compression step three
There is a caching defeated centre.
Further, the storage optimization of described analog quantity numerical value, is the method by saved differences, the ginseng of difference
The standard of examining is the minima in all numerical value, and this minima is defined as reference value, and other numerical value only record and are somebody's turn to do
The difference of reference value, the reference value of analog quantity uses nybble storage, and the data type of difference is then according to difference
Maximum selects length short data type of trying one's best to store；The data type of difference, limits and is stored as a word
Joint, two bytes or nybble data, i.e. difference maximum use less than or equal to a byte span, unification
One byte storage, difference maximum is more than a byte span, again less than or equal to two byte spans, system
One uses two byte storages, and difference maximum is more than two byte spans, and unification uses nybble storage.
Further, after managing web receives the data compression message of extension, take following steps that data are carried out
Decompression recovery processes:
Depressurization steps one: general lossless decompression, with the general lossless compression algorithm identical with car-mounted terminal to message
Body decompresses.
Depressurization steps two: data are recovered, and the information through storage optimization are recovered.Below in conjunction with summary of the invention
Further illustrate with principle:
Car-mounted terminal one three DBMS press cycles (such as 15 seconds) of definition, adopted car load network within each cycle
The data of collection, encapsulate after using following three step that data are carried out three stage compression process again and report managing web:
Step one: data filtering.
According to signal type, signal is divided into the switching value/amount of enumerating and analog quantity three class, carries out data filtering respectively.
For switching value and the amount of enumerating, owing to the data value of signal is saltus step, value number is limited, and
In actual moving process, the numerical value of switching value and the amount of enumerating typically will not frequently change, shape within some time period
State does not changes, and therefore the data filtering method for switching value He the amount of enumerating is: data value is only becoming
Just processing during change and preserve, the numerical value after the time of preservation data variation and change, otherwise, data will be dropped.
And for analog quantity, for every kind of signal, set its cycle reported to managing web, in such a week
The data gathered in phase, one data value of a Sampling hold.
Step 2: data store optimization
After data filtering, in three DBMS press cycles, for switching value and the amount of enumerating, preserve
Get off be change in value time time and numerical value, and for analog quantity, preserve is that N number of after filtering adopts
Sample time and time corresponding to first sampled value.
In view of the requirement of real-time of data, three DBMS press cycles of car-mounted terminal definition general the most all second level (as
15 seconds), so in three DBMS press cycles, the change of the change in each sampling time mainly number of seconds,
And the uniform format of the time of regulation is YY-MM-DD-hh-mm-ss (the GMT+8 time) in 808 agreements, essence
Really to the second, BCD [6] (8421 yards of 6 bytes) data type is used to deposit, the redundancy in each time
It is more, so the first of data store optimization optimizes point and be the storage optimization of time, for certain three DBMS
Press cycles, it is only necessary to time on the basis of the initial time in complete documentation cycle, other time the most only record with should
The time difference of fiducial time, deposits by a byte.For switching value, the period of change of numerical value is not fixed,
So each sampling is required for recording a time difference, and for analog quantity, owing to the sampling period is fixed, so only
Needing to record the time difference of first sampled data acquisition time, the acquisition time of subsequent sampling data, at pipe
Reason is recovered according to time difference and the sampling period of first sampled data acquisition time on website.
Second optimization point of data store optimization is the storage optimization of analog quantity numerical value, is also to pass through saved differences
Method, but the reference standard of difference is the minima in all numerical value, most descends value to be defined as reference value this,
Other numerical value only record the difference with this reference value, in order to reduce the complexity of algorithm, no matter the numerical value of analog quantity is long
Degree is defined as several bit (being currently 32 bits to the maximum), and the reference value of all analog quantitys all uses nybble to store,
The data type of difference then selects length short data type of trying one's best to store according to the maximum of difference.Examine
Consider to efficiency, the data type of difference, be defined to be stored as a byte, two bytes or nybble data, the poorest
Value maximum is less than or equal to a byte span, and unified use one byte storage, difference maximum is more than a word
Joint span, again less than or equal to two byte spans, unified use two bytes storage, difference maximum is big
In two byte spans, unified use nybble storage.If speed is the signal of 32 bits, each
Sampled value needs to take four bytes, if in certain three DBMS press cycles, the maximum difference of speed does not surpasses
Cross the span of a byte, then in this cycle, its difference the most only stores by a byte.
Step 3: general lossless compress
Use general lossless data compression algorithms that the data through data store optimization are compressed, due to terminal system
System resource difference is very big, does not the most limit specific compression algorithm, for the terminal system that resource is sufficient, can adopt
Bigger with compression ratio, but the compression algorithm that resource overhead is the biggest.
The data compression message of extension is transmitted directly to managing web, managing web by being wirelessly transferred by car-mounted terminal
After receiving the data compression message of extension, first resolve the data compression message of extension, re-use and car-mounted terminal phase
Message body is decompressed by same general lossless compression algorithm, then the information through storage optimization is recovered,
Finally store data into data base.
What the present invention provided is applicable to three DBMS compression methods of car networking, has the beneficial effects that:
Compared with the data transmission not being compressed with decompression, use this kind of three stage compression algorithm of the present invention,
Decrease the flow of wireless data transmission, reduce Channel holding time, improve terminal and the interactive efficiency of managing web,
Also campus network is reduced.Investigate from compression efficiency merely, the compression method that the method is the most optimal, because
Data store optimization part does not accomplish the limit (difference length is not accurate to bit), and the general nothing of step 3
Damage compression and the most do not limit specific lossless compression algorithm, and different compression algorithm compression ratios is different, but
It is to consider from system total demand, by rationally selecting the lossless compression algorithm of step 3, this three DBMSs compression side
Method can obtain a preferable compromise, therefore at car networking arenas in terms of compression efficiency and execution speed the two
Can be generally applicable.
Accompanying drawing explanation
Fig. 1 is the structure chart of an embodiment of the three DBMS compressor of the present invention；
Fig. 2 is the data filtering method figure of the switching value of the present invention；
Fig. 3 is the data filtering method figure of the amount of enumerating of the present invention；
Fig. 4 is the data filtering method figure of the analog quantity of the present invention；
Fig. 5 is the part Organization Chart of an embodiment of the three DBMS compressor of the present invention.
Detailed description of the invention
Describe the present invention below with reference to drawings and Examples.
Fig. 1 is the structure chart of an embodiment of the three DBMS compressor of the present invention.
Car-mounted terminal one three DBMS press cycles (such as 15 seconds) of definition, adopted car load network within each cycle
The data of collection, encapsulate after using following four steps that data are carried out three stage compression process again and report managing web:
Step one: data filtering.
According to signal type, signal is divided into switching value, the amount of enumerating and analog quantity three class, carries out data filtering respectively.
For switching value and the amount of enumerating, owing to the data value of signal is saltus step, value number is limited, and
In actual moving process, the numerical value of switching value and the amount of enumerating typically will not frequently change, shape within some time period
State does not changes, and therefore the data filtering method for switching value He the amount of enumerating is: data value is only becoming
Just processing during change and preserve, the numerical value after the time of preservation data variation and change, otherwise, data will be dropped.
Fig. 2 is the data filtering method of switching value, as in figure 2 it is shown, time point t0, t1, t2, t3, t4, t5,
T6 correspond to switching magnitude 1 respectively, and 0,0,1,1,0,0, according to the principle of the value only preserved when changing, extract
The critical data point in t0, t1, t3, t5 moment preserves, and t2, the data value in t4, t6 moment abandons and does not preserve.
Being similar to, Fig. 3 is the data filtering method of the amount of enumerating, the amount of the enumerating span in Fig. 3 be 0,3}, t0,
T1, t2, t3, t4, t5, t6 correspond to switching magnitude 1 respectively, and 2,2,3,0,0,0, according to only preserving when changing
The principle of value, extracts t0, and the critical data point in t1, t3, t4 moment preserves, and t2, the data value in t5, t6 moment
Abandon and do not preserve.
And for analog quantity, for every kind of signal, set its cycle reported to managing web, in such a week
The data gathered in phase, one data value of a Sampling hold.It is assumed that GES reports vehicle-mounted
The cycle of terminal is 100 milliseconds, and sets it to the cycle that managing web reports as 2 seconds, then the data to speed
Filter method is, often 20 data values In1, In2, In3...In20 of input, and only one meansigma methods Out1 of output is protected
Deposit process.In view of the extensibility of system, every kind of signal is variable to the cycle design that managing web reports,
Managing web can by management message to terminal issue instruction modification specify signal (or multiple signal) report week
Phase.
Step 2: data store optimization
After data filtering, in three DBMS press cycles, for switching value and the amount of enumerating, preserve
Get off be change in value time time and numerical value, and for analog quantity, preserve is that N number of after filtering adopts
Sample time and sampled value.
In view of the requirement of real-time of data, three DBMS press cycles of car-mounted terminal definition are general all in second level,
So in three DBMS press cycles, the change of the change in each sampling time mainly number of seconds, so number
First optimization point according to storage optimization is the storage optimization of time, for certain three DBMS press cycles, only
Needing the time on the basis of the initial time in complete documentation cycle, be accurate to the second, other time the most only records and this base
Time difference between Zhun Shi, deposits by a byte.For switching value, the period of change of numerical value is not fixed, institute
It is required for recording a time difference with each sampling, and for analog quantity, owing to the sampling period is fixed, so only needing
Recording the time difference of first sampled data acquisition time, the acquisition time of subsequent sampling data, in management
On website, time difference and sampling period according to first sampled data acquisition time recover.
Second optimization point of data store optimization is the storage optimization of analog quantity numerical value, is also to pass through saved differences
Method, but the reference standard of difference is the minima in all numerical value, and this minima is defined as reference value,
Other numerical value only record the difference with this reference value, in order to reduce the complexity of algorithm, no matter the numerical value of analog quantity is long
Degree is defined as several bit (being currently 32 bits to the maximum), and the reference value of all analog quantitys all uses nybble to store,
The data type of difference then selects length short data type of trying one's best to store according to the maximum of difference.Examine
Consider to efficiency, the data type of difference, be defined to be stored as a byte, two bytes or nybble data, the poorest
Value maximum is less than or equal to a byte span, and unified use one byte storage, difference maximum is more than a word
Joint span, again less than or equal to two byte spans, unified use two bytes storage, difference maximum is big
In two byte spans, unified use nybble storage.If speed is the signal of 32 bits, each
Sampled value needs to take four bytes, if in certain three DBMS press cycles, the maximum difference of speed does not surpasses
Cross the span of a byte, then in this cycle, its difference the most only stores by a byte.
During one three DBMS press cycles time-out, by starting the storage optimization of data, first record fiducial time,
The most various signals are carried out data store optimization process.
For various analog signalses, record as shown in table 1: signal numbering ID, number of samples n, first adopt
Sample time difference △ T1, sampling period T, difference length (byte number) △ VLen, reference value V, first adopt
Sample value difference △ V1... the n-th sampled value difference △ Vn.
Table 1: the data store optimization result of analog signals
| ID | n | △T1 | T | △VLen | V | △V1 | △V2 | ... | △Vn |
For various switching values and the amount of enumerating, tracer signal numbering ID as shown in table 2, number of samples n, first
Acquisition time difference △ T1, first data value V1... the n-th acquisition time difference △ Tn, nth data value Vn.
Table 2: switching value and the data store optimization result of the amount of enumerating signal
| ID | n | △T1 | V1 | △T2 | V2 | ... | △Tn | Vn |
Step 3: general lossless compress
Use general lossless data compression algorithms that the data through data store optimization are compressed, due to terminal system
System resource difference is very big, does not the most limit specific compression algorithm, for the terminal system that resource is sufficient, can adopt
Bigger with compression ratio, but the compression algorithm that resource overhead is the biggest.
In above step, compression step one and compression step two are carried out, in real time at described compression step
In three, data are to run up to certain time to carry out first compression, in compression step two or in compression step two and pressure
A caching is had in the middle of the data transmission of contracting step 3
Encapsulation extension compressed message, owing to the data compression reporting message of 808 protocol definition defines that compression algorithm is
GZIP compression algorithm, and the general lossless compression algorithm used in step 3 in the present invention does not limit, so
The present invention can extend new type of message for sending the data processed through three stage compression.In view of a car connection
Net managing web may dock with multiple terminal, so needing to define a compression algorithm class in extension compressed message head
Type-word section, for recording the type of the lossless compression algorithm that car-mounted terminal is used.
The data compression message format of extension is as shown in table 3, processes the compression data obtained for disappearing through three stage compression
Breath body, except message id and the length field of message body in message header, managing web decompresses for convenience, note
Record length and compression algorithm type before a message body compression, in addition to increase the safety of compression data, also
Need message body is verified, and in message header, record message body check value, the data compression message of extension
Message is as shown in table 3:
Table 3: the data compression message format of extension
| Message id | Message body length | Length before message body compression | Compression algorithm type | Message body check value | Message body |
The data compression message of extension is transmitted directly to managing web, managing web by being wirelessly transferred by car-mounted terminal
After receiving the data compression message of extension, such as Fig. 1, first according to the content of message header, message body is resolved also
Carry out legitimacy verification, re-use the general lossless compression algorithm identical with car-mounted terminal and message body is decompressed,
Then recover, by the temporal information of storage optimization, to recover by the number of storage optimization according to data difference according to time difference
According to value, finally the data of recovery are stored data base.
In a specific embodiment, the device utilizing the method to be formed is:
For three DBMS compressor of car networking, data compression device receives the network collection data from car load
Being sent to managing web after treatment, data compression device includes data filtering module, the data storage being sequentially connected with
Optimize module, general lossless compression modules, wherein:
Signal is divided into switching value and the amount of enumerating, analog quantity two class by data filtering module, carries out data filtering respectively；
Data store optimization module carries out the storage optimization of time and analog quantity numerical value respectively to sorted data
Storage optimization；
Data after optimizing are used general lossless data compression algorithms to deposit through data by general lossless compression modules
The data that storage optimizes are compressed；
In the present embodiment, three DBMS compressor are loaded on car-mounted terminal, in other examples, and can
To be loaded on other-end；
Data store optimization module includes cache module, and the data optimized first are stored by cache module, when reaching
During to the press cycles set, transmit together and be compressed to general lossless compression modules.
Data filtering device includes that signal type sort module, switching value and the amount of enumerating filtering module, analog quantity filter
Module: signal type sort module splits data into switching value and the amount of enumerating, analog quantity two kinds according to data type,
And connect different filtering modules respectively and filter；Switching value and the amount of enumerating filtering module are only being sent out at data value
During changing just process preserve, and preserve data variation time and change after numerical value, otherwise, data will be lost
Abandon；Analog quantity filtering module, for every kind of signal, sets its cycle reported to managing web, such a
The data gathered in cycle, one data value of a Sampling hold.
Data store optimization module, including time storage optimization module, switching value and the amount of enumerating storage optimization module,
Memorized in analogue optimizes module, connects different modules according to data type difference, time storage optimization module,
For the storage optimization of time, in each three DBMS press cycles, the initial time in a complete documentation cycle is base
Between Zhun Shi, other time the most only records the time difference with this fiducial time, deposits by a byte；Switching value and
The amount of enumerating storage optimization module, for the time of switching value He the amount of enumerating, one time of each numerical value corresponding record
Difference；Memorized in analogue optimizes module, and the time for analog quantity only records first time difference corresponding to sampled data；
For the storage optimization of analog data value, carrying out by the way of reference value and difference, the reference standard of difference is
Minima in all numerical value, is defined as reference value minima, and other numerical value only record the difference with this reference value,
Shared by difference, storage size is determined by maximum difference.
Car-mounted terminal one three DBMS press cycles of definition, such as 15 seconds, data were through the time of data compression device
In each three DBMS press cycles.
Data after compression can use the agreement of JT/T 808 prescribed by standard, is transmitted with growth data type.
Correspondingly, managing web is provided with data decompression device, below data decompression device includes being sequentially connected with
Module: general lossless decompression module, data recovery module, wherein:
General lossless decompression module, enters message body with the general lossless compression algorithm identical with data compression device
Row decompresses；Data recovery module, recovers the information through storage optimization.
Finally should be noted that: above example is only in order to illustrate that the technical scheme of this case is not intended to limit；To the greatest extent
This case has been described in detail by pipe with reference to preferred embodiment, those of ordinary skill in the field it is understood that
Still the detailed description of the invention of this case can be modified or portion of techniques feature is carried out equivalent；And not
Departing from the spirit of this case technical scheme, it all should be contained in the middle of the technical scheme scope that this case is claimed.
Claims (10)
1. three DBMS compressor for car networking, it is characterised in that data compression device receives from whole
The network collection data of car, are sent to managing web after treatment, and data compression device includes being sequentially connected with
Data filtering module, data store optimization module, general lossless compression modules, wherein:
Data filtering module, is divided into the switching value/amount of enumerating class by signal, and analog quantity is another kind of, enters respectively
Row data filtering；
Sorted data are carried out storage optimization and the analog quantity number of time by data store optimization module respectively
The storage optimization of value；
General lossless compression modules, uses general lossless data compression algorithms to through data store optimization
Data are compressed.
A kind of three DBMS compressor for car networking the most according to claim 1, it is characterised in that institute
The three DBMS compressor for car networking stated are loaded in car-mounted terminal；Described data store optimization
Module includes cache module, and the data optimized first are stored by cache module, when the compression reaching setting
During the cycle, transmit together and be compressed to general lossless compression modules.
A kind of three DBMS compressor for car networking the most according to claim 1 and 2, it is characterised in that
Managing web is provided with data decompression device, data decompression device include being sequentially connected with lower module: logical
By lossless decompression module, data recovery module, wherein:
General lossless decompression module, with the general lossless compression algorithm identical with data compression device to message
Body decompresses；
Data recovery module, recovers the information through storage optimization.
4. according to the arbitrary described a kind of three DBMS compressor for car networking of claims 1 to 3, its feature
Be, described data filtering device include signal type sort module, switching value and the amount of enumerating filtering module,
Analog quantity filtering module:
Signal type sort module splits data into the switching value/amount of enumerating, analog quantity two kinds according to data type,
And connect different filtering modules respectively and filter；
Switching value and the amount of enumerating filtering module only just process when changing at data value and preserve, and preserve
Numerical value after the time of data variation and change, otherwise, data will be dropped；
Analog quantity filtering module, for every kind of signal, sets its cycle reported to managing web, so
The data gathered in one cycle, one data value of a Sampling hold.
5. according to the arbitrary described a kind of three DBMS compressor for car networking of Claims 1-4, its feature
It is that described data store optimization module is deposited including time storage optimization module, switching value and the amount of enumerating
Storage optimizes module, memorized in analogue optimizes module, connects different modules according to data type difference,
Time storage optimization module, for the storage optimization of time, in each three DBMS press cycles, only
Time on the basis of the initial time in complete documentation cycle, other time the most only record with this fiducial time time
Between poor, deposit by a byte；
Switching value and the amount of enumerating storage optimization module, for the time of switching value He the amount of enumerating, each numerical value
One time difference of corresponding record；
Memorized in analogue optimizes module, for time of analog quantity only record first sampled data corresponding time
Between poor；For the storage optimization of analog data value, carry out by the way of reference value and difference, difference
Reference standard is the minima in all numerical value, and minima is defined as reference value, other numerical value only record with
The difference of this reference value, storage size shared by difference is determined by maximum difference.
6. three DBMS compression methods for car networking, it is characterised in that car-mounted terminal one three progression of definition
According to press cycles, data to car load network collection within each cycle, all use following three step logarithm
Managing web is reported according to encapsulating again after carrying out three stage compression process:
Compression step one: data filtering, is divided into switching value, the amount of enumerating and simulation according to signal type by signal
Measure three classes, carry out data filtering respectively；
Compression step two: data store optimization, excellent including the storage optimization of time and the storage of analog quantity numerical value
Change；
Compression step three: general lossless compress, uses general lossless data compression algorithms to store through data
The data optimized are compressed.
A kind of three DBMS compression methods for car networking the most according to claim 6, it is characterised in that pressure
Data filtering method in contracting step one is particularly as follows: for switching value and the amount of enumerating, data filtering method is:
Data value only just processes when changing and preserves, the numerical value after the time of preservation data variation and change,
Otherwise, data will be dropped；And for analog quantity, for every kind of signal, set it and report to managing web
Cycle, within such a cycle gather data, one data value of a Sampling hold.
8. according to a kind of three DBMS compression methods for car networking described in claim 6 or 7, it is characterised in that
Data store optimization method in compression step two particularly as follows:
For the storage optimization of time, in each three DBMS press cycles, initiateing of a complete documentation cycle
Time on the basis of time, other time the most only records the time difference with this fiducial time, deposits by a byte
Put；
For switching value and the amount of enumerating, one time difference of each numerical value corresponding record, and for analog quantity,
Owing to the sampling period is fixed, the time difference that only first sampled data of record is corresponding；
For the storage optimization of analog data value, carry out by the way of reference value and difference, shared by difference
Storage size is determined by maximum difference；
Compression step one and compression step two are carried out in real time, and in described compression step three, data are
Running up to certain time carries out first compression, in compression step two or at compression step two and compression step three
Data transmission in the middle of have a caching.
9. according to the arbitrary described a kind of three DBMS compression methods for car networking of claim 6 to 8, its feature
It is that the storage optimization of described analog quantity numerical value is the method by saved differences, the reference mark of difference
Standard is the minima in all numerical value, and this minima is defined as reference value, and other numerical value only record and are somebody's turn to do
The difference of reference value, the reference value of analog quantity uses nybble storage, and the data type of difference is then according to difference
The maximum of value selects length short data type of trying one's best to store；The data type of difference, restriction is deposited
Storage is that a byte, two bytes or nybble data, i.e. difference maximum are less than or equal to a byte span
, unified use one byte storage, difference maximum is more than a byte span, again less than or equal to two words
Joint span, unified use two bytes storage, difference maximum is more than two byte spans, system
One uses nybble storage.
10. according to the arbitrary described a kind of three DBMS compression methods for car networking of claim 6 to 9, its feature
It is, after managing web receives the data compression message of extension, takes following steps that data are decompressed
Recovery processes:
Depressurization steps one: general lossless decompression, with the general lossless compression algorithm pair identical with car-mounted terminal
Message body decompresses.
Depressurization steps two: data are recovered, and the information through storage optimization are recovered.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510344462.XA CN106330995A (en) | 2015-06-19 | 2015-06-19 | Three-level data compression device used for vehicle networking and method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510344462.XA CN106330995A (en) | 2015-06-19 | 2015-06-19 | Three-level data compression device used for vehicle networking and method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106330995A true CN106330995A (en) | 2017-01-11 |
Family
ID=57729273
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510344462.XA Pending CN106330995A (en) | 2015-06-19 | 2015-06-19 | Three-level data compression device used for vehicle networking and method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106330995A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107613527A (en) * | 2017-09-12 | 2018-01-19 | 华为技术有限公司 | Message treatment method, relevant device and system |
| CN109003352A (en) * | 2018-07-02 | 2018-12-14 | 南京兴鼎升电气设备有限公司 | A kind of storage recording method of train event recorder |
| CN110247886A (en) * | 2018-03-07 | 2019-09-17 | 安波福技术有限公司 | Effective time series data communication |
| CN111078755A (en) * | 2019-12-19 | 2020-04-28 | 远景智能国际私人投资有限公司 | Time sequence data storage query method and device, server and storage medium |
| CN111639072A (en) * | 2020-04-14 | 2020-09-08 | 北京中交兴路信息科技有限公司 | Data storage method and system in Internet of vehicles scene and readable storage medium |
| CN111741083A (en) * | 2020-06-06 | 2020-10-02 | 李彩云 | Communication data processing method based on edge computing and Internet of things and cloud server |
| CN111865328A (en) * | 2020-08-11 | 2020-10-30 | 珠海美佳音科技有限公司 | Compression method of environment data and related device |
| WO2021226922A1 (en) * | 2020-05-14 | 2021-11-18 | 深圳市欢太科技有限公司 | Data compression method, apparatus and device, and readable storage medium |
| CN113885810A (en) * | 2021-12-07 | 2022-01-04 | 北京智科车联科技有限公司 | Processing method and device of Internet of vehicles data, storage medium and electronic equipment |
| CN115221134A (en) * | 2022-07-18 | 2022-10-21 | 陕西天行健车联网信息技术有限公司 | Distributed real-time compression method and device for Internet of vehicles data and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102006081A (en) * | 2010-12-01 | 2011-04-06 | 南京工业大学 | Compression method of mass energy consumption information in intelligent building |
| CN103112476A (en) * | 2012-12-28 | 2013-05-22 | 中国神华能源股份有限公司 | Comprehensive detection device |
| CN203480562U (en) * | 2013-08-07 | 2014-03-12 | 合肥迈腾信息科技有限公司 | Taxi networking information collection and uploading device |
| CN103854483A (en) * | 2014-03-21 | 2014-06-11 | 广东新快易通智能信息发展有限公司 | Vehicle dynamic information system based on GPS (global position system) satellite positioning and ECU (electronic control unit) |
| US9026307B2 (en) * | 2007-01-18 | 2015-05-05 | Toyota Jidosha Kabushiki Kaisha | Vehicle control system |
-
2015
- 2015-06-19 CN CN201510344462.XA patent/CN106330995A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9026307B2 (en) * | 2007-01-18 | 2015-05-05 | Toyota Jidosha Kabushiki Kaisha | Vehicle control system |
| CN102006081A (en) * | 2010-12-01 | 2011-04-06 | 南京工业大学 | Compression method of mass energy consumption information in intelligent building |
| CN103112476A (en) * | 2012-12-28 | 2013-05-22 | 中国神华能源股份有限公司 | Comprehensive detection device |
| CN203480562U (en) * | 2013-08-07 | 2014-03-12 | 合肥迈腾信息科技有限公司 | Taxi networking information collection and uploading device |
| CN103854483A (en) * | 2014-03-21 | 2014-06-11 | 广东新快易通智能信息发展有限公司 | Vehicle dynamic information system based on GPS (global position system) satellite positioning and ECU (electronic control unit) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107613527B (en) * | 2017-09-12 | 2021-02-26 | 华为技术有限公司 | Message processing method, related equipment and system |
| CN107613527A (en) * | 2017-09-12 | 2018-01-19 | 华为技术有限公司 | Message treatment method, relevant device and system |
| CN110247886A (en) * | 2018-03-07 | 2019-09-17 | 安波福技术有限公司 | Effective time series data communication |
| CN110247886B (en) * | 2018-03-07 | 2021-12-03 | 安波福技术有限公司 | Efficient time series data communication |
| CN109003352A (en) * | 2018-07-02 | 2018-12-14 | 南京兴鼎升电气设备有限公司 | A kind of storage recording method of train event recorder |
| CN111078755A (en) * | 2019-12-19 | 2020-04-28 | 远景智能国际私人投资有限公司 | Time sequence data storage query method and device, server and storage medium |
| CN111078755B (en) * | 2019-12-19 | 2023-07-28 | 远景智能国际私人投资有限公司 | Time sequence data storage query method and device, server and storage medium |
| CN111639072A (en) * | 2020-04-14 | 2020-09-08 | 北京中交兴路信息科技有限公司 | Data storage method and system in Internet of vehicles scene and readable storage medium |
| WO2021226922A1 (en) * | 2020-05-14 | 2021-11-18 | 深圳市欢太科技有限公司 | Data compression method, apparatus and device, and readable storage medium |
| CN111741083A (en) * | 2020-06-06 | 2020-10-02 | 李彩云 | Communication data processing method based on edge computing and Internet of things and cloud server |
| CN111865328A (en) * | 2020-08-11 | 2020-10-30 | 珠海美佳音科技有限公司 | Compression method of environment data and related device |
| CN111865328B (en) * | 2020-08-11 | 2022-03-01 | 珠海美佳音科技有限公司 | Compression method of environment data and related device |
| CN113885810A (en) * | 2021-12-07 | 2022-01-04 | 北京智科车联科技有限公司 | Processing method and device of Internet of vehicles data, storage medium and electronic equipment |
| CN115221134A (en) * | 2022-07-18 | 2022-10-21 | 陕西天行健车联网信息技术有限公司 | Distributed real-time compression method and device for Internet of vehicles data and storage medium |
| CN115221134B (en) * | 2022-07-18 | 2023-05-05 | 陕西天行健车联网信息技术有限公司 | Distributed real-time compression method, device and storage medium for Internet of vehicles data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106330995A (en) | Three-level data compression device used for vehicle networking and method thereof | |
| CN101039212B (en) | Fast data storage method | |
| US20170142187A1 (en) | Method for Uploading All of In-Vehicle Data to the Cloud in an Efficient, Automated, Secure, and Reliable Fashion | |
| CN102355696B (en) | Large scale Internet of things gateway system and realization method thereof | |
| CN105991625B (en) | Method and equipment for data transmission | |
| CN110011962B (en) | Method for identifying service data of Internet of vehicles | |
| CN106788892B (en) | Multi-source heterogeneous test flight data processing method and multi-source heterogeneous test flight data processing system | |
| CN110061931B (en) | Industrial control protocol clustering method, device and system and computer storage medium | |
| CN102571820A (en) | Method used for transmitting data, compression server and terminal | |
| CN102497450B (en) | Two-stage-system-based distributed data compression processing method | |
| CN109451486A (en) | WiFi acquisition system and WiFi terminal detection method based on probe request | |
| CN111835836A (en) | Intelligent Internet of things terminal data processing device and method | |
| CN108173620A (en) | WSN abnormal datas based on compression network coding find network system realization | |
| CN101426216B (en) | Test method and device for wireless access system distributed base station | |
| CN102694730A (en) | Method and device for parallel processing | |
| CN208940010U (en) | A kind of intranet and extranet synchronization system | |
| CN102664948A (en) | Inter-gateway data storing system and method | |
| CN104038307A (en) | Data stream transmission system and method | |
| CN107070748A (en) | A kind of processing system and method for the big data that communicates | |
| CN107748728B (en) | Serial port communication distribution device and communication method thereof | |
| CN101188585B (en) | Conversion method of data sampling rate and its system in baseband signal transmission | |
| CN109299069A (en) | A kind of big data acquisition management platform based on internet data acquisition | |
| CN109617815A (en) | Data collection system, method, apparatus and the terminal device of LTE core network | |
| JP2008219264A (en) | Data compression and transfer device, data compression and transfer system, data compressing and transferring method, and data compression and transfer program | |
| CN107769998B (en) | Method and system for PTN network mass flow performance statistics storage query |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170111 |
|
| RJ01 | Rejection of invention patent application after publication |