CN107463335A - A kind of location track big data high-efficiency storage method - Google Patents
A kind of location track big data high-efficiency storage method Download PDFInfo
- Publication number
- CN107463335A CN107463335A CN201710651614.XA CN201710651614A CN107463335A CN 107463335 A CN107463335 A CN 107463335A CN 201710651614 A CN201710651614 A CN 201710651614A CN 107463335 A CN107463335 A CN 107463335A
- Authority
- CN
- China
- Prior art keywords
- time
- location
- big data
- storage method
- location track
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
- G06F11/3082—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting the data filtering being achieved by aggregating or compressing the monitored data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3438—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment monitoring of user actions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Human Computer Interaction (AREA)
- Computer Hardware Design (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention discloses a kind of location track big data high-efficiency storage method, including position Stored Procedure and position recovering process, comprise the concrete steps that：Position sequence encodes：" the position sequence label " of station acquisition point is represented into traditional location data；Time series encodes：Sequential labeling by timestamp value " serializing " into an expression period；Algorithm binding site and time series label are pieced together by bitmap, two-dimensional signal are encoded to one-dimensional；By inverse algorithm, one-dimension information is decoded into two-dimensional signal；When location track is provided by limit fixed position, memory space can be greatly saved；Numerical value is pieced together using bitmap, positive and negative calculating process is all relatively more succinct；Regulation at any time can be carried out according to nodes of locations and time granularity；Based on the numerical value that bitmap is pieced together carry out continuous time position identical calculate when (when namely data are further compressed), very simply；Only need to look for continuously incremental numerical value can, meet the use habit in the case of non-coding.
Description
Technical field
The present invention relates to a kind of location track big data high-efficiency storage method, is particularly the object based on specified label
The record storage of movement position track, it can at least save more than 1/3 memory space.
Background technology
Instantly WiFi has become the necessity in life, and Beacon equipment is also more and more in commercial location.It is similar
Market and campus etc. are deployed with the place of Wi-Fi hotspot or Beacon equipment, are obtained in real-time stream of people's heating power situation, with
And popular track circuit such as seeks at the acquisition needs of business data, it typically can all open and carry out WiFi terminal (or Beacon is whole
End) position positioning, track record and back track function now just need in real time to record the position data of terminal.
In a large-scale network, the species data will be very huge, particularly (i.e. need not in unlatching WiFi sniffs
Access WiFi can also find out user MAC function) when, the basic exponentially type of position data increases.Than wirelessly being connect if any 10,000
The campus of access point (AP), average each AP access 5 terminals, and every five seconds for example carries out a position data record, and (timestamp is general
For 4 byte Byte), it is conventional using gps data (taking 8 byte Byte), 50000*12Byte=600Kbyte is had every time.
Daily data have：600K*24*60*12=10368000K ≈ 10GByte, if opening sniff, this data will amplify 10
More than times, reach 100GByte quantity daily.
The content of the invention
For problem present in background technology, the invention provides a kind of location track big data high-efficiency storage method.
To achieve the above object, the present invention provides following technical scheme：A kind of location track big data high-efficiency storage method,
Including position Stored Procedure and position recovering process, comprise the concrete steps that：
Position sequence encodes：" the position sequence label " of station acquisition point is represented into traditional location data；
Time series encodes：Sequential labeling by timestamp value " serializing " into an expression period；
Algorithm binding site and time series label are pieced together by bitmap, two-dimensional signal are encoded to one-dimensional；
By inverse algorithm, one-dimension information is decoded into two-dimensional signal.
Wherein：Position sequence encodes and time series encodes no sequencing.
The position Stored Procedure includes：
S10：According to scene location and time encoding digit；
S20：According to storage location and time calculation code；
S30：Position stores.
The position recovering process includes：
S40：Extracting position encodes；
S50：According to coding inverse position and time；
S60：Specific service computation is carried out according to position and time.
The wherein maximum of position sequence coding and time series coding is determined according to byte number.
Compared with prior art, the beneficial effects of the invention are as follows：When location track is provided by limit fixed position, Ke Yiji
Big saving memory space；Numerical value is pieced together using bitmap, positive and negative calculating process is all relatively more succinct；Can according to nodes of locations and when
Between granularity carry out regulation at any time；Based on the numerical value that bitmap is pieced together carry out continuous time position identical calculate when (namely
When data are further compressed), very simply；Only need to look for continuously incremental numerical value can, meeting makes in the case of non-coding
With custom.
Brief description of the drawings
Fig. 1 is the position storing process schematic diagram of this method；
Fig. 2 is the position data reduction process schematic diagram of this method；
Fig. 3 is the example schematic diagram of position and time encoding；
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, rather than whole embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other under the premise of creative work is not made
Embodiment, belong to the scope of protection of the invention.
Embodiment：
Fig. 1 and Fig. 2 are referred to, the present invention provides a kind of technical scheme：A kind of location track big data high-efficiency storage method,
Including position Stored Procedure and position recovering process, comprise the concrete steps that：
Position sequence encodes：" the position sequence label " of station acquisition point is represented into traditional location data；
Time series encodes：Sequential labeling by timestamp value " serializing " into an expression period；
Algorithm binding site and time series label are pieced together by bitmap, two-dimensional signal are encoded to one-dimensional；
By inverse algorithm, one-dimension information is decoded into two-dimensional signal.
Precondition：(1) required to determine the byte number (Byte) of storage according to memory space；(2) further according to byte number, really
Determine the maximum of position sequence and time series；
For example describe to be used as storage byte using 4 bytes (Byte)：
It is encoded to a 4 byte signed integer types：4294967296(2^32).Totally 10.
It is assumed that first 5 represent position sequence (00001-42948), latter 5 represent time series (00001-99999)；With
Exemplified by upper number, if using day as unit data storage, minimum can be sequence of points (because sharing 86400 daily with 1 second
Second).
It is assumed that first 4 represent position sequence (0001-4293)；5 represent time series (000001-999999) afterwards；With
Exemplified by upper number, if using year as unit data storage, minimum can be that a sequence of points (shares because of annual with 1 minute
512640 seconds).
Embodiment 1
Location point is serialized：
(a) such as each WiFi access point (or beacon equipment point) is ranked up (in general device management software
Access point serializing will be carried), sequence words meet that precondition is set.
(b) sequencing method is not limited to described in (a), as long as can meet " position uniqueness ".
(c) such as No. 1 position is that 00001, No. 2 positions are 00002.
It will be serialized at time point：
(a) constrained according in precondition, and actual conditions requirement (such as time backtracking precision of minimum requirements 5 seconds)
Serialized.
(b) such as 5 seconds points, in accompanying drawing 3, T1 (00001)=0:00:01-0:00:05；T2 (00002)=0:
00:06-0:00:10；T3 (00003)=0:00:11-0:00:15 by that analogy.
Calculated according to scene size coding：Five time encoding numbers after preceding 5 position encoded several * 100000+.
Decoded according to cryptoprinciple：It is assumed that encoding value is X；It is position encoded：L-ID=X；Time encoding：T-ID=
X- (position L-ID) * 100000；
Embodiment 2
Remaining is same as Example 1, and difference is：
Calculated according to scene size coding：1-18bit is time encoding, and 19-32bit is position encoded.
Decoded according to cryptoprinciple：It is assumed that encoding value is X, position L-ID=X＆0xFFFC0000>>18 (＆ expressions
" step-by-step with ",>>Represent " moving to right " bit arithmetic)；Time encoding T-ID=X＆0x3FFFF (＆ represents " step-by-step with ").
Embodiment 3
Remaining is same as Example 1, and difference is：
Calculated according to scene size coding：29-32bit is that coding method (represents time encoding size, bit0:18,
bit1:19,bit2:20,bit3:21)；According to coding method self-adapting adjusted positions and the coding range of time.
Decoded according to cryptoprinciple：It is assumed that encoding value is X；Calculate time encoding bit numbers N==X＆
0xC0000000>>30 (＆ represents " step-by-step with ",>>Represent " moving to right " bit arithmetic)；Calculation position coding bit numbers M=32-N；When
Between encode T-ID=X＆2^N (＆ represents " step-by-step with ", and ^ represents to ask power to operate, non-bit arithmetic)；Position L-ID=X＆ ((2^32)
|2^N)0xFFFC0000>>M (＆ and | represent " step-by-step with " and " step-by-step or ",>>Represent " moving to right " bit arithmetic).
Based on above-mentioned, present invention has the advantage that：, can be great when location track is provided by limit fixed position
Save memory space；Numerical value is pieced together using bitmap, positive and negative calculating process is all relatively more succinct；Can be according to nodes of locations and time grain
Degree carries out regulation at any time；Based on the numerical value that bitmap is pieced together carry out continuous time position identical calculate when (namely data
When further compressing), very simply；Only need to look for continuously incremental numerical value can, meet the use habit in the case of non-coding
It is used.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all essences in the present invention
All any modification, equivalent and improvement made within refreshing and principle etc., should be included in the scope of the protection.
Claims (5)
- A kind of 1. location track big data high-efficiency storage method, it is characterised in that：Including position Stored Procedure and position recovering mistake Journey, comprise the concrete steps that：Position sequence encodes：" the position sequence label " of station acquisition point is represented into traditional location data；Time series encodes：Sequential labeling by timestamp value " serializing " into an expression period；Algorithm binding site and time series label are pieced together by bitmap, two-dimensional signal are encoded to one-dimensional；By inverse algorithm, one-dimension information is decoded into two-dimensional signal.
- A kind of 2. location track big data high-efficiency storage method according to claim 1, it is characterised in that：Position sequence is compiled Code and time series encode no sequencing.
- A kind of 3. location track big data high-efficiency storage method according to claim 1, it is characterised in that：Position storage stream Journey includes：S10：According to scene location and time encoding digit；S20：According to storage location and time calculation code；S30：Position stores.
- A kind of 4. location track big data high-efficiency storage method according to claim 1, it is characterised in that：Position recovering mistake Journey includes：S40：Extracting position encodes；S50：According to coding inverse position and time；S60：Specific service computation is carried out according to position and time.
- A kind of 5. location track big data high-efficiency storage method according to claim 1, it is characterised in that：Position sequence is compiled The maximum of code and time series coding is determined according to byte number.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710651614.XA CN107463335A (en) | 2017-08-02 | 2017-08-02 | A kind of location track big data high-efficiency storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710651614.XA CN107463335A (en) | 2017-08-02 | 2017-08-02 | A kind of location track big data high-efficiency storage method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107463335A true CN107463335A (en) | 2017-12-12 |
Family
ID=60548172
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710651614.XA Pending CN107463335A (en) | 2017-08-02 | 2017-08-02 | A kind of location track big data high-efficiency storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107463335A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110598156A (en) * | 2019-09-19 | 2019-12-20 | 腾讯科技（深圳）有限公司 | Drawing data processing method, drawing data processing device, terminal, server and storage medium |
| CN110888885A (en) * | 2019-11-25 | 2020-03-17 | 深圳广联赛讯有限公司 | Track data processing method and device, server and readable storage medium |
| CN111314392A (en) * | 2020-05-15 | 2020-06-19 | 诺领科技(南京)有限公司 | Satellite navigation positioning auxiliary ephemeris data compression and transmission method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080174485A1 (en) * | 2007-01-24 | 2008-07-24 | Carani Sherry L | Tracking System and Method with Asset Tool Bar for Polling, Message, Historic Report, Location, Map and Geo Fence Features |
| CN103744861A (en) * | 2013-12-12 | 2014-04-23 | 深圳先进技术研究院 | Lookup method and device for frequency sub-trajectories in trajectory data |
| CN104820905A (en) * | 2015-05-19 | 2015-08-05 | 威海北洋电气集团股份有限公司 | Space trajectory big data analysis-based person management and control method and system |
| CN104834990A (en) * | 2015-03-31 | 2015-08-12 | 北京首都国际机场股份有限公司 | Passenger informatization coding method and device |
| CN104951464A (en) * | 2014-03-27 | 2015-09-30 | 华为技术有限公司 | Data storage method and system |
| CN106407378A (en) * | 2016-09-11 | 2017-02-15 | 复旦大学 | Method for expressing road network trajectory data again |
-
2017
- 2017-08-02 CN CN201710651614.XA patent/CN107463335A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080174485A1 (en) * | 2007-01-24 | 2008-07-24 | Carani Sherry L | Tracking System and Method with Asset Tool Bar for Polling, Message, Historic Report, Location, Map and Geo Fence Features |
| CN103744861A (en) * | 2013-12-12 | 2014-04-23 | 深圳先进技术研究院 | Lookup method and device for frequency sub-trajectories in trajectory data |
| CN104951464A (en) * | 2014-03-27 | 2015-09-30 | 华为技术有限公司 | Data storage method and system |
| CN104834990A (en) * | 2015-03-31 | 2015-08-12 | 北京首都国际机场股份有限公司 | Passenger informatization coding method and device |
| CN104820905A (en) * | 2015-05-19 | 2015-08-05 | 威海北洋电气集团股份有限公司 | Space trajectory big data analysis-based person management and control method and system |
| CN106407378A (en) * | 2016-09-11 | 2017-02-15 | 复旦大学 | Method for expressing road network trajectory data again |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110598156A (en) * | 2019-09-19 | 2019-12-20 | 腾讯科技（深圳）有限公司 | Drawing data processing method, drawing data processing device, terminal, server and storage medium |
| CN110598156B (en) * | 2019-09-19 | 2022-03-15 | 腾讯科技（深圳）有限公司 | Drawing data processing method, drawing data processing device, terminal, server and storage medium |
| CN110888885A (en) * | 2019-11-25 | 2020-03-17 | 深圳广联赛讯有限公司 | Track data processing method and device, server and readable storage medium |
| CN111314392A (en) * | 2020-05-15 | 2020-06-19 | 诺领科技(南京)有限公司 | Satellite navigation positioning auxiliary ephemeris data compression and transmission method |
| CN111314392B (en) * | 2020-05-15 | 2020-09-15 | 诺领科技(南京)有限公司 | Satellite navigation positioning auxiliary ephemeris data compression and transmission method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107547633B (en) | User constant standing point processing method and device and storage medium | |
| CN110082699B (en) | Low-voltage transformer area intelligent electric energy meter operation error calculation method and system | |
| CN107463335A (en) | A kind of location track big data high-efficiency storage method | |
| CN104931041B (en) | A kind of location sequence Forecasting Methodology based on user trajectory data | |
| Stojkoska et al. | Data compression for energy efficient IoT solutions | |
| CN103379136B (en) | Compression method and decompression method of log acquisition data, compression apparatus and decompression apparatus of log acquisition data | |
| Zhao et al. | Spatial-temporal aggregation graph convolution network for efficient mobile cellular traffic prediction | |
| CN104380832B (en) | Compression device, decompression apparatus, compression method and decompression method | |
| CN104102790B (en) | Power supply figure automatic mapping system and method based on GIS | |
| CN103338461B (en) | Based on network plan method and the device of Traffic prediction | |
| CN114626643A (en) | Smart city government power supply regulation and control method, Internet of things system, device and medium | |
| CN108449393A (en) | Combustion gas data transmission method and Internet of things system | |
| CN116828412B (en) | New energy automobile fills and trades electric box and becomes rack wireless communication system | |
| CN104881739B (en) | Data consistency verification method is matched somebody with somebody by a kind of battalion based on IEC61970/61968 CIM standards | |
| Abdulzahra MSc et al. | Energy conservation approach of wireless sensor networks for IoT applications | |
| Prieto et al. | Balancing power consumption in IoT devices by using variable packet size | |
| CN105451173A (en) | Track-data-analysis-technology-based intelligent cluster communication resource configuration method and system | |
| CN107195110A (en) | A kind of portable power source lending system | |
| Gu et al. | A Spatial-Temporal Transformer Network for City-Level Cellular Traffic Analysis and Prediction | |
| CN104123822A (en) | Water-meter reading system employing photographing direct reading based on wireless ad hoc network | |
| CN117040542A (en) | Intelligent comprehensive distribution box energy consumption data processing method | |
| CN203192214U (en) | Quasi real-time ultra-low power consumption wireless ad hoc network photographing direct-reading water meter reading system | |
| CN103945488A (en) | Network community establishment method based on geographic position and network nodes | |
| CN103297182A (en) | Sending method and device of spectrum sensing measurement data | |
| Sacaleanu et al. | Compression scheme for increasing the lifetime of wireless intelligent sensor networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171212 |
|
| RJ01 | Rejection of invention patent application after publication |