CN104516955A - Massive vehicle-mounted machine track data storage method - Google Patents
Massive vehicle-mounted machine track data storage method Download PDFInfo
- Publication number
- CN104516955A CN104516955A CN201410779344.7A CN201410779344A CN104516955A CN 104516955 A CN104516955 A CN 104516955A CN 201410779344 A CN201410779344 A CN 201410779344A CN 104516955 A CN104516955 A CN 104516955A
- Authority
- CN
- China
- Prior art keywords
- track data
- car machine
- file
- storage means
- day
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a massive vehicle-mounted machine track data storage method which can meet the storage requirement of TB-grade track data generated every day under the condition of access of massive vehicle-mounted machine, and can search track data of a certain vehicle in a short time. The storage method takes a file system as a storage medium, the file system is divided into directory structures with three layers according to years, months and days, each vehicle-mounted machine generates a track file every day, the track file is stored in corresponding directories, and the track files in a previous day are combined and compressed at 0 o'clock every day.
Description
Technical field
The present invention relates to the technical field of vehicle data process, relate to a kind of storage means of magnanimity car machine track data particularly.
Background technology
The storage means of the car machine track data of current main flow, major part take database as medium, and the track of multiple car machine is stored in a database table.But, this storage means efficiency is extremely low, mutual lock can be produced wait for when inquiring about and update is concurrent, especially when extracting the track of single car machine of a few days, need a large amount of database block file of scanning just can extract corresponding information, therefore, this storage means can only tackle the scene of below 10,000 car machines, for the car machine access increased requirement of advancing by leaps and bounds at present, will feel simply helpless.
Summary of the invention
Technology of the present invention is dealt with problems and is: overcome the deficiencies in the prior art, a kind of storage means of magnanimity car machine track data is provided, it can access the storage demand that situation produces TB level track data lower every day by satisfying magnanimity car machine, and can retrieve the track data of a certain car at short notice.
Technical solution of the present invention is: the storage means of this magnanimity car machine track data, this storage means take file system as storage medium, it file system is the bibliographic structure of three levels by date three partition of the levels, each car machine produces a trail file, under being stored in corresponding catalogue every day; Every day 0, these trail files of the previous day are carried out merging compression process.
The present invention, by being storage medium with file system, file system is the bibliographic structure of three levels by date three partition of the levels, just can access the storage demand that situation produces TB level track data lower every day by satisfying magnanimity car machine; When retrieve certain car machine a time period in track data time, the position at trail file place can be navigated to fast according to the unique identification of time period and car machine, decrease the I/O operation of a lot of redundant information relative to the storage mode of database, improve recall precision significantly.
Accompanying drawing explanation
Fig. 1 is the schematic diagram of a preferred embodiment of storage means according to magnanimity car machine track data of the present invention.
Fig. 2 is the process flow diagram of the present invention by place, vehicle unique identification location physical machine.
Fig. 3 is the schematic diagram that main frame one standby host of the present invention stores.
Embodiment
As shown in Figure 1, the storage means of this magnanimity car machine track data, this storage means is storage medium with file system, file system is the bibliographic structure of three levels by date three partition of the levels, each car machine produces a trail file, under being stored in corresponding catalogue every day; Along with the growth of time, storage mode above can produce a large amount of trail files, take the Inodes (index node of file system, file data is all stored in " block ", so obviously, also must find the metamessage of a local store files, size of the founder of such as file, the date created of file, file etc.The region of this store files metamessage is just called inode) quantity, Inodes quantity constantly increases thereupon, can reduce IO storage efficiency, in order to address this problem, every day 0, these trail files of the previous day is carried out merging compression process.
The present invention, by being storage medium with file system, file system is the bibliographic structure of three levels by date three partition of the levels, just can access the storage demand that situation produces TB level track data lower every day by satisfying magnanimity car machine; When retrieve certain car machine a time period in track data time, the position at trail file place can be navigated to fast according to the unique identification of time period and car machine, decrease the I/O operation of a lot of redundant information relative to the storage mode of database, improve recall precision significantly.
In addition, the method is these trail file boil down to META file and DATA file.
In addition, comprise in described META file each car machine DATA file reference position, take byte quantity, the car machine on same day sum, the tracking quantity of each car machine, the summation of report to the police sum and whole tracking quantity.
In addition, create index according to business demand, index comprise report in tracking quantity, warning quantity, distance travelled, tire pressure one or more.
In addition, the track data after described DATA file store compressed, before compression, track data is character string mode, is compressed into corresponding form, the decollator of ignore character string storage mode by compression algorithm according to the type of track data before compression.
In addition, before described compression, track data is latitude and longitude information, boil down to digital form.Such as latitude and longitude information 65348879, if take 8 bytes (GBK mode is 16 bytes) according to character string storage, be revised as digital form storage and only take 4 bytes, save 50% space, the decollator of this storage mode also abridged character storage mode, therefore overall overall space 60% of saving through calculating is in left and right.
In addition, the method expands at least two physical machine hard-disc storage separate unit physical machine hard-disc storage, one main frame one standby host, as shown in Figure 3, under default situations, track read-write service is carried out on main frame, by drbd, (drbd is a kind of block device, can be used among High Availabitity (HA). it is similar to network DNC-1 function. when data are write local file system, data also will to be sent in network on another main frame. be recorded in the same manner in a file system) service data syn-chronization to standby host, when main frame breaks down, amendment LAN (Local Area Network) dns service makes it point to standby host, continued to provide track to read and write service by standby host, after fault repaired by main frame, it is made to become standby host again.
Certainly, also need before this unique identification part of vehicle to be added its track data place physical machine mark of this car, locate the flow process of place physical machine as shown in Figure 2 by vehicle unique identification, it comprises the following steps:
(1) vehicle unique identification is obtained;
(2) front 10 character strings of this mark are intercepted;
(3) according to the character string inquiry LAN (Local Area Network) dns of step (2);
(4) this wheel paths physical memory cell of storage is routed to.
The above; it is only preferred embodiment of the present invention; not any pro forma restriction is done to the present invention, every above embodiment is done according to technical spirit of the present invention any simple modification, equivalent variations and modification, all still belong to the protection domain of technical solution of the present invention.
Claims (7)
1. the storage means of a magnanimity car machine track data, it is characterized in that: this storage means take file system as storage medium, file system is the bibliographic structure of three levels by date three partition of the levels, each car machine produces a trail file, under being stored in corresponding catalogue every day; Every day 0, these trail files of the previous day are carried out merging compression process.
2. the storage means of magnanimity car machine track data according to claim 1, is characterized in that: the method is these trail file boil down to META file and DATA file.
3. the storage means of magnanimity car machine track data according to claim 2, is characterized in that: comprise in described META file each car machine DATA file reference position, take byte quantity, the car machine on same day sum, the tracking quantity of each car machine, the summation of report to the police sum and whole tracking quantity.
4. the storage means of magnanimity car machine track data according to claim 3, is characterized in that: create index according to business demand, index comprise report in tracking quantity, warning quantity, distance travelled, tire pressure one or more.
5. the storage means of magnanimity car machine track data according to claim 4, it is characterized in that: the track data after described DATA file store compressed, before compression, track data is character string mode, corresponding form is compressed into, the decollator of ignore character string storage mode according to the type of track data before compression by compression algorithm.
6. the storage means of magnanimity car machine track data according to claim 5, is characterized in that: before described compression, track data is latitude and longitude information, boil down to digital form.
7. the storage means of magnanimity car machine track data according to claim 1, it is characterized in that: the method expands at least two physical machine hard-disc storage separate unit physical machine hard-disc storage, one main frame one standby host, under default situations, track read-write service is carried out on main frame, served data syn-chronization to standby host by drbd, when main frame breaks down, amendment LAN (Local Area Network) dns service makes it point to standby host, continued to provide track to read and write service by standby host, after fault repaired by main frame, then it is made to become standby host.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410779344.7A CN104516955A (en) | 2014-12-16 | 2014-12-16 | Massive vehicle-mounted machine track data storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410779344.7A CN104516955A (en) | 2014-12-16 | 2014-12-16 | Massive vehicle-mounted machine track data storage method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104516955A true CN104516955A (en) | 2015-04-15 |
Family
ID=52792254
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410779344.7A Pending CN104516955A (en) | 2014-12-16 | 2014-12-16 | Massive vehicle-mounted machine track data storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104516955A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107451248A (en) * | 2017-07-28 | 2017-12-08 | 福建中金在线信息科技有限公司 | A kind of date storage method, device and electronic equipment |
| CN110083599A (en) * | 2019-03-28 | 2019-08-02 | 华东师范大学 | A kind of track of vehicle data index method based on temporal-spatial interpolating |
| CN112507024A (en) * | 2020-12-01 | 2021-03-16 | 浙商银行股份有限公司 | Block chain data uplink checking method, device, equipment and storage medium |
| CN113343134A (en) * | 2021-06-22 | 2021-09-03 | 中国标准化研究院 | Dynamic maintenance management method and system for administrative division codes |
| CN115617770A (en) * | 2022-11-17 | 2023-01-17 | 达芬骑动力科技(北京)有限公司 | Data disk storage management method for vehicle state signal data storage |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101459557A (en) * | 2008-11-29 | 2009-06-17 | 成都市华为赛门铁克科技有限公司 | Secure logging centralized storage method and device |
| CN202373020U (en) * | 2011-03-11 | 2012-08-08 | 杭州鸿泉数字设备有限公司 | Intelligent coach operation system realizing device based on cloud computing platform |
| CN102662610A (en) * | 2012-04-20 | 2012-09-12 | 中国科学院遥感应用研究所 | Remote-sensing data management system and method |
| CN103049556A (en) * | 2012-12-28 | 2013-04-17 | 中国科学院深圳先进技术研究院 | Fast statistical query method for mass medical data |
| CN103338261A (en) * | 2013-07-04 | 2013-10-02 | 北京泰乐德信息技术有限公司 | Storage and processing method and system of rail transit monitoring data |
-
2014
- 2014-12-16 CN CN201410779344.7A patent/CN104516955A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101459557A (en) * | 2008-11-29 | 2009-06-17 | 成都市华为赛门铁克科技有限公司 | Secure logging centralized storage method and device |
| CN202373020U (en) * | 2011-03-11 | 2012-08-08 | 杭州鸿泉数字设备有限公司 | Intelligent coach operation system realizing device based on cloud computing platform |
| CN102662610A (en) * | 2012-04-20 | 2012-09-12 | 中国科学院遥感应用研究所 | Remote-sensing data management system and method |
| CN103049556A (en) * | 2012-12-28 | 2013-04-17 | 中国科学院深圳先进技术研究院 | Fast statistical query method for mass medical data |
| CN103338261A (en) * | 2013-07-04 | 2013-10-02 | 北京泰乐德信息技术有限公司 | Storage and processing method and system of rail transit monitoring data |
Non-Patent Citations (1)
| Title |
|---|
| 钟运琴等: "大规模时空数据分布式存储方法研究", 《高技术通讯》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107451248A (en) * | 2017-07-28 | 2017-12-08 | 福建中金在线信息科技有限公司 | A kind of date storage method, device and electronic equipment |
| CN110083599A (en) * | 2019-03-28 | 2019-08-02 | 华东师范大学 | A kind of track of vehicle data index method based on temporal-spatial interpolating |
| CN110083599B (en) * | 2019-03-28 | 2023-06-23 | 华东师范大学 | Vehicle track data indexing method based on space-time interpolation |
| CN112507024A (en) * | 2020-12-01 | 2021-03-16 | 浙商银行股份有限公司 | Block chain data uplink checking method, device, equipment and storage medium |
| CN113343134A (en) * | 2021-06-22 | 2021-09-03 | 中国标准化研究院 | Dynamic maintenance management method and system for administrative division codes |
| CN115617770A (en) * | 2022-11-17 | 2023-01-17 | 达芬骑动力科技(北京)有限公司 | Data disk storage management method for vehicle state signal data storage |
| CN115617770B (en) * | 2022-11-17 | 2023-03-28 | 达芬骑动力科技(北京)有限公司 | Data disk storage management method for vehicle state signal data storage |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210152638A1 (en) | Data processing method, apparatus, and system | |
| CN104516955A (en) | Massive vehicle-mounted machine track data storage method | |
| CN104346357B (en) | The file access method and system of a kind of built-in terminal | |
| CN100399327C (en) | Managing file system versions | |
| CN107423422B (en) | Spatial data distributed storage and search method and system based on grid | |
| CN102521269B (en) | Index-based computer continuous data protection method | |
| CN102843403A (en) | File processing method based on distributed file system, system, and client | |
| CN101777017B (en) | Rapid recovery method of continuous data protection system | |
| CN105677826A (en) | Resource management method for massive unstructured data | |
| US9501506B1 (en) | Indexing system | |
| KR101496179B1 (en) | System and method for searching information based on data absence tagging | |
| CN105718455A (en) | Data query method and apparatus | |
| CN104395904A (en) | Efficient data object storage and retrieval | |
| CN104462563A (en) | File storage method and system | |
| CN103745007A (en) | File managing method and device | |
| US20150149500A1 (en) | Multi-level lookup architecture to facilitate failure recovery | |
| CN103383690A (en) | Distributed data storage method and system | |
| CN103744913A (en) | Database retrieval method based on search engine technology | |
| CN104765661A (en) | Multiple-node hot standby method of metadata service nodes in cloud storage service | |
| CN104239438A (en) | File information storage method and file information read-write method based on separate storage | |
| CN106874481A (en) | A kind of metadata of distributed type file system information-reading method and system | |
| CN107291876A (en) | A kind of DDM method | |
| CN104424219A (en) | Method and equipment of managing data documents | |
| CN104391899B (en) | A kind of data managing method and system for concentrating system for settling account | |
| CN102722450B (en) | Storage method for redundancy deletion block device based on location-sensitive hash |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150415 |