CN102495848B - Method for processing massive GPS (global positioning system) data and system - Google Patents
Method for processing massive GPS (global positioning system) data and system Download PDFInfo
- Publication number
- CN102495848B CN102495848B CN 201110365001 CN201110365001A CN102495848B CN 102495848 B CN102495848 B CN 102495848B CN 201110365001 CN201110365001 CN 201110365001 CN 201110365001 A CN201110365001 A CN 201110365001A CN 102495848 B CN102495848 B CN 102495848B
- Authority
- CN
- China
- Prior art keywords
- time
- car
- mounted terminal
- file
- unique identification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title abstract description 10
- 238000003672 processing method Methods 0.000 claims description 14
- 230000003203 everyday effect Effects 0.000 claims description 3
- 241001269238 Data Species 0.000 description 1
- PEDCQBHIVMGVHV-UHFFFAOYSA-N Glycerine Chemical compound OCC(O)CO PEDCQBHIVMGVHV-UHFFFAOYSA-N 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Landscapes
- Telephonic Communication Services (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a method for processing massive GPS (global positioning system) data and a system. The method for processing massive GPS data includes analyzing received GPS data to obtain reporting times and unique vehicular identities, storing the GPS data under storage paths named after the reporting times and in files named after the unique identities of vehicular terminals, and realizing an effect that one vehicle corresponds to one data file during one day; and when in inquiry, dividing inquiry time in receive inquiry information into a minimum unit based on a day, carrying out operation for the inquiry, positioning the files according to the unique vehicular identities, reading information of eligible files, finally, merging inquiry results, and outputting merged results to a user so that the user can check the merged results. By the aid of the method and the system, the massive GPS data can be quickly and regularly stored, required inquiry data among the massive GPS data can be accurately and quickly found, inquiry accuracy and efficiency are effectively improved, and a hundred percent of hit rate can be realized.
Description
Technical field
The present invention relates to data processing field, more particularly, relate to a kind of mass GPS data processing method and system.
Background technology
General data all adopts the relation data library storage, as Oracle, mysql, mssqlserver etc., the gps data amount is too large, car-mounted terminal is according to the regular reported data of cycle of setting, such as report once 10 seconds, 8640 data were just arranged in one day, and when car-mounted terminal quantity was many, this reported data amount was very huge, the gps data server can become very slow reaching certain data volume post-relational database library inquiry, cause search efficiency seriously low, sometimes the database server CPU usage reaches 100%, also affects other business datum storage and inquiries.
Summary of the invention
The technical problem to be solved in the present invention is how to improve storage and the search efficiency of mass GPS data, problem for the data query inefficiency of prior art provides a kind of storage of above mass GPS data and mass GPS data processing method and system of search efficiency of solving.
The technical solution adopted for the present invention to solve the technical problems is: a kind of mass GPS data processing method and system are provided.
The invention provides a kind of mass GPS data processing method, the mass data that receives for the treatment of the GPS server, this GPS server comprises receiving element, resolution unit, the first judging unit, the second judging unit, time file creation module, document creation module, storage unit, query requests receiving element, cutting unit, reading unit, output unit, it is characterized in that, comprise the following steps:
S10, receiving element receive the gps data that the GPS car-mounted terminal reports; Described gps data comprise call time, car-mounted terminal unique identification, vehicle position information, security of vehicle information, described car-mounted terminal unique identification is license plate number or telephone number;
S11, resolution unit are resolved the gps data that receives, and obtain calling time and the car-mounted terminal unique identification;
S12, the first judging unit according on call time and judge the store path that whether has in storage unit with the name of calling time on described, if this store path is arranged, if execution in step S14 is not this store path, execution in step S13;
S13, time file creation module are according to the creation-time file that calls time on described;
S14, the second judging unit call time according to the judgement of car-mounted terminal unique identification whether file with this car-mounted terminal unique identification name is arranged under the store path of name on described, if the file with this car-mounted terminal unique identification name is arranged, execution in step S16; If the file of not naming with this car-mounted terminal unique identification, execution in step S15;
S15, the newly-built file with this car-mounted terminal unique identification name of document creation module;
S16, with described gps data sequential storage in the file with car-mounted terminal unique identification name;
S17, query requests receiving element receive client's query requests, and described query requests comprises query time and car-mounted terminal unique identification;
The described query time that S18, cutting unit will receive is divided into and take in the sky as one or more inquiries of unit;
S19, reading unit are at the described file content that reads in take the sky as the minimum memory file with the name of car-mounted terminal unique identification;
S20, Query Result is merged, export by output unit (20).
In mass GPS data processing method of the present invention, described time file comprises
Time file with the time name;
Set up the month file with the month name in each described time file;
Set up the dayfile folder with name every day in each file in described month.
In mass GPS data processing method of the present invention, described query time comprises very first time point and the second time point, and the wherein said very first time is o'clock early than the second time point.
The present invention also provides a kind of mass GPS data handling system, comprising: receiving element: be used for receiving the gps data that car-mounted terminal sends over; Resolution unit: be connected with described receiving element, be used for the gps data that receiving element receives is resolved, obtain calling time and the car-mounted terminal unique identification; The first judging unit: be connected with described resolution unit, be used for according on call time and judge whether storage unit has the store path of naming to call time on described; The second judging unit: be connected with the first judging unit, whether the file with this car-mounted terminal unique identification name is arranged under the store path of the name that is used for calling time on described according to the judgement of car-mounted terminal unique identification; Time file creation module: be connected with the first judging unit, be used for according to the creation-time file that calls time in gps data; The document creation module: be connected with the second judging unit, the car-mounted terminal unique identification that be used for to create in the gps data is the file of filename;
Storage unit: be connected with the second judging unit, be used for storing described gps data;
Described gps data comprise call time, car-mounted terminal unique identification, vehicle position information, security of vehicle information; Described car-mounted terminal unique identification is license plate number or telephone number;
The query requests receiving element is used for receiving query requests, and described query requests comprises query time and car-mounted terminal unique identification; Cutting unit is connected with the query requests receiving element, and the described query time that is used for receiving is divided into and take in the sky as one or more inquiries of unit; Reading unit is connected respectively with cutting unit and storage unit, is used for the file content of naming with the car-mounted terminal unique identification in described storage unit reads take the sky as the minimum memory file; Output unit is used for the result of reading unit is merged output.
Description of drawings
The invention will be further described below in conjunction with drawings and Examples, in accompanying drawing:
Fig. 1 is: the process flow diagram of mass GPS data processing method of the present invention;
Fig. 2 is: the particular flow sheet of mass GPS data processing method step S2 of the present invention;
Fig. 3 is: the particular flow sheet of mass GPS data processing method step S4 of the present invention;
Fig. 4 is: the structural drawing of mass GPS data query system of the present invention.
Embodiment
In order to make purpose of the present invention clearer, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that specific embodiment described herein only in order to explain the present invention, is not intended to limit the present invention.
A kind of mass GPS data processing method of the present invention, the mass data that receives for the treatment of the GPS server, this GPS server comprises receiving element 10, resolution unit 11, the first judging unit 12, the second judging unit 14, time file creation module 13, document creation module 15, storage unit 16, query requests receiving element 17, cutting unit 18, reading unit 19, output unit 20, the GPS car-mounted terminal regularly is reported to the GPS server with gps data according to the cycle of setting), comprise the following steps:
S1: receiving element 11 receives the gps data that the GPS car-mounted terminal reports;
S2: storage unit 16 is stored gps data with the form of file, and according to one day corresponding data file of a car;
S3: query requests receiving element 17 receives client's query requests;
S4: according to client's query requests, cutting unit 18 is divided into take the sky as unit query requests at storage unit query;
This gps data comprise call time, car-mounted terminal unique identification, vehicle position information, security of vehicle information, described car-mounted terminal unique identification is license plate number or telephone number.
The detailed process of step S2 as shown in Figure 2, concrete steps are as follows:
S20, receiving element 10 receive the gps data that car-mounted terminal sends over; Gps data comprise call time, car-mounted terminal unique identification, vehicle position information, security of vehicle information;
S21,11 pairs of gps datas that receive of resolution unit are resolved, and obtain calling time and the car-mounted terminal unique identification;
S22, the first judging unit 12 according on call time and judge the store path that whether has in storage unit with the name of calling time on described, if this store path is arranged, if execution in step S14 is not this store path, execution in step S13;
S23, time file creation module 13 are according to the creation-time file that calls time on described, this time file comprises the time file of time name, and set up the month file of naming with month in each time file, set up in every month the dayfile folder with name every day in part file
S24, the second judging unit 14 call time according to the judgement of car-mounted terminal unique identification whether file with this car-mounted terminal unique identification name is arranged under the store path of name on described, if the file with this car-mounted terminal unique identification name is arranged, execution in step S26; If the file of not naming with this car-mounted terminal unique identification, execution in step S25;
S25, the newly-built file with this car-mounted terminal unique identification name of document creation module 15;
S26, with described gps data sequential storage in the file with car-mounted terminal unique identification name.
In concrete storing process, according to the feature of gps data, gps data is carried out cutting take the sky as unit according to the time, then data press the car cutting, finally realize data file of one day car correspondence.The bibliographic structure of file is as follows: root directory year the moon day vehicle unique number (data file).The GPS car-mounted terminal regularly reports gps data according to the time cycle of setting, and such as reporting a gps data 10 seconds, this gps data comprised and calling time, as on January 1st, 2011; Also comprise GPS car-mounted terminal unique identification, this unique identification can be the license plate number (being AAA as license plate number) of this GPS car-mounted terminal, can be also car phone number (as telephone number 15912345678), can be also the data that other can this GPS car-mounted terminal of unique identification, also comprise vehicle position information, security of vehicle information and other information, vehicle position information such as longitude and latitude, speed, direction, reference position etc.Receiving element receives this gps data, and resolution unit parses and calls time is on January 1st, 2011, and the car-mounted terminal unique identification is 15912345678.The file whether following path is arranged in the first judgment unit judges storage unit: 2011 Nian Yi Yue 1 day, if have, whether file with 15912345678 names is arranged in the second judgment unit judges 1 dayfile folder, if have, this gps data is sequentially write this document, do not cover original content, usually under circumstances, the reported data time is orderly, be directly appended to after file just passablely, repay on a small quantity data but have, write fashionable need and be inserted in the middle of file.If there is no the path in the first judgment unit judges storage unit is the 2011 Nian Yi Yue files of 1 day, time file creation module creates this document folder.If there is no the files with 15912345678 names in the second judgment unit judges 1 dayfile folder, the newly-built files with 15912345678 names of document creation module.
The detailed process of step S4 comprises the following steps as shown in Figure 3:
S40, query requests receiving element 17 receive query requests, and described query requests comprises query time and car-mounted terminal unique identification; This query time comprises very first time point and the second time point, and the wherein said very first time is o'clock early than the second time point;
The described query time that S41, cutting unit 18 will receive is divided into one day with interior one or more inquiries;
S42, reading unit 19 are at the described file content that reads in take the sky as the minimum memory file with the name of car-mounted terminal unique identification;
S43, Query Result is merged, by output unit 20 outputs.
in concrete query script, the query requests unit receives query requests, this query requests comprises query time and car-mounted terminal unique identification, such as: query time is on January 3,1 day to 2011 January in 2011, GPS car-mounted terminal unique identification is telephone number 15912345678, at first cutting unit is divided into 3 days with this query time, respectively in following path: 2011 Nian Yi Yue 1 day, 2011 Nian Yi Yue 2 days, 2011 Nian Yi Yue 3 days, search the file with the name of car-mounted terminal unique identification, such as the car-mounted terminal unique identification of inquiring about is telephone number 15912345678, if find, read this document information, and all information that satisfy condition of searching in these 3 files are merged, exporting to the user by output unit checks.Adopt this querying method, can fast query context be limited to the time of querying condition appointment, be that unit resolves into a plurality of one day to the query time section with interior inquiry and do not need to inquire about All Files when inquiry folder by the sky, then the result of multiple queries is merged is exactly the result that we want.Use this querying method, can accelerate inquiry velocity, improve search efficiency and degree of accuracy.
The present invention also provides a kind of mass GPS data handling system, comprises receiving element 10: be used for receiving the gps data that car-mounted terminal sends over; Resolution unit 11: be connected with receiving element 10, be used for the gps data that receiving element 10 receives is resolved, obtain calling time and the car-mounted terminal unique identification; The first judging unit 12: be connected with resolution unit 11, be used for according on call time and judge whether storage unit 16 has the store path of naming to call time on described; The second judging unit 14: be connected 12 with the first judging unit, whether the file with this car-mounted terminal unique identification name is arranged under the store path of the name that is used for calling time on described according to the judgement of car-mounted terminal unique identification; Time file creation module 13: be connected with the first judging unit 12, be used for according to the creation-time file that calls time in gps data; Document creation module 15: be connected with the second judging unit 14, the car-mounted terminal unique identification that be used for to create in the gps data is the file of filename; Storage unit 16: be used for storing described gps data; Query requests receiving element 17: be used for receiving query requests, query requests comprises query time and car-mounted terminal unique identification; Cutting unit 18: be connected with query requests receiving element 17, be used for the query time that receives is divided into taking in the sky as one or more inquiries of unit; Reading unit 19: are connected with storage unit with cutting unit 18 and are connected respectively, be used for read file content with the name of car-mounted terminal unique identification take the sky as the minimum memory file; Output unit 20: be connected with reading unit 19, be used for that the user is exported in the result merging of reading unit and check.Time file creation module 13 specifically comprises time file creating unit, month file creating unit, dayfile folder creating unit.The gps data that receiving element 10 receives comprise call time, car-mounted terminal unique identification, vehicle position information, security of vehicle information, this car-mounted terminal unique identification can be license plate number, telephone number, can also be that other can vehicle-mounted label of unique identification.
The above is only preferred embodiment of the present invention, not in order to limiting the present invention, all any modifications of doing within the spirit and principles in the present invention, is equal to and replaces and improvement etc., within all should being included in protection scope of the present invention.
Claims (4)
1. mass GPS data processing method, the mass data that receives for the treatment of the GPS server, this GPS server comprises receiving element (10), resolution unit (11), the first judging unit (12), the second judging unit (14), time file creation module (13), document creation module (15), storage unit (16), query requests receiving element (17), cutting unit (18), reading unit (19), output unit (20), it is characterized in that, comprise the following steps:
S10, receiving element (10) receive the gps data that the GPS car-mounted terminal reports; Described gps data comprise call time, car-mounted terminal unique identification, vehicle position information, security of vehicle information, described car-mounted terminal unique identification is license plate number or telephone number;
S11, resolution unit (11) are resolved the gps data that receives, and obtain calling time and the car-mounted terminal unique identification;
S12, the first judging unit (12) according on call time and judge the store path that whether has in storage unit with the name of calling time on described, if this store path is arranged, if execution in step S14 is not this store path, execution in step S13;
S13, time file creation module (13) are according to the creation-time file that calls time on described;
S14, the second judging unit (14) call time according to the judgement of car-mounted terminal unique identification whether file with this car-mounted terminal unique identification name is arranged under the store path of name on described, if the file with this car-mounted terminal unique identification name is arranged, execution in step S16; If the file of not naming with this car-mounted terminal unique identification, execution in step S15;
S15, the newly-built file with this car-mounted terminal unique identification name of document creation module (15);
S16, with described gps data sequential storage in the file with car-mounted terminal unique identification name;
S17, query requests receiving element (17) receive client's query requests, and described query requests comprises query time and car-mounted terminal unique identification;
The described query time that S18, cutting unit (18) will receive is divided into and take in the sky as one or more inquiries of unit;
S19, reading unit (19) read the file content with the name of car-mounted terminal unique identification in take the sky as the minimum memory file;
S20, Query Result is merged, export by output unit (20).
2. mass GPS data processing method according to claim 1, is characterized in that, described time file comprises
Time file with the time name;
Set up the month file with the month name in each described time file;
Set up the dayfile folder with name every day in each file in described month.
3. mass GPS data processing method according to claim 1, is characterized in that, described query time comprises very first time point and the second time point, and the wherein said very first time is o'clock early than the second time point.
4. a mass GPS data handling system, is characterized in that, comprising:
Receiving element (10): be used for receiving the gps data that car-mounted terminal sends over; Resolution unit (11): be connected with described receiving element (10), be used for the gps data that receiving element receives is resolved, obtain calling time and the car-mounted terminal unique identification; The first judging unit (12): be connected with described resolution unit (11), be used for according on call time and judge whether storage unit has the store path of naming to call time on described; The second judging unit (14): be connected with the first judging unit (12), whether the file with this car-mounted terminal unique identification name is arranged under the store path of the name that is used for calling time on described according to the judgement of car-mounted terminal unique identification; Time file creation module (13): be connected with the first judging unit (12), be used for according to the creation-time file that calls time in gps data; Document creation module (15): be connected with the second judging unit (14), the car-mounted terminal unique identification that be used for to create in the gps data is the file of filename;
Storage unit (16): be connected with the second judging unit (14), be used for storing described gps data;
Described gps data comprise call time, car-mounted terminal unique identification, vehicle position information, security of vehicle information, described car-mounted terminal unique identification is license plate number or telephone number;
Query requests receiving element (17) is used for receiving query requests, and described query requests comprises query time and car-mounted terminal unique identification; Cutting unit (18) is connected with the query requests receiving element, and the described query time that is used for receiving is divided into and take in the sky as one or more inquiries of unit; Reading unit (19) is connected respectively with cutting unit (18) and storage unit (16), is used for the file content of naming with the car-mounted terminal unique identification in described storage unit (16) reads take the sky as the minimum memory file; Output unit (20) is used for the result of reading unit (16) is merged output.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110365001 CN102495848B (en) | 2011-11-17 | 2011-11-17 | Method for processing massive GPS (global positioning system) data and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110365001 CN102495848B (en) | 2011-11-17 | 2011-11-17 | Method for processing massive GPS (global positioning system) data and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102495848A CN102495848A (en) | 2012-06-13 |
| CN102495848B true CN102495848B (en) | 2013-06-19 |
Family
ID=46187673
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201110365001 Expired - Fee Related CN102495848B (en) | 2011-11-17 | 2011-11-17 | Method for processing massive GPS (global positioning system) data and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102495848B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104599341B (en) * | 2013-10-31 | 2017-02-15 | 深圳市赛格导航科技股份有限公司 | Car positioning information storage method and apparatus thereof |
| CN105302867B (en) * | 2015-09-28 | 2019-06-11 | 浙江宇视科技有限公司 | A kind of search engine inquiry method and device |
| CN105653652B (en) * | 2015-12-28 | 2019-07-05 | 上海瀚银信息技术有限公司 | A kind of method of data synchronization and system |
| CN106249262A (en) * | 2015-12-29 | 2016-12-21 | 北京市交通信息中心 | A kind of storage method and device of vehicle satellite location data |
| CN106326358A (en) * | 2016-08-10 | 2017-01-11 | 东方网力科技股份有限公司 | Big data-based vehicle passing information inquiry method and device |
| CN108111557B (en) * | 2016-11-24 | 2021-06-11 | 杭州海康威视数字技术股份有限公司 | Method and device for acquiring data in cloud storage system |
| CN108959114B (en) * | 2018-05-22 | 2022-12-16 | 长沙金信诺防务技术有限公司 | Sonar detection data recording device and recording method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101370025A (en) * | 2007-08-17 | 2009-02-18 | 北京灵图软件技术有限公司 | Storing method, scheduling method and management system for geographic information data |
| CN101876992A (en) * | 2009-11-17 | 2010-11-03 | 中国科学院自动化研究所 | Method for managing image data warehouse |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB0622149D0 (en) * | 2006-11-07 | 2006-12-20 | Singlepoint Holdings Ltd | System and method to validate and authenticate digital data |
-
2011
- 2011-11-17 CN CN 201110365001 patent/CN102495848B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101370025A (en) * | 2007-08-17 | 2009-02-18 | 北京灵图软件技术有限公司 | Storing method, scheduling method and management system for geographic information data |
| CN101876992A (en) * | 2009-11-17 | 2010-11-03 | 中国科学院自动化研究所 | Method for managing image data warehouse |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102495848A (en) | 2012-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102495848B (en) | Method for processing massive GPS (global positioning system) data and system | |
| CA2895893C (en) | Searchable data archive | |
| CN102542026B (en) | Mass global position system (GPS) data access method and system | |
| CN105164674A (en) | Queries involving multiple databases and execution engines | |
| CN110413634B (en) | Data query method, system, device and computer readable storage medium | |
| RU2016116718A (en) | METHOD (OPTIONS) AND DEVICE (OPTIONS) OF INFORMATION PROCESSING, TERMINAL AND SERVER | |
| CN103902698A (en) | Data storage system and data storage method | |
| CN108804459B (en) | Data query method and device | |
| CN102880541A (en) | Log information acquisition system and log information acquisition method | |
| CN108170752B (en) | Template-based metadata management method and system | |
| WO2014063628A1 (en) | Method and device for providing item shipping route and terminal device | |
| WO2014044053A1 (en) | Data indexing method and device | |
| CN112637305A (en) | Data storage and query method, device, equipment and medium based on cache | |
| WO2019100853A1 (en) | Page query method, device, and electronic apparatus | |
| CN115145871A (en) | File query method and device and electronic equipment | |
| CN110704423B (en) | Excitation information acquisition method and device, storage medium and electronic equipment | |
| CN110162540B (en) | Block chain account book data query method, electronic device and storage medium | |
| CN108920523A (en) | Data query method, apparatus, equipment, system and medium on block chain | |
| CN111078774A (en) | Automatic data integration method | |
| CN116680315A (en) | Data offline processing method and device, electronic equipment and storage medium | |
| CN111625600B (en) | Data storage processing method, system, computer equipment and storage medium | |
| CN117493319A (en) | Data deduplication method and device, electronic equipment and storage medium | |
| CN101980190A (en) | Method and device for quickly putting service data into base | |
| CN110928868B (en) | Vehicle data retrieval method, device and computer readable storage medium | |
| CN109543079B (en) | Data query method and device, computing equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20130619 |