CN112905646A - Geographic data loading method and device based on access statistics - Google Patents
Geographic data loading method and device based on access statistics Download PDFInfo
- Publication number
- CN112905646A CN112905646A CN202110374029.6A CN202110374029A CN112905646A CN 112905646 A CN112905646 A CN 112905646A CN 202110374029 A CN202110374029 A CN 202110374029A CN 112905646 A CN112905646 A CN 112905646A
- Authority
- CN
- China
- Prior art keywords
- data block
- data
- access
- scene
- cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011068 loading method Methods 0.000 title claims abstract description 27
- 238000000034 method Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 abstract description 2
- 238000013500 data storage Methods 0.000 abstract 1
- 239000002699 waste material Substances 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 230000009191 jumping Effects 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Remote Sensing (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention relates to a geographic data loading method and a device based on access statistics, which comprises a cache system, a data storage system and a data processing system, wherein the cache system is used for counting the access heat of data blocks and the use frequency of different scene data blocks, replacing the most recently used data blocks when the cache capacity is exhausted, caching the scene data blocks with the use frequency exceeding a threshold value into a queue and simultaneously adjusting an access heat sequence; the data fetching system is used for configuring the data blocks, determining the size of the returned data block when the requested data is the size of the returned data block, calculating the prefetching time of the next data block in real time, and realizing different access strategies according to whether the data block is hit in the cache system or not. The method and the device can be used for counting which scene data blocks belong to the hot data in the geographic scene roaming process in a given time interval based on user behavior statistics, reduce network overhead by adopting cache for the hot data, and directly request non-hot data blocks in a time slice from a remote server in advance, so that unnecessary storage occupation can be reduced.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a geographic data loading method and device based on access statistics.
Background
Applications based on geographic data are increasing, such as map navigation, smart cities, 3D maps, etc. Map base maps of existing geographic applications are rendered by dynamic loading in a lazy loading mode in a tile data format, and scene models on the base maps are rendered by dynamic loading in a slicing mode.
At present, geographic application products in the market do not integrate access behaviors of tourists for dynamically loading geographic data, so that equipment cache access history is usually adopted or a remote server is requested to download the geographic data again, the mode 1) causes equipment storage waste along with the time lapse of using the product by the tourists, the mode 2) causes great network bandwidth waste due to the fact that the data is requested again at each time, particularly huge geographic data, and the two modes further cause user experience to become unfriendly.
Roaming of 3D scenes is a typical interactive mode in three-dimensional maps, and because the data volume of the 3D maps is very large, the roaming cannot be performed by requesting to home at one time. In fact, tile data are adopted in the base map of the three-dimensional map and rendering is dynamically loaded based on a lazy loading mode, and the 3D landscape model on the base map is necessarily dynamically loaded and rendered in a slicing mode. When 3D map roaming is carried out, the technical method adopted by the existing commercial products is to cache all accessed historical data blocks or not cache re-requests every time, and the two methods have disadvantages.
When geographic service software products on the market process geographic data dynamic loading, historical data are cached locally or scene data are requested again, and the former mode gradually occupies a large amount of equipment storage space along with the increase of the frequency of using the products by users; the latter approach results in a huge waste of network bandwidth due to the typically large amount of geo-scene data and may affect the user experience in weak networks.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, provides a geographic data loading method and device based on access statistics, and solves the defects of a data access loading mode in the roaming process of the existing three-dimensional scene.
The purpose of the invention is realized by the following technical scheme: a geographic data loading device based on access statistics comprises a cache system and a data acquisition system; the cache system is used for counting the access heat of the data blocks and the use frequency of different scene data blocks, replacing the data blocks which are least frequently used recently when the cache capacity is exhausted, caching the scene data blocks of which the use frequency exceeds a threshold value into a queue and simultaneously adjusting the access heat sequence;
the data fetching system is used for configuring the data blocks, determining the size of the returned data block when the requested data is the size of the returned data block, calculating the prefetching time of the next data block in real time, and realizing different access strategies according to whether the data block is hit in the cache system or not.
The cache system comprises an LRU module, a Counter module and a scene data block cache queue;
the LRU module is used for counting the access heat of the data blocks which are already stored in the scene data block cache queue and replacing the least frequently used data blocks if the cache capacity is exhausted within a set time slice;
the Counter module is used for counting the use frequency of different scene data blocks in a time slice, caching the data blocks into a queue when the use frequency of a certain scene data block generates a cache threshold value, and simultaneously adjusting the access heat sequence of the LRU module;
the scene data block cache queue is used for storing cached data blocks.
The access system comprises a scene data block configuration submodule, a roaming coordinate tracking submodule and a scene data block access strategy submodule;
the database configuration submodule is used for configuring the space coding rule of the data block and the space range of the data block and determining the size of the returned data block when the data is requested;
the roaming coordinate tracking submodule is used for calculating the prefetching time of the next data block in real time according to the space position of the data block where the tourist is currently located;
the scene data block access strategy submodule is used for prefetching from the cache system according to the data block ID, calling the Counter module and the LRU module for hot spot sequence adjustment once if the cache system is hit, calling the Counter module to count the use frequency if the cache system is hit, and pulling the data block from a remote server.
A method of geographic data loading a device based on access statistics, the method comprising: .
If the user does not enter the geographical scene for the first time, pulling the ID information of the scene data block and the time slice information used by the user from the server;
judging whether the current time slice of the user is used up, and if the time slice is not used up, judging whether the next accessed data block is cached;
if the next accessed data block is not cached, calling a Counter module and judging whether the current data block reaches a cacheable threshold value or not, and if so, judging whether a cache queue in the current time slice is full;
if the cache queue is full, replacing the ID of the data block which is accessed currently with the ID of the data block which is used least according to the LRU access sequence, caching the current data block into the cache queue, and adjusting the LRU access sequence at the same time; if the cache queue is not full, adjusting the LRU access sequence and caching the currently accessed data block into the cache queue;
and judging whether the next scene data block needs to be pre-fetched or not, and if so, continuing to execute the step of judging whether the accessed data block is cached or not when the next scene data block is pre-fetched.
If the current time slice is used up, the current data block counting period is ended, and the LRU module and the Counter module are reset, new time slices are set, and the time slices are uploaded to the server.
If the database of the next access is cached, the whole system needs to call the Counter module to update the access times of the current data block and adjust the LRU access sequence in the time slice regardless of caching.
And if the caching threshold is not reached, prefetching the specified scene data block from the remote server and performing a roaming process.
If the next scene data block does not need to be pre-fetched, the roaming process is directly performed.
If the user enters the geographical scene for the first time, the LRU module, the Counter module and the local cache queue need to be initialized, the identity information of the user is uploaded to the server, and the time slice is set.
Before determining whether the user enters the geographic scene for the first time, the scene data block configuration information including the spatial number of the scene data block and the spatial range of the scene data block needs to be initialized.
The invention has the following advantages: a geographic data loading method and device based on access statistics can count which scene data blocks belong to hot data in a geographic scene roaming process in a given time interval based on user behavior statistics, reduce network overhead by adopting cache for the hot data, and directly request non-hot data blocks in a time slice from a remote server in advance, so that unnecessary storage occupation can be reduced;
in the design of the cache system, on the premise of controlling the cache capacity to be certain, the non-hot data can be replaced by the LRU access sequence device, and the dynamic adjustment process can effectively adapt to the behavior pattern of the user in the current time slice, so that the user experience is better improved.
Drawings
FIG. 1 is a schematic diagram of the apparatus of the present invention;
FIG. 2 is a schematic flow chart of the method of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the detailed description of the embodiments of the present application provided below in connection with the appended drawings is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application. The invention is further described below with reference to the accompanying drawings.
As shown in fig. 1, the present invention relates to a geographic data loading device based on access statistics, which mainly includes the following contents:
the cache system is divided into three sub-modules, wherein the LRU module is used for counting the access heat of the data blocks already existing in the cache queue, and if the cache capacity is exhausted in a set time slice, the least frequently used data blocks are required to be replaced according to the LRU module;
the Counter module is used for counting the use frequency of different scene data blocks in a time slice, caching the data block into a queue when the use frequency of a certain scene data block exceeds a caching threshold value, and simultaneously adjusting the access heat sequence of the LRU;
the scene data block cache queue is used for storing cached data blocks, and unnecessary server data requests can be effectively reduced.
The data fetching system is also divided into three sub-modules, wherein the data block configuration sub-module is used for configuring the space coding rule of the data block, the space range of the data block and the like, and is used for determining the size of the returned data block when requesting data;
the roaming coordinate tracking submodule calculates the prefetching time of the next data block in real time according to the space position of the data block where the tourist is currently located;
and finally, the scene data block access strategy module prefetches from the cache system according to the data block ID, if the cache system is hit, the Counter module and the LRU module are sequentially called to adjust the hot spot sequence, if the cache system is not hit, the Counter sub-module is required to be called to count the frequency, and the data block is pulled from the remote server.
Wherein, the strategy has the advantages that: if the application cache capacity can be infinite, the data blocks accessed by the history can be completely cached without pulling the complete data blocks from the remote server again [ only the updating part is required for the existing data updating ]; however, the practical problem is that the hardware storage capacity is limited, for example, as the number of installed applications increases and the application use time increases, the problem of insufficient storage often occurs in our mobile phone device, and the very important reason for causing the problem is that various software applications continuously cache access data without limiting the cache capacity and regularly cleaning the infrequent data. Therefore, the strategy proposed by the patent is to calculate the access sequence of the hot data block by counting the access frequency of the geographic data block in the time slice and the LRU module through the Counter on the premise of limiting the maximum cache capacity for the application, so that the access of large data volume, such as geographic data, can cache the hot data and replace and clean the cold data in the time slice, thereby balancing the problems of cache use and network overhead.
As shown in fig. 2, another embodiment of the present invention relates to a geographic data loading method based on access statistics, which mainly includes the following steps:
and step 0, initializing configuration information of the scene data blocks, wherein the configuration information comprises the space number of the scene data blocks, the space range of the scene data blocks and the like.
Step 1, when a user enters a geographic scene, a server is needed to judge whether the user enters for the first time. And if the first time, jumping to the step 2, otherwise, jumping to the step 3.
And 2, if the geographical scene is entered for the first time, initializing the LRU module, the Counter module and the local cache queue device, uploading the identity information of the user to a server and setting a time slice.
And 3, if the user does not enter the geographical scene for the first time, pulling the information such as the ID information of the scene data block and the time slice used by the user from the server.
And 4, judging whether the current time slice of the user is used up, if the time slice is used up, skipping to the step 5, and if not, skipping to the step 6.
And 5, if the current time slice is used up, indicating that the current data block counting period is ended, resetting the LRU module and the Counter module, setting a new time slice and uploading the information to the server.
And 6, judging whether the next accessed data block is cached, if so, skipping to the step 7, otherwise, skipping to the step 8.
And 7, if the whole system is in the time slice, whether caching is carried out or not, a Counter module is required to be called to update the access times of the current data block, and an LRU access sequence is adjusted.
And 8, calling a Counter module, judging whether the current data block reaches a cacheable threshold value, if so, skipping to the step 9, and otherwise, skipping to the step 10.
And 9, judging whether the cache queue in the current time slice is full, if so, skipping to the step 12, and if not, skipping to the step 13.
And 10, prefetching a specified scene data block from a remote server.
And step 11, roaming process.
And step 12, replacing the data block ID which is accessed currently with the least recently used data block ID according to the LRU access sequence, caching the current data block into a cache queue, and adjusting the LRU access sequence.
And step 13, adjusting the LRU access sequence and caching the currently accessed data block into a cache queue.
And step 14, judging whether the next scene data block needs to be prefetched or not based on the roaming positioning module, if the next scene data block does not need to be prefetched, skipping to step 11, and if the next scene data block does not need to be prefetched, skipping to step 15.
And step 15, jumping to step 6 when the next scene data block is prefetched.
In order to buffer some cold data without excessively depending on the device buffer memory and ensure the network request expense as small as possible, the invention records the historical access data block condition of the tourist in a given time slice to count which data blocks are the hot data of the tourist and buffer the data blocks, and reduces the device storage waste through the network request for the non-hot data.
The foregoing is illustrative of the preferred embodiments of this invention, and it is to be understood that the invention is not limited to the precise form disclosed herein and that various other combinations, modifications, and environments may be resorted to, falling within the scope of the concept as disclosed herein, either as described above or as apparent to those skilled in the relevant art. And that modifications and variations may be effected by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. A geographic data loading device based on access statistics is characterized in that: the system comprises a cache system and a data acquisition system; the cache system is used for counting the access heat of the data blocks and the use frequencies of different scene data blocks, replacing the data blocks which are least frequently used recently when the cache capacity is exhausted, caching the scene data blocks of which the use frequencies exceed a threshold value into a queue and adjusting an access heat sequence at the same time;
the data fetching system is used for configuring the data blocks, determining the size of the returned data block when the requested data is the size of the returned data block, calculating the prefetching time of the next data block in real time, and realizing different access strategies according to whether the data block is hit in the cache system or not.
2. The geographic data loading device based on access statistics as claimed in claim 1, wherein: the cache system comprises an LRU module, a Counter module and a scene data block cache queue;
the LRU module is used for counting the access heat of the data blocks which are already stored in the scene data block cache queue and replacing the least frequently used data blocks if the cache capacity is exhausted within a set time slice;
the Counter module is used for counting the use frequency of different scene data blocks in a time slice, caching the data blocks into a queue when the use frequency of a certain scene data block generates a cache threshold value, and simultaneously adjusting the access heat sequence of the LRU module;
the scene data block cache queue is used for storing cached data blocks.
3. The geographic data loading device based on access statistics as recited in claim 2, wherein: the access system comprises a scene data block configuration submodule, a roaming coordinate tracking submodule and a scene data block access strategy submodule;
the database configuration submodule is used for configuring the space coding rule of the data block and the space range of the data block and determining the size of the returned data block when the data is requested;
the roaming coordinate tracking submodule is used for calculating the prefetching time of the next data block in real time according to the space position of the data block where the tourist is currently located;
the scene data block access strategy submodule is used for prefetching from the cache system according to the data block ID, calling the Counter module and the LRU module for hot spot sequence adjustment once if the cache system is hit, calling the Counter module for counting the use frequency if the cache system is not hit, and pulling the data block from a remote server.
4. A method for loading geographic data based on access statistics is characterized in that: the method comprises the following steps:
if the user does not enter the geographical scene for the first time, pulling the ID information of the scene data block and the time slice information used by the user from the server;
judging whether the current time slice of the user is used up, and if the time slice is not used up, judging whether the next accessed data block is cached;
if the next accessed data block is not cached, calling a Counter module and judging whether the current data block reaches a cacheable threshold value or not, and if so, judging whether a cache queue in the current time slice is full;
if the cache queue is full, replacing the ID of the data block which is accessed currently with the ID of the data block which is used least according to the LRU access sequence, caching the current data block into the cache queue, and adjusting the LRU access sequence at the same time; if the cache queue is not full, adjusting the LRU access sequence and caching the currently accessed data block into the cache queue;
and judging whether the next scene data block needs to be pre-fetched or not, and if so, continuing to execute the step of judging whether the accessed data block is cached or not when the next scene data block is pre-fetched.
5. The method for geographic data loading device based on access statistics as claimed in claim 4, wherein: if the current time slice is used up, the current data block counting period is ended, and the LRU module and the Counter module are reset, new time slices are set, and the time slices are uploaded to the server.
6. The method for geographic data loading device based on access statistics as claimed in claim 4, wherein: if the database of the next access is cached, the whole system needs to call the Counter module to update the access times of the current data block and adjust the LRU access sequence in the time slice regardless of caching.
7. The method for geographic data loading device based on access statistics as claimed in claim 4, wherein: and if the caching threshold is not reached, prefetching the specified scene data block from the remote server and performing a roaming process.
8. The method for geographic data loading device based on access statistics as claimed in claim 4, wherein: if the next scene data block does not need to be pre-fetched, the roaming process is directly performed.
9. The method for geographic data loading device based on access statistics according to any one of claims 4-8, characterized by that: if the user enters the geographical scene for the first time, the LRU module, the Counter module and the local cache queue need to be initialized, the identity information of the user is uploaded to the server, and the time slice is set.
10. The method for geographic data loading device based on access statistics as claimed in claim 9, wherein: before determining whether the user enters the geographic scene for the first time, the scene data block configuration information including the spatial number of the scene data block and the spatial range of the scene data block needs to be initialized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110374029.6A CN112905646A (en) | 2021-04-07 | 2021-04-07 | Geographic data loading method and device based on access statistics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110374029.6A CN112905646A (en) | 2021-04-07 | 2021-04-07 | Geographic data loading method and device based on access statistics |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112905646A true CN112905646A (en) | 2021-06-04 |
Family
ID=76110167
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110374029.6A Pending CN112905646A (en) | 2021-04-07 | 2021-04-07 | Geographic data loading method and device based on access statistics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112905646A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104834607A (en) * | 2015-05-19 | 2015-08-12 | 华中科技大学 | Method for improving distributed cache hit rate and reducing solid state disk wear |
| US20150256639A1 (en) * | 2014-03-04 | 2015-09-10 | Mobophiles, Inc., Dba Mobolize | Dynamic cache allocation and network management |
| CN109002260A (en) * | 2018-07-02 | 2018-12-14 | 深圳市茁壮网络股份有限公司 | A kind of data cached processing method and processing system |
| CN109359095A (en) * | 2018-09-11 | 2019-02-19 | 东华大学 | A kind of DLK method that big data is quickly read |
| CN110245095A (en) * | 2019-06-20 | 2019-09-17 | 华中科技大学 | A kind of solid-state disk cache optimization method and system based on data block map |
-
2021
- 2021-04-07 CN CN202110374029.6A patent/CN112905646A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150256639A1 (en) * | 2014-03-04 | 2015-09-10 | Mobophiles, Inc., Dba Mobolize | Dynamic cache allocation and network management |
| CN104834607A (en) * | 2015-05-19 | 2015-08-12 | 华中科技大学 | Method for improving distributed cache hit rate and reducing solid state disk wear |
| CN109002260A (en) * | 2018-07-02 | 2018-12-14 | 深圳市茁壮网络股份有限公司 | A kind of data cached processing method and processing system |
| CN109359095A (en) * | 2018-09-11 | 2019-02-19 | 东华大学 | A kind of DLK method that big data is quickly read |
| CN110245095A (en) * | 2019-06-20 | 2019-09-17 | 华中科技大学 | A kind of solid-state disk cache optimization method and system based on data block map |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111968212B (en) | Viewpoint-based three-dimensional urban scene data dynamic scheduling method | |
| CN106603688B (en) | Method for accessing augmented reality user context | |
| CN103279539B (en) | The display packing of interest point set, the display packing of electronic chart and device | |
| CN103607735B (en) | Base station display method and device based on map | |
| CN110599891B (en) | Map rendering method for power grid equipment | |
| US20150156609A1 (en) | Prefetching and caching map data based on mobile network coverage | |
| US20060277277A1 (en) | Method of automatically caching WAP web pages and a mobile communications device for the same | |
| CN103178989A (en) | Method and device for calculating visit hotness | |
| CN103179438A (en) | Method and device for determining of video playing first access code rate | |
| CN110582096B (en) | Data request processing method and communication network system | |
| CN107450860B (en) | Map file pre-reading method based on distributed storage | |
| CN111143733A (en) | Local data caching method and device, electronic equipment and readable storage medium | |
| CN112905646A (en) | Geographic data loading method and device based on access statistics | |
| CN104168174A (en) | Method and apparatus for information transmission | |
| CN115208907A (en) | Storage method and system based on object storage | |
| CN112003921A (en) | Method for actively caching and replacing hot data in edge computing environment | |
| CN108810167A (en) | A kind of information-pushing method and device | |
| CN109902241B (en) | Resource loading method, device and equipment | |
| CN109460293B (en) | Computing resource selection method under distributed computing environment in wireless cloud computing system | |
| CN115278290B (en) | Virtual reality video caching method and device based on edge nodes | |
| CN117950787B (en) | Advertisement display method and device, electronic equipment and storage medium | |
| CN113727128B (en) | Hot spot flow processing method and system | |
| CN114040379B (en) | Information recommendation method, device, equipment and storage medium | |
| CN118643238A (en) | Method and terminal for displaying interest surface in map | |
| CN114288647B (en) | Artificial intelligence game engine based on AI Designer, game rendering method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210604 |
|
| RJ01 | Rejection of invention patent application after publication |