CN114357104A - Non-index map slice polymerization method - Google Patents
Non-index map slice polymerization method Download PDFInfo
- Publication number
- CN114357104A CN114357104A CN202210175793.5A CN202210175793A CN114357104A CN 114357104 A CN114357104 A CN 114357104A CN 202210175793 A CN202210175793 A CN 202210175793A CN 114357104 A CN114357104 A CN 114357104A
- Authority
- CN
- China
- Prior art keywords
- level
- slice
- map slice
- map
- directory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for aggregating index-free map slices, which comprises the following steps: step S1, separately storing the map slice files of each level according to grid blocks; step S2, the user sends a request to the server to obtain the map slice information; step S3, calculating according to the map slice information; and step S4, judging whether the corresponding key exists in the redis memory database. The invention improves the query speed of the slice data, reduces the I/O times of a disk, reduces the occupation of network bandwidth, is convenient for the upgrade and maintenance operation of the map and realizes the high-performance map slice loading function by combining the server cache technology through processing the map slice blocks.
Description
Technical Field
The invention relates to a method, in particular to a method for aggregating index-free map slices, and belongs to the field of application of geographic information systems and image processing technologies.
Background
The map slicing process is a process of cutting a map within a specified geographic range into a plurality of rows and columns of fixed-size square pictures at a certain scale level. These regular pictures are map slices, also called tiles, which are a typical data of geographic information systems (also called GIS).
The map slice divides the slice level from top to bottom according to the pyramid model, and from the level 1, the higher the zoom level is, the smaller the area range is displayed, but the more detailed the ground feature description is. Map slices in a pyramid model are typically uniquely distinguished by triplets (level, row number, column number). The data volume of each level slice is 2n × 2n, where n is the slice level. For example, level 1 is 4 slices, with range covering the world; the 2 nd level is 16 slices, the slice data of the 1 st level is divided into 4 slices, and the division of each level of slices is gradually completed according to the mode.
The slice pyramid is characterized in that the scale of a single file is small, but the number of slices increases exponentially with the increase of the level of the slice pyramid, and the number of slices generated by slice data construction is huge. When the image is large or the image is complex, the browser will have the phenomena of response delay, jamming and the like when the client roams. Taking a 19-level pyramid as an example, the number of files exceeds billions of files, and thus a large amount of time is consumed for storing or calling up the slice data. And frequent I/O operations of a disk can be performed when a certain file is searched in the massive small files, so that the data loading efficiency of the GIS is reduced on the whole. Therefore, an efficient organization method for these massive small files is urgently needed to realize efficient storage and improve retrieval efficiency.
Disclosure of Invention
In order to solve the defects of the prior art, the invention provides an index-free map slice aggregation method, which improves the query speed of slice data, reduces the I/O times of a disk, reduces the occupation of network bandwidth, facilitates the upgrade and maintenance operation of a map and realizes the high-performance map slice loading function by combining the cache technology of a server.
In order to solve the technical problems, the invention adopts the technical scheme that: an index-free map slice aggregation method, comprising the steps of:
step S1, separately storing the map slice files of each level according to grid blocks;
step S2, the user sends a request to the server to obtain the map slice information;
step S3, calculating according to the map slice information;
and step S4, judging whether the corresponding key exists in the redis memory database.
Preferably, in step S1, the data is stored separately in 64 × 64 grid blocks, and the specific data organization structure is as follows:
s11, taking the map slice data set as a root directory of the map slice file data;
s12, a map slice level directory is arranged under the root directory;
s13, obtaining a row directory of the map slice matrix of the level under the map slice level directory;
s14, under the map slice level directory, the list of the map slice matrix at the level is obtained;
s15, listing a specific map slice data file;
according to the data organization structure, a plurality of slices of the third-level directory at each level are packaged into Z _ X/64_ Y/64.dat, and the dat file is a binary file which is stored firstly according to the sequence of X and then is stored according to the sequence of Y.
Preferably, the naming mode of the level directory name is as follows: "Z + level", the range of levels is: 1-n, namely Z1, Z2, Z3..
Preferably, the line directory name is named as follows: "X + line number", the line number range is: 0 to [ (2n-1)/64 ].
Preferably, the column directory names are named in the following way: "Y + column number", column number range: 0 to [ (2n-1)/64 ].
Preferably, the names of the map slice data file names are named as follows: slice "Z, X, Y", i.e., "slice level, abscissa value, ordinate value";
the first level directory is level Z, the first level directory is level number, and the level number is started from 1;
the second level directory is a line number X/64, starting from 0;
the third level directory is the column number Y/64, starting from 0;
and finally storing the map slices under a third-level catalog.
Preferably, in step S2, after the user sends the request at the client, the server first obtains the map slice information X, Y, Z according to the request address, where Z is the level, X is the abscissa, and Y is the ordinate.
In step S3, the name, address, key value of redis, and offset value of the dat file are calculated from the map slice information X, Y, Z.
In step S4, if there is a corresponding key in redis, the value is directly read, the offset and size are calculated, the slice data corresponding to X, Y, Z is read, and the byte data is returned to the client for rendering.
In step S4, if there is no corresponding key in the redis, reading data according to the file name and the path I/O, and calculating offset and size, and meanwhile saving the read binary file into the redis, caching common slice information in the redis, improving extraction efficiency, and according to the dat file and the calculated offset and size, reading slice data corresponding to X, Y, and Z in the distributed database and returning byte data to render to the client.
The invention stores the map slice files of each level separately according to 64-by-64 grid blocks. When the client calls the map slice, the client sends a request to the server, the server analyzes the map slice information according to the request address, then calls a data file from a memory database or a file system, transmits the map slice file in a byte stream form through a network and returns the map slice file to the client, and finally renders the stream data into visual image data through a client rendering engine. By adopting the method, the original map slicing operation which needs to be processed discretely for multiple times is reduced to one-time block file processing on the data packet, the time for reading a disk is greatly reduced, the network transmission consumption is reduced, the map rendering time is reduced, and the map rendering performance and the user experience are improved.
Drawings
FIG. 1 is a flow chart of the present invention.
FIG. 2 is a data organization chart of a map slice according to the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
An index-free map slice aggregation method as shown in fig. 1, comprising the steps of:
step S1, separately storing the map slice files of each level according to 64 × 64 grid blocks;
as shown in fig. 2, the specific data organization is as follows:
s11, taking the map slice data set as a root directory of the map slice file data;
s12, a map slice level directory is arranged under the root directory;
the naming mode of the level directory name is as follows: "Z + level", the range of levels is: 1-n, namely Z1, Z2, Z3..
S13, obtaining a row directory of the map slice matrix of the level under the map slice level directory;
the line directory name is named as follows: "X + line number", the line number range is: 0 to [ (2n-1)/64 ].
S14, under the map slice level directory, the list of the map slice matrix at the level is obtained;
the column directory names are named in the following way: "Y + column number", column number range: 0 to [ (2n-1)/64 ]. For example, the folder 8/0/0 (level number/(line number/64)/(column number/64)), level 8, line number 0, and map slices 0_0.png to 63_63.png are stored.
S15, listing a specific map slice data file;
the naming mode of the map slice data file name is as follows: slice "Z, X, Y", i.e., "slice level, abscissa value (row number/64), ordinate value (column number/64)";
the first level directory is level Z, the first level directory is level number, and the level number is started from 1;
the second level directory is a line number X/64, starting from 0;
the third level directory is the column number Y/64, starting from 0;
and finally storing the map slices under a third-level catalog. For example, a grid slice, named X _ y.png (or X _ y.jpg), X and Y are named according to absolute position.
According to the data organization structure, a plurality of slices of the third-level directory at each level are packaged into Z _ X/64_ Y/64.dat, and the dat file is a binary file which is stored firstly according to the sequence of X and then is stored according to the sequence of Y.
Taking the data in the 8/0/0 folder as an example, the level is 8, the row number is 0, and the map slices stored in the folders are 0_0.png to 63_63. png:
first, 0_0.png is read in a binary manner and stored in a dat file, and then map slices are stored in the order of 1_0.png.
Step S2, the user sends a request to the server to obtain the map slice information;
after a user sends a request at a client, a server acquires map slice information X, Y and Z according to a request address, wherein Z is a level, X is a horizontal coordinate, and Y is a vertical coordinate.
Step S3, calculating according to the map slice information;
from the map slice information X, Y, Z, the name, address, key value of redis, and offset value of the dat file are calculated. (Note: Offset refers to Offset).
Note: redis is an open source log-type and Key-Value database which is written by using ANSI C language, supports network, can be based on memory and can also be persistent, and provides API of multiple languages. redis is a key-value storage system. Redis is a high-performance key-value database. The Key-value database is a database that stores data in Key-value pairs, like maps in Java. The entire database can be understood as a large map, with each key corresponding to a unique value.
And step S4, judging whether the corresponding key exists in the redis memory database.
And if the corresponding key exists in the redis, directly reading the value, calculating the offset and the size, reading the slice data corresponding to the X, Y and Z, and returning the byte data to the client for rendering.
If no corresponding key exists in the redis, reading data according to the file name and the path I/O, calculating offset and size, simultaneously saving the read binary file into the redis, caching common slice information in the redis, improving extraction efficiency, reading slice data corresponding to X, Y and Z in a Ceph file system according to the dat file and the calculated offset and size, and returning byte data to the client for rendering.
Note: the Ceph is a distributed file system and has the characteristics of high expansion, high availability and high performance, and can provide object storage, block storage and file system storage.
The invention improves the copy speed of the slice data, reduces the I/O times of the disk and the occupation of network bandwidth by processing the map slice block, and is convenient for the upgrade and maintenance operation of the map. Meanwhile, the loading efficiency and the rendering efficiency of the front end and the back end of the image data are improved, the image data blocking problem is eliminated, and the user experience is improved.
The above embodiments are not intended to limit the present invention, and the present invention is not limited to the above examples, and those skilled in the art may make variations, modifications, additions or substitutions within the technical scope of the present invention.
Claims (10)
1. An index-free map slice aggregation method is characterized in that: the method comprises the following steps:
step S1, separately storing the map slice files of each level according to grid blocks;
step S2, the user sends a request to the server to obtain the map slice information;
step S3, calculating according to the map slice information;
and step S4, judging whether the corresponding key exists in the redis memory database.
2. The index-free map slice aggregation method of claim 1, wherein: in step S1, the data are stored separately according to 64 × 64 grid blocks, and the specific data organization structure is as follows:
s11, taking the map slice data set as a root directory of the map slice file data;
s12, a map slice level directory is arranged under the root directory;
s13, obtaining a row directory of the map slice matrix of the level under the map slice level directory;
s14, under the map slice level directory, the list of the map slice matrix at the level is obtained;
s15, listing a specific map slice data file;
according to the data organization structure, a plurality of slices of the third-level directory at each level are packaged into Z _ X/64_ Y/64.dat, and the dat file is a binary file which is stored firstly according to the sequence of X and then is stored according to the sequence of Y.
3. The index-free map slice aggregation method of claim 2, wherein: the naming mode of the level directory name is as follows: "Z + level", the range of levels is: 1-n, namely Z1, Z2, Z3..
4. The index-free map slice aggregation method of claim 2, wherein: the naming mode of the line directory name is as follows: "X + line number", the line number range is: 0 to [ (2n-1)/64 ].
5. The index-free map slice aggregation method of claim 2, wherein: the naming mode of the column directory name is as follows: "Y + column number", column number range: 0 to [ (2n-1)/64 ].
6. The index-free map slice aggregation method of claim 2, wherein: the naming mode of the map slice data file name is as follows: slice "Z, X, Y", i.e., "slice level, abscissa value, ordinate value";
the first level directory is level Z, the first level directory is level number, and the level number is started from 1;
the second level directory is a line number X/64, starting from 0;
the third level directory is the column number Y/64, starting from 0;
and finally storing the map slices under a third-level catalog.
7. The index-free map slice aggregation method of claim 1, wherein: in step S2, after the user sends the request at the client, the server first obtains the map slice information X, Y, Z according to the request address, where Z is the level, X is the abscissa, and Y is the ordinate.
8. The index-free map slice aggregation method of claim 1, wherein: in step S3, the name, address, key value of redis, and offset value of the dat file are calculated based on the map slice information X, Y, Z.
9. The index-free map slice aggregation method of claim 1, wherein: in step S4, if there is a corresponding key in the redis, the value is directly read, the offset and size are calculated, the slice data corresponding to X, Y, Z is read, and the byte data is returned to the client for rendering.
10. The index-free map slice aggregation method of claim 1, wherein: in step S4, if there is no key corresponding to the rdis, reading data according to the file name and the path I/O, and calculating offset and size, and meanwhile saving the read binary file into the rdis, caching common slice information in the rdis, so as to improve extraction efficiency, and according to the dat file and the calculated offset and size, reading slice data corresponding to X, Y, and Z in the distributed database, and returning byte data to render on the client.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210175793.5A CN114357104A (en) | 2022-02-25 | 2022-02-25 | Non-index map slice polymerization method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210175793.5A CN114357104A (en) | 2022-02-25 | 2022-02-25 | Non-index map slice polymerization method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114357104A true CN114357104A (en) | 2022-04-15 |
Family
ID=81093009
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210175793.5A Pending CN114357104A (en) | 2022-02-25 | 2022-02-25 | Non-index map slice polymerization method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114357104A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117891961A (en) * | 2024-03-14 | 2024-04-16 | 山东省地图院 | Data cascade sharing method and system based on map product aggregation |
-
2022
- 2022-02-25 CN CN202210175793.5A patent/CN114357104A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117891961A (en) * | 2024-03-14 | 2024-04-16 | 山东省地图院 | Data cascade sharing method and system based on map product aggregation |
| CN117891961B (en) * | 2024-03-14 | 2024-05-28 | 山东省地图院 | Data cascade sharing method and system based on map product aggregation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107423422B (en) | Spatial data distributed storage and search method and system based on grid | |
| US8285757B2 (en) | File system for a storage device, methods of allocating storage, searching data and optimising performance of a storage device file system | |
| CN111930767B (en) | Multilayer cache-based vector tile real-time slicing and updating method | |
| CN106682110B (en) | Image file storage and management system and method based on Hash grid index | |
| CN113626550B (en) | Image tile map service method based on triple bidirectional index and optimized cache | |
| US20020035660A1 (en) | Functional memory based on a trie structure | |
| CN102968456B (en) | A kind of raster data reading and processing method and device | |
| EP3944556B1 (en) | Block data storage method and apparatus, and block data access method and apparatus | |
| CN112632068A (en) | Solution for rapidly providing mass data query service | |
| CN114357104A (en) | Non-index map slice polymerization method | |
| CN113806300A (en) | Data storage method, system, device, equipment and storage medium | |
| CN109597574A (en) | Distributed data storage method, server and readable storage medium storing program for executing | |
| CN103631838A (en) | Method and system for storing tabular data in a memory-efficient manner | |
| EP3767486A1 (en) | Multi-record index structure for key-value stores | |
| KR20060095444A (en) | Entity lookup system | |
| CN116756253B (en) | Data storage and query methods, devices, equipment and media of relational database | |
| CN113849478A (en) | Cloud native big data analysis engine | |
| CN117573676A (en) | Address processing method and device based on storage system, storage system and medium | |
| CN115658693A (en) | Efficient optimized storage method suitable for massive raster tile data | |
| CN111460012A (en) | Spark-based meteorological historical station leather-following data visualization method and system | |
| CN114185934B (en) | Indexing and query method and system based on Tiandun database column storage | |
| CN116414935A (en) | Method for distributed Search space vector data based on Elastic Search | |
| CN115934794A (en) | Elastic management method for mass multi-source heterogeneous remote sensing space data query | |
| CN108021562B (en) | Disk storage method and device applied to distributed file system and distributed file system | |
| CN114996270A (en) | Method and device for inquiring paging data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |