CN113495936A - Multi-format map tile generation method and system - Google Patents

Abstract

The invention discloses a multi-format map tile generation method and a multi-format map tile generation system based on a distributed system, wherein the method comprises the steps of receiving a slicing task request, and dividing and distributing subtasks of the slicing task; task parameters in the slicing task request comprise a preset tile format and a preset storage format; and performing distributed parallel processing on the slice subtasks, generating corresponding map tiles according to the tile format, and storing the map tiles according to the storage format. By applying the scheme of the invention, the slicing task of the user can be self-adapted in various projections, various data formats, various output forms and the like, corresponding resources in the distributed system are reasonably called to quickly process data, and the method has extremely high benefits for improving the expansibility of the system and the economic benefit ratio of the system.

Description

Multi-format map tile generation method and system

[ technical field ] A method for producing a semiconductor device

The invention relates to a computer application technology, in particular to a multi-format map tile generation method and a multi-format map tile generation system.

[ background of the invention ]

With the development of mapping technology, the volume of map data is larger and more frequent, and the method is mainly used for solving the problems of difficult dynamic update and low production speed of map tiles.

A map tile is an optimization strategy for improving map browsing user experience, and comprises a series of map slice files with a scale and in a certain map range. Map tiles are subjected to unique identification through levels and row-column numbers according to the ancestors of the pyramid structure. With the development of mapping technology, the volume of map data is larger and larger, and the map is updated more and more frequently.

Generally, electronic map generation map tiles (hereinafter referred to as cut maps) require several hours for a few times and several days or even longer for a many times, and thus cannot meet actual production requirements. How to generate map tiles quickly is a common concern for many technicians.

In addition, with the acceleration of the map updating frequency, the demand for updating the local area of the map is more urgent, and more formats for outputting map tiles are provided, such as hash files, mbtiles, hbases and the like; the storage formats of the map tiles are also different in requirements, such as png (png), jpg (jpg), tif (tif), a mixed mode and the like, and by extracting various requirements, the speed of tile production is improved, and meanwhile different storage formats need to be flexibly adapted.

[ summary of the invention ]

Aspects of the present application provide a multi-format map tile generation method, system, device, and storage medium.

One aspect of the application provides a multi-format map tile generation method, which includes receiving a slicing task request, and performing subtask division and distribution on the slicing task; task parameters in the slicing task request comprise a preset tile format and a preset storage format; and performing distributed parallel processing on the slice subtasks, generating corresponding map tiles according to the tile format, and storing the map tiles according to the storage format.

The above-described aspect and any possible implementation further provide an implementation, where the task parameters in the slice task request further include: data range, slice level.

The foregoing aspects and any possible implementations further provide an implementation in which dividing and distributing the sub-tasks for the slicing task includes calculating map tiles that need to be divided into sub-tasks according to the data range and the slicing level; and calculating the number of the slicing subtasks according to the map tiles needing to be divided into the subtasks, and distributing the slicing subtasks.

The foregoing aspects and any possible implementation manners further provide an implementation manner, where performing distributed parallel processing on the slice subtasks, generating corresponding map tiles according to the tile formats and storing the map tiles according to the storage formats includes setting subtask parameters for the slice subtasks, where the subtask parameters include a data path to be processed, a slice output format, a slice storage manner, a slice processing geographical range, a slice processing hierarchy range, a slice output coordinate system, a slice resampling manner, a background value to be processed by a slice, whether a slice is to be dynamically updated, and the dynamic updating includes merging of boundaries of different areas, and merging of new areas and old areas of the same area.

The foregoing aspects and any possible implementation manners further provide an implementation manner, where performing distributed parallel processing on the slice subtasks, generating corresponding map tiles according to the tile format, and storing the map tiles according to the storage format further includes determining whether the map tiles in the subtasks already exist; if the map tiles in the subtasks do not exist in the map tile database, executing the subtasks to generate corresponding map tiles; if the map tiles in the subtasks exist in the map tile database, judging whether the tiles need to be updated, and if the tiles do not need to be updated, ending the slicing subtask; if an update is required, the subtasks are executed to generate the corresponding map tile.

The foregoing aspects and any possible implementation manners further provide an implementation manner, where performing distributed parallel processing on the slice subtasks, generating corresponding map tiles according to the tile format, storing the map tiles according to the storage format, and acquiring data to be processed from the distributed file system according to a data path to be processed and a number range of the tiles; unifying the coordinate system of the data to be processed and the slice output coordinate system in the subtask parameters; generating tile data according to a slice output format in the subtask parameters; storing the tile data.

The above-described aspect and any possible implementation further provide an implementation, wherein generating tile data according to the slice output format in the subtask parameter further includes performing dynamic update and background processing on the tile data.

In another aspect of the present invention, a multi-format map tile generating system is provided, which includes a subtask division and distribution module, configured to receive a slicing task request, and perform subtask division and distribution on the slicing task; task parameters in the slicing task request comprise a preset tile format and a preset storage format; and the subtask processing module is used for performing distributed parallel processing on the slice subtasks, generating corresponding map tiles according to the tile format and storing the map tiles according to the storage format.

In another aspect of the present invention, a computer device is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method as described above when executing the program.

In another aspect of the invention, a computer-readable storage medium is provided, on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the method as set forth above.

Based on the introduction, the scheme of the invention can carry out self-adaptation through carrying out various projections, various data formats, various output forms and the like on the slicing task of the user, reasonably call corresponding resources in a distributed system to carry out data processing quickly, and has extremely high benefits for improving the expansibility of the system and the economic benefit ratio of the system.

[ description of the drawings ]

FIG. 1 is a flow chart of a multi-format map tile generation method according to the present invention;

FIG. 2 is a block diagram of a multi-format map tile generation system in accordance with the present invention;

fig. 3 illustrates a block diagram of an exemplary computer system/server 012 suitable for use in implementing embodiments of the invention.

[ detailed description ] embodiments

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Fig. 1 is a flowchart of an embodiment of a multi-format map tile generation method according to the present invention, as shown in fig. 1, including the following steps:

step S11, receiving a slicing task request, and dividing and distributing subtasks of the slicing task; task parameters in the slicing task request comprise a preset tile format and a preset storage format;

and step S12, performing distributed parallel processing on the slice subtasks, generating corresponding map tiles according to the tile format and storing the map tiles according to the storage format.

Preferably, the execution subject of the method is a distributed system, and the distributed system includes a Web gateway, a control center, a distributed processing system, and a distributed file system. Preferably, the distributed literature system is an HDFS file system, and is implemented by performing reading and writing on the HDFS file system, and performing image blocking, hierarchical parallel slicing, and merging of the same tiles. The parallel computing framework of the distributed processing system adopts a MapReduce parallel framework or a spark parallel framework of Hadoop. The embodiment adopts a MapReduce parallel framework of Hadoop.

In one preferred implementation of step S11,

preferably, the control center of the distributed system receives a slicing task request sent by a user through a Web gateway; and dividing and distributing subtasks of the slicing task.

Preferably, the slicing task request carries task parameters of the slicing task set by the user, and the distributed system verifies the validity of the task parameters; and after the verification is passed, dividing the sub-tasks of the slices according to the task parameters.

Preferably, the task parameters include data ranges Xmin, Ymin, Xmax, Ymax of the input data, slice levels, and the like.

Preferably, the task parameters further include subtask parameters, and the subtask parameters include: the method comprises the steps of a data path to be processed, a slice output format, a slice storage mode, a slice processing geographical range, a slice processing level range, a slice output coordinate system, a slice resampling mode, a background value to be processed of a slice, whether the slice needs to be dynamically updated or not, and the dynamic updating comprises different area boundary merging, same area new-old merging and the like.

Although a geodetic coordinate system which represents positions by latitude and longitude can describe positions of points on the earth, for a scene in which map geographic data are displayed in a two-dimensional plane, points in a three-dimensional space need to be mapped into the two-dimensional space by means of projection. Map projection requires establishing a one-to-one correspondence between earth surface points and projection plane points, and mercator projection is often used in internet maps. Mercator projection is an earth projection method proposed by dutch geologist mercator in 1569, which is one of cylindrical projections. It should be noted, however, that the mercator projection is not a coordinate system, but rather a spatial mapping performed to represent the three-dimensional earth in a two-dimensional plane. Therefore, in the GIS map and the internet map, although the map viewed by the user is subjected to mercator projection, the longitude and latitude coordinates are still used to represent the position of a point on the earth, and map data needs to be presented in a projection manner when the map is drawn and visualized. For a world map projected as a plane through the mercator, the world map is divided into map units with the pixels of 256X256 by means of cutting under different map resolutions (pixel sizes of the whole world map), namely, at a slicing level, and each divided map unit is called a map tile.

The tile pyramid is used for layering data, and the resolution of adjacent layers is 2 times of the resolution; each layer is divided into tiles, and any adjacent 4 tiles in the layer with the smaller resolution form a tile matrix of 2 x2, and the position of the tile in the layer with the same geographic range in the layer with the larger resolution is exactly one tile in the layer.

Preferably, the distributed system divides the slice subtasks according to the task parameters in the slice task request.

Preferably, the number ranges TileMinX, TileMinY, TileMaxX, TileMaxY of the map tiles to be output are calculated according to the data range of the input data included in the task parameters, the slice level.

For example, in mercator projection, TileMinX ═ (int) (math. ceil ((Xmin + pi 6378173)/(2 pi 6378173/256/2)^z) 256-1)); where z is the slice level and 6378173 is the earth's equatorial radius.

And respectively calculating TileMinX, TileMinY, TileMaxX and TileMaxY to obtain the number range of the map tiles to be output. And the map tiles in the numbering range are the map tiles needing dividing the molecular task.

Preferably, the number of the slicing subtasks is calculated according to the map tiles needing to be divided into the subtasks, and the slicing subtasks are distributed.

Preferably, the map tiles are divided into slice subtasks, that is, a slice subtask is established for a logical partition composed of one or more map tiles to process. The tiles output by each logical partition are different, and each partition corresponds to one subtask. Preferably, a subtask identity is set for each subtask.

Preferably, the distributed system performs distributed processing on the slice subtasks.

Preferably, the subtasks are allocated to the processing machines in the distributed system for processing according to the resource configuration of the processing machines in the distributed system. The number m of the subtasks which are simultaneously performed by the distributed system is related to the number k of the computing nodes and the number d of the cpu cores of each computing node, and m is k x d.

Preferably, the subtask identifier is submitted to a distributed processing system, so that the distributed processing system allocates a processor for the subtask identifier to process.

MapReduce in Hadoop is a high-performance parallel computing framework based on a cluster, can automatically complete parallel processing of computing tasks, is one of a divide-and-conquer method, and can enable subtasks to independently run on a distributed processing system by using the MapReduce framework to achieve the effect of parallel processing of each subtask.

In one preferred implementation of step S12,

and performing distributed processing on the sub-tasks to generate corresponding map tiles.

Preferably, after each processor in the distributed processing system receives the distribution task of the control center, the processing of the subtask is performed, and the processing includes the following substeps:

substep S121, setting subtask parameters for the subtasks;

preferably, the subtask parameters include: the method comprises the steps of a data path to be processed, a slice output format, a slice storage mode, a slice processing geographical range, a slice processing level range, a slice output coordinate system, a slice resampling mode, a background value to be processed of a slice, whether the slice needs to be dynamically updated or not, and the dynamic updating comprises different area boundary merging, same area new-old merging and the like.

Substep S122, judging whether the map tile in the subtask already exists;

preferably, a map tile database storing generated map tiles is queried according to the number of the map tiles in the subtask to determine whether the map tiles in the subtask exist. The map tile data is stored with the generated map tiles, and the map tiles are stored in association with the numbers of the map tiles so as to be inquired.

And if the map tiles in the subtasks do not exist in the map tile database, executing the subtasks so as to generate corresponding map tiles.

If the tile is found to exist, judging whether the tile needs to be updated, and if the tile does not need to be updated, ending the slicing subtask; if an update is required, the subtasks are executed to generate the corresponding map tile.

S123, acquiring data to be processed from the distributed file system according to the data path to be processed in the subtask parameter and the number range of the tiles;

and acquiring source data corresponding to the map tiles of the subtasks to be divided.

Preferably, the source data is data obtained by means of aerial survey, satellite observation and the like, and is a tiff format file.

And a substep S124, unifying the coordinate system of the data to be processed and the slice output coordinate system in the subtask parameters.

Preferably, if the coordinate system of the data to be processed is not consistent with the slice output coordinate system, projection conversion is performed. The tile can be subjected to WGS84 longitude and latitude projection conversion, WGS84 Web mercator projection conversion, GCJ02 Web mercator projection conversion and BD09 Web mercator projection conversion, and can be applied to various different scenes to be flexibly called by various APIs.

And a substep S125 of generating tile data according to the slice output format in the subtask parameters.

After the data to be sliced are prepared, a corresponding slicing strategy device is automatically constructed through the output slicing format and the storage form, and corresponding tile data is generated through the slicing strategy device.

And a substep S126 of dynamically updating the tile data.

Preferably, whether dynamic update is needed is judged according to whether dynamic update parameters of the slices in the subtask parameters need to be performed.

If dynamic updating is needed, acquiring the existing slice, and dynamically synthesizing the slice according to the generated tile data and the existing tile;

if not, then the background process is direct.

Substep S128, performing background processing on the tile data.

Preferably, the tile data is subjected to background processing according to a background value parameter to be processed by the slice in the subtask parameters.

And a substep S137 of storing the tile data.

Preferably, the tile data is stored according to a slice storage manner in the subtask parameter.

By applying the scheme of the invention, the self-adaptation is carried out on the slicing tasks of the user in various projections, various data formats, various output forms and the like, and the corresponding resources in the distributed system are reasonably called to carry out the data processing quickly, so that the method has extremely high benefits on improving the expansibility of the system and the economic benefit ratio of the system.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.

The above is a description of method embodiments, and the embodiments of the present invention are further described below by way of apparatus embodiments.

FIG. 2 is a flowchart of an embodiment of the multi-format map tile generation system of the present invention, as shown in FIG. 2, including:

the subtask division and distribution module 21 is configured to receive a slice task request, and perform subtask division and distribution on the slice task; task parameters in the slicing task request comprise a preset tile format and a preset storage format;

and the subtask processing module 22 is configured to perform distributed parallel processing on the slice subtasks, generate corresponding map tiles according to the tile format, and store the map tiles in the distributed storage system 23 according to the storage format.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the terminal and the server described above may refer to corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processor, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the described module may refer to the corresponding process in the foregoing method embodiment, and is not described herein again.

FIG. 3 shows a schematic block diagram of an electronic device 300 that may be used to implement embodiments of the present disclosure. Apparatus 300 may be used to implement the multi-format map tile generation system of FIG. 2. As shown, device 300 includes a Central Processing Unit (CPU)301 that may perform various appropriate actions and processes in accordance with computer program instructions stored in a Read Only Memory (ROM)302 or loaded from a storage unit 308 into a Random Access Memory (RAM) 303. In the RAM 303, various programs and data necessary for the operation of the device 300 can also be stored. The CPU 301, ROM 302, and RAM 303 are connected to each other via a bus 304. An input/output (I/O) interface 305 is also connected to bus 304.

Various components in device 300 are connected to I/O interface 305, including: an input unit 306 such as a keyboard, a mouse, or the like; an output unit 307 such as various types of displays, speakers, and the like; a storage unit 308 such as a magnetic disk, optical disk, or the like; and a communication unit 309 such as a network card, modem, wireless communication transceiver, etc. The communication unit 309 allows the device 300 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The processing unit 301 performs the various methods and processes described above. For example, in some embodiments, the method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 300 via ROM 302 and/or communication unit 309. When the computer program is loaded into RAM 303 and executed by CPU 301, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the CPU 301 may be configured to perform the method by any other suitable means (e.g., by way of firmware).

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a load programmable logic device (CPLD), and the like.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Further, while operations are depicted in a particular order, this should be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims (10)

1. A multi-format map tile generation method is characterized by comprising the following steps:

receiving a slicing task request, and dividing and distributing subtasks of the slicing task; task parameters in the slicing task request comprise a preset tile format and a preset storage format;

and performing distributed parallel processing on the slice subtasks, generating corresponding map tiles according to the tile format, and storing the map tiles according to the storage format.

2. The method of claim 1,

the task parameters in the slice task request further include: data range, slice level.

3. The method of claim 2, wherein sub-task partitioning and distributing the slicing task comprises:

calculating map tiles needing dividing the molecular tasks according to the data range and the slicing level;

and calculating the number of the slicing subtasks according to the map tiles needing to be divided into the subtasks, and distributing the slicing subtasks.

4. The method of claim 3, wherein performing distributed parallel processing on the tile subtasks, generating corresponding map tiles according to the tile format and storing in the storage format comprises:

setting subtask parameters for the slice subtasks, wherein the subtask parameters comprise a data path to be processed, a slice output format, a slice storage mode, a slice processing geographical range, a slice processing level range, a slice output coordinate system, a slice resampling mode, a background value to be processed by the slices, whether the slices need to be dynamically updated or not, and the dynamic updating comprises merging of boundaries of different areas and merging of new areas and old areas.

5. The method of claim 4, wherein performing distributed parallel processing on the tile subtasks, generating corresponding map tiles according to the tile format and storing in the storage format further comprises:

judging whether the map tiles in the subtasks exist or not;

if the map tiles in the subtasks do not exist in the map tile database, executing the subtasks to generate corresponding map tiles;

if the map tiles in the subtasks exist in the map tile database, judging whether the tiles need to be updated, and if the tiles do not need to be updated, ending the slicing subtask; if an update is required, the subtasks are executed to generate the corresponding map tile.

6. The method of claim 5, wherein performing distributed parallel processing on the tile subtasks, generating corresponding map tiles according to the tile format and storing in the storage format further comprises:

acquiring data to be processed from the distributed file system according to the data path to be processed and the number range of the tiles;

unifying the coordinate system of the data to be processed and the slice output coordinate system in the subtask parameters;

generating tile data according to a slice output format in the subtask parameters;

storing the tile data.

7. The method of claim 6, wherein generating tile data according to the slice output format in the subtask parameters further comprises:

and dynamically updating and background processing the tile data.

8. A multi-format map tile generation system, comprising:

the subtask division and distribution module is used for receiving a slicing task request and dividing and distributing the subtask of the slicing task; task parameters in the slicing task request comprise a preset tile format and a preset storage format;

and the subtask processing module is used for performing distributed parallel processing on the slice subtasks, generating corresponding map tiles according to the tile format and storing the map tiles according to the storage format.

9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the method according to any one of claims 1 to 7.

Images