CN116755627A

CN116755627A - Spatial data storage method, spatial data storage device, computer equipment and storage medium

Info

Publication number: CN116755627A
Application number: CN202310746382.1A
Authority: CN
Inventors: 王旭; 陈志金; 蒋宏伟; 李一樊
Original assignee: Industrial and Commercial Bank of China Ltd ICBC
Current assignee: Industrial and Commercial Bank of China Ltd ICBC
Priority date: 2023-06-25
Filing date: 2023-06-25
Publication date: 2023-09-15

Abstract

The present application relates to a spatial data storage method, apparatus, computer device, storage medium and computer program product. The method relates to the technical field of computers, can be used in the field of financial science and technology or other related fields, and comprises the following steps: acquiring space data monitored by remote sensing equipment; the remote sensing equipment is in communication connection with the financial system; according to longitude and latitude coordinates of the space data, primarily dividing the space data, and dividing the space data in the same longitude and latitude coordinate range into the same grid to obtain a plurality of corresponding grid data; traversing the grid data according to the coordinate proximity relation among the grid data, if the data size of the current grid data is larger than the storable data size of the current storage node, performing at least one subdivision on the current grid data, and storing the sub-data obtained by the subdivision to the current storage node or the next storage node. By adopting the method, the spatial data can be stored in a distributed mode through data division with high efficiency.

Description

Spatial data storage method, spatial data storage device, computer equipment and storage medium

Technical Field

The present application relates to the field of computer technology, and may be used in the field of financial technology or other related fields, and in particular, to a spatial data storage method, apparatus, computer device, storage medium, and computer program product.

Background

In recent years, the application of spatial data is mature gradually, a banking financial system can analyze the farmland growth condition through remote sensing data, evaluate the crop planting area, the crop growth condition and the crop yield, and take the evaluation result as auxiliary decision information of agricultural related financial products. The space data has the characteristic of large volume, and the remote sensing image or the track data of the vehicle and the like often reach the GB level, so that the distributed storage becomes a necessary choice for storing massive space data.

Traditional data partitioning mainly considers the integrity of data and the balance of node loads. Data integrity is a basic requirement of data partitioning, and ensures that data is correct without loss. The existing slicing mode generally encodes data and distributes the data to each node in a balanced manner through a polling or hash algorithm mode. However, the conventional data dividing method only uses latitude and longitude information and a dichotomy to divide the space data, and has the problem that the efficiency of calling the space sub-data after the space data is divided is low.

Therefore, how to efficiently store spatial data in a distributed manner through data partitioning and improve the input or output efficiency of the spatial data is a technical problem to be solved.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a spatial data storage method, apparatus, computer device, storage medium, and computer program product that can improve storage efficiency.

In a first aspect, the present application provides a method of spatial data storage. The method comprises the following steps:

acquiring space data monitored by remote sensing equipment for monitoring crops; the remote sensing equipment is in communication connection with the financial system;

according to longitude and latitude coordinates of the space data, primarily dividing the space data, dividing the space data in the same longitude and latitude coordinate range into the same grid, and obtaining a plurality of grid data corresponding to the space data;

traversing the grid data according to the coordinate proximity relation among the grid data, and if the data size of the current grid data is larger than the storable data size of the current storage node, performing at least one subdivision on the current grid data, and storing the sub-data obtained by the subdivision to the current storage node or the next storage node;

Wherein the subdividing employs at least one of a spatial coding model, a temporal coding model, and a semantic coding model, and the spatial coding model, the temporal coding model, and the semantic coding model are configured to different priorities;

and updating the data volume which can be stored by the current storage node and the next storage node after each time of data storage is executed.

In one embodiment, the preliminary dividing the spatial data according to the longitude and latitude coordinates of the spatial data, dividing the spatial data belonging to the same longitude and latitude coordinate range into the same grid, and obtaining a plurality of grid data corresponding to the spatial data includes:

coding longitude and latitude coordinates corresponding to the data elements of the space data according to a preset coding bit number by using a spatial coding method based on a dichotomy to obtain coordinate coding values corresponding to the data elements; wherein the data element is basic unit data of the spatial data;

and merging the data elements corresponding to the same coordinate coding values into one grid data to obtain a plurality of grid data.

In one embodiment, the spatial coding model has the highest priority; the spatial proximity of the division result of the spatial coding model is higher than that of the preliminary division result;

If the data size of the current grid data is larger than the storable data size of the current storage node, performing at least one subdivision on the current grid data, and storing the sub-data obtained by the subdivision to the current storage node or the next storage node, including:

acquiring a space coding model with highest priority, and dividing the current grid data into a corresponding number of sub data based on the space coding model;

traversing the sub-data to be stored by taking the divided sub-data as the sub-data to be stored, storing the current sub-data to be stored into the current storage node if the data amount of the traversed current sub-data to be stored is smaller than or equal to the storable data amount of the current storage node, and determining whether to sub-divide the current sub-data to be stored if the data amount of the traversed current sub-data to be stored is larger than the storable data amount of the current storage node;

under the condition that the sub-division is determined, a division model of priority order is obtained, the current sub-data to be stored is divided again by adopting the division model of priority order, the sub-data to be stored is updated by a plurality of sub-data obtained by division, and the step of traversing the sub-data to be stored is carried out again until the fact that the sub-data to be stored is not divided is determined, and the current sub-data to be stored is stored to a next storage node;

Wherein, the conditions for carrying out the subdivision are as follows: the sub data to be stored currently comprises a plurality of data elements, and the storable data quantity of the current storage node is not lower than a preset value.

In one embodiment, the traversing the sub-data to be stored includes:

obtaining a target division model adopted by the sub-data to be stored;

traversing the sub-data to be stored according to the coding sequence corresponding to the target division model;

the coding sequence of the space coding model in the partitioning model is determined by the space adjacent relation among a plurality of sub-data obtained by partitioning; the coding sequence of the time coding model in the partitioning model is determined by the time relation among a plurality of sub-data obtained by partitioning; the coding order of the semantic coding model in the partition model is determined by the semantic relationship between the plurality of sub-data obtained by the partitioning.

In one embodiment, the acquiring the spatial coding model with the highest priority, dividing the current grid data into a corresponding number of sub-data based on the spatial coding model includes:

dividing the current grid data into a number of sub-data corresponding to the coding orders according to preset coding orders by using a spatial coding method based on a Hilbert curve;

And connecting and encoding the sub data with the number corresponding to the encoding orders according to the encoding sequence of the Hilbert curve to obtain a plurality of divided sub data.

In one embodiment, the obtaining the partition model of the priority, and using the partition model of the priority to sub-partition the sub-data to be stored currently includes:

under the condition that the priority division model is a time coding model, acquiring the sub-data of the same time period in the sub-data to be stored currently, and obtaining the sub-data of a plurality of time periods;

and coding the sub-data of a plurality of time periods according to the time sequence to obtain a plurality of sub-data divided by the sub-data to be stored currently.

under the condition that the priority division model is a semantic coding model, acquiring sub-data with the same semantic in the current sub-data to be stored to obtain sub-data with multiple semantic;

and encoding the sub-data with multiple semantics according to the semantic encoding model to obtain multiple sub-data after the sub-data to be stored is divided.

In one embodiment, in the case of determining to perform the sub-division, the obtaining a partition model of the priority order, and using the partition model of the priority order to sub-divide the sub-data to be stored currently includes:

and under the condition that the sub-division is determined to be performed, if the division model with the lowest priority is adopted, acquiring the space coding model, and adopting the space coding model to sub-divide the sub-data to be stored currently.

In one embodiment, after traversing the plurality of grid data according to the coordinate proximity relationship between the plurality of grid data, the method further includes:

and if the data volume of the current grid data is smaller than or equal to the storable data volume of the current storage node, storing the current grid data to the current storage node.

In one embodiment, before traversing the plurality of grid data according to the coordinate proximity relationship between the plurality of grid data, the method further includes:

acquiring the quotient of the total data amount of the space data and the number of the storage nodes;

an initial storable amount of data for each of the storage nodes is determined based on the quotient.

In a second aspect, the present application also provides a spatial data storage device. The device comprises:

The data acquisition module is used for acquiring the space data monitored by the remote sensing equipment for monitoring the crops; the remote sensing equipment is in communication connection with the financial system;

the preliminary division module is used for carrying out preliminary division on the space data according to the longitude and latitude coordinates of the space data, dividing the space data in the same longitude and latitude coordinate range into the same grid, and obtaining a plurality of grid data corresponding to the space data;

the subdivision module is used for traversing the grid data according to the coordinate proximity relation among the grid data, and if the data size of the current grid data is larger than the storable data size of the current storage node, the current grid data is subdivided at least once, and the sub data obtained by division is stored in the current storage node or the next storage node;

In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the steps of the method according to the first aspect when the processor executes the computer program.

In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method according to the first aspect.

In a fifth aspect, the present application also provides a computer program product. The computer program product comprising a computer program which, when executed by a processor, implements the steps of the method according to the first aspect.

According to the space data storage method, the device, the computer equipment, the storage medium and the computer program product, space data monitored by remote sensing equipment for monitoring crops are obtained, the space data are initially divided according to longitude and latitude coordinates of the space data, and the space data in the same longitude and latitude coordinate range are divided into the same grid to obtain a plurality of grid data corresponding to the space data; traversing the grid data according to the coordinate proximity relation among the grid data, and if the data size of the current grid data is larger than the storable data size of the current storage node, performing at least one subdivision on the current grid data, and storing the sub-data obtained by the subdivision to the current storage node or the next storage node; wherein the subdividing employs at least one of a spatial coding model, a temporal coding model, and a semantic coding model, and the spatial coding model, the temporal coding model, and the semantic coding model are configured to different priorities; and updating the data volume which can be stored by the current storage node and the next storage node after each time of data storage is executed. It can be known that the space data with large data volume can be primarily divided through the longitude and latitude coordinate ranges of the space data, and the space data can be quickly primarily divided through the coordinate range division mode. And traversing according to the coordinate proximity relation between the primarily divided grid data, and storing the traversed grid data in the nodes or performing sub-division processing. Because the space data is primarily divided, the data volume of the space data subjected to the sub-division processing is smaller, so that a divided result can be obtained more quickly, and the data division efficiency is improved. At this time, the models used for the subdivision are spatial coding models, temporal coding models and semantic coding models with different priorities, the spatial coding models have better spatial adjacency, the temporal coding models have temporal adjacency, and the semantic coding models have semantic adjacency, so that the data divided by the models have higher adjacency. When the grid data is subdivided, the grid data is correspondingly divided into multiple different division models according to preset priority and division rules, so that the data is divided into multiple stages. After data division, the space data with higher coordinate adjacency, space adjacency, time adjacency and semantic adjacency are stored in the same node, if part of the space data is read, for example, farmland space data of wheat is stored or read, because the data with high adjacency is stored in one node during data division, most of the data of the wheat only need to be read from one node, and farmland space data of the wheat does not need to be read from a plurality of nodes, thereby reducing the probability of reading the data from different nodes, and finally improving the input or output efficiency of the space data.

Drawings

FIG. 1 is a diagram of an application environment for a spatial data storage method in one embodiment;

FIG. 2 is a flow chart of a method of spatial data storage according to one embodiment;

FIG. 3 is a flow chart illustrating the grid data repartitioning step in one embodiment;

FIG. 4 is a block diagram of the architecture of a spatial data store in one embodiment;

fig. 5 is an internal structural diagram of a computer device in one embodiment.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

The spatial data storage method provided by the embodiment of the application can be applied to an application environment shown in figure 1. The remote sensing device 102 communicates with the server 104 through a network, and the remote sensing device 102 sends spatial data monitored by the remote sensing device for monitoring crops to the server 104. The data storage system may store data that the server 104 needs to process, where the server 104 performs division encoding on the spatial data, and stores the divided sub-data in the data storage system for further storage. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The remote sensing device 102 may be, but is not limited to, various remote sensing satellites, lidar, aerial photography digital cameras, imaging spectrometers, personal computers, notebook computers, smart phones, tablet computers, internet of things devices, and portable wearable devices. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.

In one embodiment, as shown in fig. 2, a spatial data storage method is provided, and the method is applied to the server 104 in fig. 1 for illustration, and includes the following steps:

s202, acquiring space data monitored by remote sensing equipment for monitoring crops; the remote sensing device is in communication connection with the financial system.

The space data is also called geometric data, which is used for representing information of various aspects such as position, form, size distribution and the like of an object, is quantitative description of things and phenomena with positioning significance existing in the current world, and is data representing the natural world by using basic space data structures such as points, lines, planes, entities and the like.

Remote sensing refers to remote sensing, which generally refers to any contactless remote detection, including detection of electromagnetic fields, force fields, mechanical waves (acoustic waves, seismic waves), and the like. The remote sensing equipment is to use modern vehicles and sensors to remotely acquire the characteristics of a target object, and the target object is identified through the characteristic information, and common remote sensing equipment comprises remote sensing satellites, laser radars, aerial photography digital cameras, imaging spectrometers and the like.

It should be understood that the remote sensing device can acquire the spatial data corresponding to the crops through the remote sensing technology, the remote sensing device transmits the acquired spatial data to the corresponding system through a network or physical connection, the connected system can be different according to different requirements, in the embodiment of the application, the financial system is adopted to be in communication connection with the remote sensing device, the spatial data shot by the remote sensing device can be received in real time through the connection of the financial system, and the spatial data is stored in a database of the financial system.

S204, according to longitude and latitude coordinates of the space data, primarily dividing the space data, and dividing the space data in the same longitude and latitude coordinate range into the same grid to obtain a plurality of grid data corresponding to the space data.

The spatial data generally comprises longitude and latitude information corresponding to the data, and the position information of crops is represented by longitude and latitude coordinates, for example, the longitude and latitude coordinates of a bit corresponding to a certain land boundary point are obtained, and the area corresponding to the land is obtained by connecting the coordinates. On the basis, the space data can be initially divided, and one space data is divided into a plurality of sub-data.

The primary division aims to divide the whole space data rapidly, take longitude and latitude coordinates as position information, further limit the division range and obtain a plurality of divided grid data rapidly. Different dividing orders can divide different amounts of grid data, the higher the dividing order is, the more the amount of the grid data is, the lower the dividing order is, and the amount of the grid data is also small.

It should be understood that the whole space data may be initially divided by a preset latitude and longitude coordinate range, for example, the partial coordinates of the space data are (0, 0), (0, 10), (10, 10) and (10, 0), and the length and width of the piece of space data are all 10, if the coordinate range is set to be 5, the piece of space data may be divided into 4 pieces of sub-data of 5*5 according to the set range, so as to obtain four grids, and the coordinate range of each grid is 5. It can be known that the division method is to divide the spatial data into one sub-data based on the spatial proximity relation corresponding to the spatial data, and further obtain a plurality of sub-data.

Alternatively, the coordinates of the position information representing the spatial data may be latitude and longitude coordinates, plane coordinates, three-dimensional coordinates, or the like.

S206, traversing the grid data according to the coordinate proximity relation among the grid data, if the data volume of the current grid data is larger than the storable data volume of the current storage node, performing at least one subdivision on the current grid data, and storing the sub-data obtained by the subdivision to the current storage node or the next storage node;

It should be understood that a certain coordinate proximity relationship exists between the preliminarily divided plurality of grid data, the coordinate proximity relationship can represent a positional relationship between the plurality of grid data, grid data with close spatial distances have a closer positional relationship, grid data with far spatial distances have a farther positional relationship, and the positional relationship between the plurality of grid data can be obtained according to the coordinate proximity relationship, so that the division order between the plurality of grid data is obtained. For example, the preliminary division divides the space data into 4*4, 16 pieces of mesh data, and at this time, it is necessary to determine the division order of the four pieces of mesh data, whether the division order of the plurality of pieces of mesh data is determined in terms of top-bottom, left-right, or clockwise or counterclockwise.

When traversing the plurality of grid data, if the data size of the current grid data is larger than the storable data size of the current storage node, the data size of the current grid data needs to be further divided, and the number of the further divisions can be one time or multiple times. The model divided at this time is different from the model corresponding to the preliminary division, and the model divided again is a model divided by at least one of a spatial coding model, a temporal coding model and a semantic coding model.

The space coding model is used for carrying out space information coding on the current grid data, and carrying out sub-division on the current grid data to obtain a plurality of space sub-data, wherein the space adjacency of the space coding model is higher than that of the preliminary division; the time coding model is used for carrying out time information coding on current grid data, and carrying out sub-division on the current grid data to obtain a plurality of time sub-data, wherein the time sub-data divided by the time coding model has time adjacency, and the principle is that grid data with similar time are divided together; the semantic coding model is used for carrying out semantic information coding on current grid data, and carrying out subdivision on the current grid data to obtain a plurality of semantic sub-data, wherein the semantic sub-data divided by the semantic coding model has semantic adjacency, and the principle is that grid data with similar semantics are divided together.

It should be appreciated that there are different priorities for the spatial coding model, the temporal coding model and the semantic coding model, and that it is necessary to determine what coding model to partition first when performing multiple subdivisions, and it is generally controlled according to the priorities what coding model to use for the first, second and third times, respectively. Therefore, different priorities need to be set for the spatial coding model, the temporal coding model, and the semantic coding model to make clear how to invoke these models for multi-level partitioning. For example, when the spatial coding model is set to be of the first priority, the temporal coding model is set to be of the second priority, and the semantic coding model is set to be of the third priority, the spatial coding model is adopted in the first subdivision, the temporal coding model is adopted in the second subdivision, and the semantic coding model is adopted in the third subdivision, so that sub-data divided three times is finally obtained.

It should be understood that when the divided grid data or the sub-data divided multiple times satisfies the storage condition, the data should be stored into the current storage node, and thus sub-data having higher spatial proximity, temporal proximity, and semantic proximity can be stored in the same storage node. And after each sub-data is stored, the storable data amount in the current storage node should be updated so that the next sub-data can be conveniently judged whether to be stored in the current storage node. If the current storage node is full of sub-data or grid data, the next storage node needs to be stored as a new current storage node until the whole divided space data is stored.

Alternatively, the result of each encoding is retained when performing the preliminary division or the re-division, and the new encoded value is connected with the previous encoded value when performing the re-division, so as to obtain the unique encoded value corresponding to the data element. The code value records the division result of multiple times at this time, so that the division result of the data element can be judged directly by reading the code value, and the effects of quick storage and reading are achieved.

According to the space data storage method, the space data with large data volume can be initially divided through the longitude and latitude coordinate range of the space data, and the space data can be rapidly initially divided through the coordinate range division mode. And traversing according to the coordinate proximity relation between the primarily divided grid data, and storing the traversed grid data in the nodes or performing sub-division processing. Because the space data is primarily divided, the data volume of the space data subjected to the sub-division processing is smaller, so that a divided result can be obtained more quickly, and the data division efficiency is improved. At this time, the models used for the subdivision are spatial coding models, temporal coding models and semantic coding models with different priorities, the spatial coding models have better spatial adjacency, the temporal coding models have temporal adjacency, and the semantic coding models have semantic adjacency, so that the data divided by the models have higher adjacency. When the grid data is subdivided, the grid data is correspondingly divided into multiple different division models according to preset priority and division rules, so that the data is divided into multiple stages. After data division, the space data with higher coordinate adjacency, space adjacency, time adjacency and semantic adjacency are stored in the same node, if the space data of the farmland are read, the farmland data with higher adjacency are stored in one node, so that most of the farmland data only need to be read from one node, and farmland space data do not need to be read from a plurality of nodes, thereby reducing the probability of reading data from different nodes, and finally improving the input or output efficiency of the space data.

In one embodiment, the preliminary dividing the spatial data according to the latitude and longitude coordinates of the spatial data, dividing the spatial data in the same latitude and longitude coordinate range into the same grid, and obtaining a plurality of grid data corresponding to the spatial data includes:

It should be understood that when the spatial data is primarily divided, a spatial coding method based on a dichotomy may be adopted to code the spatial data in the same longitude and latitude coordinate range, so as to obtain the same code, while the codes corresponding to the spatial data not in the same longitude and latitude coordinate range are different, whether the codes are the same or not is used to obtain multiple groups of divided grid data, and different spatial data are divided into the same grid based on the codes, at this time, one grid corresponds to one code, and each code corresponds to the spatial data in a certain same longitude and latitude coordinate range.

At this time, the basic unit of the spatial data is a data element, so that longitude and latitude coordinates of the data element are divided and encoded, coordinate encoding values corresponding to the encoded data element are divided, each coordinate encoding value represents one grid, a plurality of different grids are obtained, and the data elements corresponding to the coordinate encoding values are combined in one grid to obtain a plurality of grid data.

Alternatively, the binary-based spatial coding method may be a Geohash coding. Determining the number of grids according to the number of bits of the Geohash codes; and the longitude and latitude coordinates of the data elements corresponding to the space data are subjected to Geohash coding to obtain a Base32 code, a plurality of Base32 codes are finally obtained, the data elements coded by the same Base32 are divided into a data grid, the data size in each data grid is calculated, and a plurality of grid data are finally obtained. The Geohash code has non-stored timeliness, is a longitude and latitude geocoding system, and is used for binary conversion of longitude and latitude information and Base32 coding to obtain a character string code. The main idea is to use the concept of dichotomy to perform infinite interval approximation on a given value, and continuously subdivide the interval according to a given precision requirement until the precision requirement is met.

In this embodiment, by using a spatial encoding method based on a dichotomy, the data elements forming the spatial data are divided into grid data corresponding to a plurality of grids, and each grid data is composed of a plurality of data elements with the same coordinate encoding values, so that the effect of rapidly dividing the spatial data into a plurality of grid data can be achieved.

In one embodiment, the spatial coding model has the highest priority; the spatial proximity of the division result of the spatial coding model is higher than the spatial proximity of the division result of the preliminary division, as shown in fig. 3, and if the data size of the current grid data is greater than the storable data size of the current storage node, the current grid data is sub-divided at least once, and the sub-data obtained by the division is stored in the current storage node or the next storage node, including:

s302, acquiring a spatial coding model with the highest priority, and dividing the current grid data into a corresponding number of sub-data based on the spatial coding model.

It should be appreciated that the priorities of the spatial coding model, the temporal coding model, and the semantic coding model decrease in sequence, so that the spatial coding model has the highest priority, the temporal coding model has the highest priority, and the semantic coding model has the lowest priority.

Firstly, a space coding model is acquired, and current grid data is divided for the first time based on the space coding model and divided into a plurality of sub-data. Wherein the spatial proximity of the division result of the spatial coding model is higher than the spatial proximity of the division result of the preliminary division, that is, both the preliminary division model and the spatial coding model have spatial proximity, but the division result of the spatial coding model has better spatial proximity than the division result of the preliminary division.

S304, traversing the sub-data to be stored by taking the divided sub-data as the sub-data to be stored, storing the sub-data to be stored currently into the current storage node if the data size of the sub-data to be stored currently traversed is smaller than or equal to the storable data size of the current storage node, and determining whether to sub-divide the sub-data to be stored currently if the data size of the sub-data to be stored currently traversed is larger than the storable data size of the current storage node.

It should be understood that the divided sub-data needs to be determined to see whether it can be stored in the current storage node, and thus is used as the sub-data to be stored. At this time, the plurality of sub-data are traversed, and if the traversed data size of the current sub-data to be stored is smaller than or equal to the storable data size of the current storage node, the storable data size is larger than the data size of the current sub-data to be stored, so that the current sub-data to be stored can be stored in the current storage node entirely, and the current sub-data to be stored is directly stored in the current storage node. In addition, after each time when one current sub data to be stored is stored in the current storage node, the storable data amount of the current storage node should be updated, and optionally, the storable data amount after updating is the storable data amount of the current storage node minus the data amount of the current sub data to be stored.

If the traversed data size of the sub data to be stored currently is larger than the storable data size of the current storage node, determining whether to sub-divide the sub data to be stored currently. It can be understood that the data size of the current sub data to be stored is too large, the current sub data to be stored cannot be directly stored to the current storage node, and at this time, the current sub data to be stored can be considered to be continuously divided into a plurality of sub data with smaller data size, so that the judgment condition that the data size of the current sub data to be stored is smaller than the storable data size of the current storage node can be satisfied.

Meanwhile, if the current sub-data to be stored has only one data element, or the storable data volume of the current storage node is lower than a preset value, no subdivision is necessary at this time, and the condition of storing the current sub-data to be stored to the next storage node is satisfied, the current sub-data to be stored is considered to be stored to the next storage node.

That is, if the current sub-data to be stored includes a plurality of data elements and the storable data amount of the current storage node is not lower than a preset value, the current sub-data to be stored is considered to be continuously divided.

S306, under the condition that the sub-division is determined, a priority division model is obtained, the current sub-data to be stored is sub-divided by adopting the priority division model, the sub-data to be stored is updated by a plurality of sub-data obtained by division, and the step of traversing the sub-data to be stored is returned to be executed until the fact that the sub-data to be stored is not re-divided is determined, and the current sub-data to be stored is stored to a next storage node;

It should be understood that in the case where it is determined to perform the subdivision, a division model of priority order is acquired, that is, in the case where the upper-level division model is a space coding model, a time coding model of priority order is employed, and in the case where the upper-level division model is a time coding model, a semantic coding model of priority order is employed.

Dividing a plurality of sub-data of the next stage according to different coding models, taking the sub-data as sub-data to be stored, and returning to execute the step of traversing the sub-data to be stored until the current sub-data to be stored is stored to the next storage node when no division is determined. The subsequent step is the same as the step of traversing the sub-data to be stored in the step S304, and whether the current sub-data to be stored meets the condition that the data volume of the traversed current sub-data to be stored is smaller than or equal to the storable data volume of the current storage node is continuously judged, and then the current sub-data to be stored is stored in the current storage node; continuously judging whether the traversed data volume of the current sub-data to be stored is larger than the storable data volume of the current storage node, and determining whether to sub-divide the current sub-data to be stored; and continuing to judge whether the sub-data is sub-divided or stored in the next storage node.

Likewise, if the sub-division is performed, the step of traversing the sub-data to be stored is returned, and the coding model used is a lower priority coding model.

In this embodiment, through preset coding models with different priorities and determination conditions of storage data, the sub-data to be stored is circularly traversed, and the sub-data to be stored is stored in the storage node, or the sub-data to be stored is subdivided, the sub-data to be stored after the subdivision is continuously circularly traversed, and finally, all the sub-data to be stored are stored in a plurality of storage nodes, so that grid data can be divided for a plurality of times according to spatial proximity, temporal proximity and semantic proximity, a plurality of sub-data to be stored are obtained, and the sub-data to be stored with highest proximity is stored in the storage node, so that data with higher proximity is stored and read, and the read-write speed of adjacent spatial data is improved.

In one embodiment, the traversing the sub-data to be stored includes:

obtaining a target division model adopted by the sub-data to be stored;

It should be appreciated that when traversing sub-data to be stored, since each traverse of one sub-data to be stored may store that data into a storage node, the order of the traverses should be defined. In this case, it is necessary to determine the coding order at the time of division based on the information of the division model, and the coding order can represent its spatial proximity, temporal proximity, or semantic proximity.

In this embodiment, the coding sequence corresponding to the partition model is determined by different partition models, so as to obtain the traversing sequence of the subsequent traversal, and it can be ensured that sub-data stored in the same storage node have adjacency.

In one embodiment, the obtaining the spatial coding model with the highest priority, dividing the current grid data into a corresponding number of sub-data based on the spatial coding model, includes:

It should be understood that the spatial proximity of the division result of the spatial encoding method of the hilbert curve is greater than the spatial proximity of the division result corresponding to the preliminary division. The data divided by Hilbert curve coding has good spatial proximity, but timeliness is obviously reduced after the precision is increased.

To ensure the timeliness of the coding method, simple tests are carried out on the operating efficiency of the GeoHash coding and the Hilbert coding, and as shown in table 1, the timeliness of the Hilbert curve coding is found to be remarkably reduced after the 7 th order is exceeded.

Optionally, when dividing the spatial data block, the preset coding order is set to 7 or less.

Table 1 algorithm run time table

Optionally, the Hilbert curve construction algorithm of order n is as follows:

(1) Dividing the space region into space blocks, and endowing coordinates to each space block in sequence from left to right and from top to bottom;

(2) Binary conversion is carried out on the coordinates in the step (1), and the binary coordinates are combined into a character string according to the sequence;

(3) The character string is divided into strings of 2 bits long from left to right, and decimal values of each string of 2 bits long are specified. For example, "00" equals 0, "01" equals 1, "10" equals 2, "11" equals 3;

(4) For each number obtained in step (3), if all 1's appearing in the following array are changed to 3's, and all 3's appearing are changed to 1's. Then all 0's appearing in the following array are changed to 2's and all 2's appearing are changed to 0's;

(5) Each value in the array is converted into a binary representation (2-bit long string) in step (4), all strings are connected from left to right, and the decimal value is calculated.

In this embodiment, the spatial encoding method of the hilbert curve can obtain sub-data with higher spatial proximity.

In one embodiment, the obtaining the partitioning model of the priority, and the sub-data to be stored currently is subdivided by using the partitioning model of the priority, including:

It should be appreciated that the time coding model uses time stamp coding to code sub-data of the same time period, thereby dividing the sub-data of different time periods together.

Optionally, the time stamp is adopted in the scheme, that is, the total seconds from the input time of Greenwich time 1970, 1 month, 1 day, 00 minutes, 00 seconds, for example, the time stamp of the input time of 2020, 1 month, 1 day, 00 minutes, 00 seconds is: 1577808000 in 2008 to 2020, the time stamp is 10 digits, and the previous time is only less than 10 digits, so the scheme sets the code length of the time stamp to 10, and if the number of the time stamp is less than 10, 0 to 10 digits are added in front of the time stamp.

In this embodiment, the sub-data with temporal proximity can be obtained by dividing by the time coding method based on the time stamp, so that the temporal correlation of the sub-data can be effectively improved.

It should be understood that the semantic coding model encodes sub-data with the same semantic meaning, and can be divided into a plurality of sub-data sets with different semantic meanings, where each sub-data set contains sub-data with similar semantic meaning.

Optionally, in practice, the spatiotemporal data contains more semantic types, and the scheme selects several core semantics to encode, and other semantics can subsequently expand a semantic encoding table, as shown in table 2.

In this embodiment, through the semantic coding method based on the semantic table, sub data with semantic proximity can be obtained through division, and semantic similarity of the sub data can be effectively improved.

Table 2 semantic coding table

In one embodiment, in the case of determining to perform the subdivision, the method acquires a partition model of a priority order, and subdivides the sub-data to be stored currently by using the partition model of the priority order, including:

It should be understood that in the case of extremely high spatial precision of data, it may be necessary to perform the division based on the spatial coding model a second or more times, and thus the number of divisions and the coding order are preset, and according to the preset coding order, the sub-data to be stored, which is necessary to perform the division based on the spatial coding model a second or more times, is continued to be divided.

It can be understood that, if the sub data to be stored still cannot meet the condition of storing the sub data to be stored in the storage node after being divided for multiple times according to the priority order, at this time, the sub data to be stored is divided for multiple times only by using the space coding model until the sub data after being divided for multiple times meets the condition of storing the sub data to the storage node.

In this embodiment, the subdivision is performed by the spatial coding model for multiple times until no subdivision is performed, so that it can be ensured that all the sub-data to be stored can be divided into sizes satisfying the condition of storing in the storage node.

It should be appreciated that it is also possible to compare the data amount of the current mesh data with the storable data amount of the current storage node after the preliminary division, and store the current mesh data to the current storage node and update the storable data amount of the current storage node.

In this embodiment, by comparing the data amount of the mesh data with the storable data amount and determining whether or not the storage condition is met, the mesh data preliminarily divided can be stored in the current storage node.

In one embodiment, before traversing the plurality of grid data according to the coordinate proximity relationship between the plurality of grid data, the method further comprises:

It will be appreciated that in order to ensure that all spatial data can be stored quickly, the average amount of data that spatial data is allocated to all storage nodes can be calculated at this time, so that the average amount of data that should be stored per storage node can be approximated. And takes this as the initial storable data amount, thereby ensuring that each storage node has the same storage space, so that when the space data is read, the data can be uniformly read from the plurality of storage nodes. The method and the device avoid that most of space data are read in a certain storage node, so that the reading speed of the space data is reduced.

In this embodiment, by determining the initial storable data amount of the storage node for the quotient corresponding to the total data amount of the computation space data and the number of the storage nodes, it is possible to ensure that the space data can be stored in a plurality of storage nodes in an equalized manner.

In one embodiment, the specific steps of the embodiment are as follows, firstly, space multi-level division coding is performed, the number of data blocks is set as N, and the total data amount is set as M; each node can store data quantity S _i I is the node number:

(1) Selecting the coding bit number of a Geohash algorithm according to actual data, and carrying out Geohash coding on longitude and latitude coordinates of a data element to obtain a Base32 code;

(2) Dividing the data elements coded by the same Base32 into a data grid, calculating the data volume in each data grid, and setting the data volume as K _a A is the number of the data grid;

(3) Calculating an ideal data amount P, p=m+.n for each data block, initial S for each data block _i Equal to;

(4) If S _i ≥K _a Then the data grid numbered a is placed into the data block numbered i and S is recalculated _i ，S _i ＝S _i -K _a . Storing the next data grid according to the Geohash coding sequence;

(5) If S _i ≤K _a Further dividing the data grid with the number of a, dividing the data blocks by adopting a Hilbert curve algorithm, setting the order as 1, dividing the number of the divided sub-data blocks into 4 grids, and setting the data size of the sub-data grids as P _b B is Hilbert coding of the sub-data block;

(6) If P _b ≤S _i Then sub data grid P _b Put into node i and recalculate S _i ；

(7) If P _b >S _i Then P is performed _b Further decomposing, firstly, dividing the data at the same time or time period into the same data blocks by adopting a time stamp method, and if the divided data blocks are still larger than S _i Then the semantic code is used for dividing, and if the divided data still existGreater than S _i The Hilbert curve continues to be used until the sub-data block has only one data element S _i Or 0;

(8) If the data quantity of the data element is larger than S _i Storing the data element to S _i+1 In the next data block, continue with S _i+1 Comparing;

(9) When all the data grids are divided into the data blocks, the division is finished.

It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides a spatial data storage device for realizing the spatial data storage method. The implementation of the solution provided by the device is similar to that described in the above method, so specific limitations in one or more embodiments of the spatial data storage device provided below may be referred to above as limitations of the spatial data storage method, and will not be described herein.

In one embodiment, as shown in FIG. 4, there is provided a spatial data storage device 400 comprising: a data acquisition module 401, a preliminary partitioning module 402, and a subdivision module 403, wherein:

a data acquisition module 401, configured to acquire spatial data monitored by a remote sensing device for monitoring crops; the remote sensing equipment is in communication connection with the financial system;

the preliminary division module 402 is configured to perform preliminary division on the spatial data according to the longitude and latitude coordinates of the spatial data, and divide the spatial data within the same longitude and latitude coordinate range into the same grid, so as to obtain a plurality of grid data corresponding to the spatial data;

a subdivision module 403, configured to traverse the plurality of grid data according to a coordinate proximity relationship between the plurality of grid data, and if the data size of the current grid data is greater than the storable data size of the current storage node, perform at least one subdivision on the current grid data, and store the sub-data obtained by the subdivision to the current storage node or the next storage node;

Further, in one embodiment, the preliminary dividing module 402 is further configured to encode longitude and latitude coordinates corresponding to a data element of the spatial data according to a preset encoding bit number by using a spatial encoding method based on a dichotomy, so as to obtain a coordinate encoded value corresponding to the data element; wherein the data element is basic unit data of the spatial data;

Further, in one embodiment, the subdivision module 403 is further configured to obtain a spatial coding model with highest priority, and divide the current grid data into a corresponding number of sub-data based on the spatial coding model;

wherein, the conditions for carrying out the subdivision are as follows: the sub data to be stored currently comprises a plurality of data elements, and the storable data quantity of the current storage node is not lower than a preset value;

the priority of the space coding model is highest; the spatial proximity of the division result of the spatial coding model is higher than that of the preliminary division result.

Further, in one embodiment, the subdivision module 403 is further configured to obtain a target division model used to obtain the sub-data to be stored;

Further, in one embodiment, the subdivision module 403 is further configured to divide the current grid data into a number of sub-data corresponding to the coding order according to a preset coding order by using a spatial coding method based on a hilbert curve;

Further, in an embodiment, the subdivision module 403 is further configured to obtain sub-data of the same time period in the sub-data to be stored currently, and obtain sub-data of a plurality of time periods when the subdivision model of the priority order is a time coding model;

Further, in an embodiment, the subdivision module 403 is further configured to obtain sub-data with the same semantics from the sub-data to be stored currently, to obtain sub-data with multiple semantics, where the division model of the priority order is a semantic coding model;

Further, in an embodiment, the repartitioning module 403 is further configured to, in a case where it is determined to perform repartitioning, obtain the spatial coding model if the partition model with the lowest priority has been adopted, and use the spatial coding model to repartition the sub-data to be stored.

Further, in one embodiment, the preliminary partitioning module 402 is further configured to store the current grid data to the current storage node if the data size of the current grid data is less than or equal to the storable data size of the current storage node.

Further, in one embodiment, the preliminary partitioning module 402 is further configured to obtain a quotient of a total amount of the spatial data and the number of storage nodes;

The various modules in the spatial data store described above may be implemented in whole or in part in software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 5. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing the space data monitored by the remote sensing device. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a spatial data storage method.

It will be appreciated by those skilled in the art that the structure shown in FIG. 5 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the method embodiments described above.

In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.

Claims

1. A method of spatial data storage, the method comprising:

2. The method of claim 1, wherein the primarily dividing the spatial data according to the longitude and latitude coordinates of the spatial data, dividing the spatial data belonging to the same longitude and latitude coordinate range into the same grid, and obtaining a plurality of grid data corresponding to the spatial data, includes:

3. The method according to claim 1, wherein the spatial coding model has the highest priority; the spatial proximity of the division result of the spatial coding model is higher than that of the preliminary division result;

4. A method according to claim 3, wherein said traversing said sub-data to be stored comprises:

obtaining a target division model adopted by the sub-data to be stored;

5. A method according to claim 3, wherein the obtaining the spatial coding model with the highest priority, dividing the current grid data into a corresponding number of sub-data based on the spatial coding model, comprises:

6. A method according to claim 3, wherein said obtaining a prioritized partitioning model, said current sub-data to be stored being subdivided using said prioritized partitioning model, comprises:

7. A method according to claim 3, wherein said obtaining a prioritized partitioning model, said current sub-data to be stored being subdivided using said prioritized partitioning model, comprises:

8. A method according to claim 3, wherein, in the case of determining to perform the sub-division, a priority-level division model is obtained, and the sub-data to be stored currently is sub-divided by using the priority-level division model, comprising:

9. The method of claim 1, wherein traversing the plurality of grid data according to the coordinate proximity relationship between the plurality of grid data further comprises:

10. The method of claim 1, wherein before traversing the plurality of grid data according to the coordinate proximity relationship between the plurality of grid data, further comprising:

11. A spatial data storage device, the device comprising:

12. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 10 when the computer program is executed.

13. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 10.

14. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any one of claims 1 to 10.