CN117688194A - Vector slice graph management method based on Redis cache - Google Patents

Abstract

The invention relates to a vector slice graph management method based on Redis cache, which uses power grid vector data to construct a pbf format vector slice, stores the pbf format vector slice in a Redis system, and can manage effective time, space limitation and elimination strategies of the vector slice by setting a cache strategy; when the client sends a request, according to whether corresponding vector slice data exists in the cache, a query result is returned or related data is acquired from the database and converted into a pbf format file to be written into a Redis system, and the pbf format file is returned to the client, so that the vector slice request of the client is responded quickly, frequent query on a back-end database is reduced, and the query efficiency is improved; the method can also convert the request parameters into geographic elements according to the request parameters, and perform range matching according to the geographic range of the stored vector slices, so that the query efficiency is improved, and the query speed and the user experience can be further improved through optimization such as preloading common vector slices and data compression.

Description

Vector slice graph management method based on Redis cache

Technical Field

The invention relates to the technical field of Internet, in particular to a vector slice graph management method based on Redis cache.

Background

The grid graphic system is an important part of the development of the smart grid, and with the growth speed of the mass level of the grid data and the rapid growth of the scale of the grid data, the traditional grid slice static picture cannot meet the requirements of users on dynamic, editable and controllable. The vector slice has the advantages of smooth roaming, good user body feeling, editable data, controllable layers, customizable theme and the like, so that the vector slice is widely applied to a power grid graphic system.

However, when the traditional vector slice graph method processes huge power grid data, the response time is long due to the excessive data volume and the requirement of frequent query, and the user experience is seriously affected. In order to improve the query efficiency and reduce the response time, a vector slice graph management method based on Redis cache is generated.

Disclosure of Invention

In order to solve the problems existing in the prior art, the application provides a vector slice graph management method based on Redis cache.

The technical scheme of the application is as follows:

a vector slice graph management method based on Redis cache, the method comprising:

determining parameters of the vector slice, and constructing a pbf format vector slice by using power grid vector data, wherein the parameters comprise coordinate projection, slice size and slice level;

writing the vector slice into a Redis system, and setting a cache strategy in the Redis system, wherein the cache strategy comprises the effective time of cache, the maximum space size of cache and a cache elimination strategy;

when a client sends a vector slice request, resolving the request to obtain a request parameter, and searching corresponding vector slice data from a Redis system according to the request parameter, wherein the method specifically comprises the following steps:

if the corresponding vector slice data exists, the vector slice data is taken out and returned to the client; if the corresponding vector slice data does not exist, converting the request parameters into corresponding geographic elements, inquiring a Postgres space database according to the geographic elements, converting the inquiring result into a pbf format file, returning the pbf format file to the client, and writing the file into a Redis system;

and the client analyzes the returned pbf format vector slice or pbf format file to obtain a corresponding layer and renders the layer.

Preferably, determining parameters of the vector slice to construct the pbf format vector slice is specifically to divide the power grid vector data into equal-sized small blocks, hierarchically organize the small blocks, name each level according to a grid number, and generate a corresponding pbf format file, wherein the grid number comprises a slice line number X, a slice column number Y and a slice level Z.

Preferably, the method further comprises registering and storing configuration information of the vector slice when the vector slice is written into the Redis system, and recording registration time and service state of the vector slice, wherein the configuration information comprises service address, geographical range and registration period of the slice, and when the vector slice is in the registration period, the service state is valid, otherwise, the service state is invalid.

Preferably, the setting of the cache policy in the Redis is specifically:

setting a cache valid time according to the instantaneity and the access frequency of the vector slice data;

setting the maximum space size of the cache according to the size of the Redis system memory and the service requirement;

and setting a cache elimination strategy according to the service requirement and the query mode, wherein the cache elimination strategy adopts an LRU algorithm or an LFU algorithm to eliminate useless data.

Preferably, the method further comprises performing range matching according to the geographical range of the stored vector slice when converting the request parameters into the corresponding geographical elements, merging query results in the same range, and returning to the client, wherein if the service state of the vector slice matched with the range is invalid, the service state is not available.

Preferably, the Redis system is directly connected with the Postgres space database, and vector data including grid space information and attribute data are stored in the Postgres space database, wherein the grid space information is a geographic element, and the attribute data is a corresponding attribute of the geographic element.

Preferably, when vector data in the Postgres space database changes, a message queue mechanism is used to update the corresponding pbf format file in the Redis system in time.

Preferably, the method further comprises preloading a portion of the commonly used vector slices according to the service requirements and the historical access patterns at the start-up of the Redis system.

Preferably, the method further comprises the steps that when the query result is converted into the pbf format file and returned to the client, the pbf file is subjected to gzip compression, and when the client receives the pbf file, the client firstly performs gzip decompression, and then performs analysis and rendering.

The application also provides a vector slice graph management platform based on Redis cache, the platform comprises a client, a gateway layer, a slice service layer, a front-end routing service layer and a data layer, wherein:

the client is used for responding to a vector slicing request input by a user and sending the vector slicing request to the gateway layer, wherein the client is also used for gzip decompression, pbf format data analysis and layer rendering;

the gateway layer is used for receiving a vector slicing request, analyzing the request to obtain request parameters, and sending the request parameters to the slicing service layer;

the slice service layer is used for searching the Redis system according to the request parameters, and if the corresponding vector slice data exists, the vector slice data is taken out and returned to the client; if the corresponding vector slice data does not exist, converting the request parameters into corresponding geographic elements, transmitting the geographic elements to a preposed routing service layer, receiving a range matching result returned by the preposed routing service layer, inquiring a Postgres space database according to the geographic elements, merging the inquiry results in the same range, converting the inquiry results into a pbf format file, carrying out gzip compression processing on the pbf format file, returning the gzip compression packet to a client, and simultaneously writing the file into a Redis system, wherein if the service state of the vector slice matched by the range is invalid, the file is unavailable;

the prepositive routing service layer is used for registering and storing the registration time and service state of the vector slice, wherein the configuration information comprises the service address, the geographic range and the registration period of the slice; when the request parameters are converted into corresponding geographic elements, the front-end routing service layer performs range matching according to the geographic range of the stored vector slice, and returns a matching result to the slice service layer, wherein the matching result comprises a range partition of the vector slice and a corresponding service state;

the data layer comprises a Redis system and a Postgres space database, and is used for storing power grid vector data in a pbf format.

Compared with the prior art, the invention has the beneficial effects that:

1) The invention provides a vector slice graph management method based on Redis cache, which stores vector slice data in a Redis system, sets a cache strategy, can quickly respond to a vector slice request of a client, reduces frequent inquiry to a back-end database, and improves inquiry efficiency; the effective time, space limitation and elimination strategy of the vector slice are managed, so that the management and utilization efficiency of data is improved;

2) The invention provides a vector slice graph management method based on Redis cache, which can update vector data change in a database to a pbf format file in a Redis system in time through connection with a Postgres space database and a message queue mechanism, so that the real-time property of the data is maintained;

3) The invention provides a vector slice graph management method based on Redis cache, which converts request parameters into geographic elements, performs range matching by utilizing the geographic range of a stored vector slice, improves query efficiency, combines query results in the same range, and further optimizes query.

Drawings

FIG. 1 is a flow chart of a method of an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention is provided to facilitate understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and all the inventions which make use of the inventive concept are protected by the spirit and scope of the present invention as defined and defined in the appended claims to those skilled in the art.

The invention provides the following technical scheme: a vector slice graph management method based on Redis cache.

Example 1

S1, determining parameters of a vector slice, and constructing a pbf format vector slice by using power grid vector data, wherein the parameters comprise coordinate projection, slice size and slice level;

s11, determining parameters of vector slicing:

s111 coordinate projection: selecting a proper projection method for slicing according to the characteristics and the use requirements of the power grid data, wherein the projection method comprises longitude and latitude coordinate projection (such as WGS 84) and plane coordinate projection (such as cutterhead projection);

s112, slice size: according to the complexity and application requirements of the power grid data, a proper slice size is selected, the size of the vector slice determines the granularity of the slice, and in general, the small-size slice can provide more detailed data, but the quantity and the storage space of the data are increased;

s113, slice level: determining the number and the range of slice levels according to the range and the detail requirement of the power grid data, wherein the slice levels represent the zoom level of map data and generally range from a global large scale to a local small scale, and the number of slice levels determines the level of map which can be enlarged or reduced;

s12, according to the determined parameters, vector slices in the pbf format can be constructed by using the power grid vector data, and specifically:

s121, converting the power grid vector data into a corresponding projection coordinate system according to the selected coordinate projection;

s122, dividing the converted vector slice data into small blocks with equal size by using a grid division method or a tile division method, organizing the small blocks in a layered manner according to the number of slice levels, naming each level according to a grid number, and generating a corresponding pbf format file, wherein the grid number comprises a slice line number X, a slice column number Y and a slice level Z, and storing the small block data on each level as an independent pbf format file to ensure that the naming mode of the file can reflect the position and the level relation of the slice;

s2, writing the vector slice, namely the pbf format file, into a Redis system, registering and storing configuration information of the vector slice, and recording registration time and service state of the vector slice, wherein the configuration information comprises service addresses, geographical ranges and registration periods (units: minutes) of the slice, and when the vector slice is in the registration period, the service state is valid, otherwise, the service state is invalid;

s21, storing the pbf format files of all the vector slices in a Hash object of Redis. Assigning a unique name or ID for each vector slice, taking the unique name or ID as a Key, and taking the data of the pbf format file as a Value;

s22, registering each vector slice in the Sorted Set object of Redis by using the name or ID of the vector slice as a Key. The sonted Set uses slice names or IDs as members, and stores metadata such as configuration information as scores. The configuration information includes the service address, geographical range and registration period (in minutes) of the slice. The registration period is to register every other time to ensure the availability of vector slicing service;

s23, recording the registration time and the service state of each vector slice. At the time of registration of the vector slice, the registration time thereof is recorded. Meanwhile, the service state of the vector slice is judged according to the registration period. If the time difference between the current time and the last registration time is smaller than the registration period, the vector slice service state is valid; otherwise, the vector slice is in an invalid service state;

storing the pbf files of all the vector slices and the configuration information thereof in a Redis system through the step S2, conveniently inquiring and managing all the vector slices, and simultaneously supporting dynamic addition, deletion and updating of the configuration information of the vector slice service;

s3, setting a cache strategy in the Redis system, wherein the cache strategy comprises the effective time of cache, the maximum space size of cache and a cache elimination strategy;

setting a cache valid time according to the real-time property and the access frequency of the vector slice data, if the vector slice data is changed frequently or needs to be updated in real time, a shorter valid time can be set, for example, a few minutes or a few hours; if the vector slice data is less variable and relatively stable, the active time may be extended, such as a day or more;

setting the maximum space of the cache according to the memory size and service requirement of the Redis system, wherein the vector slice data may occupy larger storage space, so that the Redis system needs to have enough memory to store the cache data, and setting the maximum space of the cache according to actual conditions, thereby avoiding performance degradation or service unavailability caused by insufficient memory;

and setting a cache elimination strategy according to the service requirement and the query mode, wherein the cache elimination strategy adopts an LRU (least recently used) algorithm or an LFU (least recently used) algorithm to eliminate useless data, the LRU algorithm refers to preferentially eliminating data which is not accessed for the longest time, and the LFU algorithm refers to preferentially eliminating data which is accessed for the least times. Selecting a proper elimination strategy according to the importance and the use frequency of the cached data so as to ensure that the most useful data is stored in the cache;

the cache strategy is reasonably set through the step S3, so that the query efficiency of vector slice data and the performance of a system are improved, and meanwhile, storage resources are fully utilized;

s4, when the client sends a vector slice request, corresponding data is obtained from a Hash object of the Redis according to a vector slice name or ID in a request parameter as a Key, wherein when the Redis system is started, partial common vector slices are preloaded according to service requirements and a history access mode;

s31, if the corresponding vector slice data exists, the vector slice data are taken out and returned to the client;

s32, if no corresponding vector slice data exists, generating and acquiring the data according to the request parameters:

A. converting X, Y, Z in the request parameters into corresponding geographic elements, such as longitude and latitude coordinates or plane coordinates;

B. performing range matching on the geographic elements according to the geographic range of the stored vector slice, and judging the area of the geographic elements;

C. querying a Postgres space database by using the geographic elements, querying corresponding vector slice data, dividing the vector slices into areas according to the judging result, and merging the vector slice data of the same area;

D. converting the combined result into a pbf format file;

E. gzip compression is carried out on the pbf format file;

F. returning the gzip compression package to the client, and writing the file into the Redis system. When writing Redis, the service state of the matched vector slice needs to be checked. If the service state is invalid, the service state is not written into Redis, so that unusable data is avoided;

s4, the client analyzes the returned pbf format vector slice or the pbf format file in the gzip compression packet to obtain a corresponding layer and render the layer;

s41, obtaining information such as a layer, element data, attribute data and the like contained in the file through analysis;

s42, screening out required layers from the layers obtained through analysis according to the requirements of the client, wherein different layers possibly contain different element types, such as points, lines, planes and the like, and possibly also contain different attribute data;

s43, converting the coordinate value of each element into screen coordinates so as to render; meanwhile, converting attribute data of the elements into corresponding text, color, quantity and other information for rendering;

and S44, rendering according to the requirements of the client and the characteristics of the data by using a related rendering library or tool. For example, the shape of a circle, a square, or the like may be selected to be used when rendering the point data, and the attribute value may be represented by a different color or size. Line width, line type, color, etc. attributes may be set when rendering line data. The properties of filling style, color, etc. can be set when rendering the surface data. Different rendering modes can be selected according to different requirements;

through step S4, the client can acquire the data of the required layer according to the returned pbf format vector slice or pbf format file, and perform corresponding rendering operation. In this way, the data of the vector slice can be visually presented to a user, and related information such as a map and the like can be conveniently displayed.

Example two

The embodiment provides a vector slice graph management platform based on Redis cache, which comprises a client, a gateway layer, a slice service layer, a front-end routing service layer and a data layer, wherein:

the client is used for responding to a vector slicing request input by a user and sending the vector slicing request to the gateway layer, wherein the client is also used for gzip decompression, pbf format data analysis and layer rendering;

the gateway layer is used for receiving a vector slicing request, analyzing the request to obtain request parameters, and sending the request parameters to the slicing service layer;

the slice service layer is used for searching the Redis system according to the request parameters, and if the corresponding vector slice data exists, the vector slice data is taken out and returned to the client; if the corresponding vector slice data does not exist, converting the request parameters into corresponding geographic elements, transmitting the geographic elements to a preposed routing service layer, receiving a range matching result returned by the preposed routing service layer, inquiring a Postgres space database according to the geographic elements, merging the inquiry results in the same range, converting the inquiry results into a pbf format file, carrying out gzip compression processing on the pbf format file, returning the gzip compression packet to a client, and simultaneously writing the file into a Redis system, wherein if the service state of the vector slice matched by the range is invalid, the file is unavailable;

the prepositive routing service layer is used for registering and storing the registration time and service state of the vector slice, wherein the configuration information comprises the service address, the geographic range and the registration period of the slice; when the request parameters are converted into corresponding geographic elements, the front-end routing service layer performs range matching according to the geographic range of the stored vector slice, and returns a matching result to the slice service layer, wherein the matching result comprises a range partition of the vector slice and a corresponding service state;

the data layer comprises a Redis system and a Postgres space database, and is used for storing power grid vector data in a pbf format.

It should be noted that, the platform and the method according to the first embodiment are based on the same inventive concept, and are not described herein.

The foregoing description is only illustrative of the present invention and is not intended to limit the scope of the invention, and all equivalent structures or equivalent processes or direct or indirect application in other related technical fields are included in the scope of the present invention.

Claims (10)

1. The vector slice graph management method based on Redis cache is characterized by comprising the following steps:

determining parameters of the vector slice, and constructing a pbf format vector slice by using power grid vector data, wherein the parameters comprise coordinate projection, slice size and slice level;

writing the vector slice into a Redis system, and setting a cache strategy in the Redis system, wherein the cache strategy comprises the effective time of cache, the maximum space size of cache and a cache elimination strategy;

when a client sends a vector slice request, resolving the request to obtain a request parameter, and searching corresponding vector slice data from a Redis system according to the request parameter, wherein the method specifically comprises the following steps:

if the corresponding vector slice data exists, the vector slice data is taken out and returned to the client; if the corresponding vector slice data does not exist, converting the request parameters into corresponding geographic elements, inquiring a Postgres space database according to the geographic elements, converting the inquiring result into a pbf format file, returning the pbf format file to the client, and writing the file into a Redis system;

and the client analyzes the returned pbf format vector slice or pbf format file to obtain a corresponding layer and renders the layer.

2. The Redis cache-based vector slice graph management method according to claim 1, wherein determining parameters of vector slices to construct pbf format vector slices is specifically to divide power grid vector data into equal-sized small blocks, hierarchically organize the small blocks, name each level according to grid numbers, and generate corresponding pbf format files, wherein the grid numbers comprise slice row numbers X, slice column numbers Y and slice level Z.

3. The method for managing vector slice graphs based on Redis cache according to claim 1, further comprising registering and storing configuration information of the vector slice when the vector slice is written into the Redis system, and recording registration time and service state of the vector slice, wherein the configuration information comprises service address, geographical range and registration period of the slice, and when the vector slice is in the registration period, the service state is valid, otherwise, the service state is invalid.

4. The vector slice graph management method based on Redis cache according to claim 1, wherein the setting of the cache policy in Redis is specifically:

setting a cache valid time according to the instantaneity and the access frequency of the vector slice data;

setting the maximum space size of the cache according to the size of the Redis system memory and the service requirement;

and setting a cache elimination strategy according to the service requirement and the query mode, wherein the cache elimination strategy adopts an LRU algorithm or an LFU algorithm to eliminate useless data.

5. The method for managing vector slice graphs based on Redis cache according to claim 3, further comprising performing range matching according to the geographic range of the stored vector slice when converting the request parameters into corresponding geographic elements, merging query results in the same range, and returning to the client, wherein if the service state of the vector slice matched by the range is invalid, the service state is not available.

6. The vector slice graph management method based on Redis cache according to claim 1, wherein the Redis system is directly connected with the Postgres space database, vector data including grid space information and attribute data are stored in the Postgres space database, the grid space information is a geographic element, and the attribute data is a geographic element corresponding attribute.

7. The vector slice graph management method based on Redis cache as claimed in claim 1, wherein when vector data in the Postgres spatial database changes, a message queue mechanism is used to update the corresponding pbf format file in the Redis system in time.

8. The method for managing vector slice graphs based on Redis cache as claimed in claim 1, wherein the method further comprises preloading a part of the commonly used vector slices according to service requirements and historical access patterns when the Redis system is started.

9. The method for managing vector slice graphs based on Redis cache according to claim 1, wherein the method further comprises converting the query result into a pbf format file, returning the pbf format file to the client, performing gzip compression processing on the pbf file, performing gzip decompression on the pbf file when the client layer receives the pbf file, and then performing parsing and rendering.

10. The vector slicing graph management platform based on Redis cache is characterized by comprising a client, a gateway layer, a slicing service layer, a front-end routing service layer and a data layer, wherein:

the client is used for responding to a vector slicing request input by a user and sending the vector slicing request to the gateway layer, wherein the client is also used for gzip decompression, pbf format data analysis and layer rendering;

the gateway layer is used for receiving a vector slicing request, analyzing the request to obtain request parameters, and sending the request parameters to the slicing service layer;

the slice service layer is used for searching the Redis system according to the request parameters, and if the corresponding vector slice data exists, the vector slice data is taken out and returned to the client; if the corresponding vector slice data does not exist, converting the request parameters into corresponding geographic elements, transmitting the geographic elements to a preposed routing service layer, receiving a range matching result returned by the preposed routing service layer, inquiring a Postgres space database according to the geographic elements, merging the inquiry results in the same range, converting the inquiry results into a pbf format file, carrying out gzip compression processing on the pbf format file, returning the gzip compression packet to a client, and simultaneously writing the file into a Redis system, wherein if the service state of the vector slice matched by the range is invalid, the file is unavailable;

the prepositive routing service layer is used for registering and storing the registration time and service state of the vector slice, wherein the configuration information comprises the service address, the geographic range and the registration period of the slice; when the request parameters are converted into corresponding geographic elements, the front-end routing service layer performs range matching according to the geographic range of the stored vector slice, and returns a matching result to the slice service layer, wherein the matching result comprises a range partition of the vector slice and a corresponding service state;

the data layer comprises a Redis system and a Postgres space database, and is used for storing power grid vector data in a pbf format.