CN110990612A - Method and terminal for rapidly displaying vector big data - Google Patents
Method and terminal for rapidly displaying vector big data Download PDFInfo
- Publication number
- CN110990612A CN110990612A CN201911278725.6A CN201911278725A CN110990612A CN 110990612 A CN110990612 A CN 110990612A CN 201911278725 A CN201911278725 A CN 201911278725A CN 110990612 A CN110990612 A CN 110990612A
- Authority
- CN
- China
- Prior art keywords
- vector
- vector data
- data
- pyramid
- grade
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/54—Browsing; Visualisation therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/56—Information retrieval; Database structures therefor; File system structures therefor of still image data having vectorial format
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Controls And Circuits For Display Device (AREA)
- Image Generation (AREA)
Abstract
The invention discloses a method and a terminal for rapidly displaying vector big data.A pyramid grade is established for a vector data source to obtain the vector data after grading, each grade corresponds to different pixel resolutions, and the vector data source comprises a linear graph or a planar graph; thinning the classified vector data according to the size of the circumscribed rectangle of the linear graph or the planar graph and the resolution of each classified pixel to obtain thinned vector data; displaying vector data of different pyramid grades according to the thinned vector data; the vector big data graph is displayed quickly, the data volume of the small scale display request is reduced, the thinning effect meets the requirement of displaying the data volume, the preprocessing time is shortened, and the display performance is improved. The problems of overlong preprocessing time, incapability of dynamically modifying display styles, limited display data size and performance and the like are solved at one time. The vector big data can be displayed dynamically and rapidly.
Description
Technical Field
The invention relates to the field of graphic display, in particular to a method and a terminal for rapidly displaying vector big data.
Background
In the information construction of various fields at the national level, especially in the development construction of a national soil foundation spatial information platform, the data volume of a professional map layer, such as national basic farmland pattern data, can often reach tens of millions or even more than one hundred million records. The fast display of linear graph and planar graph vector big data is a basic requirement and a difficult technology. Existing solutions include:
one, grid tile technology
The grid tile technology saves the time for reading and rendering vector data by establishing a plurality of levels of tile sets with different scales and different pixel resolutions in advance, and achieves the purpose of quickly responding to a user display request.
The grid tile technology has the advantages of high display efficiency and stable performance; the disadvantages are that the preprocessing time is long, the user needs to wait after the data is put in storage, and the display style is static after the tile is generated, and the user cannot dynamically modify the display style.
Two, vector slicing technique
The vector slice technology establishes a vector slice set which is subjected to thinning and cutting and has different scales at multiple levels in advance, so that a system can quickly read a proper amount of vector slice data according to the current display request coordinate range of a user, transmit the vector slice data to a client and render the vector slice data at the client to finish display.
The vector slicing technology has the advantages that the requirement of dynamically modifying the display style by a user is met, and the query and calculation requirements can be simultaneously supported by vector data at a client; the method has the disadvantages that the reduction effect of thinning and cutting on vector data quantity is limited, the pressure of data transmission and client display data can be reduced only to a certain extent, the fast display of big data cannot be really solved, and the requirement on the hardware display capacity of the client is higher because the data is rendered at the client, so that the requirements of different hardware conditions of a user in the internet environment cannot be met.
Three, grid tile and dynamic display mixing technology
Compared with the raster tile technology, the raster tile and dynamic display mixing technology only generates a raster tile set at a small scale level in advance for responding to a small scale display request of a client, and for a large scale display request, because the data range of the client once display request is small at the moment, the related data volume is small, and the response is completed by adopting a dynamic data reading and dynamic rendering mode.
The grid tile and dynamic display hybrid technology has the advantages that the display performance is guaranteed, and meanwhile, the problem that the grid tile is long in preprocessing time is solved; the disadvantage is that the display style of the tiles is static and the user cannot dynamically modify the display style.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: a method and a terminal for rapidly displaying vector big data are provided, which can dynamically and rapidly display the vector big data.
In order to solve the technical problems, the invention adopts a technical scheme that:
a method for rapidly displaying vector big data comprises the following steps:
s1, establishing pyramid classification for a vector data source to obtain classified vector data, wherein each classification corresponds to different pixel resolutions, and the vector data source comprises a linear graph or a planar graph;
s2, thinning the classified vector data according to the size of the circumscribed rectangle of the linear graph or the planar graph and the resolution of each classified pixel to obtain thinned vector data;
and S3, displaying the vector data of different pyramid grades according to the thinned vector data.
In order to solve the technical problem, the invention adopts another technical scheme as follows:
a terminal for rapidly displaying vector big data comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to realize the following steps:
s1, establishing pyramid classification for a vector data source to obtain classified vector data, wherein each classification corresponds to different pixel resolutions, and the vector data source comprises a linear graph or a planar graph;
s2, thinning the classified vector data according to the size of the circumscribed rectangle of the linear graph or the planar graph and the resolution of each classified pixel to obtain thinned vector data;
and S3, displaying the vector data of different pyramid grades according to the thinned vector data.
The invention has the beneficial effects that: by establishing pyramid classification for a vector data source and adopting the size of an external rectangle based on a linear graph or a planar graph to dilute vector data, the data volume of a small scale display request is reduced, the thinning effect meets the requirement of the display data volume, and the connection between the thinned data and the pyramid classification is easier to establish, so that the method is convenient for programming realization, reduces the preprocessing time, improves the display performance, solves the problems that the preprocessing time is too long, the display style cannot be dynamically modified, the display data volume and the performance are limited and the like at one time, and can dynamically and quickly display large vector data.
Drawings
FIG. 1 is a flowchart illustrating steps of a method for fast displaying vector big data according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a RDD block-based distributed rendering in vector big data fast display according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a terminal for fast displaying vector big data according to an embodiment of the present invention;
description of reference numerals:
1. a terminal for rapidly displaying vector big data; 2 a memory; 3. a processor;
Detailed Description
In order to explain technical contents, achieved objects, and effects of the present invention in detail, the following description is made with reference to the accompanying drawings in combination with the embodiments.
Referring to fig. 1, a method for fast displaying vector big data includes the steps of:
s1, establishing pyramid classification for a vector data source to obtain classified vector data, wherein each classification corresponds to different pixel resolutions, and the vector data source comprises a linear graph or a planar graph;
s2, thinning the classified vector data according to the size of the circumscribed rectangle of the linear graph or the planar graph and the resolution of each classified pixel to obtain thinned vector data;
and S3, displaying the vector data of different pyramid grades according to the thinned vector data.
From the above description, the beneficial effects of the present invention are: by establishing pyramid classification for the vector data source and adopting the size of the external rectangle based on the linear graph or the planar graph to rarefy the vector data, the data volume of the small scale display request is reduced, the rarefying effect meets the requirement of the display data volume, and the connection between the rarefying data and the pyramid classification is easier to establish, so that the programming realization is facilitated, the preprocessing time is reduced, and the display performance is improved. The problems that the preprocessing time is too long, the display style cannot be dynamically modified, the display data size and the performance are limited and the like are solved at one time, and the vector big data can be dynamically and rapidly displayed.
Further, the step S1 of establishing a pyramid hierarchy for the vector data source includes:
and carrying out different numbers of partitions on the vector data source, wherein the resolution ratio of each partition is the same, and the number of the partitions corresponding to different pyramid grades is different.
As can be seen from the above description, each pyramid level corresponds to a different number of partitions, so that each level has a different pixel resolution, and the pyramid level can be conveniently found according to the pixel resolution, which facilitates the subsequent establishment of the association between the rarefied data and the pyramid level.
Further, the step S2 includes:
s21, for each hierarchical vector data, respectively:
traversing each graphic object, and calculating the size of a circumscribed rectangle of the graphic object, wherein the size comprises the area or the perimeter;
comparing the size of the circumscribed rectangle of each graphic object with the pixel resolution corresponding to the grade of the circumscribed rectangle, and counting the total number and the total size of the graphic objects which are less than or equal to the pixel resolution corresponding to the grade of the circumscribed rectangle;
s22, presetting a maximum thinning threshold and a minimum thinning threshold, and determining a maximum vector compression level and a minimum vector compression level according to the maximum thinning threshold, the minimum thinning threshold, the total number and the total size of the graphic objects which are corresponding to each grade and are less than or equal to the pixel resolution of the graphic objects;
and S23, determining the data source of each pyramid grade according to the maximum vector compression grade and the minimum vector compression grade to obtain the diluted vector data.
It can be known from the above description that, a maximum vector compression level and a minimum vector compression level are determined based on a preset maximum thinning threshold and a preset minimum thinning threshold, and a total number and a total size of the graphics objects corresponding to each level and smaller than or equal to the pixel resolution, and a data source of each pyramid level is determined according to the maximum vector compression level and the minimum vector compression level, so as to obtain thinned vector data, and a maximum vector compression ratio that does not affect the overall display effect of the graphics and a minimum vector compression ratio that can perform the vector compression function can be accurately determined, so that not only can the data amount of the displayed graphics objects be reduced, but also the vector compression function is performed, the overall display effect of the graphics layer is not affected when the full-view is displayed, and the display effect of the graphics is ensured while the compression is ensured.
Further, when the size of the circumscribed rectangle of each hierarchical vector data is calculated, and the total number and the total size of the graphic objects with the pixel resolution ratio which is smaller than or equal to the hierarchical position of the circumscribed rectangle are counted, the data source which needs to be calculated or counted is pushed to a plurality of working nodes to carry out distributed parallel calculation;
and receiving the data after the calculation of each working node is completed, and summarizing to obtain the total number and the total size of the graphic objects which correspond to each grade and are less than or equal to the pixel resolution of the graphic objects.
According to the description, when the statistical calculation is carried out, the vector data sources corresponding to all the grades are pushed to the plurality of working nodes for distributed parallel calculation, so that the data calculation time length can be effectively shortened, and the efficiency of quick display can be further improved.
Further, the step S3 includes:
s31, segmenting the thinned vector data source according to the coordinate range of the vector data source, and constructing a plurality of block RDDs, wherein each block RDD is mapped to one working node;
s32, receiving a display request, and determining a corresponding pyramid grade according to the display request;
s33, determining a corresponding diluted vector data source according to pyramid grading, and determining an RDD block related to the display request according to the coordinate range of the RDD block;
and S34, rendering on the corresponding working nodes according to the related RDD blocks, receiving and summarizing the pictures rendered by the corresponding working nodes, and displaying the pictures.
According to the description, the vector data source after thinning is divided, a plurality of RDD blocks are constructed, each RDD block is mapped to one working node, when rendering is carried out, the RDD blocks have coordinate ranges and are only one part of the whole coordinate range, the pictures generated by rendering of the corresponding working nodes are one part of standard pictures, each working node only needs to render the corresponding part of the pictures, and the standard pictures do not need to be rendered like the prior standard pictures.
Referring to fig. 3, a terminal for fast displaying vector big data includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor executes the computer program to implement the following steps:
s1, establishing pyramid classification for a vector data source to obtain classified vector data, wherein each classification corresponds to different pixel resolutions, and the vector data source comprises a linear graph or a planar graph;
s2, thinning the classified vector data according to the size of the circumscribed rectangle of the linear graph or the planar graph and the resolution of each classified pixel to obtain thinned vector data;
and S3, displaying the vector data of different pyramid grades according to the thinned vector data.
From the above description, the beneficial effects of the present invention are: by establishing pyramid classification for the vector data source and adopting the size of the external rectangle based on the linear graph or the planar graph to rarefy the vector data, the data volume of the small scale display request is reduced, the rarefying effect meets the requirement of the display data volume, and the connection between the rarefying data and the pyramid classification is easier to establish, so that the programming realization is facilitated, the preprocessing time is reduced, and the display performance is improved. The problems that the preprocessing time is too long, the display style cannot be dynamically modified, the display data size and the performance are limited and the like are solved at one time, and the vector big data can be dynamically and rapidly displayed.
Further, the step S1 of establishing a pyramid hierarchy for the vector data source includes:
and carrying out different numbers of partitions on the vector data source, wherein the resolution ratio of each partition is the same, and the number of the partitions corresponding to different pyramid grades is different.
As can be seen from the above description, each pyramid level corresponds to a different number of partitions, so that each level has a different pixel resolution, and the pyramid level can be conveniently found according to the pixel resolution, which facilitates the subsequent establishment of the association between the rarefied data and the pyramid level.
Further, the step S2 includes:
s21, for each hierarchical vector data, respectively:
traversing each graphic object, and calculating the size of a circumscribed rectangle of the graphic object, wherein the size comprises the area or the perimeter;
comparing the size of the circumscribed rectangle of each graphic object with the pixel resolution corresponding to the grade of the circumscribed rectangle, and counting the total number and the total size of the graphic objects which are less than or equal to the pixel resolution corresponding to the grade of the circumscribed rectangle;
s22, presetting a maximum thinning threshold and a minimum thinning threshold, and determining a maximum vector compression level and a minimum vector compression level according to the maximum thinning threshold, the minimum thinning threshold, the total number and the total size of the graphic objects which are corresponding to each grade and are less than or equal to the pixel resolution of the graphic objects;
and S23, determining the data source of each pyramid grade according to the maximum vector compression grade and the minimum vector compression grade to obtain the diluted vector data.
It can be known from the above description that, a maximum vector compression level and a minimum vector compression level are determined based on a preset maximum thinning threshold and a preset minimum thinning threshold, and a total number and a total size of the graphics objects corresponding to each level and smaller than or equal to the pixel resolution, and a data source of each pyramid level is determined according to the maximum vector compression level and the minimum vector compression level, so as to obtain thinned vector data, and a maximum vector compression ratio that does not affect the overall display effect of the graphics and a minimum vector compression ratio that can perform the vector compression function can be accurately determined, so that not only can the data amount of the displayed graphics objects be reduced, but also the vector compression function is performed, the overall display effect of the graphics layer is not affected when the full-view is displayed, and the display effect of the graphics is ensured while the compression is ensured.
Further, when the size of the circumscribed rectangle of each hierarchical vector data is calculated, and the total number and the total size of the graphic objects with the pixel resolution ratio which is smaller than or equal to the hierarchical position of the circumscribed rectangle are counted, the data source which needs to be calculated or counted is pushed to a plurality of working nodes to carry out distributed parallel calculation;
and receiving the data after the calculation of each working node is completed, and summarizing to obtain the total number and the total size of the graphic objects which correspond to each grade and are less than or equal to the pixel resolution of the graphic objects.
According to the description, when the statistical calculation is carried out, the vector data sources corresponding to all the grades are pushed to the plurality of working nodes for distributed parallel calculation, so that the data calculation time length can be effectively shortened, and the efficiency of quick display can be further improved.
Further, the step S3 includes:
s31, segmenting the thinned vector data source according to the coordinate range of the vector data source, and constructing a plurality of block RDDs, wherein each block RDD is mapped to one working node;
s32, receiving a display request, and determining a corresponding pyramid grade according to the display request;
s33, determining a corresponding diluted vector data source according to pyramid grading, and determining an RDD block related to the display request according to the coordinate range of the RDD block;
and S34, rendering on the corresponding working nodes according to the related RDD blocks, receiving and summarizing the pictures rendered by the corresponding working nodes, and displaying the pictures.
According to the description, the vector data source after thinning is divided, a plurality of RDD blocks are constructed, each RDD block is mapped to one working node, when rendering is carried out, the RDD blocks have coordinate ranges and are only one part of the whole coordinate range, the pictures generated by rendering of the corresponding working nodes are one part of standard pictures, each working node only needs to render the corresponding part of the pictures, and the standard pictures do not need to be rendered like the prior standard pictures.
Example one
Referring to fig. 1, a method for fast displaying vector big data includes:
s1, establishing pyramid classification for a vector data source to obtain classified vector data, wherein each classification corresponds to different pixel resolutions, and the vector data source comprises a linear graph or a planar graph;
the vector pyramid is a technology for converting a vector data source to obtain a plurality of different-level data sources with different data volumes according to display requirements of different scales, and establishing pyramid classification for the vector data source comprises the following steps:
and carrying out different numbers of partitions on the vector data source, wherein the resolution ratio of each partition is the same, and the number of the partitions corresponding to different pyramid grades is different.
Specifically, 23 pyramid hierarchies may be established: assuming a global latitude and longitude range (-180: 180, -90: 90), the 0 level corresponds to 2 partitions, namely (-180: 0, -90: 90) and (0: 180, -90: 90), then each time, one current partition is divided into four (0 level is special except for), the 1 level corresponds to 8 partitions, the 2 level corresponds to 32 partitions, and the like, until 22 levels are reached, so that a vector pyramid of 0 level to 22 levels and a total of 23 levels is established, and a pixel resolution array A can be used for storing the pixel resolution corresponding to each level; assuming each partition is displayed using 256 x 256 pixel pictures, pixel resolution array a is: {0.7,0.35,0.17,0.08,0.04,0.02,0.01,0.005,0.0025,0.0012,0.0006,0.0003,0.00015,0.00008,0.00004,0.00002,0.00001,0.000005,0.0000025,0.0000012,0.0000006,0.0000003,0.00000015}, this pixel resolution array representing 23 pyramid hierarchies of 0-22, array a can also be expressed as { a0, a1, a2, …, a22 };
s2, thinning the classified vector data according to the size of the circumscribed rectangle of the linear graph or the planar graph and the resolution of each classified pixel to obtain thinned vector data;
s21, for each hierarchical vector data, respectively:
traversing each graphic object, and calculating the size of a circumscribed rectangle of the graphic object, wherein the size comprises the area or the perimeter;
comparing the size of the circumscribed rectangle of each graphic object with the pixel resolution corresponding to the grade of the circumscribed rectangle, and counting the total number and the total size of the graphic objects which are less than or equal to the pixel resolution corresponding to the grade of the circumscribed rectangle;
specifically, on the basis of the pixel resolution array A, a new null value array B is established, wherein the new null value array B is { B0, B1, B2, …, B22} and a new null value array C is { C0, C1, C2, …, C22 }, wherein each element bi in the array B is used for storing the total size of the graphic objects with the circumscribed rectangle size smaller than or equal to the pixel resolution corresponding to the vector data corresponding to the i +1 th hierarchical pyramid, namely the total size of the graphic objects with the circumscribed rectangle size smaller than or equal to the corresponding pixel resolution (namely the area statistic value or the perimeter statistic value), and each element C in the array C is used for storing the total size of the graphic objects with the circumscribed rectangle size smaller than or equal to ai (namely the area statistic value or the perimeter statistic value)iThe total number of the graphic objects with the size of the circumscribed rectangle less than or equal to the pixel resolution corresponding to the vector data for storing the (i + 1) th hierarchical pyramid, that is, the size of the circumscribed rectangle is less than or equal to aiThe total number of graphic objects of (1);
for example, b22 stores the total area of the graphic objects of which the area of the circumscribed rectangle is less than 0.00000015 or the total circumference of the graphic objects of which the circumference of the circumscribed rectangle is less than 0.00000015 in the vector data corresponding to 23 gradations;
c22 storing the total number of graphic objects of which the area of the circumscribed rectangle is less than 0.00000015 or the total number of graphic objects of which the perimeter of the circumscribed rectangle is less than 0.00000015 in the vector data corresponding to the 23-level classification;
meanwhile, a variable bt is newly established for storing the total size of all the graphic objects, namely the sum of the areas or the sum of the circumferences of all the graphic objects, and a variable ct is newly established for storing the total number of all the graphic objects;
during statistics, for the vector data corresponding to each grade, respectively counting the total area value or the total perimeter of the graphic object of which the area or the perimeter of the circumscribed rectangle under the grade is smaller than the corresponding pixel resolution of the grade, and then writing the total area value or the total perimeter into the corresponding element in the array B; counting the total number of the graphic objects of which the areas or the circumferences of the circumscribed rectangles under the grading are smaller than the corresponding grading pixel resolution, and writing the graphic objects into corresponding elements in the array C;
preferably, when the size of the circumscribed rectangle of the graphic object is calculated for each hierarchical vector data and the statistics of the total number and the total size of the graphic objects with the pixel resolution ratio smaller than or equal to the hierarchical position of the graphic object is calculated, the data source to be calculated or counted is pushed to a plurality of working nodes for distributed parallel calculation;
and receiving the data after the calculation of each working node is completed, and summarizing to obtain the total number and the total size of the graphic objects which correspond to each grade and are less than or equal to the pixel resolution of the graphic objects.
Specifically, under the environment of spark cluster and hbase cluster, a JavaRDD object of an original data source is constructed at a driver node, and a Shuffle method of the JavaRDD object is used for transferring the data source to each worker node, so that distributed statistical calculation is realized;
in distributed statistical calculation, each graph object of a data source is subjected to cyclic traversal, the size of an external rectangle of the data source is obtained through calculation, the counted area, or perimeter, and the counted area, or perimeter, are written into an array B, and the number of the array B is written into an array C (each graph is 1); after driver node collectdata are counted, the counted data are combined to obtain a complete array B, a complete variable bt, a complete array C and a complete variable ct;
s22, presetting a maximum thinning threshold and a minimum thinning threshold, and determining a maximum vector compression level and a minimum vector compression level according to the maximum thinning threshold, the minimum thinning threshold, the total number and the total size of the graphic objects which are corresponding to each grade and are less than or equal to the pixel resolution of the graphic objects;
specifically, in general, 30% of the small patch area is deleted, the overall display effect of the planar patch image layer is not affected when the whole image is displayed, and a percentage T (which may be set to 30% in this embodiment) is set as the maximum thinning threshold;
from the array B and the maximum thinning threshold T, a value, for example B6, is found, and assuming that the following condition is satisfied, i.e. the statistical value corresponding to B6 is less than or equal to bt × T, and the previous value B5 is greater than bt × T, the value B6 represents the meaning: if the graphic object with the circumscribed rectangle size less than or equal to a6(0.01) is deleted, the area or the perimeter of the deleted graphic object is just less than or equal to T, the overall display effect of the graphic is just not influenced, the vector compression ratio corresponding to a5 is too large, obvious data is lost during display, and a6 represents the size of the largest circumscribed rectangle which can be deleted by the user;
on the basis of a6, assuming that the compressed data source corresponding to a6 is d6, a0-a6 should all use d6 as the data source for display;
in general, if the vector compression ratio (the ratio of the number of graphics) is less than 50%, the meaning of compression is lost, and a percentage U (which may be set to 50% in this embodiment) is set as the minimum thinning threshold;
according to the array C and the minimum rarefaction threshold U, C6, C7 and C8 … are checked in sequence to find a value, such as C7, which satisfies the following conditions: that is, the statistical value corresponding to c7 is greater than or equal to ct × U, and the statistical value corresponding to c8 is less than ct × U, the value c7 represents the following meaning: if we delete the graphic objects whose bounding rectangle is less than or equal to a7(0.005), then the number of the deleted graphic objects is just greater than or equal to U, and just can play the role of vector compression, and the vector compression ratio corresponding to a8 is too small, so that the meaning of compression is lost, and a7 represents the size of the smallest bounding rectangle that we can delete;
on the basis of a7, assuming that the compressed data source corresponding to a7 is d7, the display data sources corresponding to d8-d22 are original data sources and do not need to be compressed;
through the above calculation, we have calculated the maximum compression level a6 and the minimum compression level a 7;
and S23, determining the data source of each pyramid grade according to the maximum vector compression grade and the minimum vector compression grade to obtain the diluted vector data.
Specifically, on the basis of the pixel resolution array A, a new null value array D is established, { D0, D1, D2, …, D22, }, and each element in the array D represents what data source is adopted by the pyramid at the corresponding level to obtain a complete array D according to the above summary: { d6, d6, d6, d6, d6, d6, d6, d7, d, d, …, d }, where d represents an original data source, that is, d0-d6 should use a compressed data source corresponding to a6, d7 uses a compressed data source corresponding to a7, and d8 to d22 do not need to be compressed, and they can use the original data source directly;
according to the array D, only two stages of compressed data sources of D6 and D7 need to be calculated, and compressed data can be obtained after graphic objects with the sizes of the external rectangles smaller than a6 or a7 in the corresponding levels are deleted according to the numerical values of a6 and a7 during calculation.
The calculated two-stage compressed data sources of d6 and d7 are stored in an hbase database together with the d of the original data source as a thinned vector data source; then, the corresponding relationship between the data of different grades and the vector data source after thinning can be obtained according to the array D, that is, the data of the 1 st to 7 th grades is the compressed data corresponding to a6, the data of the 8 th grade is the compressed data corresponding to a7, and the data of other grades is the original data source without compression.
Example two
The present embodiment is different from the first embodiment in that the step S3 includes:
s31, dividing the thinned vector data source according to the coordinate range of the vector data source, and constructing a plurality of partitioned RDDs (resource Distributed databases), wherein each partitioned RDD is mapped to a worker working node;
specifically, referring to fig. 2, when the system is initialized, the original data source d, and the compressed data sources d6 and d7 are divided into a plurality of partitions according to the total number of cluster cores, and a partitioned RDD is respectively constructed at the driver node according to the partition coordinate range, and the partitioned RDD requires to support the memory and the hard disk cache, so that the data traversal performance can be ensured. And the block RDD coordinate range is a part of the whole RDD coordinate range, and each block RDD is mapped to one worker working node. At the moment, pictures generated by rendering on worker nodes are a part of standard pictures, the sum of the pictures generated by rendering on all worker nodes is equal to the size of the standard pictures, the size of the pictures collected by transmission is fixed and unchanged, and the worker nodes can be infinitely expanded according to the size of data volume and performance requirements until the performance requirements are met.
S32, receiving a display request, and determining a corresponding pyramid grade according to the display request;
specifically, when a client display request is received, the pyramid hierarchy currently required to be used is calculated according to client request parameters.
S33, determining a corresponding diluted vector data source according to pyramid grading, and determining an RDD block related to the display request according to the coordinate range of the RDD block;
specifically, a target data source is obtained according to the array D, and then the RDD blocks related to the client display request at this time are searched according to the RDD block coordinate range of the target data source.
And S34, rendering on the corresponding working nodes according to the related RDD blocks, receiving and summarizing the pictures rendered by the corresponding working nodes, and displaying the pictures.
Specifically, rendering is completed at a worker node according to RDD blocks related to the client display request at this time, and pictures are collected and summarized at a driver node and transmitted back to the client to complete display.
When distributed parallel rendering is carried out, an RDD blocking strategy is adopted, the dynamic rendering performance of vector data is improved, the rapid display capability of large vector data is enhanced, and the parallel rendering performance and the supportable data volume can be linearly increased along with the increase of cluster nodes.
EXAMPLE III
Referring to fig. 3, a terminal 1 for fast displaying vector big data includes a memory 2, a processor 3, and a computer program stored in the memory 2 and capable of running on the processor 3, where the processor 3 implements the steps of the first embodiment or the second embodiment when executing the computer program.
In summary, in the method and the terminal for rapidly displaying the vector big data provided by the present invention, in the whole process of rapidly displaying the vector big data, a vector pyramid is established through a vector compression technology, a distributed parallel technology is adopted, the vector data is thinned based on the size of the circumscribed rectangle of the linear graph or the planar graph, a plurality of RDD blocks are constructed according to the thinned vector data, and the distributed parallel rendering is performed, so that the overall display effect of the graph layer is not affected when the whole graph is displayed, the vector compression effect is also achieved, the data volume displayed by the graph object is reduced, the thinning effect meets the requirement of the data volume to be displayed, and the hierarchical relation between the thinned data and the pyramid is easier to be established, thereby facilitating the programming implementation, reducing the preprocessing time, and improving the display performance; by partitioning a plurality of RDDs and performing distributed parallel rendering, the dynamic rendering performance of vector data is improved, the rapid display capability of large vector data is enhanced, and the rapid display of larger data volume can be supported; the problems that the preprocessing time is too long, the display style cannot be dynamically modified, the display data size and the performance are limited and the like are solved at one time, and the vector big data can be dynamically and rapidly displayed.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all equivalent changes made by using the contents of the present specification and the drawings, or applied directly or indirectly to the related technical fields, are included in the scope of the present invention.
Claims (10)
1. A method for rapidly displaying vector big data is characterized by comprising the following steps:
s1, establishing pyramid classification for a vector data source to obtain classified vector data, wherein each classification corresponds to different pixel resolutions, and the vector data source comprises a linear graph or a planar graph;
s2, thinning the classified vector data according to the size of the circumscribed rectangle of the linear graph or the planar graph and the resolution of each classified pixel to obtain thinned vector data;
and S3, displaying the vector data of different pyramid grades according to the thinned vector data.
2. The method for rapidly displaying vector big data according to claim 1, wherein the step S1 of building a pyramid hierarchy for the vector data source comprises:
and carrying out different numbers of partitions on the vector data source, wherein the resolution ratio of each partition is the same, and the number of the partitions corresponding to different pyramid grades is different.
3. The method for rapidly displaying vector big data according to claim 1, wherein the step S2 comprises:
s21, for each hierarchical vector data, respectively:
traversing each graphic object, and calculating the size of a circumscribed rectangle of the graphic object, wherein the size comprises the area or the perimeter;
comparing the size of the circumscribed rectangle of each graphic object with the pixel resolution corresponding to the grade of the circumscribed rectangle, and counting the total number and the total size of the graphic objects which are less than or equal to the pixel resolution corresponding to the grade of the circumscribed rectangle;
s22, presetting a maximum thinning threshold and a minimum thinning threshold, and determining a maximum vector compression level and a minimum vector compression level according to the maximum thinning threshold, the minimum thinning threshold, the total number and the total size of the graphic objects which are corresponding to each grade and are less than or equal to the pixel resolution of the graphic objects;
and S23, determining the data source of each pyramid grade according to the maximum vector compression grade and the minimum vector compression grade to obtain the diluted vector data.
4. The method for rapidly displaying vector big data according to claim 2, wherein when the size of the circumscribed rectangle of the graphic object is calculated for each hierarchical vector data and the total number and the total size of the graphic objects with the pixel resolution less than or equal to the hierarchical pixel resolution where the hierarchical pixel resolution is located are counted, the data source to be calculated or counted is pushed to a plurality of working nodes for distributed parallel calculation;
and receiving the data after the calculation of each working node is completed, and summarizing to obtain the total number and the total size of the graphic objects which correspond to each grade and are less than or equal to the pixel resolution of the graphic objects.
5. The method for rapidly displaying vector big data according to any one of claims 1 to 4, wherein the step S3 comprises:
s31, segmenting the thinned vector data source according to the coordinate range of the vector data source, and constructing a plurality of block RDDs, wherein each block RDD is mapped to one working node;
s32, receiving a display request, and determining a corresponding pyramid grade according to the display request;
s33, determining a corresponding diluted vector data source according to pyramid grading, and determining an RDD block related to the display request according to the coordinate range of the RDD block;
and S34, rendering on the corresponding working nodes according to the related RDD blocks, receiving and summarizing the pictures rendered by the corresponding working nodes, and displaying the pictures.
6. A terminal for rapidly displaying vector big data, comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to realize the following steps:
s1, establishing pyramid classification for a vector data source to obtain classified vector data, wherein each classification corresponds to different pixel resolutions, and the vector data source comprises a linear graph or a planar graph;
s2, thinning the classified vector data according to the size of the circumscribed rectangle of the linear graph or the planar graph and the resolution of each classified pixel to obtain thinned vector data;
and S3, displaying the vector data of different pyramid grades according to the thinned vector data.
7. The terminal for fast displaying vector big data according to claim 6, wherein the step S1 of building a pyramid hierarchy for the vector data source comprises:
and carrying out different numbers of partitions on the vector data source, wherein the resolution ratio of each partition is the same, and the number of the partitions corresponding to different pyramid grades is different.
8. The terminal for fast displaying vector big data according to claim 6, wherein the step S2 comprises:
s21, for each hierarchical vector data, respectively:
traversing each graphic object, and calculating the size of a circumscribed rectangle of the graphic object, wherein the size comprises the area or the perimeter;
comparing the size of the circumscribed rectangle of each graphic object with the pixel resolution corresponding to the grade of the circumscribed rectangle, and counting the total number and the total size of the graphic objects which are less than or equal to the pixel resolution corresponding to the grade of the circumscribed rectangle;
s22, presetting a maximum thinning threshold and a minimum thinning threshold, and determining a maximum vector compression level and a minimum vector compression level according to the maximum thinning threshold, the minimum thinning threshold, the total number and the total size of the graphic objects which are corresponding to each grade and are less than or equal to the pixel resolution of the graphic objects;
and S23, determining the data source of each pyramid grade according to the maximum vector compression grade and the minimum vector compression grade to obtain the diluted vector data.
9. The terminal for rapidly displaying vector big data according to claim 7, wherein when the size of the circumscribed rectangle of the graphic object is calculated for each hierarchical vector data and the total number and the total size of the graphic objects with the pixel resolution less than or equal to the hierarchical pixel resolution where the hierarchical pixel resolution is located are counted, the data source to be calculated or counted is pushed to a plurality of working nodes for distributed parallel calculation;
and receiving the data after the calculation of each working node is completed, and summarizing to obtain the total number and the total size of the graphic objects which correspond to each grade and are less than or equal to the pixel resolution of the graphic objects.
10. The terminal for fast displaying vector big data according to any of claims 6 to 9, wherein the step S3 comprises:
s31, segmenting the thinned vector data source according to the coordinate range of the vector data source, and constructing a plurality of block RDDs, wherein each block RDD is mapped to one working node;
s32, receiving a display request, and determining a corresponding pyramid grade according to the display request;
s33, determining a corresponding diluted vector data source according to pyramid grading, and determining an RDD block related to the display request according to the coordinate range of the RDD block;
and S34, rendering on the corresponding working nodes according to the related RDD blocks, receiving and summarizing the pictures rendered by the corresponding working nodes, and displaying the pictures.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911278725.6A CN110990612B (en) | 2019-12-13 | 2019-12-13 | Method and terminal for rapidly displaying vector big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911278725.6A CN110990612B (en) | 2019-12-13 | 2019-12-13 | Method and terminal for rapidly displaying vector big data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110990612A true CN110990612A (en) | 2020-04-10 |
| CN110990612B CN110990612B (en) | 2022-08-02 |
Family
ID=70093077
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911278725.6A Active CN110990612B (en) | 2019-12-13 | 2019-12-13 | Method and terminal for rapidly displaying vector big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110990612B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111857627A (en) * | 2020-07-16 | 2020-10-30 | 北京奇虎科技有限公司 | Graphic processing method, apparatus, device and storage medium |
| CN112817545A (en) * | 2021-03-11 | 2021-05-18 | 福州大学 | Method and system for storing and managing data of on-line analysis-while-analyzing image and grid cube |
| CN114998108A (en) * | 2022-08-01 | 2022-09-02 | 广州城市信息研究所有限公司 | Vector data optimization processing method and system |
| CN115018712A (en) * | 2022-08-01 | 2022-09-06 | 广州城市信息研究所有限公司 | Distributed scheduling method and system for map data |
| CN116069833A (en) * | 2022-12-30 | 2023-05-05 | 北京航天测控技术有限公司 | Mass data processing and visualizing method based on dynamic multi-level resolution self-adaptive analysis |
| CN116450872A (en) * | 2023-05-04 | 2023-07-18 | 易智瑞信息技术有限公司 | Spark distributed vector grid turning method, system and equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2533279A1 (en) * | 2005-01-18 | 2006-07-18 | Ryan Eccles | System and method for processing map data |
| CN103986763A (en) * | 2014-05-13 | 2014-08-13 | 中国科学院地理科学与资源研究所 | Vector data stream transmission method and system oriented to Web visualization |
| CN105760529A (en) * | 2016-03-03 | 2016-07-13 | 福州大学 | Spatial index and cache construction method for vector data of mobile terminal |
| CN107133272A (en) * | 2017-04-07 | 2017-09-05 | 南京南瑞集团公司 | A kind of Web ends magnanimity power network resources GIS data dynamic dispatching and rendering intent |
| CN110032610A (en) * | 2019-03-12 | 2019-07-19 | 紫光云数科技有限公司 | A kind of scale vector method for exhibiting data and system |
-
2019
- 2019-12-13 CN CN201911278725.6A patent/CN110990612B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2533279A1 (en) * | 2005-01-18 | 2006-07-18 | Ryan Eccles | System and method for processing map data |
| CN103986763A (en) * | 2014-05-13 | 2014-08-13 | 中国科学院地理科学与资源研究所 | Vector data stream transmission method and system oriented to Web visualization |
| CN105760529A (en) * | 2016-03-03 | 2016-07-13 | 福州大学 | Spatial index and cache construction method for vector data of mobile terminal |
| CN107133272A (en) * | 2017-04-07 | 2017-09-05 | 南京南瑞集团公司 | A kind of Web ends magnanimity power network resources GIS data dynamic dispatching and rendering intent |
| CN110032610A (en) * | 2019-03-12 | 2019-07-19 | 紫光云数科技有限公司 | A kind of scale vector method for exhibiting data and system |
Non-Patent Citations (1)
| Title |
|---|
| 王枫: "矢量数据分级显示方法研究", 《中国优秀硕士学位论文全是数据库 基础科学辑》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111857627A (en) * | 2020-07-16 | 2020-10-30 | 北京奇虎科技有限公司 | Graphic processing method, apparatus, device and storage medium |
| CN112817545A (en) * | 2021-03-11 | 2021-05-18 | 福州大学 | Method and system for storing and managing data of on-line analysis-while-analyzing image and grid cube |
| CN114998108A (en) * | 2022-08-01 | 2022-09-02 | 广州城市信息研究所有限公司 | Vector data optimization processing method and system |
| CN115018712A (en) * | 2022-08-01 | 2022-09-06 | 广州城市信息研究所有限公司 | Distributed scheduling method and system for map data |
| CN114998108B (en) * | 2022-08-01 | 2022-11-15 | 广州城市信息研究所有限公司 | Vector data optimization processing method and system |
| CN115018712B (en) * | 2022-08-01 | 2022-11-15 | 广州城市信息研究所有限公司 | Distributed scheduling method and system for map data |
| CN116069833A (en) * | 2022-12-30 | 2023-05-05 | 北京航天测控技术有限公司 | Mass data processing and visualizing method based on dynamic multi-level resolution self-adaptive analysis |
| CN116069833B (en) * | 2022-12-30 | 2024-04-19 | 北京航天测控技术有限公司 | Mass data processing and visualizing method based on dynamic multi-level resolution self-adaptive analysis |
| CN116450872A (en) * | 2023-05-04 | 2023-07-18 | 易智瑞信息技术有限公司 | Spark distributed vector grid turning method, system and equipment |
| CN116450872B (en) * | 2023-05-04 | 2024-02-20 | 易智瑞信息技术有限公司 | Spark distributed vector grid turning method, system and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110990612B (en) | 2022-08-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110990612B (en) | Method and terminal for rapidly displaying vector big data | |
| CN108664619B (en) | Primitive storage and scheduling method for mass line-drawing topographic map of tile-like technology | |
| US10262392B2 (en) | Distributed and parallelized visualization framework | |
| US20020113797A1 (en) | Systems and methods for representing and displaying graphics | |
| CN111862292B (en) | Data rendering method and device for transmission line corridor and computer equipment | |
| CN112115534A (en) | Method for converting three-dimensional house model into two-dimensional vector plane with height attribute | |
| CN113032513B (en) | Slicing method and rendering method for spatial geographic data | |
| CN110990516A (en) | Map data processing method and device and server | |
| CN110597900B (en) | Method for generating vector slice by GDB data in real time according to needs | |
| CN111090716A (en) | Vector tile data processing method, device, equipment and storage medium | |
| CN111899323A (en) | Three-dimensional earth drawing method and device | |
| CN112115226B (en) | Map rendering method and map rendering device | |
| Goldin et al. | Georouting and delta-gathering: Efficient data propagation techniques for geosensor networks | |
| CN111581233B (en) | Real-time slicing method for vector graphic data of power grid equipment and storage medium | |
| CN115018712B (en) | Distributed scheduling method and system for map data | |
| CN110727746A (en) | Buffer analysis parallel scheduling method and device based on secondary task division strategy | |
| US11907283B2 (en) | Multi-resolution raster data access method, apparatus, electronic device, and computer storage medium | |
| CN115408382A (en) | Vector space information gridding storage management method for large-scale satellite data | |
| CN116975170A (en) | Map display method, map data generation method, map display device and electronic equipment | |
| CN116797747A (en) | Underwater detection data visualization method, device, computer equipment and storage medium | |
| CN115761119A (en) | Neighborhood voxel storage method and device, computer equipment and storage medium | |
| Zhou et al. | Database support for spatial generalisation for WWW and mobile applications | |
| CN117237503B (en) | Geographic element data accelerated rendering and device | |
| CN113658033B (en) | GPU method for calculating internal point set of given region | |
| CN112446959B (en) | Oblique photography model optimization method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |