CN111782745B - Space-time big data grid coding efficient visualization method and system - Google Patents
Space-time big data grid coding efficient visualization method and system Download PDFInfo
- Publication number
- CN111782745B CN111782745B CN202010594839.8A CN202010594839A CN111782745B CN 111782745 B CN111782745 B CN 111782745B CN 202010594839 A CN202010594839 A CN 202010594839A CN 111782745 B CN111782745 B CN 111782745B
- Authority
- CN
- China
- Prior art keywords
- data
- space
- visual
- coding
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007794 visualization technique Methods 0.000 title claims abstract description 26
- 230000000007 visual effect Effects 0.000 claims abstract description 247
- 238000013079 data visualisation Methods 0.000 claims abstract description 72
- 238000012800 visualization Methods 0.000 claims abstract description 66
- 238000012545 processing Methods 0.000 claims abstract description 20
- 238000000034 method Methods 0.000 claims description 37
- 230000008859 change Effects 0.000 claims description 13
- 230000009467 reduction Effects 0.000 claims description 13
- 230000007246 mechanism Effects 0.000 claims description 8
- 239000003086 colorant Substances 0.000 claims description 7
- 238000010276 construction Methods 0.000 claims description 5
- 238000003672 processing method Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 8
- 238000007726 management method Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 238000012544 monitoring process Methods 0.000 description 5
- 230000004044 response Effects 0.000 description 5
- 229910052792 caesium Inorganic materials 0.000 description 4
- TVFDJXOCXUVLDH-UHFFFAOYSA-N caesium atom Chemical compound [Cs] TVFDJXOCXUVLDH-UHFFFAOYSA-N 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 229910002056 binary alloy Inorganic materials 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000005906 menstruation Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Software Systems (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a space-time big data grid coding efficient visualization method and a system, wherein the system comprises a space-time big data visualization model, a space-time database, an acquisition module, a grid coding module and a visual domain data visualization processing module, and a grid coding index data structure is constructed on the space-time big data visualization model; the visual domain selection module is arranged on the space-time big data visualization model, and the visual domain data visualization processing module comprises a visualization main thread module and a data request sub-thread module. The visual field can be selected or set on the space-time big data visualization model according to needs, the data request sub-thread queries the space-time data of the corresponding visual field according to the visual field coding index data set and transmits the space-time data to the visualization main thread for visualization expression, and the visual domain coding index data set visualization method has the advantages of data query, visualization expression, high visualization loading quality efficiency and the like, and remarkably improves the data loading and visualization fluency.
Description
Technical Field
The invention relates to the technical field of space-time big data visualization, in particular to a space-time big data grid coding efficient visualization method and system.
Background
Spatiotemporal big data is big data that is based on a unified spatiotemporal reference, activities in the space-time are directly or indirectly associated with locations. The data set is the 'sum' of the quantity and quality characteristics of each element (phenomenon) of the space structure and the space relation of the real geographic world including human activities and the data set changed in space and time, and has the characteristics of space, time, attributes, multi-source (meta) isomerism, multi-dimensional dynamics and the like. The visualization of the space-time big data Web end generally comprises the steps that a front-end browser sends a data request, a back-end server receives the front-end request to perform data query and data processing, then a result is returned to the front end, and the front end receives data and completes data loading and visualization. Due to the fact that the space-time big data are large in quantity and various in acquisition modes, the data request quantity of the Web end is very large when the Web end acquires the back-end data, the data query usually needs more time, meanwhile, the data analysis and processing also need time, the visualization time consumption is large, and the visualization efficiency of the whole space-time big data is low.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a space-time big data grid coding efficient visualization method and system.
The purpose of the invention is realized by the following technical scheme:
a space-time big data grid coding efficient visualization method comprises the following steps:
A. establishing a space-time big data visualization model, constructing a gridding coding index data structure on the space-time big data visualization model, simultaneously obtaining a coding index data set, and storing the coding index data set into a space-time database, wherein the coding index data set comprises codes, indexes and data corresponding relations; selecting or setting a visible area on the space-time big data visualization model, and obtaining a visible area coding index data set corresponding to the visible area;
B. setting a visual main thread and a data request sub-thread aiming at a visual domain of a space-time big data visual model; the method comprises the steps that spatiotemporal data are collected, grid coding dimension reduction projection processing is carried out on the spatiotemporal data, the spatiotemporal data corresponding to visible areas are inquired by the data request sub-thread according to a visible area coding index data set and are transmitted to a visual main thread, and the visual main thread carries out visual expression on the spatiotemporal data of the visible areas;
C. the method comprises the steps of collecting visual field space-time data in real time, carrying out intersection operation on the corresponding space-time data before and after the visual field is updated, clearing the space-time data outside the visual field, transmitting newly-added space-time data in the visual field to a visual main thread by a data request sub-thread, and updating visual expression of the space-time data of the visual field in the visual field of a space-time big data visual model by the visual main thread.
The optimal gridding coding index data structure construction method of the space-time big data grid coding high-efficiency visualization method comprises the following steps:
a1, according to the principle of equal longitude difference and equal latitude difference, subdividing a gridded space-time big data visualization model into a plurality of sub-level grid units according to a quadtree grid, constructing a multi-level gridding coding index data structure, and carrying out gridding coding expression on longitude and latitude data by utilizing the multi-level gridding coding index data structure.
The optimal grid coding dimension reduction projection processing method of the space-time big data grid coding high-efficiency visualization method of the invention is as follows:
and A2, carrying out grid coding dimension reduction projection processing on the collected space-time data according to a grid coding index data structure to obtain the codes and indexes of each space-time data.
The optimal visual field coding index data set matching or obtaining method of the space-time big data grid coding efficient visualization method is as follows:
a3 warp difference l in step A1 gridding0Weft difference b0Visual field gridding mean diameter difference l1Weft difference b1And acquiring a visible field coding row and column number set X and Y:
visible field coded line number set
Visual field coding column number set
Wherein x represents a mesh coding level;
and A4, performing interleaved bit-fetching coding on the binary representations of the row numbers and the column numbers of the visible field coding set according to the Morton coding rule to obtain a visible field coding index data set of the visible field.
The optimal step B of the space-time big data grid coding high-efficiency visualization method also comprises the following steps:
b1, establishing a batch data request management mechanism: the visual main thread distributes data request tasks to data request sub-threads in batches, and the data request sub-threads inquire the time-space data of the corresponding visual field according to the visual field coding index data set and transmit the inquiry result to the visual main thread in batches;
and B2, the visualization main thread and the data request sub-thread are separated and synchronously performed, and the visualization main thread performs visualization expression in the visual domain of the space-time big data visualization model after receiving the query result and the data of the data request sub-thread.
The optimal step C of the space-time big data grid coding high-efficiency visualization method also comprises the following steps: and C, performing intersection operation on the coding set A and the coding set B, eliminating the space-time data corresponding to the [ B- (A & n & gtB) ] coding set, and loading and visualizing the newly-added space-time data corresponding to the [ A- (A & n & gtB) ] coding set in the visual domain of the space-time big data visualization model through a visualization main line.
The optimal gridding coding index data structure of the space-time big data grid coding high-efficiency visualization method forms a plurality of visualization grid units with equal longitude differences and equal latitude differences after the quad-tree grid is divided, so that the visualization grid units in the visual field of the space-time big data visualization model are obtained, the visualization main thread updates and visualizes the space-time data of the visual field on each visualization grid unit in the visual field, and the visualization expression mode comprises the adoption of numbers, colors and graphs.
A space-time big data grid coding efficient visualization method comprises the following steps:
A. establishing a space-time big data visualization model, and establishing a gridding coding index data structure on the space-time big data visualization model, wherein the construction method of the gridding coding index data structure comprises the following steps: according to the principle of equal longitude difference and equal latitude difference, a plurality of sub-level grid units are divided for the time-space big data visualization model according to the quad-tree grid;
according to the warp difference l in gridding0Weft difference b0Visual field gridding mean diameter difference l1Weft difference b1Obtaining a set of visible field code row and column numbers X and Y:
visible field coded line number set
Visual field coding column number set
Wherein x represents a mesh coding level;
carrying out staggered bit-fetching coding on binary representations of row numbers and column numbers of a visible field coding set according to a Morton coding rule to obtain a visible field coding index data set of a visible field, wherein the visible field coding index data set comprises codes and corresponding relations between indexes and data;
B. selecting or setting a visible area on the space-time big data visualization model and openly uploading the visible area to an internet web end, or uploading the space-time big data visualization model to the internet web end and selecting or setting the visible area on the internet web end; setting a visual main thread and a data request sub-thread aiming at a visual domain of a space-time big data visual model; establishing a batch data request management mechanism, synchronously performing a visual main thread and a data request sub-thread, and distributing data request tasks to the data request sub-thread in batches by the visual main thread;
C. the method comprises the steps that space-time data are collected in real time, a data request sub-thread codes the space-time data according to longitude and latitude, the data request sub-thread inquires and collects space-time data corresponding to a visual field according to a visual field code index data set, the data request sub-thread performs intersection operation on codes of the space-time data before and after the visual field is updated, space-time data outside the visual field are cleared, the data request sub-thread transmits newly added space-time data in the visual field to a visual main thread, and the visual main thread updates and visually expresses the space-time data in the visual field on a space-time big data visual model.
The optimal step C of the space-time big data grid coding high-efficiency visualization method also comprises the following steps: in the step C, the code set B before updating and the code set A after updating perform intersection operation on the code set A and the code set B, firstly, the space-time data corresponding to the [ B- (A N B) ] code set is eliminated, and then, the newly added space-time data corresponding to the [ A- (A N B) ] code set is loaded and visualized through a visualization main thread; the visual expression is correspondingly updated on each visual grid unit in the visual domain, and the visual expression mode comprises the adoption of numbers, colors and graphs;
the acquired space-time data is stored in a space-time database, and new increment, accumulated amount and change rate are counted according to time dimension and are visually expressed.
A space-time big data grid coding efficient visualization system comprises a space-time big data visualization model, a space-time database, an acquisition module, a grid coding module and a visual domain data visualization processing module, wherein a gridding coding index data structure is constructed on the space-time big data visualization model, the gridding coding index data structure divides the gridded space-time big data visualization model into a plurality of sub-level grid units according to a quadtree and grid according to the principle of equal warp difference and equal weft difference, a multi-level gridding coding index data structure is constructed, and a coding index data set is obtained and stored in the space-time database;
the space-time big data visualization model is provided with a visual field selection module, and the visual field selection module is used for selecting or setting a visual field on the space-time big data visualization model and obtaining a visual field coding index data set corresponding to the visual field; the acquisition module is used for acquiring space-time data, and the grid coding module is used for carrying out grid coding processing on the space-time data;
the visual domain data visualization processing module comprises a visual main thread module and a data request sub-thread module, the data request sub-thread module is used for inquiring the space-time data corresponding to the visual domain according to the visual domain code index data set, then performing intersection operation on the codes of the corresponding space-time data before and after the visual domain is updated, obtaining the newly added space-time data in the visual domain and then transmitting the newly added space-time data to the visual main thread, and the visual main thread module is used for performing update visualization expression on the space-time data in the visual domain of the space-time big data visualization model.
Compared with the prior art, the invention has the following advantages and beneficial effects:
(1) according to the method, a space-time big data visualization model is established, a gridding coding index data structure is established on the space-time big data visualization model, a coding index data set is obtained, a user selects or sets a visual field on the space-time big data visualization model according to needs, and meanwhile, a data request sub-thread inquires space-time data corresponding to the visual field according to the visual field coding index data set and transmits the space-time data to a visual main thread for visual expression.
(2) According to the invention, a batch data request management mechanism is established, and a visual main thread and a data request sub-thread are separated and synchronously performed, so that the data request query efficiency and visual expression and loading speed are improved, the response time is greatly reduced, and the smoothness of system loading and visualization is improved.
(3) The invention unifies data query and visual expression by utilizing a quadtree grid index, namely, the grid coding index is utilized to improve the data request and data query efficiency, and grid units are used as data visual analysis expression units; the grid unit reduces the difficulty of data organization, multi-source data fusion analysis and visualization, and is widely applied to the fields of track density analysis, urban traffic situation monitoring and the like.
(4) The invention can freely select and set the visible field and upload the visible field to the web end of the Internet so as to facilitate the setting of the visible field and the viewing and tracking of visual contents, realizes the intersection operation of codes of corresponding time-space data before and after the updating of the visible field and can quickly obtain newly-added time-space data in the visible field, can realize the operations of quick query, updating, visual loading and the like of the time-space data in the visible field, conveniently realizes the visual expression and real-time tracking of the visible field, and can also realize the statistics of new increment, accumulated amount and change rate according to time dimension and carry out visual expression.
Drawings
FIG. 1 is a schematic flow chart of a method according to a second embodiment;
FIG. 2 is a schematic reference diagram of the generation of a quadtree coding index according to the third embodiment;
FIG. 3 is a diagram illustrating Morton encoding and dimension reduction according to a third embodiment;
FIG. 4 is a schematic illustration of a main thread and a sub-thread in the third embodiment;
FIG. 5 is a diagram illustrating trellis-encoded dimension reduction projection of spatiotemporal data according to a third embodiment;
FIG. 6 is a diagram illustrating an effect of visually expressing track points in grid cells with thematic color symbols according to the fourth embodiment;
FIG. 7 is a schematic diagram illustrating a first-level geogrid defined according to the principle of equal warp and difference weft in accordance with a fourth embodiment;
FIG. 8 is a diagram showing the effect of the visualization update expression in the fourth embodiment;
FIG. 9 is an effect diagram after the visualization is updated according to the fourth embodiment;
FIG. 10 is a software schematic block diagram of the system of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the following examples:
example one
A space-time big data grid coding efficient visualization method comprises the following steps:
A. establishing a space-time big data visualization model, constructing a gridding coding index data structure on the space-time big data visualization model, simultaneously obtaining a coding index data set, and storing the coding index data set into a space-time database, wherein the coding index data set comprises codes, indexes and data corresponding relations; selecting or setting a visible area on the space-time big data visualization model, and obtaining a visible area coding index data set corresponding to the visible area;
B. setting a visual main thread and a data request sub-thread aiming at a visual domain of a space-time big data visual model; the method comprises the steps that spatiotemporal data are collected, grid coding dimension reduction projection processing is carried out on the spatiotemporal data, the spatiotemporal data corresponding to visible areas are inquired by the data request sub-thread according to a visible area coding index data set and are transmitted to a visual main thread, and the visual main thread carries out visual expression on the spatiotemporal data of the visible areas;
C. the method comprises the steps of collecting visual field space-time data in real time, carrying out intersection operation on the corresponding space-time data before and after the visual field is updated, clearing the space-time data outside the visual field, transmitting newly-added space-time data in the visual field to a visual main thread by a data request sub-thread, and updating visual expression of the space-time data of the visual field in the visual field of a space-time big data visual model by the visual main thread.
The method for constructing the data structure of the gridding coding index in the embodiment is as follows:
a1, according to the principle of equal longitude difference and equal latitude difference, subdividing a gridded space-time big data visualization model into a plurality of sub-level grid units according to a quadtree grid, constructing a multi-level gridding coding index data structure, and carrying out gridding coding expression on longitude and latitude data by utilizing the multi-level gridding coding index data structure.
The grid coding dimension reduction projection processing method in the embodiment is as follows:
and A2, carrying out grid coding dimension reduction projection processing on the collected space-time data according to a grid coding index data structure to obtain the codes and indexes of each space-time data.
The matching or obtaining method for the visible encoding index data set in this embodiment is as follows:
a3 warp difference l in step A1 gridding0Weft difference b0Visual field gridding mean diameter difference l1Weft difference b1And acquiring a visible field coding row and column number set X and Y:
visible field coded line number set
Visual field coding column number set
Wherein x represents a mesh coding level;
and A4, performing interleaved bit-fetching coding on the binary representations of the row numbers and the column numbers of the visible field coding set according to the Morton coding rule to obtain a visible field coding index data set of the visible field.
In this embodiment, step B further includes:
b1, establishing a batch data request management mechanism: the visual main thread distributes data request tasks to data request sub-threads in batches, and the data request sub-threads inquire the time-space data of the corresponding visual field according to the visual field coding index data set and transmit the inquiry result to the visual main thread in batches;
and B2, the visualization main thread and the data request sub-thread are separated and synchronously performed, and the visualization main thread performs visualization expression in the visual domain of the space-time big data visualization model after receiving the query result and the data of the data request sub-thread.
In this embodiment, step C further includes the following steps: and C, performing intersection operation on the coding set A and the coding set B, eliminating the space-time data corresponding to the [ B- (A & n & gtB) ] coding set, and loading and visualizing the newly-added space-time data corresponding to the [ A- (A & n & gtB) ] coding set in the visual domain of the space-time big data visualization model through a visualization main line.
In this embodiment, the grid coding index data structure forms a plurality of visual grid units with equal longitude differences and equal latitude differences after the quadtree grid is divided, so as to obtain the visual grid units in the visual domain of the space-time big data visual model, the visual main thread updates and visually expresses the space-time data of the visual domain on each visual grid unit in the visual domain, and the visual expression mode includes the adoption of numbers, colors, graphs or other forms.
As shown in fig. 10, a space-time big data grid coding efficient visualization system includes a space-time big data visualization model, a space-time database, an acquisition module, a grid coding module and a visual domain data visualization processing module, wherein a grid coding index data structure is constructed on the space-time big data visualization model, and the grid coding index data structure divides a plurality of sub-level grid units according to a quadtree and a grid on the grid of the grid space-time big data visualization model by an equal warp difference and equal weft difference principle, constructs a multi-level grid coding index data structure, and obtains a coding index data set to store in the space-time database;
the space-time big data visualization model is provided with a visual field selection module, and the visual field selection module is used for selecting or setting a visual field on the space-time big data visualization model and obtaining a visual field coding index data set corresponding to the visual field; the grid coding module is used for carrying out grid coding processing on the space-time data collected by the collection module;
the visual domain data visualization processing module comprises a visual main thread module and a data request sub-thread module, the data request sub-thread module is used for inquiring the space-time data corresponding to the visual domain according to the visual domain code index data set, then performing intersection operation on codes of the corresponding space-time data before and after the visual domain is updated to obtain newly-added space-time data in the visual domain, and transmitting the newly-added space-time data to the visual main thread, and the visual main thread module is used for performing update visualization expression on the space-time data in the visual domain of the space-time big data visualization model; the gridding coding index data structure divides a gridding space-time big data visualization model into a plurality of sub-level grid units according to a quadtree and a grid according to the principle of equal longitude difference and equal latitude difference; constructing a gridding coding index data structure according to the warp difference and the weft difference of grid units, wherein the gridding coding index data structure comprises a plurality of visual grid units with equal warp difference and equal weft difference; the grid coding index data structure forms a plurality of visual grid units with equal longitude differences and equal latitude differences after the quad-tree grid is divided, so that the visual grid units in the visual domain of a space-time big data visual model are obtained, the visual main thread module updates and visually expresses space-time data of the visual domain on each visual grid unit in the visual domain, and the visual expression mode comprises the mode of adopting numbers, colors, graphs or other forms.
Example two
A space-time big data grid coding efficient visualization method comprises the following steps:
A. establishing a space-time big data visualization model, and establishing a gridding coding index data structure on the space-time big data visualization model, wherein the construction method of the gridding coding index data structure comprises the following steps: according to the principle of equal longitude difference and equal latitude difference, a plurality of sub-level grid units are divided for the time-space big data visualization model according to the quad-tree grid;
according to the warp difference l in gridding0Weft difference b0Visual field gridding mean diameter difference l1Weft difference b1Obtaining a set of visible field code row and column numbers X and Y:
visible field coded line number set
Visual field coding column number set
Wherein x represents a mesh coding level;
carrying out staggered bit-fetching coding on binary representations of row numbers and column numbers of a visible field coding set according to a Morton coding rule to obtain a visible field coding index data set of a visible field, wherein the visible field coding index data set comprises codes and corresponding relations between indexes and data; and constructing a gridding coding index data structure according to the warp difference and the weft difference of the grid units, wherein the gridding coding index data structure comprises a plurality of visual grid units with equal warp difference and equal weft difference.
B. Selecting or setting a visible area on the space-time big data visualization model and openly uploading the visible area to an internet web end, or uploading the space-time big data visualization model to the internet web end and selecting or setting the visible area on the internet web end; setting a visual main thread and a data request sub-thread aiming at a visual domain of a space-time big data visual model; establishing a batch data request management mechanism, synchronously performing a visual main thread and a data request sub-thread, and distributing data request tasks to the data request sub-thread in batches by the visual main thread;
C. the method comprises the steps that space-time data are collected in real time, a data request sub-thread codes the space-time data according to longitude and latitude, the data request sub-thread inquires and collects space-time data corresponding to a visual field according to a visual field code index data set, the data request sub-thread performs intersection operation on codes of the space-time data before and after the visual field is updated, space-time data outside the visual field are cleared, the data request sub-thread transmits newly added space-time data in the visual field to a visual main thread, and the visual main thread updates and visually expresses the space-time data in the visual field on a space-time big data visual model. Step C of this embodiment further includes the following steps: in the step C, the code set B before updating and the code set A after updating perform intersection operation on the code set A and the code set B, firstly, the space-time data corresponding to the [ B- (A N B) ] code set is eliminated, and then, the newly added space-time data corresponding to the [ A- (A N B) ] code set is loaded and visualized through a visualization main thread; the visual expression is correspondingly updated on each visual grid unit in the visual domain, and the visual expression mode comprises the adoption of numbers, colors, graphs or other forms;
the acquired space-time data is stored in a space-time database, and new increment, accumulated amount and change rate are counted according to time dimension and are visually expressed.
EXAMPLE III
As shown in fig. 1, a space-time big data trellis coding efficient visualization method includes the following steps:
s1, establishing a space-time big data visualization model, establishing a local area quadtree grid coding index on the space-time big data visualization model, carrying out grid coding dimension reduction expression on the space-time big data, realizing the integration of coding-index-data, and acquiring a geospatial coding set (also called coding index data set) corresponding to a visual field;
s2, a method for separating a gridding space-time big data front-end data request and a visualization thread is provided, a sub-thread (also called a data request sub-thread) queries gridding space-time data according to a visible geospatial coding set, then a query result is transmitted to a main thread (also called a visualization main thread) through thread communication, and finally the main thread finishes visualization of gridding space-time big data;
s3, monitoring the change of the geographic scene of the visual domain in real time, performing intersection operation on the space coding sets (also called coding sets) corresponding to the visual domain before and after updating, clearing the space-time data outside the visual domain, matching the space-time data corresponding to the newly added coding sets, completing the self-adaptive space matching of the gridding space-time data, realizing the quick updating of the space-time data in the visual domain and performing the visual expression in the visual domain of the space-time big data visual model.
In step S1, a geospatial range (i.e., a level 1 grid range) is defined according to the equal-warp-difference-weft-difference principle, then quad-tree grid division is performed on the geographic space range, a division level and a grid unit size are determined according to a visualization requirement, the finest grid code is used as a quad-tree coding index, a space mapping relationship between the grid unit and the space-time big data is established, and the space-time big data quad-tree coding index generation method is as follows:
the geography grid code corresponding to the visual field is to convert longitude and latitude into row number, then generate one-dimensional Morton code according to Morton coding rule, see figure 2:
visible field coded line number set
Visual field coding column number set
Where x denotes the mesh coding level, b0、l0The latitude difference and the longitude difference of the first-level geographic grid are respectively obtained when the geographic range is defined. b1、l1The latitude difference and the longitude difference of the visual field range respectively change along with the change of the visual field.
As shown in fig. 3, the Morton code adopts a Z curve to perform two-dimensional series connection, thereby realizing dimension reduction expression. And respectively representing the elements of the visible field coding line number set and the elements of the column number set as binary systems according to a Morton coding rule, and then carrying out staggered bit-fetching coding to obtain a visible field grid coding set, namely a spatiotemporal data request query condition.
In step S1, the content of the data request in the current geographic scene is determined, but when the grid cell is small and the grid density is large, the amount of concurrent requests for data is large, and thus thread blocking is easily caused. In step S2, a batch data request management mechanism is established according to the grid density, and a method for separating the grid spatio-temporal big data front-end data request from the visualization thread is proposed. As shown in fig. 4, the main thread distributes the data request task to the sub-threads in batches, the sub-threads are independently responsible for the data request and the data query task, and the query result is returned to the main thread through thread communication to complete the space-time big data visualization. According to the embodiment, a front-end multithreading technical method is adopted, so that the data request and visualization can be executed in an independent thread mode, the response speed of a front-end system is greatly improved, and the visualization efficiency is improved.
As shown in fig. 5, when the user browsing operation causes a continuous update change of the scene, a method for adaptive space matching of the gridded spatio-temporal data is proposed in step S3, which monitors the change of the geographic scene in real time, matches the coding set B before the scene update, obtains the coding set a after the scene update, performs intersection operation on the set a and the set B, and removes and recovers spatio-temporal data corresponding to the [ B- (a ═ n ∞ B) ] coding set. And matching the newly added grid set according to the method of the step S1, determining whether data request and visualization are required to be executed in batches according to the scene change scale in the step S2, and completing the loading visualization of the space-time data corresponding to the newly added [ A- (A &) B) ] coding set through a data request and visualization thread separation method. By the technical method, the front-end system only retains the gridded spatio-temporal data in the visual field range, the system breakdown caused by continuous increase of data due to scene updating is avoided, the data request batch processing is carried out according to the scene change scale, and the front-end blockage caused by overlarge single data request quantity is avoided.
Example four
The space-time big data grid coding efficient visualization method provided by the invention can be suitable for various space-time big data, and the technical idea of the invention is introduced by taking visualization of grid trajectory density as an example based on a Cesium open source visualization platform. The open source geographic visualization platform of the Cesium is an open source JavaScript graphic library facing the three-dimensional earth and produced by AGI company, the Cesium is concentrated on the visualization of space-time data, and is widely applied in the fields of traffic, planning, city management, terrain simulation and the like, and the Cesium platform is used as a basic geographic information platform implemented by the method. The visualization of the gridding track density is to map the original taxi track data into a grid, as shown in fig. 6, count the number of track points in a grid unit, and perform visualization display by thematic color symbols.
An efficient visualization method for a space-time big data grid coding Web end adopts fine granularity to perform efficient visualization expression of grid track density, and comprises the following steps:
s1, constructing a quadtree grid coding index, and self-adapting grid geographic space matching;
in this embodiment, a beijing main city area is taken as an example research area, a first-level geogrid (see a rectangular frame shown in fig. 7) is defined according to the principle of equal longitude and difference latitude, longitude and latitude coordinates of four corners of a first-level quadtree grid are shown in table 1, and then the average side length of a first-level grid code can be calculated to be 69360.6916m and the grid area can be calculated to be 4733365829.9152m2(average side length is the average of long side and short side, and grid area is the product of long side and short side). And (4) carrying out geographic space division according to a quadtree grid subdivision principle to construct a quadtree grid coding index. According to the visualization requirement of the gridding track density, when the grid unit is divided to about 30 × 30m, the grid level is about 12 levels, the visualization requirement of the gridding track density is met, the space division is stopped, and the 12 th level grid code is used as a quad-tree grid index Key value.
TABLE 1
Selecting historical taxi track data in Beijing city, wherein the time is from 11 month 1 day in 2019 to 11 month 7 day in 2019, the total data amount is 4656 thousands of track points in one week, a quadtree index Key value field is added in the data processing process, an index relation between a grid unit and the track points is established through a Key value, the invention content of the Key value calculation method is already described, and the partial data display result is shown in Table 2:
TABLE 2
Obtaining a current geographic scene range through a Camera in an initialization state, calculating the longitude difference and latitude difference of grid units, and obtaining a grid set, wherein the method comprises the following steps:
step 1: acquiring an initialization Camera field of view range:
var recinit＝viewer.camera.computeViewRectangle()；
step 2: acquisition of field of view geographic range:
west longitude: var west is rectangle, west/math.pi 180;
north latitude: var normal, normal/math, pi 180;
east longitude: var east is rectangle, east/math.pi 180;
south latitude: var south ═ rectangle.
Step 3: obtaining the warp difference and weft difference of grid units in the geographic range of the current visual field:
visual field menstruation difference: l1＝east-west
Visual field weft difference: b1＝north-south
Step 4: calculating visible field coding row number and column number set
Visible field coded line number set
Visual field coding column number set
Step 5: the initialization can be generated according to the Morton coding method according to the line number and the column number
The visible grid code set A is the content of the front-end data request.
S2, separating the front-end data request from the visualization thread, and completing visualization of the gridding space-time data request and the gridding space-time big data;
the front-end visualization system of the embodiment performs space-time data visualization by adopting fine-grained grids, the number of the grids is up to 3000, the length of a subset of a single data request can be adjusted according to the data query efficiency, the grid units are controlled within 500 in the tested example, the response time of the system is within 0.3s, and the response speed is high, so that 500 is used as the length of the subset. Data requests are established every 500 times until all data requests are completed.
The embodiment is based on a Web Worker front-end multithreading technology, and realizes the separation of a data request and a visual thread. The method comprises the steps of creating a Worker object in a main thread (namely, the visual main thread of the invention), generating a Worker sub-thread (namely, the data request sub-thread of the invention), and communicating the main thread and the Worker sub-thread through postMessage and onMessage. The main thread is used as a master command to be responsible for sending a data request task to the sub-threads; the sub-thread receives the task and establishes data connection with the database to obtain a query result, and the result is transmitted back to the main thread; and the main thread receives the data result and completes the visualization of the track density. As shown in fig. 8, the embodiment implements asynchronous non-blocking execution of a front-end data request and data visualization, reduces front-end latency, and improves page response speed, and the specific steps are as follows:
the main thread creates a worker object and determines the content of the data request:
step 1: creating worker instances
var worker＝new Worker("worker.js")；
Step 2: determine if a batch data request is required (Single load)
if(postTDData.length<500){
workerdata is a set of codes to be requested
Step 3: sending data request to worker sub-thread
worker.postMessage(workerdata)；
Step 4: receiving data returned by worker sub thread
worker.onmessage＝function(evt){
var data＝evt.data；
Step 5: visualization of trajectory density based on returned results
for(var m＝0；m<data.length；m++){
And calling a ceium function method to finish result visual drawing
Step 2: determining if a batch data request is required (batch load)
Step 3: determining batch times
queryNum＝Math.ceil(postTDData.length/500)；
Step 4: sending data request to worker sub-thread
worker.postMessage(workerdata)；
Step 5: receiving data returned by worker sub thread
worker.onmessage＝function(evt){
var data＝evt.data；
Step 6: visualization of the trajectory density based on the returned results (as above)
Js: the method for receiving the request and inquiring the data by the sub thread is as follows:
step 1: building ajax data requests
var xhr＝new XMLHttpRequest()；
Step 2: setting data request mode and path, adopting synchronous request method
xhr.open('post','/selectredu/',false)；
xhr.setRequestHeader("Content-type", 'application/json；charset＝UTF-8')；
Step 3: sending a data request
xhr.send(request)；
Step 4: receiving data return result and sending main thread
And S3, monitoring scene change in real time, removing and recovering data outside the view after the scene is updated, and matching with the newly added grid data set.
The initialization space-time data request and the visual loading are completed through the step S2, the user browsing operation generates very dense vision field updating, a Cesum mouse moving monitoring function is called, the vision field state is monitored in real time, the grid set B in the current view field is obtained according to the self-adaptive space-time data matching method in the step S1, then the grid set A and the grid set B are subjected to intersection calculation, the grid set to be loaded and the grid set to be cleared are obtained, the space-time data outside the view field is cleared, meanwhile, a data request is sent to complete the visual updating, the effect is shown in figure 9, and the specific method is as the following function method:
step 1: monitoring a camera moving event, and when the camera finishes moving, carrying out the next operation:
viewer.scene.camera.moveEnd.addEventListener(function(){
step 2: obtaining a set of grids to be loaded and cleared
Set a is initialized grid set, or grid set before the view field change
The set B is a grid set in the translation rear view domain;
set Sclean＝[A-(A∩B)]// Note: to-be-cleaned grid set
Removing S by calling Remove methodcleanCorresponding spatio-temporal data;
Step3：
and (3) grid set to be loaded: sload＝[B-(A∩B)]
Calculating the number of times of data to be requested:
method for separating front-end data request and visual thread to complete space-time data to be loaded
And (6) visualization.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (7)
1. A space-time big data grid coding efficient visualization method is characterized by comprising the following steps:
A. establishing a space-time big data visualization model, constructing a gridding coding index data structure on the space-time big data visualization model, simultaneously obtaining a coding index data set, and storing the coding index data set into a space-time database, wherein the coding index data set comprises codes, indexes and data corresponding relations; selecting or setting a visible area on the space-time big data visualization model, and obtaining a visible area coding index data set corresponding to the visible area;
the construction method of the gridding coding index data structure comprises the following steps:
a1, according to the principle of equal longitude difference and equal latitude difference, subdividing a gridded space-time big data visualization model into a plurality of sub-level grid units according to a quadtree grid, constructing a multi-level gridding coding index data structure, and carrying out gridding coding expression on longitude and latitude data by utilizing the multi-level gridding coding index data structure;
the grid coding dimension reduction projection processing method comprises the following steps:
a2, carrying out grid coding dimension reduction projection processing on the collected space-time data according to a grid coding index data structure to obtain the codes and indexes of each space-time data;
the method for obtaining the visible area coding index data set comprises the following steps:
a3 warp difference l in step A1 gridding0Weft difference b0Visual field gridding mean diameter difference l1Weft difference b1And acquiring a visible field coding row and column number set X and Y:
visible field coded line number set
Visual field coding column number set
Wherein x represents a mesh coding level;
a4, performing staggered bit-fetching coding on binary representations of row numbers and column numbers of the visible field coding set according to Morton coding rules to obtain a visible field coding index data set of visible fields;
B. setting a visual main thread and a data request sub-thread aiming at a visual domain of a space-time big data visual model; the method comprises the steps that spatiotemporal data are collected, grid coding dimension reduction projection processing is carried out on the spatiotemporal data, the spatiotemporal data corresponding to visible areas are inquired by the data request sub-thread according to a visible area coding index data set and are transmitted to a visual main thread, and the visual main thread carries out visual expression on the spatiotemporal data of the visible areas;
C. the method comprises the steps of collecting visual field space-time data in real time, carrying out intersection operation on the corresponding space-time data before and after the visual field is updated, clearing the space-time data outside the visual field, transmitting newly-added space-time data in the visual field to a visual main thread by a data request sub-thread, and updating visual expression of the space-time data of the visual field in the visual field of a space-time big data visual model by the visual main thread.
2. The space-time big data grid coding efficient visualization method according to claim 1, characterized in that: the step B further comprises the following steps:
b1, establishing a batch data request management mechanism: the visual main thread distributes data request tasks to data request sub-threads in batches, and the data request sub-threads inquire the time-space data of the corresponding visual field according to the visual field coding index data set and transmit the inquiry result to the visual main thread in batches;
and B2, the visualization main thread and the data request sub-thread are separated and synchronously performed, and the visualization main thread performs visualization expression in the visual domain of the space-time big data visualization model after receiving the query result and the data of the data request sub-thread.
3. The space-time big data grid coding efficient visualization method according to claim 1, characterized in that: step C also includes the following: and C, performing intersection operation on the coding set A and the coding set B, eliminating the space-time data corresponding to the [ B- (A & n & gtB) ] coding set, and loading and visualizing the newly-added space-time data corresponding to the [ A- (A & n & gtB) ] coding set in the visual domain of the space-time big data visualization model through a visualization main line.
4. The space-time big data grid coding efficient visualization method according to claim 1, characterized in that: the gridding coding index data structure forms a plurality of visual grid units with equal longitude differences and equal latitude differences after the quad-tree grid is divided, so that the visual grid units in the visual domain of a space-time big data visual model are obtained, the visual main thread updates and visually expresses space-time data of the visual domain on each visual grid unit in the visual domain, and the visual expression mode comprises the adoption of numbers, colors and graphs.
5. A space-time big data grid coding efficient visualization method is characterized in that: the method comprises the following steps:
A. establishing a space-time big data visualization model, and establishing a gridding coding index data structure on the space-time big data visualization model, wherein the construction method of the gridding coding index data structure comprises the following steps: according to the principle of equal longitude difference and equal latitude difference, a plurality of sub-level grid units are divided for the time-space big data visualization model according to the quad-tree grid;
according to the warp difference l in gridding0Weft difference b0Visual field gridding mean diameter difference l1Weft difference b1Obtaining a set of visible field code row and column numbers X and Y:
visible field coded line number set
Visual field coding column number set
Wherein x represents a mesh coding level;
carrying out staggered bit-fetching coding on binary representations of row numbers and column numbers of a visible field coding set according to a Morton coding rule to obtain a visible field coding index data set of a visible field, wherein the visible field coding index data set comprises codes and corresponding relations between indexes and data;
B. selecting or setting a visible area on the space-time big data visualization model and openly uploading the visible area to an internet web end, or uploading the space-time big data visualization model to the internet web end and selecting or setting the visible area on the internet web end; setting a visual main thread and a data request sub-thread aiming at a visual domain of a space-time big data visual model; establishing a batch data request management mechanism, synchronously performing a visual main thread and a data request sub-thread, and distributing data request tasks to the data request sub-thread in batches by the visual main thread;
C. the method comprises the steps that space-time data are collected in real time, a data request sub-thread codes the space-time data according to longitude and latitude, the data request sub-thread inquires and collects space-time data corresponding to a visual field according to a visual field code index data set, the data request sub-thread performs intersection operation on codes of the space-time data before and after the visual field is updated, space-time data outside the visual field are cleared, the data request sub-thread transmits newly added space-time data in the visual field to a visual main thread, and the visual main thread updates and visually expresses the space-time data in the visual field on a space-time big data visual model.
6. The space-time big data grid coding efficient visualization method according to claim 5, characterized in that: step C also includes the following: in the step C, the code set B before updating and the code set A after updating perform intersection operation on the code set A and the code set B, firstly, the space-time data corresponding to the [ B- (A N B) ] code set is eliminated, and then, the newly added space-time data corresponding to the [ A- (A N B) ] code set is loaded and visualized through a visualization main thread; the visual expression is correspondingly updated on each visual grid unit in the visual domain, and the visual expression mode comprises the adoption of numbers, colors and graphs;
the acquired space-time data is stored in a space-time database, and new increment, accumulated amount and change rate are counted according to time dimension and are visually expressed.
7. A space-time big data grid coding efficient visualization system is characterized in that: the large space-time data visualization model is constructed with a gridding coding index data structure, the gridding coding index data structure divides the gridded large space-time data visualization model into a plurality of sub-level grid units according to a quadtree and a grid according to the principle of equal warp difference and equal weft difference, a multi-level gridding coding index data structure is constructed, and coding index data are obtained and stored in the space-time database in a set manner; the method for obtaining the visible area coding index data set comprises the following steps:
a3 warp difference in gridding0Weft difference b0Visual field gridding mean diameter difference l1Weft difference b1And acquiring a visible field coding row and column number set X and Y:
visible field coded line number set
Visual field coding column number set
Wherein x represents a mesh coding level;
a4, performing staggered bit-fetching coding on binary representations of row numbers and column numbers of the visible field coding set according to Morton coding rules to obtain a visible field coding index data set of visible fields;
the space-time big data visualization model is provided with a visual field selection module, and the visual field selection module is used for selecting or setting a visual field on the space-time big data visualization model and obtaining a visual field coding index data set corresponding to the visual field; the acquisition module is used for acquiring space-time data, and the grid coding module is used for carrying out grid coding processing on the space-time data;
the visual domain data visualization processing module comprises a visual main thread module and a data request sub-thread module, the data request sub-thread module is used for inquiring the space-time data corresponding to the visual domain according to the visual domain code index data set, then performing intersection operation on the codes of the corresponding space-time data before and after the visual domain is updated, obtaining the newly added space-time data in the visual domain and then transmitting the newly added space-time data to the visual main thread, and the visual main thread module is used for performing update visualization expression on the space-time data in the visual domain of the space-time big data visualization model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010594839.8A CN111782745B (en) | 2020-06-28 | 2020-06-28 | Space-time big data grid coding efficient visualization method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010594839.8A CN111782745B (en) | 2020-06-28 | 2020-06-28 | Space-time big data grid coding efficient visualization method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111782745A CN111782745A (en) | 2020-10-16 |
| CN111782745B true CN111782745B (en) | 2021-08-03 |
Family
ID=72760569
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010594839.8A Active CN111782745B (en) | 2020-06-28 | 2020-06-28 | Space-time big data grid coding efficient visualization method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111782745B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112015851A (en) * | 2020-11-02 | 2020-12-01 | 中科星图股份有限公司 | Space-time data storage and retrieval method based on Beidou grid code |
| CN112860689B (en) * | 2021-01-11 | 2024-02-20 | 北斗伏羲信息技术有限公司 | Platform system based on space-time grid data |
| CN115774803B (en) * | 2023-02-10 | 2023-04-18 | 中大体育产业集团股份有限公司 | Visual identification method and system for physical examination site |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109992636A (en) * | 2019-03-22 | 2019-07-09 | 中国人民解放军战略支援部队信息工程大学 | Space-time code method, temporal index and querying method and device |
| CN111198975A (en) * | 2019-12-25 | 2020-05-26 | 上海杰狮信息技术有限公司 | Grid-based space-time big data visualization method and visualization system |
| CN111241167A (en) * | 2020-01-15 | 2020-06-05 | 广东省城乡规划设计研究院 | Large-data-volume space-time data visualization method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109871418A (en) * | 2019-01-04 | 2019-06-11 | 广州市城市规划勘测设计研究院 | A kind of space index method and system of space-time data |
-
2020
- 2020-06-28 CN CN202010594839.8A patent/CN111782745B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109992636A (en) * | 2019-03-22 | 2019-07-09 | 中国人民解放军战略支援部队信息工程大学 | Space-time code method, temporal index and querying method and device |
| CN111198975A (en) * | 2019-12-25 | 2020-05-26 | 上海杰狮信息技术有限公司 | Grid-based space-time big data visualization method and visualization system |
| CN111241167A (en) * | 2020-01-15 | 2020-06-05 | 广东省城乡规划设计研究院 | Large-data-volume space-time data visualization method |
Non-Patent Citations (1)
| Title |
|---|
| 城市地理空间基础数据库更新方法研究;李宗华;《城市勘测》;20061231(第1期);17 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111782745A (en) | 2020-10-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111782745B (en) | Space-time big data grid coding efficient visualization method and system | |
| CN110502594B (en) | Real-time fusion and online updating method for map tiles of remote sensing image | |
| CN108446293B (en) | Method for constructing city portrait based on city multi-source heterogeneous data | |
| CN106909644B (en) | A kind of multistage tissue and indexing means towards mass remote sensing image | |
| CN109977179A (en) | A kind of poly- scattered method for visualizing of grid dividing realization magnanimity atural object mark point of layering | |
| US10262392B2 (en) | Distributed and parallelized visualization framework | |
| CN110610258B (en) | Urban air quality refined estimation method and device fusing multi-source space-time data | |
| CN109448361A (en) | Resident's traffic trip volume forecasting system and its prediction technique | |
| CN103020222A (en) | Visual mining method for vehicle GPS (global positioning system) data analysis and abnormality monitoring | |
| CN109299298A (en) | Construction method, device, application method and the system of image fusion model | |
| CN110060471B (en) | Vehicle OD flow prediction model construction method and vehicle OD flow prediction method | |
| CN113570275A (en) | Water resource real-time monitoring system based on BIM and digital elevation model | |
| CN104007948A (en) | Method and device based on distributed computation and visualized display of three-dimensional GIS mass data | |
| CN104199949B (en) | Under a kind of real time environment extensive dynamic object method for visualizing is realized using grid | |
| WO2024108975A1 (en) | Slicing method for spatial model data | |
| CN111475565B (en) | Visual playback system and method for target historical geographic information data | |
| CN116011322A (en) | Urban information display method, device, equipment and medium based on digital twinning | |
| CN112328722A (en) | Method and system for rapidly loading three-dimensional GIS (geographic information System) and model data of power transformer substation | |
| CN112380302A (en) | Thermodynamic diagram generation method and device based on track data, electronic equipment and storage medium | |
| CN103118102A (en) | System and method for counting and controlling spatial data access laws under cloud computing environment | |
| CN107995278B (en) | A kind of scene intelligent analysis system and method based on metropolitan area grade Internet of Things perception data | |
| CN109255004A (en) | The online interaction formula method for visualizing of accuracy guarantee towards geographical spatial data | |
| CN114079498A (en) | Space-time gridding satellite-borne data storage method and system | |
| CN116681847A (en) | Management and service system for urban three-dimensional geological data | |
| Whigham | Hierarchies of space and time |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |