CN108804602A - A kind of distributed spatial data storage computational methods based on SPARK - Google Patents
A kind of distributed spatial data storage computational methods based on SPARK Download PDFInfo
- Publication number
- CN108804602A CN108804602A CN201810530638.4A CN201810530638A CN108804602A CN 108804602 A CN108804602 A CN 108804602A CN 201810530638 A CN201810530638 A CN 201810530638A CN 108804602 A CN108804602 A CN 108804602A
- Authority
- CN
- China
- Prior art keywords
- distributed
- spatial
- data
- space
- hbase
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The distributed spatial data storage computational methods based on SPARK that the invention discloses a kind of, on the basis of multiplexing traditional GIS software function, with cloud computing software, high-performance and High Availabitity are brought for spatial storage methods and calculating, realization method includes being encoded to spatial index with the elongated GeoHash for taking spatial dimension into account, and the distributed spatial data storage table structure of compatible a variety of Spatial data types is realized on HBase；The file of memory space data is converted into the corresponding storage format HFile of HBase using MapReduce methods, realizes the Quick Pretreatment and storage of distributed space data；Spatial dimension coarse search is carried out to the spatial data stored in HBase using elongated GeoHash codings；It will be mapped as elastic data collection object between the distributed space in SPARK from the spatial data obtained in HBase；The traditional GIS software function of multiplexing realizes the distributed space data Processing Interface of elastic data collection between distributed space, and fine screen choosing and Distributed Calculation are carried out to the space data sets that spatial dimension slightly inquires.
Description
Technical field
The invention belongs to GIS-Geographic Information System fields, are related to a kind of distributed spatial data storage computational methods.
Background technology
In recent years, with the fast development of sensor technology and earth observation technology, geographical space big data has become
The important component of big data.The data volume of space information system management and processing increases to PB or even EB from TB grades
Grade.Space big data also increases geography in the characteristics of type, picking rate, value density, accuracy and variability etc.
The complexity of spatial data management and processing, while also implying that traditional data management system and computing capability are difficult to meet this
A little demands.Traditional GIS-Geographic Information System (GIS) GML data storage management depends directly on existing database (such as
PostGIS, Oracle Spatial), or structure Spatial Data Engine middleware (such as ArcSDE) on it.But these schemes exist
Distributed geographical space big data management and the upper Shortcomings of calculating.Distributed space data tissue towards cloud environment in recent years
Management has become the trend of space big data management.
The distributed storage under cloud environment, distributed computing framework Hadoop and its modified version are moved towards with GIS data
Cloud computing technologies and the software facilities such as SPARK, distributed file system HDFS, distributed data base HBase, between distributed space
Data organization and management brings foreground.Wherein SPARK frames use the distributed treatment pattern based on memory, present ratio
Hadoop better performances and fault-tolerance realize that efficient spatial data management and computational methods seem and very must on this basis
It wants.
How traditional GIS software to be moved to using smaller cost using SPARK as under the distributed cloud environment of representative, for cloud
The development of GIS platform software is of great significance.Has the space data set for partly working and taking up under SPARK environment at present
It knits.However, existing work still lacks the multiplexing to having GIS software function, the solution of GIS functions is built from the beginning
It is often with high costs, take time and effort and error-prone；Secondly, it mostly uses distributed file system currently based on the research of SPARK and deposits
Spatial data is stored up, and using HBase distributeds database purchase and accesses the method that SPARK calculated, it is empty due to existing
Between data distribution formula storage organization and Index Design, spatial data table to elastic data collection between SPARK distributed spaces mapping etc.
A series of difficult points still have to be achieved.
Invention content
Thus to solve the above problems, the present invention provides a kind of distributed spatial data storage and meter based on SPARK
Calculation method, it is spatial data to use advanced cloud computing software on the basis of multiplexing traditional GIS software function
Storage brings high-performance and High Availabitity with calculating.
The technical solution adopted in the present invention is a kind of distributed spatial data storage computational methods based on SPARK,
It is spatial storage methods and calculating band with cloud computing software on the basis of the traditional GIS software function of multiplexing
Coming high-performance and High Availabitity, realization method includes the following steps,
Step 1, spatial index is encoded to the elongated GeoHash for taking spatial dimension into account, realizes that compatibility is a variety of on HBase
The distributed spatial data storage table structure of Spatial data types；
Step 2, the file of memory space data is converted into the corresponding storage formats of HBase using MapReduce methods
HFile realizes the Quick Pretreatment and storage of distributed space data；
Step 3, it is encoded using elongated GeoHash and spatial dimension coarse search is carried out to the spatial data stored in HBase；
Step 4, elastic data set pair between the distributed space in SPARK will be mapped as from the spatial data obtained in HBase
As；
Step 5, it is multiplexed the distributed space that traditional GIS software function realizes elastic data collection between distributed space
Between data-processing interface, the space data sets slightly inquired to spatial dimension carry out fine screen choosing and Distributed Calculation.
Moreover, in step 1, each Space Elements class is set and corresponds to a table, for different Space Elements geometric types,
Different row clusters is defined, the row in row cluster are made of the attribute of spatial object, and versions of data is indicated with timestamp；Each space
Object corresponds to a line unit, is combined and is constituted by elongated GeoHash indexes and element identification symbol ID.
Moreover, in step 3, determine that the GeoHash that coarse search uses is encoded according to the maximum length and width of retrieval rectangle frame first
Length ensures that the grid corresponding to the code length is sized to that retrieval rectangle frame is completely covered；Then, rectangle frame center of gravity is calculated
The grid at place and its adjacent all directions are encoded to the GeoHash of grid, to judge and obtain retrieval rectangle frame fall lattice
Net range and the encoding list；It is prefix to be matched in HBase spatial tables with the coding or its subset+" _ " according to corresponding coding
Row is strong, and corresponding spatial data is obtained from HBase.
Moreover, in step 4, mapping process is as follows,
First, it is obtained in HBase table after spatial index coarse sizing using the HBase read-write interfaces built in SPARK
Data generate Hadoop distributed elastic data set HadoopRDD objects, and the object for managing and handling distribution in memory
The data set obtained from Hadoop distributed file systems/database under formula environment；
Then, empty for parsing in a distributed manner by calling the distributed mapping Processing Interface Map operations of HadoopRDD
Between data object and index, generate subregion and map elastic data collection MapPartitionRDD objects, the object is for managing and locate
Manage the data set generated by Map operations under distributed environment；
Finally, customized translation interface is called, the spatial data object and its directoried data set that parsing is obtained transmit
To elastic data collection SpatialRDD objects between newly-built distributed space, completes MapPartitionRDD objects and arrive
The conversion of SpatialRDD objects.
Moreover, SpatialRDD objects are further converted into distributed point elasticity data set PointRDD structures, distribution
Formula linear elasticity data set PolylineRDD structures and distributed surface elastic data set PolygonRDD, including in a distributed manner will
All spatial data objects of undefined geometric type add specified geometric type in SpatialRDD, complete from distributed general
Conversion of the logical geometric type data set to distributed particular geometric categorical data collection；The distribution point elasticity data set
PointRDD structures, distributed linear elasticity data set PolylineRDD structures and distributed surface elastic data set PolygonRDD
Structure is respectively used to management and the point Point under processing distributed environment, line Polyline knead dough Polygon types in memory
Space data sets.
It is an advantage of the invention that：
(1) a set of distribution GIS kernel construction methods are proposed, massive spatial data can be carried out efficient storage and
Processing；
(2) it has been compatible with traditional geography in formation software kernel, such as GeoStar (GeoStar), has improved the multiplexing of software
Property, save great amount of cost；
(3) it is that spatial storage methods and calculating bring height to have used advanced cloud computing software (HBase, SPARK etc.)
The characteristic of performance and High Availabitity has important market value.
Description of the drawings
Fig. 1 is that the overall of the embodiment of the present invention realizes Organization Chart.
Fig. 2 is the HBase GML data storage table structure charts of the embodiment of the present invention.
Fig. 3 is the elongated GeoHash encoding schemes schematic diagram for taking spatial dimension into account of the embodiment of the present invention.
Fig. 4 is the spatial dimension search strategy schematic diagram of the embodiment of the present invention encoded based on GeoHash.
Fig. 5 is that the HBase data of the embodiment of the present invention map flow chart to elastic data collection between SPARK distributed spaces.
Fig. 6 is the SPARK distributed space data process charts of the embodiment of the present invention.
Specific implementation mode
The present invention is understood and implemented for the ease of those of ordinary skill in the art, and the present invention is made into one with reference to example
Step detailed description, it should be understood that case study on implementation described herein is merely to illustrate and explain the present invention, and is not used to limit this
Invention.
The distributed spatial data storage computational methods based on SPARK that the present invention provides a kind of, from number between distributed space
It sets out according to the design and realization of storage organization, is asked for the keys such as distributed storage therein and distributed memory object designs
Topic devises the spatial data table structure towards row storage and the distributed memory space pair based on elasticity distribution formula data set
As realizing the method that HBase tables of data Mapping and Converting SPARK distributed memory spatial objects are operated, and pass through multiplexing
Traditional GIS software kernel realizes the operation to distributed memory spatial object, to be provided for storage and processing space big data
A kind of more comprehensive solution covering interior external memory design.
Embodiment totally realizes framework referring to attached drawing 1.Cloud computing resources layer provides hardware money as the bottom of entire frame
Source and resource management capacity are, it can be achieved that node administration, fault tolerant mechanism, load balancing, log management etc.；External memory layer is using distributed
Database and distributed file system (such as HBase or HDFS) and design space table and spatial index；Memory computation layer includes then
The distributed GIS kernels of multiplexing traditional GIS kernel and build the distributed spatial manipulation operator on kernel.Key is
It is spatial storage methods and meter that advanced cloud computing software is used on the basis of the traditional GIS software function of multiplexing
Calculation brings high-performance and High Availabitity, includes the following steps：
Step 1：Elongated geographical cryptographic Hash (GeoHash) to take spatial dimension into account is encoded to spatial index, on HBase
Realize the distributed spatial data storage table structure of a variety of Spatial data types such as compatible point, line, surface；
On the basis of HBase table structure, in conjunction with OGC elementary factor spatial data structures, the storage of design space data
Table structure.Similar with traditional spatial data table, each Space Elements class corresponds to a table, for different Space Elements geometry
Type defines different row clusters.The table structure of realization is shown in attached drawing 2, and the row in row cluster are made of the attribute of spatial object, the used time
Between stamp indicate versions of data (Ts in figure), rear historical data is updated to a spatial object in this way and still can be used as reference.
Each spatial object corresponds to a line unit, accords with ID by elongated GeoHash indexes and element identification and combines and constitutes, by underscore every
Open that (line unit of such as figure Spatial Objects 1, spatial object 2 is expressed as GeoHash_1, GeoHash_2, with " _ " by two
Coded is separated, is also possible to prevent the situation that geohash encoded radios are empty, such as when spatial object minimum enclosed rectangle is horizontal
Across 0 degree of latitude and longitude, geohash codings are just sky), the geometry category of geometry field (WKB) memory space object in a binary format
Property (such as solid 1, solid 2, respectively the geometric coordinate data of memory space object 1,2), multiple attribute fields are (such as every
A spatial object have respectively attribute 1, attribute 2 ..., the various attribute values of attribute n) then memory space objects.Wherein, GeoHash
For coding by dividing two-dimentional latitude and longitude coordinates by grid, each grid indicates certain warp by given unique encodings
Latitude scope, and to get over the corresponding longitude and latitude range of wavelength grid just about small for code length.Therefore, by using elongated GeoHash
Coding, can be preferably with respect to the different spaces range of different spaces object.
See attached drawing 3, to realize the elongated GeoHash codings for taking spatial dimension into account, ensures the corresponding longitude and latitude model of coding
It can includes completely the spatial object to enclose, and the present invention seeks four vertex encodings of minimum enclosed rectangle of spatial object first, then asks
The longest-prefix intersection (the prefix intersection of such as w4gh12, w4gk12, w4gj12 and w4gm12 are w4g) of this 4 codings, represents
Completely include the minimal geographical grid range of the spatial object.GeoHash in such as figure1、GeoHash2、GeoHash3、
GeoHash4The GeoHash codings of grid where respectively minimum external four vertex, and the range corresponding to its intersection is then
Dotted box portion can ensure to completely include the element.
Step 2：Space large data files are converted into the corresponding storage formats of HBase using MapReduce methods
HFile realizes the Quick Pretreatment and storage of distributed space data；
The file of memory space data (such as shapefile, csv file) is stored on HDFS first；In HBase
The GML data storage table that middle foundation step 1 is realized；Spatial data files are parsed using MapReduce frames, are calculated
HFile is write data into after the GeoHash indexes of each spatial object；HFile is finally imported into corresponding HBase spatial datas
In storage table.
Step 3：Spatial dimension coarse search is carried out to the spatial data stored in HBase using elongated GeoHash codings；
GeoHash code lengths that coarse search uses are determined according to the maximum length and width of retrieval rectangle frame, ensure the volume first
Grid corresponding to code length is sized to that retrieval rectangle frame is completely covered；Then, calculate rectangle frame center of gravity where grid and
Its adjacent all directions to the GeoHash of grid encode, to judge and obtain retrieval rectangle frame fall grid range and coding
List；Since step 1 is set, line unit, which by elongated GeoHash indexes and element identification accords with ID and combines, to be constituted, by underscore every
It opens, this step is matched in HBase spatial tables according to corresponding coding and is good for for the row of prefix with the coding or its subset+" _ ", is such as compiled
Code " w4gh " will be matched (crosses over the rectangle frame meeting of 0 longitude and 0 latitude area with " w4gh ", " w4g_ ", " w4_ ", " w_ " and " _ "
Occur four vertex without common prefix the case where) be prefix line unit, so as to obtain corresponding space number from HBase
According to.In attached drawing 4, frame retrieval covers 4 minimum grid, and coding is respectively w4gj, w4gm, w4gh and w4gk, then by this 4
The corresponding range of coding retrieves corresponding data as screening range from HBase.
Step 4：It will be mapped as elastic data set pair between the distributed space in SPARK from the spatial data obtained in HBase
As；
Seeing attached drawing 5, the present invention is based on the kernel data structures of SPARK --- distributed elastic data set RDD is (a kind of read-only
Record partitioning set, for managing in memory and handling distributed data collection, the conversion between different type RDD is claimed
For conversion operation, and the relationship between the RDD generated by conversion operation is referred to as father and son and traces to the source relationship), devise distribution
Spatial elastic data set SpatialRDD structures manage and handle the space data sets under distributed environment in memory.Together
When, the present invention by the distributed point elasticity data set PointRDD structures of the further refinement of SpatialRDD structures, distribution
Formula linear elasticity data set PolylineRDD structures and distributed surface elastic data set PolygonRDD structures, are respectively used to memory
The space data sets of middle management and point Point, line Polyline knead dough Polygon types under processing distributed environment.This hair
It is bright also set up from HBase create SpatialRDD interface (CreateSpatialRDDFromHBase), the interface execute with
Lower operation：
First, it is obtained in HBase table after spatial index coarse sizing using the HBase read-write interfaces built in SPARK
Data generate Hadoop distributed elastic data set HadoopRDD objects, and the object for managing and handling distribution in memory
The data set obtained from Hadoop distributed file systems/database under formula environment；
Then, by calling the distributed mapping Processing Interface Map of HadoopRDD to operate, a kind of conversion operation of RDD,
For analytic space data object and index in a distributed manner, generates subregion and maps elastic data collection MapPartitionRDD objects,
The object is used to manage and handle the data set generated by Map operations under distributed environment；
Finally, customized translation interface is called, the spatial data object and its directoried data set that parsing is obtained transmit
To newly-built SpatialRDD objects, conversion of the MapPartitionRDD objects to SpatialRDD objects is completed.
Further, the present invention be also a series of interfaces of SpatialRDD structure designs realize by SpatialRDD objects into
One step is converted to PointRDD objects, PolylineRDD objects and PolygonRDD objects.Implementation method is in a distributed manner will
All spatial data objects of undefined geometric type add specified geometric type in SpatialRDD, complete from distributed general
Conversion of the logical geometric type data set to distributed particular geometric categorical data collection.
Step 5：Traditional GIS software (such as GeoStar GeoStar) function of multiplexing realizes bullet between distributed space
At the distributed space datas of (including SpatialRDD, PointRDD, PolylineRDD and PolygonRDD) such as property data set
Manage interface (circular scope query interface between rectangular extent query interface, distributed space, distributed spatial object between such as distributed space
Cluster interface etc.), fine screen choosing and Distributed Calculation are carried out to the space data sets that spatial dimension slightly inquires.
See attached drawing 6, utilizes the thick Directory Enquiries data of spatial index from HBase table according to query context input by user first
And by calling interface generate SpatialRDD (can be further converted into according to mission requirements PointRDD, PolylineRDD and
PolygonRDD subregion) and in a manner of dividing equally by data set is carried out, makes the data volume that each computing unit is assigned to as far as possible
It is identical, it is to be noted that a computer node may include several computing units, a computing unit according to its process cores calculation
The same time can only handle the data of a data partition；Operator is selected to be distributed to respectively with spatial data handling operator data fine screen
Computing unit where a data partition, these operators are realized by being multiplexed traditional GIS software function, such as lucky difficult to understand
Star GeoStar kernels provide spatial object ask friendship judge, spatial object distance calculate etc. functions；It is finally single in each calculating
Further it is calculated and is handled after accurately filtering out the spatial object in query context in member, as space clustering,
Cuclear density analysis etc..
When it is implemented, computer software technology, which can be used, in the above flow realizes automatic running.
Claims (5)
1. a kind of distributed spatial data storage computational methods based on SPARK, it is characterised in that：In the traditional geography information of multiplexing
On the basis of system software function, with cloud computing software, high-performance and High Availabitity are brought with calculating for spatial storage methods,
Realization method includes the following steps,
Step 1, spatial index is encoded to the elongated GeoHash for taking spatial dimension into account, compatible a variety of spaces is realized on HBase
The distributed spatial data storage table structure of data type；
Step 2, the file of memory space data is converted into the corresponding storage formats of HBase using MapReduce methods
HFile realizes the Quick Pretreatment and storage of distributed space data；
Step 3, it is encoded using elongated GeoHash and spatial dimension coarse search is carried out to the spatial data stored in HBase；
Step 4, elastic data collection object between the distributed space in SPARK will be mapped as from the spatial data obtained in HBase；
Step 5, it is counted between the distributed space of elastic data collection between the traditional GIS software function realization distributed space of multiplexing
According to Processing Interface, fine screen choosing and Distributed Calculation are carried out to the space data sets that spatial dimension slightly inquires.
2. the distributed spatial data storage computational methods based on SPARK according to claim 1, it is characterised in that：Step 1
In, each Space Elements class one table of correspondence is set, different row clusters is defined for different Space Elements geometric types, arranges
Row in cluster are made of the attribute of spatial object, and versions of data is indicated with timestamp；Each spatial object corresponds to a line unit,
It is combined and is constituted by elongated GeoHash indexes and element identification symbol ID.
3. the distributed spatial data storage computational methods based on SPARK according to claim 1, it is characterised in that：Step 3
In, GeoHash code lengths that coarse search uses are determined according to the maximum length and width of retrieval rectangle frame, ensure the code length first
Corresponding grid is sized to that retrieval rectangle frame is completely covered；Then, the grid and its adjacent where rectangle frame center of gravity is calculated
From all directions to the GeoHash of grid encode, to judge and obtain retrieval rectangle frame fall grid range and the encoding list；Root
, from HBase acquisition pair strong for the row of prefix with the coding or its subset+" _ " is matched in HBase spatial tables according to coding is corresponded to
The spatial data answered.
4. the distributed spatial data storage computational methods based on SPARK according to claim 1, it is characterised in that：Step 4
In, mapping process is as follows,
First, the data in HBase table after spatial index coarse sizing are obtained using the HBase read-write interfaces built in SPARK,
Hadoop distributed elastic data set HadoopRDD objects are generated, the object for managing and handling distributed ring in memory
The data set obtained from Hadoop distributed file systems/database under border；
Then, by calling the distributed mapping Processing Interface Map operations of HadoopRDD, for analytic space number in a distributed manner
It according to object and index, generates subregion and maps elastic data collection MapPartitionRDD objects, the object is for managing and handling point
The data set generated by Map operations under cloth environment；
Finally, customized translation interface is called, the spatial data object and its directoried data set that parsing obtains are passed to newly
Elastic data collection SpatialRDD objects between the distributed space built complete MapPartitionRDD objects to SpatialRDD pairs
The conversion of elephant.
5. the distributed spatial data storage computational methods based on SPARK according to claim 4, it is characterised in that：It will
SpatialRDD objects are further converted into distributed point elasticity data set PointRDD structures, distributed linear elasticity data set
PolylineRDD structures and distributed surface elastic data set PolygonRDD, including in a distributed manner will be uncertain in SpatialRDD
All spatial data objects of adopted geometric type add specified geometric type, complete from distributed common geometric type data set
To the conversion of distributed particular geometric categorical data collection；The distribution point elasticity data set PointRDD structures, distributed line
Elastic data collection PolylineRDD structures and distributed surface elastic data set PolygonRDD structures, are respectively used to memory middle pipe
The space data sets of reason and point Point, line Polyline knead dough Polygon types under processing distributed environment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810530638.4A CN108804602A (en) | 2018-05-25 | 2018-05-25 | A kind of distributed spatial data storage computational methods based on SPARK |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810530638.4A CN108804602A (en) | 2018-05-25 | 2018-05-25 | A kind of distributed spatial data storage computational methods based on SPARK |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108804602A true CN108804602A (en) | 2018-11-13 |
Family
ID=64090765
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810530638.4A Pending CN108804602A (en) | 2018-05-25 | 2018-05-25 | A kind of distributed spatial data storage computational methods based on SPARK |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108804602A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109597857A (en) * | 2018-12-06 | 2019-04-09 | 中电工业互联网有限公司 | A kind of Internet of Things big data calculation method based on Spark |
| CN109885632A (en) * | 2019-01-22 | 2019-06-14 | 中国科学院空间应用工程与技术中心 | A kind of space science and application data retrieval method, system, medium and equipment |
| CN110175175A (en) * | 2019-05-29 | 2019-08-27 | 大连大学 | Secondary index and range query algorithm between a kind of distributed space based on SPARK |
| CN110377624A (en) * | 2019-07-31 | 2019-10-25 | 象辑知源（武汉）科技有限公司 | A kind of storage and querying method to the geographic information data with multidimensional properties such as time and spaces |
| CN110597935A (en) * | 2019-08-05 | 2019-12-20 | 北京云和时空科技有限公司 | Space analysis method and device |
| WO2020192225A1 (en) * | 2019-03-22 | 2020-10-01 | 深圳先进技术研究院 | Remote sensing data indexing method for spark, system and electronic device |
| CN112270412A (en) * | 2020-10-15 | 2021-01-26 | 北京百度网讯科技有限公司 | Network operator processing method and device, electronic equipment and storage medium |
| CN112380222A (en) * | 2020-11-26 | 2021-02-19 | 吉林师范大学 | Method and device for generating and processing global unique identifier of geospatial data |
| CN112487125A (en) * | 2020-12-09 | 2021-03-12 | 武汉大学 | Distributed space object organization method for space-time big data calculation |
| CN113204553A (en) * | 2021-05-07 | 2021-08-03 | 北京连山科技股份有限公司 | Vector data compiling method based on GeoHash |
| CN113626207A (en) * | 2021-10-12 | 2021-11-09 | 苍穹数码技术股份有限公司 | Map data processing method, device, equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104199986A (en) * | 2014-09-29 | 2014-12-10 | 国家电网公司 | Vector data space indexing method base on hbase and geohash |
| CN105205169A (en) * | 2015-10-12 | 2015-12-30 | 中国电子科技集团公司第二十八研究所 | Distributed image index and retrieval method |
-
2018
- 2018-05-25 CN CN201810530638.4A patent/CN108804602A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104199986A (en) * | 2014-09-29 | 2014-12-10 | 国家电网公司 | Vector data space indexing method base on hbase and geohash |
| CN105205169A (en) * | 2015-10-12 | 2015-12-30 | 中国电子科技集团公司第二十八研究所 | Distributed image index and retrieval method |
Non-Patent Citations (3)
| Title |
|---|
| PENG YUE: "Big spatial data processing with Apache Spark", 《INSTITUTE OF ELECTRIC AND ELECTRONICS ENGINEERS》 * |
| PENG YUE: "Geospatial data storage based on HBase and MapReduce", 《INSTITUTE OF ELECTRIC AND ELECTRONICS ENGINEERS》 * |
| 侯志通: "条带状公路运营管理空间大数据降维组织及混合存储关键技术研究", 《中国博士学位论文全文数据库 工程科技Ⅱ辑》 * |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109597857A (en) * | 2018-12-06 | 2019-04-09 | 中电工业互联网有限公司 | A kind of Internet of Things big data calculation method based on Spark |
| CN109885632A (en) * | 2019-01-22 | 2019-06-14 | 中国科学院空间应用工程与技术中心 | A kind of space science and application data retrieval method, system, medium and equipment |
| CN109885632B (en) * | 2019-01-22 | 2021-02-12 | 中国科学院空间应用工程与技术中心 | Space science and application data retrieval method, system, medium and equipment |
| WO2020192225A1 (en) * | 2019-03-22 | 2020-10-01 | 深圳先进技术研究院 | Remote sensing data indexing method for spark, system and electronic device |
| CN110175175B (en) * | 2019-05-29 | 2023-05-09 | 大连大学 | SPARK-based distributed space secondary index and range query algorithm |
| CN110175175A (en) * | 2019-05-29 | 2019-08-27 | 大连大学 | Secondary index and range query algorithm between a kind of distributed space based on SPARK |
| CN110377624A (en) * | 2019-07-31 | 2019-10-25 | 象辑知源（武汉）科技有限公司 | A kind of storage and querying method to the geographic information data with multidimensional properties such as time and spaces |
| CN110597935A (en) * | 2019-08-05 | 2019-12-20 | 北京云和时空科技有限公司 | Space analysis method and device |
| CN112270412A (en) * | 2020-10-15 | 2021-01-26 | 北京百度网讯科技有限公司 | Network operator processing method and device, electronic equipment and storage medium |
| CN112270412B (en) * | 2020-10-15 | 2023-10-27 | 北京百度网讯科技有限公司 | Network operator processing method and device, electronic equipment and storage medium |
| CN112380222A (en) * | 2020-11-26 | 2021-02-19 | 吉林师范大学 | Method and device for generating and processing global unique identifier of geospatial data |
| CN112380222B (en) * | 2020-11-26 | 2023-08-29 | 吉林师范大学 | Geospatial data global unique identifier generation processing method and device |
| CN112487125B (en) * | 2020-12-09 | 2022-08-16 | 武汉大学 | Distributed space object organization method for space-time big data calculation |
| CN112487125A (en) * | 2020-12-09 | 2021-03-12 | 武汉大学 | Distributed space object organization method for space-time big data calculation |
| CN113204553A (en) * | 2021-05-07 | 2021-08-03 | 北京连山科技股份有限公司 | Vector data compiling method based on GeoHash |
| CN113626207A (en) * | 2021-10-12 | 2021-11-09 | 苍穹数码技术股份有限公司 | Map data processing method, device, equipment and storage medium |
| CN113626207B (en) * | 2021-10-12 | 2022-03-08 | 苍穹数码技术股份有限公司 | Map data processing method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108804602A (en) | A kind of distributed spatial data storage computational methods based on SPARK | |
| Lv et al. | BIM big data storage in WebVRGIS | |
| CN106909644B (en) | A kind of multistage tissue and indexing means towards mass remote sensing image | |
| CN109992636A (en) | Space-time code method, temporal index and querying method and device | |
| CN103412962B (en) | A kind of storage method of magnanimity tile data and read method | |
| CN104199986B (en) | Vector data space index method based on hbase and geohash | |
| CN108446293A (en) | A method of based on urban multi-source isomeric data structure city portrait | |
| US10055808B1 (en) | Distributed and parallelized visualization framework | |
| CN105045869B (en) | Natural resources geographical spatial data method for organizing based on multiple data centers and system | |
| CN102509022B (en) | Method for quickly constructing raster database facing to Virtual Earth | |
| CN104281701B (en) | Multiscale Distributed Spatial data query method and system | |
| CN109635068A (en) | Mass remote sensing data high-efficiency tissue and method for quickly retrieving under cloud computing environment | |
| CN109299298A (en) | Construction method, device, application method and the system of image fusion model | |
| Wang et al. | Research and implementation on spatial data storage and operation based on Hadoop platform | |
| Zalipynis | Chronosdb: distributed, file based, geospatial array dbms | |
| CN105608222A (en) | Rapid building method of tile pyramid for large-scale raster data set | |
| CN107766471A (en) | The organization and management method and device of a kind of multi-source data | |
| US10558665B2 (en) | Network common data form data management | |
| CN110147377A (en) | General polling algorithm based on secondary index under extensive spatial data environment | |
| CN108009265B (en) | Spatial data indexing method in cloud computing environment | |
| CN109492060A (en) | A kind of map tile storage method based on MBTiles | |
| CN110175175A (en) | Secondary index and range query algorithm between a kind of distributed space based on SPARK | |
| CN108268614A (en) | A kind of distribution management method of forest reserves spatial data | |
| CN116860905A (en) | Space unit coding generation method of city information model | |
| CN109359205A (en) | A kind of remote sensing image cutting method and equipment based on geographical grid |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181113 |