CN107423431A - A kind of remotely-sensed data storage method and system based on distributed file system - Google Patents

A kind of remotely-sensed data storage method and system based on distributed file system Download PDF

Info

Publication number
CN107423431A
CN107423431A CN201710656126.8A CN201710656126A CN107423431A CN 107423431 A CN107423431 A CN 107423431A CN 201710656126 A CN201710656126 A CN 201710656126A CN 107423431 A CN107423431 A CN 107423431A
Authority
CN
China
Prior art keywords
data
access
remote sensing
hbase
mongodb
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710656126.8A
Other languages
Chinese (zh)
Inventor
陈广胜
程益群
景维鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northeast Forestry University
Original Assignee
Northeast Forestry University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northeast Forestry University filed Critical Northeast Forestry University
Priority to CN201710656126.8A priority Critical patent/CN107423431A/en
Publication of CN107423431A publication Critical patent/CN107423431A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/13—File access structures, e.g. distributed indices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present invention proposes a kind of remotely-sensed data storage method and system based on distributed file system, the inventive method is by establishing the distributed storage framework towards remote sensing application, store remote sensing raw video data and metadata and slice of data, and define two access interfaces, respectively MongoDB access interfaces and the direct access interfaces of HDFS/Hbase, can compatible existing Remote Sensing Data Processing platform.With the method for the invention it is achieved that the distributed storage of polytype remotely-sensed data, and provide access and processing of the access interface realization to remotely-sensed data.

Description

A kind of remotely-sensed data storage method and system based on distributed file system
Technical field
The present invention relates to remote sensing image technical field of memory, more particularly to a kind of remote sensing number based on distributed file system According to storage method and system.
Background technology
The remotely-sensed data storage system based on distributed file system that there is currently belongs to the letter to existing system mostly Singly piece together, lack the support of main flow big data processing framework, the processing to remotely-sensed data brings inconvenience.Simultaneously as Remotely-sensed data includes all diversiform datas in itself, and existing distributed file system can not be directly for polytype remote sensing number According to being stored, also lack unified interface and multiple types of data is managed, the larger inconvenience that the use to user is brought.
The content of the invention
In order to solve the above problems, the invention discloses a kind of remotely-sensed data storage method based on distributed file system And system, by the way that HDFS/Hbase and MongoDB is applied in combination, the distributed storage of polytype remotely-sensed data is realized, and And access interface is provided to realize access and processing of the Hadoop/Spark platforms to remotely-sensed data.
The present invention realizes by the following method：A kind of remotely-sensed data storage method based on distributed file system, including：
Remote sensing raw video data are stored using HDFS, store remote sensing metadata and slice of data using HBase, structure is distant Feel data distribution formula storing framework, be specially：
Using HDFS, remote sensing raw video data are stored in multiple DataNode nodes；
Select image_scale parameters or automatically determined by auto_scale () function to cut figure size, entered using GDAL storehouses Row distribution cuts figure；
Using improved Hilbert coded systems, expand Hilbert and be encoded to fixed-length coding and be defined as rowkey, institute It is HBase indexes to state rowkey；
Using format_file () function, the parallel storage of slice of data is realized；
Using put orders, the in-stockroom operation of remote sensing metadata is realized；
Define distributed access interface：The distributed access interface is respectively MongoDB access interfaces and HDFS/ The direct access interfaces of Hbase, the access mode of the distributed access interface are specially：
Judge whether to meet one of isMDB or isHDB states, if it is, determining whether, otherwise data write-in is local Disk；
Further determine whether to meet isHDB states, if it is, calling interface IHdfsOut (), realizes data access； Otherwise, call function mongo_convert (), by data buffer storage to MongoDB, IMongoOut () interface is called to realize data Access.
In described method, the image_scale parameters cut figure size parameter for determination, described to cut figure size parameter Including：128x128,256x256,512x512,1024x1024；The auto_scale () function is used to automatically determine to cut figure Size；
The format_file () function is cut map file for serializing and is put in storage；The put orders are used to add in table Add a line text data.
In described method, the isMDB states are permission MongoDB storehouses access state；The isHDB states are permission Directly access HDFS/HBase states.
In described method, the IHdfsOut () is used to realize the access based on HDFS/HBase data；
The mongo_convert () is used to realize remote sensing image data depositing to MongoDB document databases in HBase Storage and the storage of remote sensing metadata in HBase to MongoDB；
The IMongoOut () is used to realize the data access based on MongoDB.
In described method, the expansion Hilbert is encoded to fixed-length coding and is specially：Escape character and Hilbert are compiled Using 1 filling, it is 2 to expand length for code, wherein escape characterCut segment number- Length (Hilbert), being encoded to after the completion of expansion 2Cut segment number。
The present invention also proposes a kind of storage system：A kind of remotely-sensed data storage system based on distributed file system, bag Include：
Remotely-sensed data distributed storage framework：Remote sensing raw video data are stored using HDFS, remote sensing is stored using HBase Metadata and slice of data, remotely-sensed data distributed storage framework is built, be specially：
Remote sensing raw video data memory module, for using HDFS, it is former that remote sensing to be stored in multiple DataNode nodes Beginning image data；
Remote sensing metadata and slice of data memory module：Select image_scale parameters or by auto_scale () function Automatically determine and cut figure size, carrying out distribution using GDAL storehouses cuts figure；
Using improved Hilbert coded systems, expand Hilbert and be encoded to fixed-length coding and be defined as rowkey, institute It is HBase indexes to state rowkey；
Using format_file () function, the parallel storage of slice of data is realized；
Using put orders, the in-stockroom operation of remote sensing metadata is realized；
Distributed access interface：The distributed access interface is respectively MongoDB access interfaces and HDFS/Hbase straight Connect access interface；
Judge whether to meet one of isMDB or isHDB states, if it is, determining whether, otherwise data write-in is local Disk；
Further determine whether to meet isHDB states, if it is, calling interface IHdfsOut (), realizes data access； Otherwise, call function mongo_convert (), by data buffer storage to MongoDB, IMongoOut () interface is called to realize data Access.
In described system, the image_scale parameters cut figure size parameter for determination, described to cut figure size parameter Including：128x128,256x256,512x512,1024x1024；The auto_scale () function is used to automatically determine to cut figure Size；
The format_file () function is cut map file for serializing and is put in storage；The put orders are used to add in table Add a line text data.
In described system, the isMDB states are permission MongoDB storehouses access state；The isHDB states are permission Directly access HDFS/HBase states.
In described system, the IHdfsOut () is used to realize the access based on HDFS/HBase data；
The mongo_convert () is used to realize remote sensing image data depositing to MongoDB document databases in HBase Storage and the storage of remote sensing metadata in HBase to MongoDB；
The IMongoOut () is used to realize the data access based on MongoDB.
In described system, the expansion Hilbert is encoded to fixed-length coding and is specially：Escape character and Hilbert are compiled Using 1 filling, it is 2 to expand length for code, wherein escape characterCut segment number- Length (Hilbert), being encoded to after the completion of expansion 2Cut segment number。
The invention has the advantages that the distributed storage of polytype remotely-sensed data can be realized, and due to existing remote sensing Data processing platform (DPP) do not support directly to access mostly, therefore the system realizes and allows to carry out HDFS/HBase to MongoDB Data exchange, realize data access.
The present invention proposes that a kind of remotely-sensed data storage method and system, the inventive method based on distributed file system are led to The distributed storage framework established towards remote sensing application is crossed, stores remote sensing raw video data and metadata and slice of data, and Two access interfaces, respectively MongoDB access interfaces and the direct access interfaces of HDFS/Hbase are defined, can be compatible existing Remote Sensing Data Processing platform.With the method for the invention it is achieved that the distributed storage of polytype remotely-sensed data, and provide Access interface realizes the access and processing to remotely-sensed data.
Brief description of the drawings
, below will be to embodiment or prior art in order to illustrate more clearly of technical scheme of the invention or of the prior art The required accompanying drawing used is briefly described in description, it should be apparent that, drawings in the following description are only in the present invention Some embodiments recorded, for those of ordinary skill in the art, on the premise of not paying creative work, can be with Other accompanying drawings are obtained according to these accompanying drawings.
Fig. 1 is that remotely-sensed data distributed storage is built in a kind of remotely-sensed data storage method based on distributed file system Frame embodiment flow chart；
Fig. 2 is a kind of access side of distributed access interface in remotely-sensed data storage method based on distributed file system Formula embodiment flow chart；
Fig. 3 is a kind of remotely-sensed data memory system architecture schematic diagram based on distributed file system.
Embodiment
In order that those skilled in the art more fully understand the technical scheme in the embodiment of the present invention, and make the present invention's Above-mentioned purpose, feature and advantage can be more obvious understandable, technical scheme in the present invention made below in conjunction with the accompanying drawings further detailed Thin explanation.
The invention discloses a kind of remotely-sensed data storage method and system based on distributed file system, this, which crosses combination, makes With HDFS/Hbase and MongoDB, realize the distributed storage of polytype remotely-sensed data, and provide access interface with Realize access and processing of the Hadoop/Spark platforms to remotely-sensed data.
The present invention realizes by the following method：A kind of remotely-sensed data storage method based on distributed file system, including：
Remote sensing raw video data are stored using HDFS, store remote sensing metadata and slice of data using HBase, structure is distant Data distribution formula storing framework is felt, as shown in figure 1, being specially：
S101：Using HDFS, remote sensing raw video data are stored in multiple DataNode nodes；
S102：Select image_scale parameters or automatically determined by auto_scale () function to cut figure size, use GDAL Storehouse carries out distribution and cuts figure；
S103：Using improved Hilbert coded systems, expand Hilbert and be encoded to fixed-length coding and be defined as Rowkey, the rowkey are HBase indexes；
S104：Using format_file () function, the parallel storage of slice of data is realized；
S105：Using put orders, the in-stockroom operation of remote sensing metadata is realized；
The put command syntaxs are：
put<table>,<rowkey>,<family:column>,<value>,<timestamp>.Wherein table, which refers to, to be needed The table to be inserted, rowkey are index, and value is value, and timestamp is timestamp.
More careful explanation is done to step：Remote sensing metadata is the explanation data of remote sensing images, includes adopting for remote sensing images Collect time, size, satellite and the information such as sensor, collecting region.These information are particularly significant to explaining remotely-sensed data.Herein Table is rs_data tables, preserves remote sensing metadata.Rowkey is consistent with the remote sensing metadata key that HDFS is stored.Column is Metadata type, value are metadata values.
Define distributed access interface：The distributed access interface is respectively MongoDB access interfaces and HDFS/ The direct access interfaces of Hbase, the access mode of the distributed access interface is as shown in Fig. 2 be specially：
S201：Judge whether to meet one of isMDB or isHDB states, if it is, determining whether, otherwise data are write Enter local disk；
S202：Further determine whether to meet isHDB states, if it is, performing S203, otherwise, perform S204；
S203：Calling interface IHdfsOut (), realizes data access；
S204：Call function mongo_convert (), by data buffer storage to MongoDB, call IMongoOut () interface Realize data access.
Because existing Remote Sensing Data Processing platform is not supported directly to access HDFS/HBase mostly, therefore the system allows Carry out HDFS/HBase to MongoDB data exchange.
IHdfsOut () is defined in Output.java with IMongoOut () interface.Mongo_convert () function It is defined in Output.cpp.The map process correlative codes of the function are as follows：
public void map(Text key,Text value,Context context)throws IOException,InterruptedException{
String [] fields=value.toString () .split (" t ")；
String type=fields [0]；
String value=fields [1]；
String time=fields [2]；
BSONObject b=new BasicBSONObject ()；
b.put("type",type)；
b.put("date",date)；
b.put("time",time)；
context.write(new ObjectId(),b)；
}
In described method, the image_scale parameters cut figure size parameter for determination, are selected by user, The figure size parameter of cutting includes：128x128,256x256,512x512,1024x1024, remote sensing images cut the general root of figure size It is 128x128,256x256,512x512 according to the selection of raw video size of data, tetra- kinds of 1024x1024, to meet different resolutions The reading demand of rate image；The auto_scale () function is used to automatically determine to cut figure size；
Auto_scale () function determines to cut figure size according to raw video size of data and required resolution ratio.If do not specify Required resolution ratio, then auto_scale () take the maximum that current raw video data allow automatically.In view of sometimes user The diagram data of cutting of particular size is needed, system also allows user voluntarily to specify image_scale parameter values.
Auto_scale () function is defined in scale.java files, and correlation function code is as follows：
Level1-4 is the cut off value of raw video size and required resolution sizes ratio.Once count counted ratio value In a certain section, then corresponding image_scale values are returned.System default value is 1024, for not specifying required resolution ratio When value.
The format_file () function is cut map file for serializing and is put in storage；The put orders are used to add in table Add a line text data.
In described method, the isMDB states are permission MongoDB storehouses access state；The isHDB states are permission Directly access HDFS/HBase states.
In described method, the IHdfsOut () is used to realize the access based on HDFS/HBase data；
The mongo_convert () is used to realize remote sensing image data depositing to MongoDB document databases in HBase Storage and the storage of remote sensing metadata in HBase to MongoDB；
The IMongoOut () is used to realize the data access based on MongoDB.
In described method, the expansion Hilbert is encoded to fixed-length coding and is specially：Escape character and Hilbert are compiled Using 1 filling, it is 2 to expand length for code, wherein escape characterCut segment number- Length (Hilbert), being encoded to after the completion of expansion 2Cut segment number.Which all cuts segment to ensure that coding can accommodate.
The present invention also proposes a kind of storage system：A kind of remotely-sensed data storage system based on distributed file system, such as Shown in Fig. 3, including：
Remotely-sensed data distributed storage framework 100：Remote sensing raw video data are stored using HDFS, are stored using HBase Remote sensing metadata and slice of data, remotely-sensed data distributed storage framework is built, be specially：
Remote sensing raw video data memory module 110, for using HDFS, remote sensing to be stored in multiple DataNode nodes Raw video data；
Remote sensing metadata and slice of data memory module 120：Select image_scale parameters or by auto_scale () letter Number, which automatically determines, cuts figure size, and carrying out distribution using GDAL storehouses cuts figure；
Using improved Hilbert coded systems, expand Hilbert and be encoded to fixed-length coding and be defined as rowkey, institute It is HBase indexes to state rowkey；
Using format_file () function, the parallel storage of slice of data is realized；
Using put orders, the in-stockroom operation of remote sensing metadata is realized；
Distributed access interface 200：The distributed access interface is respectively MongoDB access interfaces 210 and HDFS/ The direct access interfaces 220 of Hbase；
Judge whether to meet one of isMDB or isHDB states, if it is, determining whether, otherwise data write-in is local Disk；
Further determine whether to meet isHDB states, if it is, calling interface IHdfsOut (), realizes data access； Otherwise, call function mongo_convert (), by data buffer storage to MongoDB, IMongoOut () interface is called to realize data Access.
In described system, the image_scale parameters cut figure size parameter for determination, described to cut figure size parameter Including：128x128,256x256,512x512,1024x1024；The auto_scale () function is used to automatically determine to cut figure Size；
The format_file () function is cut map file for serializing and is put in storage；The put orders are used to add in table Add a line text data.
In described system, the isMDB states are permission MongoDB storehouses access state；The isHDB states are permission Directly access HDFS/HBase states.
In described system, the IHdfsOut () is used to realize the access based on HDFS/HBase data；
The mongo_convert () is used to realize remote sensing image data depositing to MongoDB document databases in HBase Storage and the storage of remote sensing metadata in HBase to MongoDB；
The IMongoOut () is used to realize the data access based on MongoDB.
In described system, the expansion Hilbert is encoded to fixed-length coding and is specially：Escape character and Hilbert are compiled Using 1 filling, it is 2 to expand length for code, wherein escape characterCut segment number- Length (Hilbert), being encoded to after the completion of expansion 2Cut segment number。
The invention has the advantages that the distributed storage of polytype remotely-sensed data can be realized, and due to existing remote sensing Data processing platform (DPP) do not support directly to access mostly, therefore the system realizes and allows to carry out HDFS/HBase to MongoDB Data exchange, realize data access.
The present invention proposes that a kind of remotely-sensed data storage method and system, the inventive method based on distributed file system are led to The distributed storage framework established towards remote sensing application is crossed, stores remote sensing raw video data and metadata and slice of data, and Two access interfaces, respectively MongoDB access interfaces and the direct access interfaces of HDFS/Hbase are defined, can be compatible existing Remote Sensing Data Processing platform.With the method for the invention it is achieved that the distributed storage of polytype remotely-sensed data, and provide Access interface realizes the access and processing to remotely-sensed data.
Each embodiment in this specification is described by the way of progressive, identical similar portion between each embodiment Divide mutually referring to what each embodiment stressed is the difference with other embodiment.Although pass through embodiment The present invention is depicted, it will be appreciated by the skilled addressee that the present invention has the essence of many deformations and change without departing from the present invention God, it is desirable to which appended claim includes these deformations and changes the spirit without departing from the present invention.

Claims (10)

  1. A kind of 1. remotely-sensed data storage method based on distributed file system, it is characterised in that including：
    Remote sensing raw video data are stored using HDFS, remote sensing metadata and slice of data is stored using HBase, builds remote sensing number According to distributed storage framework, it is specially：
    Using HDFS, remote sensing raw video data are stored in multiple DataNode nodes；
    Select image_scale parameters or automatically determined by auto_scale () function to cut figure size, divided using GDAL storehouses Cloth cuts figure；
    Using improved Hilbert coded systems, expand Hilbert and be encoded to fixed-length coding and be defined as rowkey, it is described Rowkey is HBase indexes；
    Using format_file () function, the parallel storage of slice of data is realized；
    Using put orders, the in-stockroom operation of remote sensing metadata is realized；
    Define distributed access interface：The distributed access interface is respectively MongoDB access interfaces and HDFS/Hbase straight Access interface is connect, the access mode of the distributed access interface is specially：
    Judge whether to meet one of isMDB or isHDB states, if it is, determining whether, otherwise data write this earth magnetism Disk；
    Further determine whether to meet isHDB states, if it is, calling interface IHdfsOut (), realizes data access；It is no Then, call function mongo_convert (), by data buffer storage to MongoDB, IMongoOut () interface is called to realize that data are visited Ask.
  2. 2. the method as described in claim 1, it is characterised in that the image_scale parameters cut figure size ginseng for determination Number, the figure size parameter of cutting include：128x128,256x256,512x512,1024x1024；The auto_scale () letter Number is used to automatically determine to cut figure size；
    The format_file () function is cut map file for serializing and is put in storage；The put orders are used to add one in table Style of writing notebook data.
  3. 3. the method as described in claim 1, it is characterised in that the isMDB states are permission MongoDB storehouses access state； The isHDB states directly access HDFS/HBase states for permission.
  4. 4. the method as described in claim 1, it is characterised in that the IHdfsOut (), which is used to realize, is based on HDFS/HBase numbers According to access；
    The mongo_convert () be used to realizing remote sensing image data in HBase to MongoDB document databases storage with Storage of the remote sensing metadata to MongoDB in HBase；
    The IMongoOut () is used to realize the data access based on MongoDB.
  5. 5. the method as described in claim 1, it is characterised in that the expansion Hilbert is encoded to fixed-length coding and is specially：Expand Character and Hilbert codings are filled, wherein for escape character using 1 filling, it is 2 to expand lengthCut segment number- Length (Hilbert), 2 are encoded to after the completion of expansionCut segment number。
  6. A kind of 6. remotely-sensed data storage system based on distributed file system, it is characterised in that including：
    Remotely-sensed data distributed storage framework：Remote sensing raw video data are stored using HDFS, remote sensing member number is stored using HBase According to and slice of data, build remotely-sensed data distributed storage framework, be specially：
    Remote sensing raw video data memory module, for using HDFS, the original shadow of remote sensing to be stored in multiple DataNode nodes As data；
    Remote sensing metadata and slice of data memory module：Select image_scale parameters or automatic by auto_scale () function It is determined that cutting figure size, carrying out distribution using GDAL storehouses cuts figure；
    Using improved Hilbert coded systems, expand Hilbert and be encoded to fixed-length coding and be defined as rowkey, it is described Rowkey is HBase indexes；
    Using format_file () function, the parallel storage of slice of data is realized；
    Using put orders, the in-stockroom operation of remote sensing metadata is realized；
    Distributed access interface：The distributed access interface is respectively that MongoDB access interfaces and HDFS/Hbase are directly visited Ask interface；
    Judge whether to meet one of isMDB or isHDB states, if it is, determining whether, otherwise data write this earth magnetism Disk；
    Further determine whether to meet isHDB states, if it is, calling interface IHdfsOut (), realizes data access；It is no Then, call function mongo_convert (), by data buffer storage to MongoDB, IMongoOut () interface is called to realize that data are visited Ask.
  7. 7. system as claimed in claim 6, it is characterised in that the image_scale parameters cut figure size ginseng for determination Number, the figure size parameter of cutting include：128x128,256x256,512x512,1024x1024；The auto_scale () letter Number is used to automatically determine to cut figure size；
    The format_file () function is cut map file for serializing and is put in storage；The put orders are used to add one in table Style of writing notebook data.
  8. 8. system as claimed in claim 6, it is characterised in that the isMDB states are permission MongoDB storehouses access state； The isHDB states directly access HDFS/HBase states for permission.
  9. 9. system as claimed in claim 6, it is characterised in that the IHdfsOut (), which is used to realize, is based on HDFS/HBase numbers According to access；
    The mongo_convert () be used to realizing remote sensing image data in HBase to MongoDB document databases storage with Storage of the remote sensing metadata to MongoDB in HBase；
    The IMongoOut () is used to realize the data access based on MongoDB.
  10. 10. system as claimed in claim 6, it is characterised in that the expansion Hilbert is encoded to fixed-length coding and is specially： Using 1 filling, it is 2 to expand length for escape character and Hilbert codings, wherein escape characterCut segment number-Length (Hilbert) 2, are encoded to after the completion of expansionCut segment number。
CN201710656126.8A 2017-08-03 2017-08-03 A kind of remotely-sensed data storage method and system based on distributed file system Pending CN107423431A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710656126.8A CN107423431A (en) 2017-08-03 2017-08-03 A kind of remotely-sensed data storage method and system based on distributed file system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710656126.8A CN107423431A (en) 2017-08-03 2017-08-03 A kind of remotely-sensed data storage method and system based on distributed file system

Publications (1)

Publication Number Publication Date
CN107423431A true CN107423431A (en) 2017-12-01

Family

ID=60437310

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710656126.8A Pending CN107423431A (en) 2017-08-03 2017-08-03 A kind of remotely-sensed data storage method and system based on distributed file system

Country Status (1)

Country Link
CN (1) CN107423431A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109086649A (en) * 2018-05-29 2018-12-25 国网新疆电力有限公司信息通信公司 Satellite remote sensing images identifying water boy method
CN109710572A (en) * 2018-12-29 2019-05-03 北京赛思信安技术股份有限公司 A kind of file sharding method based on HBase
CN113495876A (en) * 2020-03-19 2021-10-12 中科星图股份有限公司 Spark-based image pyramid distributed slicing system and method
CN113722274A (en) * 2021-08-09 2021-11-30 河南农业大学 Efficient R-tree index remote sensing data storage model

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103281376A (en) * 2013-05-31 2013-09-04 武汉大学 Method for automatic caching construction of massive timing sequence remote-sensing images in cloud environment
CN103595791A (en) * 2013-11-14 2014-02-19 中国科学院深圳先进技术研究院 Cloud accessing method for mass remote sensing data
CN104331453A (en) * 2014-10-30 2015-02-04 北京思特奇信息技术股份有限公司 Distributed file system and constructing method thereof
CN104820714A (en) * 2015-05-20 2015-08-05 国家电网公司 Mass small tile file storage management method based on hadoop
US9171042B1 (en) * 2013-02-25 2015-10-27 Emc Corporation Parallel processing database tree structure

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9171042B1 (en) * 2013-02-25 2015-10-27 Emc Corporation Parallel processing database tree structure
CN103281376A (en) * 2013-05-31 2013-09-04 武汉大学 Method for automatic caching construction of massive timing sequence remote-sensing images in cloud environment
CN103595791A (en) * 2013-11-14 2014-02-19 中国科学院深圳先进技术研究院 Cloud accessing method for mass remote sensing data
CN104331453A (en) * 2014-10-30 2015-02-04 北京思特奇信息技术股份有限公司 Distributed file system and constructing method thereof
CN104820714A (en) * 2015-05-20 2015-08-05 国家电网公司 Mass small tile file storage management method based on hadoop

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
张晓兵: "基于HBase的弹性可视化遥感影像存储系统", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *
赵鹏举 等: "一种基于地理信息元数据标准的空间索引体系", 《科技导报》 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109086649A (en) * 2018-05-29 2018-12-25 国网新疆电力有限公司信息通信公司 Satellite remote sensing images identifying water boy method
CN109710572A (en) * 2018-12-29 2019-05-03 北京赛思信安技术股份有限公司 A kind of file sharding method based on HBase
CN109710572B (en) * 2018-12-29 2021-02-02 北京赛思信安技术股份有限公司 HBase-based file fragmentation method
CN113495876A (en) * 2020-03-19 2021-10-12 中科星图股份有限公司 Spark-based image pyramid distributed slicing system and method
CN113722274A (en) * 2021-08-09 2021-11-30 河南农业大学 Efficient R-tree index remote sensing data storage model

Similar Documents

Publication Publication Date Title
CN107423431A (en) A kind of remotely-sensed data storage method and system based on distributed file system
US8099421B2 (en) File system, and method for storing and searching for file by the same
CN102446184B (en) Industrial data storage and index method based on time series
US20100322301A1 (en) Image processor, image generator and computer program
CN110096507A (en) A kind of page complexity table rendering method, system, terminal and medium
CN104750859A (en) Network storing method
CN103037344B (en) A kind of ticket De-weight method and device
CN104951482B (en) A kind of method and device of the image file of operation Sparse formats
CN106095923A (en) A kind of method and system adding data of being on the list in orderly list
CN100507921C (en) Words display process in embedded system and system thereof
CN103067441A (en) Method, device and equipment of picture sharing
CN105354236A (en) Reconciliation information generation method and system
CN111639120B (en) Method, device and equipment for imaging architecture view and readable storage medium
CN110019636A (en) A kind of storage mode of GIS tile map
CN103034677A (en) Organizing and run coding index method for multidate tile data set
CN102810115A (en) Method for implementing multi-layer distributed document management system
CN110969000A (en) Data merging processing method and device
CN103761194B (en) A kind of EMS memory management process and device
CN109344412A (en) A kind of language transfer method, device, equipment and storage medium
CN102169504B (en) Database indexing method for monitoring satellite ground equipment
CN110109884A (en) A kind of file reading, device, equipment and medium
JP4821287B2 (en) Structured document encoding method, encoding apparatus, encoding program, decoding apparatus, and encoded structured document data structure
CN101639846A (en) Method for user-defined character font
CN115080563A (en) Data capturing method, device, system, electronic equipment and storage medium
CN101345531A (en) Data processing method for ogg format stream

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20171201

RJ01 Rejection of invention patent application after publication