CN110263017B - Key value storage method and device based on spherical grid remote sensing data file set - Google Patents
Key value storage method and device based on spherical grid remote sensing data file set Download PDFInfo
- Publication number
- CN110263017B CN110263017B CN201910500955.6A CN201910500955A CN110263017B CN 110263017 B CN110263017 B CN 110263017B CN 201910500955 A CN201910500955 A CN 201910500955A CN 110263017 B CN110263017 B CN 110263017B
- Authority
- CN
- China
- Prior art keywords
- remote sensing
- sensing data
- file
- source file
- data source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000012545 processing Methods 0.000 claims abstract description 19
- 230000006870 function Effects 0.000 claims description 17
- 238000004458 analytical method Methods 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a key value storage method and device based on a spherical grid remote sensing data file set. The method comprises the following steps: acquiring the file name and the file size of each remote sensing data source file, and determining the data format of the remote sensing data source file according to the file name; calling a subdivision processing function corresponding to the data format according to the data format to analyze the remote sensing data in the remote sensing data source file; and if the file size is judged to be larger than the threshold value, splitting and storing the remote sensing data of the remote sensing data source file after being analyzed into a database according to a preset storage format according to a regular polyhedron spherical mesh splitting method. The device is used for executing the method. The key value storage method and device based on the spherical grid remote sensing data file set provided by the embodiment of the invention improve the storage efficiency of the remote sensing data.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a key value storage method and device based on a spherical grid remote sensing data file set.
Background
The remote sensing technology is widely applied to various fields such as national economy, national defense and military, ecological protection and the like, such as meteorological observation, mapping, military investigation, ecological condition assessment and the like.
With the wide application of remote sensing technology, a large number of remote sensing data are obtained by utilizing aircrafts such as remote sensing satellites and airplanes, and the remote sensing data are various in types and are not uniform in format. In the prior art, the received remote sensing data is directly stored, and the storage efficiency is low due to the fact that a source file of single remote sensing data is large. In addition, after the source file of the remote sensing data is analyzed, the stored remote sensing data file is usually named according to the channel of the remote sensing data and the time of generating the remote sensing data, and the file name is used as a key field in the inquiry process of the remote sensing data, which is not beneficial to the access to the remote sensing data during the calculation and analysis of the remote sensing data.
Therefore, how to provide a method for storing remote sensing data, which can rapidly store the remote sensing data to improve the storage efficiency of the remote sensing data, is an important issue to be solved in the industry.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a key value storage method and device based on a spherical grid remote sensing data file set.
On one hand, the invention provides a key value storage method based on a spherical mesh remote sensing data file set, which comprises the following steps:
acquiring the file name and the file size of each remote sensing data source file, and determining the data format of the remote sensing data source file according to the file name;
calling a subdivision processing function corresponding to the data format according to the data format to analyze the remote sensing data in the remote sensing data source file;
and if the file size is judged to be larger than the threshold value, splitting and storing the remote sensing data of the remote sensing data source file after being analyzed into a database according to a preset storage format according to a regular polyhedron spherical mesh splitting method.
Further, the method further comprises the step of storing the remote sensing data of the remote sensing data source file after analysis into a database according to a preset storage format if the file size is judged to be smaller than or equal to the threshold value.
Further, the splitting and storing the analyzed remote sensing data of the remote sensing data source file into a database according to a preset storage format according to the regular polyhedron spherical mesh splitting method comprises:
dividing the remote sensing data of the remote sensing data source file into a plurality of pieces of remote sensing data according to a regular polyhedron spherical mesh subdivision method, continuously storing the plurality of pieces of remote sensing data, wherein each piece of remote sensing data corresponds to part of continuous remote sensing data of the remote sensing data source file, and the size of each piece of remote sensing data does not exceed a preset value.
The method further comprises the steps of encoding the block remote sensing data by using a filling curve code to enable each block remote sensing data to obtain a unique code, taking the code of the block remote sensing data as an access identifier of the block remote sensing data, and enabling the access identifier of each block remote sensing data to form an index; and storing the key and the value into the database by taking the index as a key and the access identifier of the block remote sensing data as a value.
Further, the method comprises receiving an access request for remote sensing data, wherein the access request comprises at least one index field;
finding the access identifier of the block remote sensing data corresponding to the index field in the index according to the index field; wherein, the corresponding relation between the index field and the access identifier is preset;
and acquiring the remote sensing data corresponding to the index field according to the access identifier of each block of remote sensing data.
Further, the method further comprises the step of establishing a receiving schedule of the remote sensing data source file when the remote sensing data source file is received.
In another aspect, the present invention provides a key value storage device based on a spherical mesh remote sensing data file set, including:
the acquisition unit is used for acquiring the file name and the file size of each remote sensing data source file and determining the data format of the remote sensing data source file according to the file name;
the calling unit is used for calling a subdivision processing function corresponding to the data format according to the data format to analyze the remote sensing data in the remote sensing data source file;
and the first storage unit is used for splitting and storing the remote sensing data of the analyzed remote sensing data source file into a database according to a preset storage format according to a regular polyhedron spherical mesh splitting method if the file size is judged to be larger than a threshold value.
The device further comprises a second storage unit, which is used for storing the remote sensing data of the remote sensing data source file after analysis into a database according to a preset storage format if the file size is judged to be smaller than or equal to the threshold value.
In yet another aspect, the present invention provides an electronic device comprising: a processor, a memory, and a communication bus, wherein:
the processor and the memory are communicated with each other through the communication bus;
the storage stores program instructions executable by the processor, and the processor calls the program instructions to execute the key value storage method based on the spherical mesh remote sensing data file set provided by the embodiments.
In yet another aspect, the present invention provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method for storing key values based on a spherical mesh remote sensing data file set according to the embodiments.
According to the key value storage method and device based on the spherical mesh remote sensing data file set, the file name and the file size of each remote sensing data source file can be obtained, the data format of the remote sensing data source file is determined according to the file name, then the remote sensing data in the remote sensing data source file is analyzed by calling the subdivision processing function corresponding to the data format according to the data format, and after the fact that the file size is larger than a threshold value is judged and obtained, the remote sensing data of the remote sensing data source file after being analyzed is stored in a database in a split mode according to the regular polyhedron spherical mesh subdivision method, and storage efficiency of the remote sensing data is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
Fig. 1 is a schematic flowchart of a key value storage method based on a spherical mesh remote sensing data file set according to an embodiment of the present invention;
FIG. 2 is a schematic flowchart of a key value storage method based on a spherical mesh remote sensing data file set according to another embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a key value storage device based on a spherical mesh remote sensing data file set according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a key value storage device based on a spherical mesh remote sensing data file set according to another embodiment of the present invention;
fig. 5 is a schematic physical structure diagram of an electronic device according to an embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic flowchart of a key value storage method based on a spherical mesh remote sensing data file set according to an embodiment of the present invention, and as shown in fig. 1, the key value storage method based on the spherical mesh remote sensing data file set provided by the present invention includes:
s101, obtaining a file name and a file size of each remote sensing data source file, and determining a data format of each remote sensing data source file according to the file name;
specifically, when receiving remote sensing data from aircraft such as remote sensing satellites and airplanes, a key value storage device (hereinafter referred to as a storage device) based on a spherical mesh remote sensing data file set can obtain the file name and the file size of each remote sensing data source file. The storage device can obtain the data format of the remote sensing data source file by analyzing the file name.
S102, calling a subdivision processing function corresponding to the data format according to the data format to analyze remote sensing data in the remote sensing data source file;
specifically, after obtaining the data format of the remote sensing data source file, the storage device calls a subdivision processing function corresponding to the data format according to the data format, and analyzes the remote sensing data in the remote sensing data source file. Wherein the subdivision processing function is preset.
S103, if the file size is judged to be larger than the threshold value, splitting and storing the remote sensing data of the analyzed remote sensing data source file into a database according to a preset storage format according to a regular polyhedron spherical mesh splitting method.
Specifically, the storage device compares the file size with the threshold, if the file size is larger than the threshold, the remote sensing data of the remote sensing data source file after analysis is split according to a regular polyhedron spherical mesh splitting method, then the remote sensing data is stored in a database according to a preset storage format, finally, the remote sensing data in each remote sensing data source file with the file size larger than the threshold is split into block remote sensing data, and each block remote sensing data has a corresponding access identifier.
The preset storage format is set according to actual experience, and the embodiment of the invention is not limited.
The database may be a peer-to-peer distributed database, and the peer-to-peer distributed database is a distributed database using a P2P network, and has the characteristics of high reliability and high access efficiency because there is no difference between the master node and the slave node.
According to the key value storage method based on the spherical mesh remote sensing data file set, the file name and the file size of each remote sensing data source file can be obtained, the data format of the remote sensing data source file is determined according to the file name, then the remote sensing data in the remote sensing data source file is analyzed by calling the subdivision processing function corresponding to the data format according to the data format, and after the fact that the file size is larger than a threshold value is judged and obtained, the remote sensing data of the remote sensing data source file after being analyzed is stored in a database in a split mode according to the regular polyhedron spherical mesh subdivision method, and storage efficiency of the remote sensing data is improved.
On the basis of the foregoing embodiments, further, the method for storing key values based on a spherical mesh remote sensing data file set provided by the present invention further includes:
and if the file size is judged to be smaller than or equal to the threshold value, storing the remote sensing data of the remote sensing data source file after analysis into the database according to the preset storage format.
Specifically, the storage device compares the file size with the threshold, and if the file size is smaller than or equal to the threshold, the storage device stores the remote sensing data obtained after the remote sensing data source file is analyzed into a database according to the preset storage format without splitting the remote sensing data source file.
On the basis of the foregoing embodiments, further, the splitting, according to a regular polyhedron spherical mesh subdivision method, the parsed remote sensing data of the remote sensing data source file into data blocks according to a preset storage format, and storing the data blocks in a database includes:
dividing the remote sensing data of the remote sensing data source file into a plurality of pieces of remote sensing data according to a regular polyhedron spherical mesh subdivision method, continuously storing the plurality of pieces of remote sensing data, wherein each piece of remote sensing data corresponds to part of continuous remote sensing data of the remote sensing data source file, and the size of each piece of remote sensing data does not exceed a preset value.
Specifically, mesh subdivision is to generate a group of fine-grained sub-meshes from a coarse-grained parent mesh by adopting a uniform method to form a tree-shaped hierarchical structure among the meshes, so that the polyhedral surface is divided into a multi-resolution discrete mesh set. The regular polyhedron spherical mesh generation method adopted by the embodiment of the invention is, for example, an HTM spherical layered triangular mesh generation method, the spherical mesh is divided into four parts continuously by adopting the idea of a quadtree, and a multi-resolution hierarchical triangular mesh structure is formed by connecting large arcs, the invariability of the area and the shape of a triangular mesh unit is kept as much as possible in each generation process, and the recursive multi-resolution structure enables the HTM spherical triangular mesh to support the uniform organization of unevenly distributed data.
The storage device can divide the remote sensing data of the remote sensing data source file into a plurality of pieces of remote sensing data according to a regular polyhedron spherical mesh subdivision method. When storing each block remote sensing data, continuously storing each block remote sensing data, wherein each block remote sensing data is continuous data, namely the block remote sensing data is partially continuous remote sensing data from a remote sensing data source file, the block remote sensing data corresponds to the partially continuous remote sensing data of the remote sensing data source file, and the size of each block remote sensing data is not more than a preset value. The preset value is set according to actual experience, and the embodiment of the invention is not limited.
On the basis of the foregoing embodiments, further, the method for storing key values based on a spherical mesh remote sensing data file set provided by the present invention further includes:
encoding the block remote sensing data by using a filling curve code to enable each block remote sensing data to obtain a unique code, taking the code of the block remote sensing data as an access identifier of the block remote sensing data, and forming an index by the access identifier of each block remote sensing data; and storing the key and the value into the database by taking the index as a key and the access identifier of the block remote sensing data as a value.
Specifically, after the storage device obtains the block remote sensing data, the block remote sensing data is encoded by using a filling curve code, such as a Hilbert curve code, and after the block remote sensing data is encoded, each block remote sensing data obtains a unique code, that is, the block remote sensing data and the codes correspond to each other one by one, the codes of the block remote sensing data are used as access identifiers of the block remote sensing data, and the access identifiers of the block remote sensing data form an index. And storing the key and the value into the database by taking the index as a key and the access identifier of the block remote sensing data as a value.
Fig. 2 is a schematic flowchart of a key value storage method based on a spherical mesh remote sensing data file set according to another embodiment of the present invention, and as shown in fig. 2, on the basis of the foregoing embodiments, further, the key value storage method based on a spherical mesh remote sensing data file set according to the present invention further includes:
s201, receiving an access request for remote sensing data, wherein the access request comprises at least one index field;
in particular, the server may send an access request for telemetric data to the storage device, and the storage device may receive the access request, the access request including at least one of the index fields. Wherein the index field includes, but is not limited to, name, number, identification, and the like.
S202, finding the access identifier of the block remote sensing data corresponding to the index field in the index according to the index field; wherein, the corresponding relation between the index field and the access identifier is preset;
specifically, after receiving the access request, the storage device searches the index field in the index according to the index field, and finds the access identifier of the block remote sensing data corresponding to the index field. And the corresponding relation between the index field and the access identifier is preset.
S203, obtaining the remote sensing data corresponding to the index field according to the access identification of each block of remote sensing data.
Specifically, after obtaining the access identifier of the block remote sensing data corresponding to the index field, the storage device may find the block remote sensing data corresponding to each block according to the access identifier of each block remote sensing data, where each block remote sensing data found is the remote sensing data corresponding to the index field. The storage device can acquire the remote sensing data corresponding to the index field and send the remote sensing data corresponding to the index field to the server. And the corresponding relation between the index field and the access identifier is preset.
On the basis of the foregoing embodiments, further, on the basis of the foregoing embodiments, the method for storing key values based on a spherical mesh remote sensing data file set provided by the present invention further includes:
and when the remote sensing data source file is received, establishing a receiving schedule of the remote sensing data source file.
Specifically, when receiving the remote sensing data source file, the storage device may record a time of each received remote sensing data source file, where the time is a receiving time, and store a filename of each remote sensing data source file in correspondence with the receiving time. And the storage device establishes a receiving time table of the remote sensing data source files according to the file names of the remote sensing data source files and the receiving times corresponding to the file names.
Fig. 3 is a schematic structural diagram of a key value storage device based on a spherical mesh remote sensing data file set according to an embodiment of the present invention, and as shown in fig. 3, the key value storage device based on a spherical mesh remote sensing data file set provided by the present invention includes an obtaining unit 301, a calling unit 302, and a first storage unit 303, where:
the obtaining unit 301 is configured to obtain a file name and a file size of each remote sensing data source file, and determine a data format of each remote sensing data source file according to the file name; the calling unit 302 is configured to call a subdivision processing function corresponding to the data format according to the data format to analyze remote sensing data in the remote sensing data source file; the first storage unit 303 is configured to split and store the remote sensing data of the analyzed remote sensing data source file into a database according to a preset storage format according to a regular polyhedron spherical mesh partitioning method if it is determined that the size of the file is larger than a threshold value.
Specifically, when receiving remote sensing data from an aircraft such as a remote sensing satellite or an airplane, the obtaining unit 301 may obtain a file name and a file size of each remote sensing data source file. The obtaining unit 301 may obtain the data format of the remote sensing data source file by analyzing the file name.
After obtaining the data format of the remote sensing data source file, the calling unit 302 calls a subdivision processing function corresponding to the data format according to the data format, and analyzes the remote sensing data in the remote sensing data source file. Wherein the subdivision processing function is preset.
The first storage unit 303 compares the file size with the threshold, if the file size is larger than the threshold, the first storage unit 303 splits the remote sensing data of the analyzed remote sensing data source file according to a regular polyhedron spherical mesh splitting method, then stores the remote sensing data in a database according to a preset storage format, and finally the remote sensing data in each remote sensing data source file with the file size larger than the threshold is split into block remote sensing data, and each block remote sensing data has a corresponding access identifier. The preset storage format is set according to actual experience, and the embodiment of the invention is not limited. Wherein the database may be a peer-to-peer distributed database.
According to the key value storage device based on the spherical mesh remote sensing data file set, the file name and the file size of each remote sensing data source file can be obtained, the data format of the remote sensing data source file is determined according to the file name, then the remote sensing data in the remote sensing data source file is analyzed by calling the subdivision processing function corresponding to the data format according to the data format, and after the fact that the file size is larger than a threshold value is judged and obtained, the remote sensing data of the remote sensing data source file after being analyzed is stored in a database in a split mode according to the regular polyhedron spherical mesh subdivision method, and storage efficiency of the remote sensing data is improved.
Fig. 4 is a schematic structural diagram of a key value storage device based on a spherical mesh remote sensing data file set according to another embodiment of the present invention, and as shown in fig. 4, the key value storage device based on a spherical mesh remote sensing data file set provided by the present invention further includes a second storage unit 304, where:
the second storage unit 304 is configured to, if it is determined that the file size is smaller than or equal to the threshold, store the remote sensing data of the remote sensing data source file after being analyzed into a database according to a preset storage format.
Specifically, the second storage unit 304 compares the file size with the threshold, and if the file size is smaller than or equal to the threshold, the second storage unit 304 stores the remote sensing data obtained after the remote sensing data source file is analyzed into the database according to the preset storage format without splitting the remote sensing data source file.
The embodiment of the apparatus provided in the present invention may be specifically configured to execute the processing flows of the above method embodiments, and the functions of the apparatus are not described herein again, and refer to the detailed description of the above method embodiments.
Fig. 5 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention, as shown in fig. 5, the electronic device includes a processor (processor)501, a memory (memory)502, and a communication bus 503;
the processor 501 and the memory 502 complete communication with each other through a communication bus 503;
the processor 501 is configured to call program instructions in the memory 502 to perform the methods provided by the above-described method embodiments, including, for example: acquiring the file name and the file size of each remote sensing data source file, and determining the data format of the remote sensing data source file according to the file name; calling a subdivision processing function corresponding to the data format according to the data format to analyze the remote sensing data in the remote sensing data source file; and if the file size is judged to be larger than the threshold value, splitting and storing the remote sensing data of the remote sensing data source file after being analyzed into a database according to a preset storage format according to a regular polyhedron spherical mesh splitting method.
The present embodiment discloses a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the method provided by the above-mentioned method embodiments, for example, comprising: acquiring the file name and the file size of each remote sensing data source file, and determining the data format of the remote sensing data source file according to the file name; calling a subdivision processing function corresponding to the data format according to the data format to analyze the remote sensing data in the remote sensing data source file; and if the file size is judged to be larger than the threshold value, splitting and storing the remote sensing data of the remote sensing data source file after being analyzed into a database according to a preset storage format according to a regular polyhedron spherical mesh splitting method.
The present embodiments provide a non-transitory computer-readable storage medium storing computer instructions that cause the computer to perform the methods provided by the above method embodiments, for example, including: acquiring the file name and the file size of each remote sensing data source file, and determining the data format of the remote sensing data source file according to the file name; calling a subdivision processing function corresponding to the data format according to the data format to analyze the remote sensing data in the remote sensing data source file; and if the file size is judged to be larger than the threshold value, splitting and storing the remote sensing data of the remote sensing data source file after being analyzed into a database according to a preset storage format according to a regular polyhedron spherical mesh splitting method.
In addition, the logic instructions in the memory may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer (which may be a personal computer, an apparatus, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (9)
1. A key value storage method based on a spherical grid remote sensing data file set is characterized by comprising the following steps:
acquiring the file name and the file size of each remote sensing data source file, and determining the data format of the remote sensing data source file according to the file name;
calling a subdivision processing function corresponding to the data format according to the data format to analyze the remote sensing data in the remote sensing data source file;
if the file size is judged to be larger than the threshold value, splitting and storing the remote sensing data of the remote sensing data source file after being analyzed into a database according to a preset storage format according to a regular polyhedron spherical mesh splitting method;
the splitting and storing the remote sensing data of the remote sensing data source file after being analyzed into a database according to a preset storage format according to the regular polyhedron spherical mesh splitting method comprises the following steps:
dividing the remote sensing data of the remote sensing data source file into a plurality of pieces of remote sensing data according to a regular polyhedron spherical mesh subdivision method, continuously storing the plurality of pieces of remote sensing data, wherein each piece of remote sensing data corresponds to part of continuous remote sensing data of the remote sensing data source file, and the size of each piece of remote sensing data does not exceed a preset value.
2. The key value storage method based on the spherical mesh remote sensing data file set according to claim 1, further comprising:
and if the file size is judged to be smaller than or equal to the threshold value, storing the remote sensing data of the remote sensing data source file after analysis into a database according to a preset storage format.
3. The key value storage method based on the spherical mesh remote sensing data file set according to claim 2, further comprising:
encoding the block remote sensing data by using a filling curve code to enable each block remote sensing data to obtain a unique code, taking the code of the block remote sensing data as an access identifier of the block remote sensing data, and forming an index by the access identifier of each block remote sensing data; and storing the key and the value into the database by taking the index as a key and the access identifier of the block remote sensing data as a value.
4. The key value storage method based on the spherical mesh remote sensing data file set according to claim 3, further comprising:
receiving an access request for remote sensing data, wherein the access request comprises at least one index field;
finding the access identifier of the block remote sensing data corresponding to the index field in the index according to the index field; wherein, the corresponding relation between the index field and the access identifier is preset;
and acquiring the remote sensing data corresponding to the index field according to the access identifier of each block of remote sensing data.
5. The method for storing key values based on the spherical mesh remote sensing data file set according to any one of claims 1 to 4, further comprising:
and when the remote sensing data source file is received, establishing a receiving schedule of the remote sensing data source file.
6. A key value storage device based on a spherical grid remote sensing data file set is characterized by comprising:
the acquisition unit is used for acquiring the file name and the file size of each remote sensing data source file and determining the data format of the remote sensing data source file according to the file name;
the calling unit is used for calling a subdivision processing function corresponding to the data format according to the data format to analyze the remote sensing data in the remote sensing data source file;
the first storage unit is used for splitting and storing the remote sensing data of the analyzed remote sensing data source file into a database according to a preset storage format according to a regular polyhedron spherical mesh splitting method if the file size is judged to be larger than a threshold value;
the splitting and storing the remote sensing data of the remote sensing data source file after being analyzed into a database according to a preset storage format according to the regular polyhedron spherical mesh splitting method comprises the following steps:
dividing the remote sensing data of the remote sensing data source file into a plurality of pieces of remote sensing data according to a regular polyhedron spherical mesh subdivision method, continuously storing the plurality of pieces of remote sensing data, wherein each piece of remote sensing data corresponds to part of continuous remote sensing data of the remote sensing data source file, and the size of each piece of remote sensing data does not exceed a preset value.
7. The spherical mesh remote sensing data file set-based key value storage device of claim 6, further comprising:
and the second storage unit is used for storing the remote sensing data of the remote sensing data source file after analysis into a database according to a preset storage format if the file size is judged to be smaller than or equal to the threshold value.
8. An electronic device, comprising: a processor, a memory, and a communication bus, wherein:
the processor and the memory are communicated with each other through the communication bus;
the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of any of claims 1 to 5.
9. A non-transitory computer-readable storage medium storing computer instructions that cause a computer to perform the method of any one of claims 1 to 5.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810623509X | 2018-06-15 | ||
| CN201810623509 | 2018-06-15 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110263017A CN110263017A (en) | 2019-09-20 |
| CN110263017B true CN110263017B (en) | 2021-09-14 |
Family
ID=67917509
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910500955.6A Active CN110263017B (en) | 2018-06-15 | 2019-06-11 | Key value storage method and device based on spherical grid remote sensing data file set |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110263017B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111026706B (en) * | 2019-10-21 | 2023-10-13 | 武汉神库小匠科技有限公司 | Warehouse entry method, device, equipment and medium for power system data |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101964157A (en) * | 2010-09-27 | 2011-02-02 | 中国人民解放军国防科学技术大学 | Sphere rhombic grid recursive subdivision method |
| CN103428202A (en) * | 2013-07-24 | 2013-12-04 | 浙江大学 | Automatic monitored storage method of remote sensing data product based on file transfer protocol (FTP) and extensible markup language (XML) |
| CN103491185A (en) * | 2013-09-25 | 2014-01-01 | 浙江大学 | Remote sensing data cloud storage method based on image block organization |
| CN103714184A (en) * | 2014-01-13 | 2014-04-09 | 许昌学院 | Quick parallel processing method and device of remote-sensing images |
| CN103984759A (en) * | 2014-05-30 | 2014-08-13 | 中国科学院遥感与数字地球研究所 | Display-oriented multi-source remote sensing data set partition and reconstruction method |
| CN104159129A (en) * | 2014-08-08 | 2014-11-19 | 北京大学 | Method for progressively transmitting partial splitting blocks under remote sensing data limited bandwidth |
| CN107463706A (en) * | 2017-08-18 | 2017-12-12 | 国网上海市电力公司 | A kind of storage of magnanimity recorder data and parsing method and system based on Hadoop |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7007035B2 (en) * | 2001-06-08 | 2006-02-28 | The Regents Of The University Of California | Parallel object-oriented decision tree system |
| CN107066562A (en) * | 2017-03-31 | 2017-08-18 | 山东农业大学 | A kind of storage method of satellite remote-sensing image data |
-
2019
- 2019-06-11 CN CN201910500955.6A patent/CN110263017B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101964157A (en) * | 2010-09-27 | 2011-02-02 | 中国人民解放军国防科学技术大学 | Sphere rhombic grid recursive subdivision method |
| CN103428202A (en) * | 2013-07-24 | 2013-12-04 | 浙江大学 | Automatic monitored storage method of remote sensing data product based on file transfer protocol (FTP) and extensible markup language (XML) |
| CN103491185A (en) * | 2013-09-25 | 2014-01-01 | 浙江大学 | Remote sensing data cloud storage method based on image block organization |
| CN103714184A (en) * | 2014-01-13 | 2014-04-09 | 许昌学院 | Quick parallel processing method and device of remote-sensing images |
| CN103984759A (en) * | 2014-05-30 | 2014-08-13 | 中国科学院遥感与数字地球研究所 | Display-oriented multi-source remote sensing data set partition and reconstruction method |
| CN104159129A (en) * | 2014-08-08 | 2014-11-19 | 北京大学 | Method for progressively transmitting partial splitting blocks under remote sensing data limited bandwidth |
| CN107463706A (en) * | 2017-08-18 | 2017-12-12 | 国网上海市电力公司 | A kind of storage of magnanimity recorder data and parsing method and system based on Hadoop |
Non-Patent Citations (1)
| Title |
|---|
| 遥感图像快速压缩传输系统的设计与实现;陈隆等;《软件》;20120228;第5-8页，第11页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110263017A (en) | 2019-09-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11481440B2 (en) | System and method for processing metadata to determine an object sequence | |
| CN104252481A (en) | Dynamic check method and device for consistency of main and salve databases | |
| CN113536081B (en) | Data center data management method and system based on artificial intelligence | |
| CN105100050A (en) | User permission management method and system | |
| CN112130891B (en) | Method and equipment for continuously deploying database | |
| CN114117160A (en) | Threat analysis map generation and application method and device based on threat information | |
| CN110737719A (en) | Data synchronization method, device, equipment and computer readable storage medium | |
| CN113183759A (en) | Method and device for displaying characters of instrument panel | |
| CN111090669A (en) | Data query method and device based on space-time collision | |
| CN110263017B (en) | Key value storage method and device based on spherical grid remote sensing data file set | |
| CN111159142B (en) | Data processing method and device | |
| CN113986828A (en) | Method and device for storing mass files, electronic equipment and storage medium | |
| CN113127482A (en) | Data quality analysis method and device, computer equipment and storage medium | |
| CN114610825A (en) | Method and device for confirming associated grid set, electronic equipment and storage medium | |
| CN112068812B (en) | Micro-service generation method and device, computer equipment and storage medium | |
| CN113111351B (en) | Test method, test device and computer readable storage medium | |
| CN112433757A (en) | Method and device for determining interface calling relationship | |
| CN110598072B (en) | Feature data aggregation method and device | |
| US20170139969A1 (en) | Method for filtering and analyzing big data, electronic device, and non-transitory computer-readable storage medium | |
| CN112181825A (en) | Test case library construction method and device, electronic equipment and medium | |
| CN102609419B (en) | Similar data de-duplication method | |
| CN112052341A (en) | Knowledge graph pruning method and device | |
| CN106168983B (en) | Mixed resource processing method and device | |
| CN110727532A (en) | Data restoration method, electronic device and storage medium | |
| CN112115148B (en) | Method, device and equipment for determining data comparison result |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |