CN106909644B - A kind of multistage tissue and indexing means towards mass remote sensing image - Google Patents
A kind of multistage tissue and indexing means towards mass remote sensing image Download PDFInfo
- Publication number
- CN106909644B CN106909644B CN201710092425.3A CN201710092425A CN106909644B CN 106909644 B CN106909644 B CN 106909644B CN 201710092425 A CN201710092425 A CN 201710092425A CN 106909644 B CN106909644 B CN 106909644B
- Authority
- CN
- China
- Prior art keywords
- data
- sub
- attribute
- multistage
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
Abstract
A kind of multistage tissue and indexing means towards mass remote sensing image, comprising the following steps: step A: obtain wide cut image raster data, pre-processed；Step B: establishing the caching pyramid of disk space, and the level indexing structure of the regular partition quaternary tree of sub-scene, connection ID and attribute database are added using scene pyramid；Step C: combined data organizational form realizes attribute data carrying, and blocking organization completes multistage storage in HBase database and establishes index；And step D: resource of distributing according to need whenever and wherever possible is realized by virtualization, scalability and sharing capability storage pool.
Description
Technical field
The present invention relates to remote sensing image data process field, in particular to a kind of multistage tissue towards mass remote sensing image
And indexing means.
Background technique
China obtains a large amount of remote sensing image data in recent years, and platform covers satellite platform, unmanned plane etc., resolution ratio from
Meter level is to Centimeter Level.Data have high resolution, the variation of imaging breadth greatly, image data amount greatly, the spies such as imaging pattern is diversified
Point, so that object judgement, wider acquisition of information and more diversified information that these data can be used for more refining generate
Deng, according to many kinds of parameters such as target type, image sensor type, time phase, matching degree combine, obtain relevant remote sensing shadow
It is data as and according to the different relevant object of attributive classification acquisition such as target relevant time, position and shape and target
Excavate one of very important information source.It needs to carry out classification processing to the remote sensing image of magnanimity, establishes point of multiple resolution ratio
Tomographic image realizes multi-level tissue, quick indexing and screening, improves mass data processing speed and Image Intelligence analysis efficiency.
Currently, register seldom the research achievement of remotely-sensed data organization and administration both at home and abroad, existing remotely-sensed data organizer
Method be mostly data-oriented storage organization and management method, be mostly solve the problems, such as remotely-sensed data how efficient storage, adopt more
Storage organization is carried out with the mode that database is combined with file system, metadata is stored in the database table of static structure,
Data entity is stored in file system, and remotely-sensed data is retrieved and read with data access access interface by Retrieval Interface
Write access.For example, Wang Mi et al. (Wang Mi, Gong Jianya, Li Deren, data organization that the space of large-scale Remote Sensing Image Database is seamless,
Wuhan University Journal information science version, 2001,26 (5): 419-423) it is directed to orthography, it proposes and utilizes a point band memory module
Realize the seamless organization management of image data.Song Shuhua, Cheng Chengqi et al. (remote sensing shadow of the such as Song Shuhua, Cheng Chengqi based on EMD
As new data organization model geography and Geographical Information Sciences .2013,29 (3): 21-25), proposition is expanded complete based on sheet line system
Remotely-sensed data is carried out fragment storage and tissue by the code identification of subdivision dough sheet by ball subdivision model EMD.Lai Jibao, Luo Xiao
A kind of beautiful et al. (remote sensing image data tissue model research computer science for supporting cloud computing of the such as Lai Jibao, Luo Xiaoli
.2013,40 (7): 80-84).The remote sensing image data tissue model RSC-DOM for proposing support cloud computing, in conjunction with image pyramid
The thought of piecemeal storage completes remote sensing image data storage and tissue under cloud computing framework.However, current pyramid level mould
Type cannot reflect difference of the image data on source sensor and time dimension, largely limit global image data
Effective use.It does not fully consider locating geographical location, be concerned grade and data accumulation degree, increase Non-hotspot region
Access pressure.
Google releases Google Earth software platform, it is all made of together with Google Map platform and fixes 18 numbers of plies
According to organizational structure, vector data terrestrial reference can be shown by having, geometric types, grating image superposition and the three-dimensional such as including point, line, surface
Model virtual functions.In terms of remote sensing image data Organization And Management, and using pre-generated map tile pyramid (Tile
Map Image) and each Tile be 256 × 256 pixel PNG files, image data is managed.Wherein, it uses map
Tile pyramid is that the image file in order to which data are quickly shown, using 256 × 256 pixels is in order to which calculator memory is efficient
Management, and using PNG file format is then in order to which computer network efficiently transmits.WorldWind is by NASA (American National
Aviation and space travel office) release a satellite photo for checking the earth virtually globe software, using plane Carree
Image data, length-width ratio 2:1 are expressed using rectangle.Using etc. four points of the partition patterns in the longitudes and latitudes whole world, the whole world is divided into
Image data (512 × 512 pixel) in the Tile of multilayer different scale, each Tile is identified and by floor group by ranks number
It knits and manages.WorldWind constructs linear quadtree on the basis of tile pyramid to manage tile index and tile data.Number
When according to retrieval, by the relationship between longitude and latitude and the ranks number of Tile, ranks number are retrieved to realize SkyLine company, the U.S.
Also TerraBuilder tool is devised for user, it can by aerophotograph, satellite image, digital elevation model and various vectors
Data are managed, are created at a single optimization compressed file, convenient for issuing 3-D data set in a streaming manner.Subtract according to the sequence of magnitude
Few runing time.Nevertheless, haveing the defects that certain, single width since the file access mode of standard obtains remote sensing image data
The data volume of remote sensing images is larger, and the I/O ability of storage system and network bandwidth limit file access speed, leads to these
Software reads and shows that the time delay of a width product image is quite big, the serious efficiency for limiting later period application and reliable
Property.
In addition, existing indexed links parser can not using the PageRank of Google as the hyperlink analysis of representative
Understanding content also can not just accomplish personalization.The country is based on the indexed mode of bibliographic structure according to the direct rope of the classification divided
Draw, but is unable to satisfy the document index with a certain specific information.And the document index mode based on query string can be than calibrated
Text information required for user really is found out, but since remotely-sensed data dimension is more, more complicated, the data scale of construction is bigger, this
A little Indexing Mechanisms can not all adapt to the efficient access of magnanimity multi-source multiattribute data.
Summary of the invention
In view of existing scheme there are the problem of, in order to overcome the shortcomings of above-mentioned prior art, the invention proposes one
Multistage tissue and indexing means of the kind towards mass remote sensing image.
According to an aspect of the invention, there is provided a kind of multistage tissue and indexing means towards mass remote sensing image,
The following steps are included: step A: obtaining wide cut image raster data, pre-processed；Step B: the caching gold of disk space is established
Word tower adds the level indexing structure of the regular partition quaternary tree of sub-scene, connection ID and attribute database using scene pyramid；
Step C: combined data organizational form realizes attribute data carrying, and blocking organization completes multistage storage in HBase database
It is indexed with establishing；And step D: it is realized whenever and wherever possible by virtualization, scalability and sharing capability storage pool
Distribution according to need resource.
It can be seen from the above technical proposal that the invention has the following advantages:
The present invention has the Local Characteristic for reinforcing raster data, reduces the amount of access of data, reduces the access of disk
Span, total amount of access reduce by 40%；Reasonable data-block cache mechanism is constructed, completes to need the efficient process of large scale data
It asks, the real-time display for each image processing and interpretation tool operation provides guarantee.Reasonable data are constructed in different scale
Block caching mechanism greatly improves the access efficiency of raster data.
The present invention is to make up the deficiency of traditional pyramid hierarchical model, the non-homogeneous compound pyramid hierarchical model of proposition,
Sensor and time dimension is anisotropically added in the other block data of each stage resolution ratio, fully demonstrates global image data
Multidimensional characteristic, preferably organize and utilize image data.The image pyramid for dynamically establishing different accuracy, reduces data
The amount of access of Non-hotspot region.In scene pyramid in the atural object data organization of each sub-scene, rule-based stroke is used
The quaternary tree index structure divided, avoids the multiple storage of atural object entity identifier, and greatly reduces complicated spatial relationship
It calculates.
The present invention devises multiple index mechanism, can build according to sensor, image resolution, imagery zone and time difference
Lithol draws, and is stored in HBase database.It distributes according to need resource, realizes magnanimity multi-source, multiple dimensioned, multidate image data
High efficiency of transmission and access.
Detailed description of the invention
Fig. 1 is multistage pyramid construction schematic illustration of the embodiment of the present invention；
Fig. 2 is the flow chart of multistage tissue and indexing means of the embodiment of the present invention towards mass remote sensing image；
Fig. 3 is the flow chart of Fig. 2 step A；
Fig. 4 is the flow chart of Fig. 2 step B；
Fig. 5 is the flow chart of Fig. 4 step B3；
Fig. 6 is the flow chart of Fig. 2 step C；
Fig. 7 is self-adapting compressing and progressive transmission schematic diagram in step D；
Fig. 8 is the flow chart of Fig. 2 step D.
Specific embodiment
Certain embodiments of the invention will be done referring to appended attached drawing in rear and more comprehensively describe to property, some of but not complete
The embodiment in portion will be shown.In fact, various embodiments of the present invention can be realized in many different forms, and should not be construed
To be limited to this several illustrated embodiment；Relatively, these embodiments are provided so that the present invention meets applicable legal requirement.
In the present specification, following various embodiments for describing the principle of the invention only illustrate, should not be with any
Mode is construed to the range of limitation invention.Referring to attached drawing the comprehensive understanding described below that is used to help by claim and its equivalent
The exemplary embodiment of the present invention that object limits.Described below includes a variety of details to help to understand, but these details are answered
Think to be only exemplary.Therefore, it will be appreciated by those of ordinary skill in the art that not departing from scope and spirit of the present invention
In the case where, embodiment described herein can be made various changes and modifications.In addition, for clarity and brevity,
The description of known function and structure is omitted.In addition, running through attached drawing, same reference numerals are used for identity function and operation.
To make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with specific embodiment, and reference
Attached drawing, the present invention is described in more detail.
In order to support rapid tissue and the retrieval of remote sensing image data, the embodiment of the present invention provides one kind towards magnanimity remote sensing
The multistage tissue and indexing means of image, the non-homogeneous compound Groupe de la Pyramide of specially a kind of dynamic and indexing means are using multistage
Pyramid Optimization Mechanism effectively improves image processing efficiency, and multistage pyramid construction principle is as shown in Figure 1, include image
Index, attribute and data (are write on one by pretreatment, the sampling of special tune ginseng bilinear interpolation, customized pyramid data file
Rise) and the Method of Data Organization based on goal systems etc..
Fig. 2 is the flow chart of multistage tissue and indexing means of the embodiment of the present invention towards mass remote sensing image, referring to attached
Fig. 2, a kind of multistage tissue and indexing means towards mass remote sensing image specifically includes the following steps:
Step A: obtaining wide cut image raster dataset, and completes the pretreatment of image raster data.
Wherein image raster dataset includes coordinate, projection, wave band, the image data of quantization, altitude data, grating map
Deng.
Step A specifically includes following sub-step, as shown in Figure 3:
Sub-step A1: image raster data breadth is larger, needs to be classified piecemeal to it, carries out coding to block data and retouches
It states；
By using the pyramid method of classification piecemeal, efficiency is exchanged for space, and establish buffer area in memory, according to number
Made according to geocoding piecemeal, four kinds of piecemeals generalling use production form include: NASA WorldWind,
GoogleEarth, MIP, sentence leveling platform, can by a set of TilesetConfig parameter to a variety of different block forms into
The unified description of row, is defined as follows:
Title
Geographic unit (degree, rice, pixel ...)
Geographic range (northern Nan Xidong)
Coordinate system
0th layer of block size (unit used with " geographic range ")
Top level number
Bottom level number
The Pixel Dimensions of data block
The file format of data block
Multi-zone supervision
Sub-step A2: the block data after Coding and description is cut and is merged under global geographic space frame.
Using 360 degree of longitudes of digital earth and 180 degree latitude as standard, plane is divided into stratification by different scale, with ranks
Mode stores latitude and longitude value, and each layer of block-by-block is encoded.The level that block data is currently located is first determined whether, then according to certainly
The upward column arrangement mode in bottom recurrence layer by layer is cut according to the column sequence of layer after image block, and it is suitable to assign to each piece
Block number, row number and the number of plies.
Sub-step A3: being read out loading processing to altitude data, and wherein altitude data is that image raster data is concentrated
Data；
Caching process is carried out to altitude data block, elevation is superimposed to image, is inquired to global different resolution level not
With the elevation put.Specifically, if elevation array of the elevation of point not in block, calculates the point height by bilinear interpolation.
Scheduling to altitude data is while to maintain a buffering about altitude data in memory by multithread scheduling
Area realizes the quick access to altitude data.
Sub-step A4: the visualization that tags of block data is realized.
Using view label function, topography and geomorphology in the visual in image display highlighting region of real-time visual technology, to screen
Curtain window display area is quickly positioned, and is directly targeted to map window according to the longitude and latitude of user's input or information of place names
Corresponding position.
Step B: establishing the caching pyramid of disk space, and the regular partition quaternary tree of sub-scene is added using scene pyramid
Level indexing structure, connection ID and attribute database.
Processing is split and simplified to image raster data, by reasonable efficient coding, establishes the slow of disk space
Deposit pyramid.The level indexing structure for being added the regular partition quaternary tree of sub-scene atural object using scene pyramid, can quickly be determined
Position and the scene pyramid for facilitating calculating neighbouring relations accelerate remotely-sensed data storage organization, and connection ID and attribute database.
Step B specifically includes following sub-step, as shown in Figure 4:
Sub-step B1: encoding the deblocking loaded, establish buffer area in system memory according to buffer memory principle, benefit
The ginseng i.e. data of the bilinear interpolation sampling processing proximate region of experience adjustments parameter are adjusted with special, realize dynamic pyramid weight
Structure.
Sub-step B2: defining the derivative model of remote sensing metadata, distinguishes storing data attribute and data in database table ranks
Entity, wherein database list row is classified as the database table structure of storage image database.
Meta data category device is constructed in training set, when new remotely-sensed data introduces, is included into existing remotely-sensed data classification,
If any remotely-sensed data classification can not be included into, new metadata type will be derived by the method for calculating information gain.Define S
For metadata sample training collection, c different class C are definedi(i=1 ... c), CI, sIt is the tuple-set of class in S, S is relative to c
The entropy of a attribute is defined as:
Wherein, piIt is that tuple belongs to classification C in SiProbability,Represent the summation operation from 1 to c, log2Indicate with 2 be
The logarithm operation at bottom.
Information gain Gain (S, A) of the defined attribute A with respect to sample set S is defined as:
Where it is assumed that dividing the tuple in S by attribute A, A has v different value { a according to the observation of training data1,
a2... av, S is divided into v subset { S by attribute A1, S2... Sv, SjTuple in representative sample training set S, on attribute A
Value be aj。The summation operation from 1 to v is represented, | S | indicate the number of tuple in sample training collection S, | Sj| indicate set Sj
The number of middle tuple,Represent the weight of j-th of Attribute transposition.It selects the attribute A of highest information gain to divide as classification to belong to
Property, i.e., new classification state is formed on original sample training collection S.
Sub-step B3: define remotely-sensed data correlation model, utilize " regular quaternary tree " representation object between be subordinate to,
The relationships such as arranged side by side, establish customized pyramid data file, data, attribute and index are associated together.
Sub-step B3 specifically includes following secondary sub-step, as shown in Figure 5:
Secondary sub-step B3.1: the five-tuple of the incidence relation of remotely-sensed data is defined；
Using time, space, observation object classification as the essential attribute of associated data, the incidence relation of remotely-sensed data can be with
It is defined as the five-tuple such as formula (3),
S=<ARt, ARs, ARo, TF, SF>(3)
Wherein, ARt, ARs, ARo are one group of correlation rules, one group of logical deduction by reasoning rule can be expressed as, for example, by condition
X, which is derived, asserts Y, is indicated with X → Y.ARt is the correlation rule of time dimension, and ARs is the correlation rule of Spatial Dimension, and ARo is
Correlation rule based on observation object.TF is temporal characteristics, and such as the life cycle of data, time identifier etc., SF is space spy
Sign, such as the spatial position of data, regional scope etc..
Secondary sub-step B3.2: data correlation tissue model is established；
Data correlation tissue model is indicated using formula (4), it indicates to contain the binary group of vertex set and side collection composition
Θ=<V, E>(4)
Wherein,
V={ v1, v2... E={ e1, e2... (5)
V is the data vertex set of non-empty, and each element representation meets remotely-sensed data classification or the remote sensing number of correlation rule
According to.E is the set of incidence relation, and the association that meets correlation rule of each element representation in some or certain several dimensions is closed
System, E are equivalent to the S in formula (3).If E is empty set, indicate all to close without association between remotely-sensed data in any dimension of definition
It is to be isolated between data unrelated.If E non-empty, then it represents that between remotely-sensed data in some dimension, under certain correlation rule
There are incidence relations.
Sub-step B3.3: by Bayesian learning, completing the update of association tissue model, is closed with adapting to real world association
It is the status of dynamic change, to realize the autonomous learning of model.
Serial number n is introduced with ΘnThe state of incidence relation between the remotely-sensed data of expression current procedures n, with Xn={ x1, x2…
xnIndicate current data set, with P (Θ0) indicate correlation model original state probability measure, it is assumed that XnData are mutually only in collecting
It is vertical, then there is posterior probability
Wherein, P (Θn) indicate the prior probability of not no training data, P (Xn) indicate the training data to be observed priori
Probability, P (Xn|Θn) indicate to assume ΘnData X is observed in the case where establishmentnProbability.In view of remotely-sensed data it is mutual solely
Vertical property, then have P (Xn|Xn-1, Θn)=P (xn, Θn), P (xn|Xn-1) it is normaliztion constant, then haveAvailable following formula:
Conditional probability P (xn|Θn) indicate Current observation data XnLikelihood score, P (Θn|Xn-1) indicate current learning procedure
Prior probability, and as n=1 the prior probability be probability P (Θ0).It is obtained down according to the aposterior knowledge learnt every time
The priori knowledge once learnt, to be made up of the state renewal process of Infinite Cyclic Knowledge delivery.It indicates are as follows:
The transfer process for forming a recursive form illustrates the study and renewal process of incidence relation state.
Step C: combined data organizational form realizes attribute data carrying, and blocking organization is completed in HBase database
Multistage storage and foundation index.
HBase is a kind of frame in big data storing data library based on Hadoop.Used the frame as number here
According to hoc solutions.
Suitable resolution match strategy is selected, unordered search procedure is avoided, multistage is completed in HBase database and is deposited
Storage and secondary index avoid a large amount of memory headroom from being occupied by temporarily useless index data.With the data based on goal systems
Organizational form and interrelational form are combined closely, and realize the access of Large Volume Data.
Step C specifically includes following steps, as shown in Figure 6:
Sub-step C1: combined data organizational form realizes attribute data carrying, blocking organization；
Image has unique ID identification information, connect with the attribute data of the image as independent spatial entities, and
It establishes and being associated with referring to objects system.All entities are managed using entity table in attribute database.Each entity exists
A record is uniquely corresponded in entity table, the essential information of storage entity, multistage is stored in HBase database.For task
Scale provides multiple dimensioned remote sensing image data, multi-scale data organization, realizes the inspection of related objective adequate resolution data
Rope, plotting, positioning.
Sub-step C2: completing multistage storage in HBase database and establishes index.
For multi-scale image data blocking organization, different multi-stage data scheduling schemes is provided.First is that with file
The data-selected scheme of folder form storage, second is that the storing data in the way of big file index, while data are provided to both data
Server.Under the support of HBase database, second level is established according to sensor, image resolution, imagery zone and time difference
Index.Space based on object of reference-attribute interactive query is realized corresponding coordinate position by referring to object physical model, is sat
It marks range or meets the fast data search in entity ID range of attributes.
Step D: it distributes according to need whenever and wherever possible by the storage pool of virtualization, scalability and sharing capability to realize
Resource, wherein referring to that storage pool is data warehouse, i.e. memory space.
Image raster data and other data need online migration in classification storage, this just needs to consider that data are mobile to preceding
The performance of platform I/O load influences.The variation that data Autonomic Migration Framework is loaded according to foreground I/O, to adjust Data Migration rate, so that
Influence to the service quality of storage system of Data Migration movement itself is very small, at the same enable data migration task as early as possible
It completes.The major technique that data Autonomic Migration Framework storage is related to has: the rate control and scheduling, Data Migration of Data Migration are to application
Latency hiding, file access block position sequence prediction etc..Data Migration needs to solve data compression and progressive transmission, such as schemes
Shown in 7.In order to support geospatial intelligence data to realize high efficiency of transmission on all types of networks, the variation to guarantee network bandwidth with
Delay not will use family and stagnate or fall into a long wait data.
Step D includes following sub-step, as shown in Figure 8:
Sub-step D1: according to the bandwidth in request source, suitable spatial data compression and expression way is selected, intelligence is passed through
The adaptive strategy of change is allocated the transmission mode of data.
Broadband uses Fast Compression and long data block transmission mode, and narrowband uses quick static compression and streaming transmission mode.
According to the parsing of terminal request task as a result, bandwidth, loading condition, concurrent access situations, terminal based on real transmission network
The multiple-factors tale quale such as data volume size cases of request designs efficient network transmission channel and estimates with unreliable channel bandwidth
Meter and bandwidth control, the comprehensive intelligent adaptive strategy study for carrying out transmission network.
Sub-step D2: reasonably dividing transmission data progress by different level, in batches, whole transmission data is divided into several
Relatively independent basic unit carries out compressed encoding to basic unit and continuously transmits, carries out basic unit again in client
It receives, decoding and initial data synthesize.
For raster data, most mature processing mode is exactly the processing of Pyramid technology piecemeal, compression algorithm
It is compressed using JPEG2000.For vector data, data volume is relatively fewer, the principle of multi-scale expression can be followed, using space
Information form general model and node LOD model handle different scale map, comprehensive Web Service, Plug-in and
The Technology designs progressive transmission architecture such as Script carries out coding tissue using the SVG of W3C, empty as issuing on browser
Between Vector Message carrier.
Sub-step D3: the data structure of Streaming Media expression and efficient index design is supported to guarantee network progressive streaming.
RTP is a kind of agreement for providing end-to-end real-time multimedia data and transmitting, and mating agreement is RTCP (Real-
Time Transfer Control Protocol), it is the transmission for monitoring real time data.They both guarantee the effect of transmission
Rate, and can according to need the quality for limitedly guaranteeing transmission.Minimum transmission unit each of next to transmission, will be by solution
The process that code, index are extracted, reconstructed.The size of resolution ratio and indication range that component is shown according to terminal, automatically determines terminal
The best information load capacity for showing screen carries out automatic Reconstruction on backstage to the information data of transmission.
So far, the present invention is finished towards the multistage tissue of mass remote sensing image and indexing means introduction.
Discribed process or method can be by including hardware (for example, circuit, special logic etc.), consolidating in the attached drawing of front
Part, software (for example, being carried on the software in non-transient computer-readable media), or both combined processing logic hold
Row.Although process or method are described according to the operation of certain sequences above, however, it is to be understood that described certain operation energy
It is executed with different order.In addition, can concurrently rather than be sequentially performed some operations.
It should be noted that in attached drawing or specification text, the implementation for not being painted or describing is affiliated technology
Form known to a person of ordinary skill in the art, is not described in detail in field.In addition, the above-mentioned definition to each element and method is simultaneously
It is not limited only to various specific structures, shape or the mode mentioned in embodiment, those of ordinary skill in the art can carry out letter to it
It singly changes or replaces, such as:
(1) bilinear interpolation calculates the point height and can also use other Interpolations in sub-step B1；
(2) compression algorithm in step D is compressed using JPEG2000；
(3) SVG for the W3C that step D is used carries out coding tissue, can also be replaced with other Vector Message carriers.
Particular embodiments described above has carried out further in detail the purpose of the present invention, technical scheme and beneficial effects
Describe in detail bright, it should be understood that the above is only a specific embodiment of the present invention, is not intended to restrict the invention, it is all
Within the spirit and principles in the present invention, any modification, equivalent substitution, improvement and etc. done should be included in protection of the invention
Within the scope of.
Claims (5)
1. a kind of multistage tissue and indexing means towards mass remote sensing image, which comprises the following steps:
Step A: wide cut image raster data is obtained, is pre-processed；
Step B: establishing the caching pyramid of disk space, and the two of the regular partition quaternary tree of sub-scene are added using scene pyramid
Grade index structure, connection ID and attribute database；
Step C: combined data organizational form realizes attribute data carrying, and blocking organization is completed multistage in HBase database
Storage and foundation index；And
Step D: it is provided by the storage pool of virtualization, scalability and sharing capability to realize to distribute according to need whenever and wherever possible
Source；
Step A includes:
Sub-step A1: piecemeal is classified to image raster data, and Coding and description is carried out to block data；
Sub-step A2: the block data after Coding and description is cut and is merged；
Sub-step A3: being read out loading processing to altitude data, and wherein altitude data is the data that image raster data is concentrated；
And
Sub-step A4: the visualization that tags of block data is realized；
Step B includes:
Sub-step B1: dynamic Pyramid Reconstruction is carried out to the deblocking coding loaded；
Sub-step B2: defining the derivative model of remote sensing metadata, distinguishes storing data attribute and data entity in database table ranks；
And
Sub-step B3: defining remotely-sensed data correlation model, indicates the relationship between object using regular quad-tree structure, establishes certainly
The pyramid data file of definition, data, attribute and index are associated together；
Wherein database list row is classified as the database table structure of storage image database；
Meta data category device is constructed in training set, when new remotely-sensed data introduces, existing remotely-sensed data classification is included into, if nothing
Method is included into any remotely-sensed data classification, will derive new metadata type by the method for calculating information gain, and defining S is
Metadata sample training collection defines c different class Ci(i=1 ... c), CI, sIt is the tuple-set of class in S, S is relative to c
The entropy of attribute is defined as:
Wherein, piIt is that tuple belongs to classification C in SiProbability,Represent the summation operation from 1 to c, log2It indicates with 2 to be bottom
Logarithm operation；
Information gain Gain (S, A) of the defined attribute A with respect to sample set S is defined as:
Where it is assumed that dividing the tuple in S by attribute A, A has v different value { a according to the observation of training data1, a2...
av, S is divided into v subset { S by attribute A1, S2... Sv, SjTuple in representative sample training set S, the value on attribute A are
aj, The summation operation from 1 to v is represented, | S | indicate the number of tuple in sample training collection S, | Sj| indicate set SjMiddle member
The number of group,Represent the weight of j-th of Attribute transposition；Select the attribute A of highest information gain as classification Split Attribute,
New classification state is formed on original sample training collection S；
Sub-step B3: defining remotely-sensed data correlation model, indicates the relationship between object using regular quad-tree structure, establishes certainly
The pyramid data file of definition, data, attribute and index are associated together；
Step C includes:
Sub-step C1: combined data organizational form realizes attribute data carrying, blocking organization；And
Sub-step C2: completing multistage storage in HBase database and establishes index；
For multi-scale image data blocking organization, the data-selected scheme stored in the form of file is provided, big file is utilized
Indexed mode storing data and the data server that both data are provided；Under the support of HBase database, according to sensing
Device, image resolution, imagery zone and time difference establish secondary index；Space based on object of reference-attribute interactive query leads to
It crosses object of reference physical model and realizes that corresponding coordinate position, coordinate range or the data that meet in entity ID range of attributes are fast
Quick checking is looked for.
2. multistage tissue according to claim 1 and indexing means, which is characterized in that sub-step B3 includes:
Secondary sub-step B3.1: the five-tuple of the incidence relation of remotely-sensed data is defined；
Secondary sub-step B3.2: data correlation tissue model is established；And
Sub-step B3.3: by Bayesian learning, the update of association tissue model is completed.
3. multistage tissue according to claim 2 and indexing means, which is characterized in that the five-tuple is following formula table
Show:
S=< ARt, ARs, ARo, TF, SF >
Wherein, ARt, ARs, ARo are one group of correlation rules, can be expressed as one group of logical deduction by reasoning rule, ARt is time dimension
Correlation rule, ARs are the correlation rules of Spatial Dimension, and ARo is the correlation rule based on observation object, and TF is temporal characteristics, SF
It is space characteristics.
4. multistage tissue according to claim 2 and indexing means, which is characterized in that the data correlation tissue model is
Contain the binary group of vertex set and side collection composition
Θ=<V, E>
Wherein,
V={ v1, v2... E={ e1, e2...
V is the data vertex set of non-empty, and each element representation meets the remotely-sensed data classification or remotely-sensed data of correlation rule, E
It is the set of incidence relation, the incidence relation that meets correlation rule of each element representation in some or certain several dimensions.
5. multistage tissue according to claim 1 and indexing means, which is characterized in that step D includes:
Sub-step D1: according to the bandwidth in request source, suitable spatial data compression and expression way are selected, by intelligentized
Adaptive strategy is allocated the transmission mode of data；
Sub-step D2: reasonably dividing transmission data progress by different level, in batches, whole transmission data is divided into several opposite
Independent basic unit compressed encoding and continuously transmit to basic unit, client carry out again basic unit reception,
Decoding and initial data synthesis；And
Sub-step D3: the data structure of Streaming Media expression and efficient index design is supported to guarantee network progressive streaming.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710092425.3A CN106909644B (en) | 2017-02-21 | 2017-02-21 | A kind of multistage tissue and indexing means towards mass remote sensing image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710092425.3A CN106909644B (en) | 2017-02-21 | 2017-02-21 | A kind of multistage tissue and indexing means towards mass remote sensing image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106909644A CN106909644A (en) | 2017-06-30 |
| CN106909644B true CN106909644B (en) | 2019-03-26 |
Family
ID=59207895
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710092425.3A Active CN106909644B (en) | 2017-02-21 | 2017-02-21 | A kind of multistage tissue and indexing means towards mass remote sensing image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106909644B (en) |
Families Citing this family (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107515952B (en) * | 2017-09-21 | 2023-10-27 | 北京星闪世图科技有限公司 | Method and system for point cloud data storage, parallel computing and real-time retrieval |
| CN107861983A (en) * | 2017-09-30 | 2018-03-30 | 中国电子科技集团公司第三十二研究所 | Remote sensing image storage system for high-speed remote sensing image processing |
| CN108090865B (en) * | 2017-12-15 | 2021-06-15 | 武汉大学 | Optical satellite remote sensing image on-orbit real-time streaming processing method and system |
| CN108510441B (en) * | 2018-03-19 | 2022-03-08 | 东南大学 | Image multi-resolution display method based on staggered grids |
| CN108959352A (en) * | 2018-04-27 | 2018-12-07 | 北京天机数测数据科技有限公司 | Time-space data analysis platform and processing method based on time and Spatial Data Model |
| CN108829836A (en) * | 2018-06-19 | 2018-11-16 | 长光卫星技术有限公司 | Isomery remote sensing big data application platform |
| CN109344188B (en) * | 2018-09-11 | 2022-10-04 | 北京航空航天大学 | Artificial intelligence statistical method for remote sensing satellite efficiency |
| CN109165307B (en) * | 2018-09-19 | 2021-02-02 | 腾讯科技（深圳）有限公司 | Feature retrieval method, device and storage medium |
| CN110968555B (en) * | 2018-09-30 | 2023-07-04 | 北京国双科技有限公司 | Dimension data processing method and device |
| CN109710715B (en) * | 2018-12-24 | 2021-04-13 | 南京泛在地理信息产业研究院有限公司 | Geometric algebraic coding and representation method of remote sensing image metadata |
| CN109783665B (en) * | 2018-12-29 | 2022-10-14 | 武汉大学 | Design method for realizing Hbase database remote sensing big data storage model based on Google S2 |
| CN110278163B (en) * | 2019-01-25 | 2022-11-15 | 中国航空无线电电子研究所 | Method for airborne information processing and organization transmission of unmanned aerial vehicle |
| CN111061729B (en) * | 2019-11-29 | 2021-04-16 | 武汉大学 | Granularity-considered vector tile organization method |
| CN111782742B (en) * | 2020-06-06 | 2021-01-26 | 中国科学院电子学研究所苏州研究院 | Large-scale geographic space data oriented storage and retrieval method and system thereof |
| CN111898001B (en) * | 2020-06-24 | 2023-08-08 | 四川大学 | Count matching method based on attribute filtering degree |
| CN112380367B (en) * | 2020-10-27 | 2023-01-10 | 中南大学 | Entropy-based remote sensing image data screening method |
| CN112817733B (en) * | 2021-02-26 | 2023-12-12 | 北京控制工程研究所 | Efficient satellite autonomous task planning task pool design method and system |
| CN112966134B (en) * | 2021-03-26 | 2022-06-28 | 北京亿耘科技有限公司 | Remote sensing image data processing method, system and storage medium |
| CN113094527B (en) * | 2021-04-25 | 2022-09-27 | 华中师范大学 | Generalized cloud control space-time spectrum big database and remote sensing image processing method |
| CN113223042B (en) * | 2021-05-19 | 2021-11-05 | 自然资源部国土卫星遥感应用中心 | Intelligent acquisition method and equipment for remote sensing image deep learning sample |
| CN115100541B (en) * | 2022-07-21 | 2023-06-06 | 米脂县宇宝北斗农业发展有限公司 | Satellite remote sensing data processing method, system and cloud platform |
| CN114969402B (en) * | 2022-07-25 | 2022-11-08 | 广州海普网络科技有限公司 | Vector geographic information acquisition method based on remote sensing image |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103595791A (en) * | 2013-11-14 | 2014-02-19 | 中国科学院深圳先进技术研究院 | Cloud accessing method for mass remote sensing data |
| CN105653694A (en) * | 2015-12-30 | 2016-06-08 | 北京汇通金财信息科技有限公司 | Storage and access method for Beidou message information |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9047847B2 (en) * | 2013-02-05 | 2015-06-02 | Facebook, Inc. | Displaying clusters of media items on a map using representative media items |
-
2017
- 2017-02-21 CN CN201710092425.3A patent/CN106909644B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103595791A (en) * | 2013-11-14 | 2014-02-19 | 中国科学院深圳先进技术研究院 | Cloud accessing method for mass remote sensing data |
| CN105653694A (en) * | 2015-12-30 | 2016-06-08 | 北京汇通金财信息科技有限公司 | Storage and access method for Beidou message information |
Non-Patent Citations (3)
| Title |
|---|
| 三维GIS中大规模场景数据获取、组织及调度方法的研究与实现;翟巍;《中国博士学位论文全文数据库》;20030315;第22页到第23页第3.2节，第37页第4.4节，第40页第4.5.1节，第45页第4.6节，第54页第5.5.2节，第65页到第67页第6.2节,附图6.1 * |
| 海量遥感影像数据组织模型研究;李峰等;《高分辨率对地观测学术年会》;20140813;第10页到第13页第4.2节 * |
| 翟巍.三维GIS中大规模场景数据获取、组织及调度方法的研究与实现.《中国博士学位论文全文数据库》.2003,第22页到第23页第3.2节，第45页第4.6节，第65页到第67页第6.2节,附图6.1. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106909644A (en) | 2017-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106909644B (en) | A kind of multistage tissue and indexing means towards mass remote sensing image | |
| CN106709067B (en) | Multisource heterogeneous space data circulation method based on Oracle database | |
| CN104820714B (en) | Magnanimity tile small documents memory management method based on hadoop | |
| US10372705B2 (en) | Parallel querying of adjustable resolution geospatial database | |
| CN104657436B (en) | Static tile pyramid parallel constructing method based on MapReduce | |
| Nishimura et al. | -HBase: design and implementation of an elastic data infrastructure for cloud-scale location services | |
| CN105786942B (en) | A kind of geography information storage system based on cloud platform | |
| CN110599490B (en) | Remote sensing image data storage method and system | |
| Nishimura et al. | MD-HBase: A scalable multi-dimensional data infrastructure for location aware services | |
| CN108446293A (en) | A method of based on urban multi-source isomeric data structure city portrait | |
| US10262392B2 (en) | Distributed and parallelized visualization framework | |
| CN109977192A (en) | The quick loading method of unmanned plane tile map, system, equipment and storage medium | |
| CN102509022B (en) | Method for quickly constructing raster database facing to Virtual Earth | |
| Mostak | An overview of MapD (massively parallel database) | |
| CN111052105A (en) | Scalable spatio-temporal density data fusion | |
| CN105608222A (en) | Rapid building method of tile pyramid for large-scale raster data set | |
| CN115238015A (en) | Space-time big data platform based on micro-service | |
| Guo et al. | A spatially adaptive decomposition approach for parallel vector data visualization of polylines and polygons | |
| CN104699826B (en) | A kind of the pyramid laminar storage method and Spatial Database Systems of image data | |
| Yu et al. | A new algorithm based on Region Partitioning for Filtering candidate viewpoints of a multiple viewshed | |
| Gao et al. | A multi-source spatio-temporal data cube for large-scale geospatial analysis | |
| Jing et al. | Improved U-Net model for remote sensing image classification method based on distributed storage | |
| Warushavithana et al. | A transfer learning scheme for time series forecasting using facebook prophet | |
| Weng et al. | Efficient visualization techniques for high resolution remotely sensed data in a network environment | |
| Xu et al. | GeoMapViz: a framework for distributed management and geospatial data visualization based on massive spatiotemporal data streams |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20201222 Address after: 100190 No. 19 West North Fourth Ring Road, Haidian District, Beijing Patentee after: Research Institute of aerospace information innovation, Chinese Academy of Sciences Address before: 100190 No. 19 West North Fourth Ring Road, Haidian District, Beijing Patentee before: Institute of Electronics, Chinese Academy of Sciences Effective date of registration: 20201222 Address after: 250101 No.9, Kuangyuan Road, Gongye North Road, Wangsheren street, Licheng District, Jinan City, Shandong Province Patentee after: Jigang Defense Technology Co.,Ltd. Address before: 100190 No. 19 West North Fourth Ring Road, Haidian District, Beijing Patentee before: Research Institute of aerospace information innovation, Chinese Academy of Sciences |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A multi-level organization and indexing method for massive remote sensing images Effective date of registration: 20230331 Granted publication date: 20190326 Pledgee: Ji'nan rural commercial bank Limited by Share Ltd. high tech branch Pledgor: Jigang Defense Technology Co.,Ltd. Registration number: Y2023980036938 |