CN116049482B - Remote sensing image overall planning method based on time-space domain - Google Patents
Remote sensing image overall planning method based on time-space domain Download PDFInfo
- Publication number
- CN116049482B CN116049482B CN202310331522.9A CN202310331522A CN116049482B CN 116049482 B CN116049482 B CN 116049482B CN 202310331522 A CN202310331522 A CN 202310331522A CN 116049482 B CN116049482 B CN 116049482B
- Authority
- CN
- China
- Prior art keywords
- image
- generated
- repaired
- grid
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000012216 screening Methods 0.000 claims abstract description 10
- 230000006978 adaptation Effects 0.000 claims description 22
- 239000011159 matrix material Substances 0.000 claims description 18
- 230000004927 fusion Effects 0.000 claims description 12
- 238000005070 sampling Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000004888 barrier function Effects 0.000 abstract 1
- 238000004364 calculation method Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000007726 management method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005336 cracking Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000000452 restraining effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/73—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/7867—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Library & Information Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a remote sensing image overall planning method based on a time-space domain, which relates to the technical field of remote sensing, and comprises the following steps: s1, acquiring a target area and a remote sensing image library; s2, acquiring a user query condition, and screening based on the user query condition to obtain a first image set; s3, taking the first image set as a first overall result, recommending the first overall result to a user if the first overall result meets the judging condition, and turning to the step S4 if the first overall result does not meet the judging condition; s4, determining an area to be filled in the target area based on the first overall result, filling the area to be filled in through a first filling strategy to obtain a second image set, taking the union set of the first image set and the second image set as a second overall result, and recommending the second overall result to a user. According to the time-space domain information of the region to be filled, the filling scheme of the uncovered region is automatically provided, so that a nonprofessional user is helped to break a remote sensing knowledge barrier, and an overall result which meets the user expectation is obtained.
Description
Technical Field
The invention relates to the technical field of remote sensing, in particular to a remote sensing image overall planning method based on a time-space domain.
Background
In the remote sensing image overall management system, image overall is one of main functions, and the function can achieve overall optimal recommendation through combination of partial images. The remote sensing image is a commodity with time-space domain characteristics, and is different from the general commodity in nature, so that the special time-space domain characteristics of the remote sensing image should be considered when the image is integrated, the space domain characteristics of the remote sensing image are represented by the fact that each image corresponds to an actual geographic area, and the time domain characteristics of the remote sensing image are represented by the fact that each image corresponds to an actual time node.
And obtaining a group of images according to the search condition of the user, if the group of images can completely cover the area, completing the query, and if the group of images can not completely cover the area, failing the query. In the face of query failure, the traditional solution is that a user reasonably modifies the original search condition to obtain an effective coverage remote sensing image, wherein the effective coverage refers to that the remote sensing image searched by the modified search condition can meet the requirement of filling an uncovered area in space and the requirement similar to the time phase of an image group searched by the user in time, namely the remote sensing image searched again can meet the requirement of a time-space domain. In order to enable the modified search image to meet the time-space domain requirement, the traditional solution is that a professional user who needs to master professional knowledge is required to finish when the original search condition is modified, so that the modified search condition can be ensured to search the remote sensing image meeting the time-space domain requirement.
At present, along with the richness and popularization of remote sensing image data, remote sensing image applications gradually move from serving professional users to mass applications. For non-professional users without rich industry knowledge, the modification direction of the original search condition cannot be mastered, and when the query failure problem is faced, the search condition cannot be accurately modified to meet the time-space domain requirement, so that an intelligent overall management method is needed, the non-professional users are helped to automatically acquire remote sensing images meeting the time-space domain requirement when the query failure problem is faced, and the uncovered area is filled, so that the space integrity and the time consistency of an overall result are ensured.
Disclosure of Invention
The invention provides a remote sensing image overall method based on a time-space domain, which aims at solving the problem that a user cannot obtain a group of remote sensing images meeting the time-space domain requirement according to query conditions.
The invention provides a remote sensing image overall planning method based on a time-space domain, which comprises the following steps:
s1, acquiring a target area and a remote sensing image library, wherein the remote sensing image library comprises remote sensing images;
s2, acquiring a user query condition, and performing image query on a remote sensing image library based on the user query condition to obtain a first image set;
s3, taking the first image set as a first overall result, recommending the first overall result to a user if the first overall result meets the judging condition, and turning to the step S4 if the first overall result does not meet the judging condition;
s4, determining an area to be filled in the target area based on the first overall result, filling the area to be filled in through a first filling strategy to obtain a second image set, taking the union set of the first image set and the second image set as a second overall result, and recommending the second overall result to a user.
Preferably, the method further comprises:
determining a region to be generated in the target region based on the second overall result;
generating images of the area to be generated through a second filling strategy to obtain a generated image set;
and recommending the third overall result to the user by taking the union of the first image set, the second image set and the generated image set as the third overall result.
Specifically, the first padding policy includes:
expanding the query condition of the user through an entry expanding method to obtain an expanded query condition, wherein the entry expanding method comprises synonymous conversion and manual replacement;
inquiring the remote sensing image library based on the outward expansion inquiry condition to obtain an outward expansion image set;
taking the first image set as a reference object, taking the external expansion image set as a target object, and calculating the time-space domain adaptation degree between the target object and the reference object through the following formula:
wherein r is a reference object, d is a target object, h 1 、h 2 Respectively the first parameter and the second parameter are fixed values, mu t Mu, as time weighting parameter t ＜1，U r 、V r For two chrominance components of a reference object, U d 、V d For two chrominance components of the target object, x r 、y r X is the abscissa and ordinate, x, respectively, of the projected coordinate system of the centroid of the reference object d 、y d Respectively the abscissa and the ordinate of the projection coordinate system of the mass center of the target object;
wherein ,for convolution operation, Y r For the luminance component of the reference object, Y d For the luminance component of the target object, F x Three-dimensional matrix, F, being the width of the spatial dimension y Three-dimensional matrix, F, being the height of the spatial dimension t A three-dimensional matrix which is a time dimension;
and arranging the outward expansion image sets according to the sequence of the time-space domain adaptation degree from large to small, and screening out a preset number of outward expansion images according to the positive sequence to serve as a second image set.
Specifically, generating the image of the region to be generated through the second filling strategy to obtain a generated image set, including:
sampling a region to be generated by using a moving window with the window length of W and the step length of S to obtain N grids to be repaired, and adding a grid set to be repaired, wherein S is more than 0 and less than W;
generating grids to be repaired by a space domain-based generation method to obtain N generated grids which are in one-to-one correspondence with the grids to be repaired, and adding the N generated grids into a generated grid set;
and carrying out connected domain marking on the generated grid set through a connected domain marking algorithm to obtain K connected domains, wherein K is smaller than N, taking each 1 connected domain as 1 generated image to obtain K generated images, and adding the K generated images into the generated image set, wherein the connected domain marking algorithm comprises a two-step method and a seed filling method.
Specifically, the spatial domain-based generation method includes:
step one, taking the remote sensing images except for the second overall result in the remote sensing image library as a third image set;
step two, sampling a third image set by using a moving window with the window length of W and the step length of W to obtain a candidate time sequence grid;
randomly extracting 1 grid to be repaired from the grid set to be repaired to serve as a current grid to be repaired;
establishing a buffer area around the current grid to be repaired based on the width i, and taking the buffer area as an auxiliary reference object, wherein the buffer area comprises a sub-area of a first image and a sub-area of a second image;
step five, taking the current grid to be repaired as a reference object, taking the candidate time sequence grid as a target object, and calculating the time-space domain adaptation degree between the target object and the reference object through the following formula:
wherein r is a reference object, ar is an auxiliary reference object, d is a target object, h 1 、h 2 Respectively the first parameter and the second parameter are fixed values, mu s Mu, which is the space weight-adjusting parameter s ＞1，U ar 、V ar For two chrominance components of the secondary reference object, U d 、V d For two chrominance components of the target object, x r 、y r X is the abscissa and ordinate, x, respectively, of the projected coordinate system of the centroid of the reference object d 、y d Respectively the abscissa and the ordinate of the projection coordinate system of the mass center of the target object;
wherein ,for convolution operation, Y ar For the luminance component of the secondary reference object, Y d A luminance component that is a target object; f (F) x Three-dimensional matrix, F, being the width of the spatial dimension y Three-dimensional matrix, F, being the height of the spatial dimension t A three-dimensional matrix which is a time dimension;
step six, arranging target objects according to the sequence from the big time-space domain adaptation degree to the small time-space domain adaptation degree, and screening 3 target objects according to the positive sequence to serve as 3 time sequence grids of the current grid to be repaired;
inputting 3 time sequence grids of the current grids to be repaired into a multi-phase generator for learning, outputting 1 grid generated by the current grids to be repaired, adding a generated grid set, and screening the current grids to be repaired from the grid set to be repaired;
and step eight, iteratively executing the steps three to seven until the grid set to be repaired is an empty set, and outputting the current generated grid set.
Specifically, the multi-temporal phase generator includes a 3-layer combination generator, the 1 st-layer combination generator includes 3 base generators, the 2 nd-layer combination generator includes 3 base generators, the 3 rd-layer combination generator includes 1 base generator, and the base generator includes one encoder and one decoder.
Specifically, the seventh step includes:
inputting 3 time sequence grids of the current grid to be repaired into 3 basic generators of a 1 st layer combination generator one by one to obtain 3 first generation data;
combining the 3 first generation data pairwise to obtain 3 groups of first generation data pairwise, and combining the first generation data pairwise based on channel dimensions to obtain 3 first fusion data;
inputting the 3 first fusion data into 3 basic generators of a 2 nd layer combination generator one by one to obtain 3 second generation data;
and merging the 3 second generation data based on the channel dimension to obtain 1 second fusion data, and inputting the second fusion data into 1 basic generator of the 3 rd layer combination generator to obtain 1 generation grid of the current grid to be repaired.
Specifically, the determination condition is that the union of the first overall results satisfies that the spatial coverage rate of the target area reaches 100%.
Specifically, determining the region to be padded in the target region based on the first overall result includes: and taking the subarea which is not covered by the union of the first overall results in the target area as an area to be filled.
Specifically, determining the region to be generated in the target region based on the second overall result includes: and taking the subarea which is not covered by the union of the second overall result in the target area as an area to be generated.
The invention has the beneficial effects that:
1. according to the method, the remote sensing image is constrained from two dimensions of a space domain and a time domain through calculation of the time-space domain adaptation degree, so that the constraint dimension is increased, the remote sensing image is further accurately constrained, and the constraint result meets the space domain requirement and the time domain requirement;
2. the method expands the image query range on the basis of restraining the time-space domain range of the remote sensing image through the first filling strategy and the restraint based on the time-space domain adaptation degree, solves the problem that a non-professional user cannot reasonably modify the original retrieval condition, replaces expert guidance, and automatically gives the retrieval result;
3. the second filling strategy is used for automatically generating the areas which cannot be covered by the first image set and the second image set to obtain the generated image set.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a remote sensing image overall planning method based on a time-space domain.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which are derived by a person skilled in the art based on the embodiments of the invention, fall within the scope of protection of the invention.
Example 1
Referring to fig. 1, the invention provides a remote sensing image overall method based on a time-space domain, which comprises the following steps:
s1, acquiring a target area and a remote sensing image library, wherein the remote sensing image library comprises remote sensing images;
s2, acquiring a user query condition, and performing image query on a remote sensing image library based on the user query condition to obtain a first image set;
s3, taking the first image set as a first overall result, recommending the first overall result to a user if the first overall result meets the judging condition, and turning to the step S4 if the first overall result does not meet the judging condition;
s4, determining an area to be filled in the target area based on the first overall result, filling the area to be filled in through a first filling strategy to obtain a second image set, taking the union set of the first image set and the second image set as a second overall result, and recommending the second overall result to a user.
In specific implementation, taking a subarea which is not covered by the union of the first overall results in the target area as an area to be filled;
in this embodiment, the user query condition includes a spatial location, an acquisition time, a resolution, and a sensor type. And the judging condition is that the union of the first overall results meets the spatial coverage rate of the target area reaching 100%.
When the remote sensing image library is rich, a group of remote sensing images which meet the expectations of the user are obtained only through the query conditions of the user, namely, a first overall result. However, in reality, when the target area of the user is too large or the acquisition time interval is too small, the remote sensing image library cannot meet all the query conditions of the user, so that overall planning fails. And filling the target area which cannot be covered by the first overall result, namely the area to be filled, according to the first filling strategy.
In a specific implementation, the first padding policy includes;
expanding the query condition of the user through an entry expanding method to obtain an expanded query condition, wherein the entry expanding method comprises synonymous conversion and manual replacement;
inquiring the remote sensing image library based on the outward expansion inquiry condition to obtain an outward expansion image set;
taking the first image as a reference object, taking the expanded image as a target object, and calculating the time-space domain adaptation degree between the target object and the reference object through the following formula:
wherein r is a reference object, d is a target object, h 1 、h 2 Respectively the first parameter and the second parameter are fixed values, mu t Mu, as time weighting parameter t ＜1，U r 、V r For two chrominance components of a reference object, U d 、V d For two chrominance components of the target object, x r 、y r X is the abscissa and ordinate, x, respectively, of the projected coordinate system of the centroid of the reference object d 、y d Respectively the abscissa and the ordinate of the projection coordinate system of the mass center of the target object;
wherein ,for convolution operation, Y r For the luminance component of the reference object, Y d For the luminance component of the target object, F x Three-dimensional matrix, F, being the width of the spatial dimension y Three-dimensional matrix, F, being the height of the spatial dimension t A three-dimensional matrix which is a time dimension;
and arranging the outward expansion image sets according to the sequence of the time-space domain adaptation degree from large to small, and screening out a preset number of outward expansion images according to the positive sequence to serve as a second image set.
In a specific implementation, the preset number is typically set to 10% of the total number of the expanded images.
In a specific implementation, the first filling strategy performs the image retrieval based on the expansion query condition by performing the expansion on the query condition, and has the advantages that the function of automatically expanding the image set is realized without manual assistance of an expert, but the defect is obvious, the expansion direction is random and cannot be determined, so that the expansion image set cannot be ensured to meet the requirement of image overall arrangement in space and time, and the expansion image set needs to be restrained. Therefore, the first filling strategy carries out time-space domain constraint on the outward-expansion image set through the time-space domain adaptation degree.
The method comprises the following steps of regarding the set thought of time adjustment parameters in a calculation formula of time-space domain adaptation degree: firstly, determining the effect of the calculation of the time-space domain adaptation degree in a first filling strategy, secondly, determining the influence factors according to the effect, and finally, determining the positions of the influence factors according to the effect.
According to the thought, setting time weight adjustment parameters: in the first filling strategy, the function of a calculation formula of the time-space domain adaptation degree is to restrict the outward expansion image set, select a second image set meeting the space domain requirement from the outward expansion image set, and if the outward expansion image set is lostInter-constraint, the first filling policy may cause "invalid filling", that is, the obtained second image set cannot increase coverage of the first uncovered area, and the first filling policy cannot complete the mission of "filling"; if the time constraint is lost to the expanded image set, the first filling policy may cause "violent filling", that is, the obtained second image set fills the first uncovered area, but due to the overlarge time phase difference between the second image set and the first image set, the time consistency of the overall image results may be achieved, for example, the acquisition time of the first image set (that is, the image is queried according to the acquisition time input by the user) is in winter, the feature of the first image set is all displayed as obvious winter features, and due to the lack of the time constraint to the expanded image set, the feature of the second image set may have obvious other season features, so that the time consistency of the overall results cannot be ensured. In summary, the influencing factors are a time domain and a time domain, under the two assumptions of losing the space constraint and losing the time constraint, it can be seen that losing the time constraint seriously reduces the quality of the image overall arrangement, but losing the space constraint can not increase the number of the image overall arrangement, so that the influencing factors of the time domain are set as dominant factors based on the principle of "rather than misuse", and the specific method is to set the time adjustment parameters as mu t And < 1, the weight of the time-related parameter is improved.
Example two
The invention provides a remote sensing image overall planning method based on a time-space domain, which further comprises the following steps:
determining a region to be generated in the target region based on the second overall result;
generating images of the area to be generated through a second filling strategy to obtain a generated image set;
and recommending the third overall result to the user by taking the union of the first image set, the second image set and the generated image set as the third overall result.
In this embodiment, a sub-region of the target region that is not covered by the union of the second overall result is taken as the region to be generated. And when the union of the second overall results does not completely cover the target area, taking the uncovered area in the target area as the area to be generated.
In a specific implementation, when the first filling policy cannot complete the 'filling' mission, it proves that the images close to the first image set when the remote sensing image set is short, then the images with larger time phase distances from the first image set in the remote sensing image set are used as third images, as the time phase requirements are known to be not up to standard, the third images cannot be directly used for 'filling', but the third images can store abundant ground object information, multiple time phase image generation can be performed by utilizing multiple third images with the same spatial domain information, and the second filling policy completes image generation by a multiple time phase generator based on information complementation among the multiple third images, so that a generated image set capable of 'filling' a region to be generated is obtained.
Specifically, generating the image of the region to be generated through the second filling strategy to obtain a generated image set, including:
sampling a region to be generated by using a moving window with the window length of W and the step length of S to obtain N grids to be repaired, and adding a grid set to be repaired, wherein S is more than 0 and less than W;
generating grids to be repaired based on a space domain generating method to obtain N generating grids which are in one-to-one correspondence with the grids to be repaired, and adding the N generating grids into a generating grid set;
and carrying out connected domain marking on the generated grid set through a connected domain marking algorithm to obtain K connected domains, wherein K is smaller than N, taking each 1 connected domain as 1 generated image to obtain K generated images, and adding the K generated images into the generated image set, wherein the connected domain marking algorithm comprises a two-step method and a seed filling method.
In the concrete implementation, when S=2/3W, the sampling effect is better, and when S is more than 2/3W, when the connected domain marks of the generated grids are carried out, the boundary information difference between the generated grids in the connected domain is overlarge, and the transition is hard, so that the whole connected domain has stronger cracking sense; and when S is less than 2/3W, the resource waste is caused by excessive sampling.
Specifically, the spatial domain-based generation method includes:
step one, taking the remote sensing images except for the second overall result in the remote sensing image library as a third image set;
step two, sampling a third image set by using a moving window with the window length of W and the step length of W to obtain a candidate time sequence grid;
randomly extracting 1 grid to be repaired from the grid set to be repaired to serve as a current grid to be repaired;
step four, a buffer area is established around the current grid to be repaired based on the width i, and the buffer area is used as an auxiliary reference object; in this embodiment, the buffer area includes a sub-area of the first image and a sub-area of the second image.
Step five, taking the grid to be repaired as a reference object, taking the candidate time sequence grid as a target object, and calculating the time-space domain adaptation degree between the target object and the reference object through the following formula:
wherein r is a reference object, ar is an auxiliary reference object, d is a target object, h 1 、h 2 Respectively the first parameter and the second parameter are fixed values, mu s Mu, which is the space weight-adjusting parameter s ＞1，U ar 、V ar For two chrominance components of the secondary reference object, U d 、V d For two chrominance components of the target object, x r 、y r X is the abscissa and ordinate, x, respectively, of the projected coordinate system of the centroid of the reference object d 、y d Respectively the abscissa and the ordinate of the projection coordinate system of the mass center of the target object;
wherein ,for convolution operation, Y ar For the luminance component of the secondary reference object, Y d A luminance component that is a target object; f (F) x Three-dimensional matrix, F, being the width of the spatial dimension y Three-dimensional matrix, F, being the height of the spatial dimension t A three-dimensional matrix which is a time dimension;
step six, arranging target objects according to the sequence from the big time-space domain adaptation degree to the small time-space domain adaptation degree, and screening 3 target objects according to the positive sequence to serve as 3 time sequence grids of the current grid to be repaired;
inputting 3 time sequence grids of the current grids to be repaired into a multi-phase generator for learning, outputting 1 grid generated by the current grids to be repaired, adding a generated grid set, and screening the current grids to be repaired from the grid set to be repaired;
and step eight, iteratively executing the steps three to seven until the grid set to be repaired is an empty set, and outputting the current generated grid set.
In this embodiment, since the area to be generated is already cut into the grids to be repaired with the window length of W by moving the window, in order to realize "one-to-one" filling of the grids to be repaired, the present invention uses the window with the window length of W to cut the third image into the candidate time sequence grids with the same size as the grids to be repaired, screens from the candidate time sequence grids through the steps three to six to obtain 3 time sequence grids meeting the time-space domain requirement, and uses the 3 time sequence grids to generate multiple time phases through the step seven to obtain 1 generated grid, and the generated grid can well fill the grids to be repaired.
In a specific implementation, the third image does not meet the requirement of time or space, so that the third image cannot be directly used as an overall result, but the third image contains a lot of ground object information, and can be generated based on the third image. However, since the time phase information of the third image is greatly different from the existing overall result (the second overall result), the generated data obtained by the one-to-one generation is difficult to be kept consistent with the existing overall result visually, and since the third image is easy to have large noise, the generated data obtained by the one-to-one generation is lack of objectivity, and the ground object information of the real region to be generated is difficult to restore, therefore, the image generation by using a plurality of third images is considered, the noise and the time phase deviation caused by the single image are neutralized, the objectivity and the uniformity of the generated data are ensured, and the relation of many-to-one is that the plurality of time sequence data are generated into one generated data, wherein the generation of the plurality of time sequence data is required to be generated by a multi-time phase generator.
In a specific implementation, the multi-temporal phase generator includes a 3-layer combination generator, the 1 st-layer combination generator includes 3 base generators, the 2 nd-layer combination generator includes 3 base generators, the 3 rd-layer combination generator includes 1 base generator, the base generator includes an encoder and a decoder, and the seventh step includes:
inputting 3 time sequence grids of the current grid to be repaired into 3 basic generators of a 1 st layer combination generator one by one to obtain 3 first generation data;
combining the 3 first generation data pairwise to obtain 3 groups of first generation data pairwise, and combining the first generation data pairwise based on channel dimensions to obtain 3 first fusion data;
inputting the 3 first fusion data into 3 basic generators of a 2 nd layer combination generator one by one to obtain 3 second generation data;
and merging the 3 second generation data based on the channel dimension to obtain 1 second fusion data, and inputting the second fusion data into 1 basic generator of the 3 rd layer combination generator to obtain 1 generation grid of the current grid to be repaired.
It can be found from the first and second embodiments that the time-space domain adaptation degree calculation formulas in the first and second filling strategies are highly similar, and the difference μ between them t (μ s ) For parameter tuning, the tuning parameters may be used to adjust the specific gravity of the time and space domain, if desiredThe influence factors are dominant, and the weight adjustment parameter mu is selected in the (- ≡1) interval; if it is desired that the influence factor of the spatial domain is dominant, the tuning parameter μmay be selected to be a value within the (1, +) interval.
Claims (8)
1. A remote sensing image overall method based on a time-space domain is characterized by comprising the following steps:
s1, acquiring a target area and a remote sensing image library, wherein the remote sensing image library comprises remote sensing images;
s2, acquiring a user query condition, and performing image query on a remote sensing image library based on the user query condition to obtain a first image set;
s3, taking the first image set as a first overall result, recommending the first overall result to a user if the first overall result meets the judging condition, and turning to the step S4 if the first overall result does not meet the judging condition;
s4, determining an area to be filled in a target area based on the first overall result, filling the area to be filled through a first filling strategy to obtain a second image set, taking the union of the first image set and the second image set as a second overall result, and recommending the second overall result to a user;
the judging condition is that the union of the first overall results meets the spatial coverage rate of the target area reaching 100%;
the first padding policy includes: expanding the query condition of the user through an entry expanding method to obtain an expanded query condition, wherein the entry expanding method comprises synonymous conversion and manual replacement;
inquiring the remote sensing image library based on the outward expansion inquiry condition to obtain an outward expansion image set;
taking the first image set as a reference object, taking the external expansion image set as a target object, and calculating the time-space domain adaptation degree between the target object and the reference object through the following formula:
wherein r is a reference pairImage, d is the target object, h 1 、h 2 Respectively the first parameter and the second parameter are fixed values, mu t Mu, as time weighting parameter t ＜1，U r 、V r For two chrominance components of a reference object, U d 、V d For two chrominance components of the target object, x r 、y r X is the abscissa and ordinate, x, respectively, of the projected coordinate system of the centroid of the reference object d 、y d Respectively the abscissa and the ordinate of the projection coordinate system of the mass center of the target object;
wherein ,for convolution operation, Y r For the luminance component of the reference object, Y d For the luminance component of the target object, F x Three-dimensional matrix, F, being the width of the spatial dimension y Three-dimensional matrix, F, being the height of the spatial dimension t A three-dimensional matrix which is a time dimension;
and arranging the outward expansion image sets according to the sequence of the time-space domain adaptation degree from large to small, and screening out a preset number of outward expansion images according to the positive sequence to serve as a second image set.
2. The method according to claim 1, characterized in that the method further comprises:
determining a region to be generated in the target region based on the second overall result;
generating images of the area to be generated through a second filling strategy to obtain a generated image set;
and recommending the third overall result to the user by taking the union of the first image set, the second image set and the generated image set as the third overall result.
3. The method of claim 2, wherein generating the image of the region to be generated by the second padding policy to obtain the generated image set, comprising:
sampling a region to be generated by using a moving window with the window length of W and the step length of S to obtain N grids to be repaired, and adding a grid set to be repaired, wherein S is more than 0 and less than W;
generating grids to be repaired by a space domain-based generation method to obtain N generated grids which are in one-to-one correspondence with the grids to be repaired, and adding the N generated grids into a generated grid set;
and carrying out connected domain marking on the generated grid set through a connected domain marking algorithm to obtain K connected domains, wherein K is smaller than N, taking each 1 connected domain as 1 generated image to obtain K generated images, and adding the K generated images into the generated image set, wherein the connected domain marking algorithm comprises a two-step method and a seed filling method.
4. A method according to claim 3, wherein the spatial domain based generation method comprises:
step one, taking the remote sensing images except for the second overall result in the remote sensing image library as a third image set;
step two, sampling a third image set by using a moving window with the window length of W and the step length of W to obtain a candidate time sequence grid;
randomly extracting 1 grid to be repaired from the grid set to be repaired to serve as a current grid to be repaired;
establishing a buffer area around the current grid to be repaired based on the width i, and taking the buffer area as an auxiliary reference object, wherein the buffer area comprises a sub-area of a first image and a sub-area of a second image;
step five, taking the current grid to be repaired as a reference object, taking the candidate time sequence grid as a target object, and calculating the time-space domain adaptation degree between the target object and the reference object through the following formula:
wherein r is a reference object, ar is an auxiliary reference object, d is a target object, h 1 、h 2 Respectively the first parameter and the second parameter are fixed values, mu s Mu, which is the space weight-adjusting parameter s ＞1，U ar 、V ar For two chrominance components of the secondary reference object, U d 、V d For two chrominance components of the target object, x r 、y r X is the abscissa and ordinate, x, respectively, of the projected coordinate system of the centroid of the reference object d 、y d Respectively the abscissa and the ordinate of the projection coordinate system of the mass center of the target object;
wherein ,for convolution operation, Y ar For the luminance component of the secondary reference object, Y d A luminance component that is a target object; f (F) x Three-dimensional matrix, F, being the width of the spatial dimension y Three-dimensional matrix, F, being the height of the spatial dimension t A three-dimensional matrix which is a time dimension;
step six, arranging target objects according to the sequence from the big time-space domain adaptation degree to the small time-space domain adaptation degree, and screening 3 target objects according to the positive sequence to serve as 3 time sequence grids of the current grid to be repaired;
inputting 3 time sequence grids of the current grids to be repaired into a multi-phase generator for learning, outputting 1 grid generated by the current grids to be repaired, adding a generated grid set, and screening the current grids to be repaired from the grid set to be repaired;
and step eight, iteratively executing the steps three to seven until the grid set to be repaired is an empty set, and outputting the current generated grid set.
5. The method of claim 4, wherein the multi-temporal phase generator comprises a 3-layer combination generator, the 1 st-layer combination generator comprises 3 base generators, the 2 nd-layer combination generator comprises 3 base generators, the 3 rd-layer combination generator comprises 1 base generator, and the base generator comprises an encoder and a decoder.
6. The method of claim 5, wherein step seven comprises:
inputting 3 time sequence grids of the current grid to be repaired into 3 basic generators of a 1 st layer combination generator one by one to obtain 3 first generation data;
combining the 3 first generation data pairwise to obtain 3 groups of first generation data pairwise, and combining the first generation data pairwise based on channel dimensions to obtain 3 first fusion data;
inputting the 3 first fusion data into 3 basic generators of a 2 nd layer combination generator one by one to obtain 3 second generation data;
and merging the 3 second generation data based on the channel dimension to obtain 1 second fusion data, and inputting the second fusion data into 1 basic generator of the 3 rd layer combination generator to obtain 1 generation grid of the current grid to be repaired.
7. The method of claim 1, wherein determining the region to be padded in the target region based on the first overall result comprises: and taking the subarea which is not covered by the union of the first overall results in the target area as an area to be filled.
8. The method of claim 2, wherein determining the region to be generated in the target region based on the second orchestration result comprises: and taking the subarea which is not covered by the union of the second overall result in the target area as an area to be generated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310331522.9A CN116049482B (en) | 2023-03-31 | 2023-03-31 | Remote sensing image overall planning method based on time-space domain |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310331522.9A CN116049482B (en) | 2023-03-31 | 2023-03-31 | Remote sensing image overall planning method based on time-space domain |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116049482A CN116049482A (en) | 2023-05-02 |
| CN116049482B true CN116049482B (en) | 2023-10-20 |
Family
ID=86131600
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310331522.9A Active CN116049482B (en) | 2023-03-31 | 2023-03-31 | Remote sensing image overall planning method based on time-space domain |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116049482B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114896437A (en) * | 2022-07-14 | 2022-08-12 | 北京数慧时空信息技术有限公司 | Remote sensing image recommendation method based on available domain |
| CN115248876A (en) * | 2022-08-18 | 2022-10-28 | 北京数慧时空信息技术有限公司 | Remote sensing image overall planning recommendation method based on content understanding |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107563438B (en) * | 2017-08-31 | 2019-08-30 | 西南交通大学 | A kind of multi-modal Remote Sensing Images Matching Method and system of fast robust |
-
2023
- 2023-03-31 CN CN202310331522.9A patent/CN116049482B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114896437A (en) * | 2022-07-14 | 2022-08-12 | 北京数慧时空信息技术有限公司 | Remote sensing image recommendation method based on available domain |
| CN115248876A (en) * | 2022-08-18 | 2022-10-28 | 北京数慧时空信息技术有限公司 | Remote sensing image overall planning recommendation method based on content understanding |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116049482A (en) | 2023-05-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Parding et al. | GCMeval–An interactive tool for evaluation and selection of climate model ensembles | |
| Grün et al. | Photogrammetric reconstruction of the great Buddha of Bamiyan, Afghanistan | |
| CN103955891B (en) | Image restoration method based on block matching | |
| KR102225024B1 (en) | Apparatus and method for image inpainting | |
| CN109034454A (en) | Route method for digging, device, computer readable storage medium and computer equipment | |
| CN109242769B (en) | A kind of image processing method and device | |
| CN114119884A (en) | Building LOD1 model construction method based on high-score seven-satellite image | |
| CN110348352A (en) | Training method, terminal and storage medium for human face image age migration network | |
| CN116049482B (en) | Remote sensing image overall planning method based on time-space domain | |
| CN111149101B (en) | Target pattern searching method and computer readable storage medium | |
| CN116861540A (en) | Method for constructing personalized artificial intelligent assistant for shear wall layout design | |
| CN112446951B (en) | Three-dimensional reconstruction method, three-dimensional reconstruction device, electronic equipment and computer storage medium | |
| CN106296604B (en) | A kind of image repair method and device | |
| CN111026938A (en) | Space-time big data integration analysis method, device, equipment and storage medium | |
| CN106649776A (en) | Method of semi-automating comprehensive vector polygon | |
| CN108197613B (en) | Face detection optimization method based on deep convolution cascade network | |
| Gong et al. | Improvement of an oak canopy model extracted from digital photogrammetry | |
| CN112686803B (en) | Spatio-temporal super-resolution mapping based on consideration of point spread function effect | |
| CN113034368A (en) | Image super-resolution model training method and device, computer equipment and medium | |
| CN112215778A (en) | Image restoration method based on three-level region division | |
| KR20060062154A (en) | Building extraction apparatus and method using hough transformation | |
| CN113256513B (en) | Face beautifying method and system based on antagonistic neural network | |
| Marcheix et al. | A persistent naming system based on graph transformation rules | |
| KR102390005B1 (en) | Learning method, learning device for generating high-resolution weather data and testing method, testing device using the same | |
| JP4198316B2 (en) | Fractal Darwinian object generation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |