CN117556079B - Remote sensing image content retrieval method, remote sensing image content retrieval device, electronic equipment and medium - Google Patents
Remote sensing image content retrieval method, remote sensing image content retrieval device, electronic equipment and medium Download PDFInfo
- Publication number
- CN117556079B CN117556079B CN202410044899.0A CN202410044899A CN117556079B CN 117556079 B CN117556079 B CN 117556079B CN 202410044899 A CN202410044899 A CN 202410044899A CN 117556079 B CN117556079 B CN 117556079B
- Authority
- CN
- China
- Prior art keywords
- image
- remote sensing
- vector
- segmented
- scale
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 239000013598 vector Substances 0.000 claims abstract description 230
- 230000004927 fusion Effects 0.000 claims abstract description 28
- 238000001914 filtration Methods 0.000 claims abstract description 25
- 238000013508 migration Methods 0.000 claims abstract description 17
- 230000005012 migration Effects 0.000 claims abstract description 17
- 238000007781 pre-processing Methods 0.000 claims abstract description 15
- 238000012549 training Methods 0.000 claims description 19
- 238000003860 storage Methods 0.000 claims description 14
- 230000009467 reduction Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 238000005520 cutting process Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 12
- 238000012360 testing method Methods 0.000 description 8
- 238000000605 extraction Methods 0.000 description 7
- 238000005070 sampling Methods 0.000 description 7
- 238000012795 verification Methods 0.000 description 7
- 230000000903 blocking effect Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 101100261000 Caenorhabditis elegans top-3 gene Proteins 0.000 description 2
- 101100153591 Cricetulus griseus TOP1 gene Proteins 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/5846—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using extracted text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/096—Transfer learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/19007—Matching; Proximity measures
- G06V30/19093—Proximity measures, i.e. similarity or distance measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
- G06V30/1918—Fusion techniques, i.e. combining data from various sources, e.g. sensor fusion
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention provides a remote sensing image content retrieval method, a device, electronic equipment and a medium, which relate to the technical field of remote sensing image retrieval and comprise the following steps: performing migration learning on the multi-modal large model based on the remote sensing image description dataset to obtain a cross-modal large model; preprocessing the remote sensing large-scale image to obtain a segmented image; coding the segmented image based on the cross-modal large model to obtain an image feature vector; encoding the image content description text based on the cross-modal large model to obtain text feature vectors, and filtering the image feature vectors in the vector library based on the geographic range; and calculating cosine similarity of the text feature vector and the filtered image feature vector, acquiring a preset number of image feature vectors and metadata information based on the cosine similarity, and finally carrying out multi-scale spatial fusion on the segmented image to obtain a remote sensing image retrieval result. The invention improves the speed and the precision of remote sensing image content retrieval.
Description
Technical Field
The present invention relates to the field of remote sensing image retrieval technologies, and in particular, to a remote sensing image content retrieval method, device, electronic apparatus, and medium.
Background
The remote sensing image retrieval refers to a process of quickly and accurately finding out related images from large-scale remote sensing images according to query conditions by using computer vision and image processing technology. The technology can acquire the remote sensing image matched with a specific area, a specific target or a specific characteristic, thereby realizing the efficient utilization of the remote sensing image data, providing powerful data support for the fields of environment monitoring, resource management, urban planning, agricultural development, military national defense and the like, and being an indispensable important link in remote sensing application.
Along with the rapid development of space earth observation technology in the global scope, the remote sensing big data age has come, and how to rapidly and accurately extract interesting images from massive remote sensing images becomes a problem to be solved.
Disclosure of Invention
Accordingly, the present invention is directed to a remote sensing image content retrieval method, apparatus, electronic device and medium, so as to improve the speed and accuracy of remote sensing image content retrieval.
In order to achieve the above object, the technical scheme adopted by the embodiment of the invention is as follows:
in a first aspect, an embodiment of the present invention provides a remote sensing image content retrieval method, including:
Performing migration learning on the multi-modal large model based on the remote sensing image description dataset to obtain a cross-modal large model;
acquiring a remote sensing large-scale image, and preprocessing the remote sensing large-scale image to obtain a block image and metadata information of the block image;
coding the segmented image based on the cross-modal large model to obtain an image feature vector, and storing the image feature vector of the segmented image and corresponding metadata information into a vector library;
acquiring an image content description text and a geographic range to be searched, encoding the image content description text based on a cross-modal large model to obtain text feature vectors, and filtering the image feature vectors in a vector library based on the geographic range to obtain filtered image feature vectors;
calculating cosine similarity of the text feature vector and the filtered image feature vector, and acquiring a preset number of image feature vectors and metadata information based on the cosine similarity;
and carrying out multi-scale space fusion on the segmented images corresponding to the preset number of image feature vectors and the metadata information to obtain remote sensing image retrieval results.
In one embodiment, performing migration learning on the multi-modal large model based on the remote sensing image description dataset to obtain a cross-modal large model includes:
Acquiring original pre-training weights of the multi-mode large model; inputting the image data and the text data of the remote sensing image description dataset into a multi-mode large model for encoding to obtain an image vector and a text vector;
and performing iterative optimization on the original pre-training weight of the multi-modal large model by adopting a contrast loss function based on the distance between the matched image vector and the text vector and the distance between the unmatched image vector and the text vector to obtain the cross-modal large model.
In one embodiment, preprocessing a remote sensing large-scale image to obtain a segmented image and metadata information of the segmented image includes:
performing image bit number conversion on the remote sensing large-scale image by adopting a percentage cut-off method, and performing noise reduction operation on the remote sensing large-scale image;
performing downsampling operation on the remote sensing large-scale image after noise reduction to obtain a multi-scale image;
and carrying out block cutting on the multi-scale image to obtain a block image, and acquiring metadata information of the block image.
In one embodiment, encoding a segmented image based on a cross-modal large model to obtain an image feature vector, and storing the image feature vector of the segmented image and corresponding metadata information into a vector library, including:
Inputting the segmented image into a cross-modal large model for feature coding to obtain an image feature vector of the segmented image, and storing the image feature vector into a vector library;
metadata information of the segmented image is obtained, the metadata information of the segmented image is stored in a vector library, and a spatial index is established based on longitude and latitude coordinates of a central point of the metadata information.
In one embodiment, filtering the image feature vector in the vector library based on the geographic range to obtain a filtered image feature vector includes:
and taking the geographic range as a filtering condition, and filtering the image feature vector of the longitude and latitude coordinates of the central point in the geographic range based on the spatial index in a vector library to obtain a filtered image feature vector.
In one embodiment, acquiring a predetermined number of image feature vectors and metadata information based on cosine similarity includes:
and selecting a preset number of image feature vectors and corresponding metadata information from large to small according to the ordering result of the cosine similarity descending order.
In one embodiment, performing multi-scale spatial fusion on a segmented image corresponding to a preset number of image feature vectors and metadata information to obtain a remote sensing image retrieval result, where the remote sensing image retrieval result comprises:
Judging whether the ranges of the plurality of segmented images are intersected or not based on the latitude and longitude coordinate ranges of the segmented images in the metadata information;
if the plurality of segmented images have the intersecting ranges and the scale levels of the segmented images with the intersecting ranges are the same, performing space fusion on the plurality of segmented images with the intersecting ranges, and taking the fused images as remote sensing image retrieval results;
if the multiple segmented images are intersected in scope and the segmented images with the intersected scope are different in scale level, the segmented image with the highest scale level is used as a remote sensing image retrieval result.
In a second aspect, an embodiment of the present invention provides a remote sensing image content retrieval device, including:
the model training module is used for performing migration learning on the multi-modal large model based on the remote sensing image description data set to obtain a cross-modal large model;
the preprocessing module is used for acquiring a remote sensing large-scale image, preprocessing the remote sensing large-scale image to obtain a block image and metadata information of the block image;
the image coding module is used for coding the segmented image based on the cross-modal large model to obtain an image feature vector, and storing the image feature vector of the segmented image and corresponding metadata information into a vector library;
The text coding module is used for acquiring the image content description text to be searched and the geographic range, coding the image content description text based on the cross-modal large model to obtain text feature vectors, and filtering the image feature vectors in the vector library based on the geographic range to obtain filtered image feature vectors;
the similarity calculation module is used for calculating cosine similarity of the text feature vector and the filtered image feature vector, and acquiring a preset number of image feature vectors and metadata information based on the cosine similarity;
and the fusion module is used for carrying out multi-scale space fusion on the segmented images corresponding to the preset number of image feature vectors and the metadata information to obtain remote sensing image retrieval results.
In a third aspect, an embodiment of the present invention provides an electronic device comprising a processor and a memory storing computer executable instructions executable by the processor to perform the steps of the method of any one of the first aspects described above.
In a fourth aspect, embodiments of the present invention provide a computer readable storage medium having a computer program stored thereon, which when executed by a processor performs the steps of the method of any of the first aspects provided above.
The embodiment of the invention has the following beneficial effects:
according to the remote sensing image content retrieval method, the remote sensing image content retrieval device, the electronic equipment and the medium, firstly, a multi-modal large model is subjected to migration learning based on a remote sensing image description data set to obtain a cross-modal large model; secondly, acquiring a remote sensing large-scale image, and preprocessing the remote sensing large-scale image to obtain a block image and metadata information of the block image; coding the segmented image based on the cross-modal large model to obtain an image feature vector, and storing the image feature vector of the segmented image and corresponding metadata information into a vector library; then acquiring an image content description text and a geographic range to be searched, encoding the image content description text based on a cross-modal large model to obtain text feature vectors, and filtering the image feature vectors in a vector library based on the geographic range to obtain filtered image feature vectors; then, calculating cosine similarity of the text feature vector and the filtered image feature vector, and acquiring a preset number of image feature vectors and metadata information based on the cosine similarity; and finally, carrying out multi-scale space fusion on the segmented images corresponding to the preset number of image feature vectors and the metadata information to obtain remote sensing image retrieval results.
According to the method, parameter fine adjustment is carried out on the multi-modal large model through a migration learning strategy to obtain the cross-modal large model suitable for the remote sensing image, and the extraction capability of the remote sensing image characteristics is enhanced; secondly, carrying out multi-scale sampling on the remote sensing large-scale image, extracting image feature vectors in a blocking mode and storing the image feature vectors, and carrying out space fusion on search results at the same time, so that the accuracy of remote sensing image content search is improved; in addition, the method filters the image feature vectors in the vector library by utilizing the geographic range, so that not only can the images meeting the conditions in the specific geographic range be queried, but also the retrieval efficiency can be greatly improved, and the high-performance image content retrieval can be realized.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a remote sensing image content retrieval method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a migration learning principle and an effect provided by an embodiment of the present invention;
FIG. 3 is a schematic diagram of image preprocessing and multi-scale sampling block according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of image feature vector encoding and binning according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of text encoding and image feature vector filtering according to an embodiment of the present invention;
FIG. 6 is a schematic diagram illustrating a text feature vector and image feature vector similarity calculation according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of multi-scale spatial fusion of a segmented image range according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a remote sensing image content retrieval effect according to an embodiment of the present invention;
FIG. 9 is a schematic diagram illustrating verification of remote sensing image content retrieval efficiency according to an embodiment of the present invention;
FIG. 10 is a flowchart of a method for searching content of a high-performance cross-modal remote sensing image based on a multi-scale feature vector library according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of a remote sensing image content retrieval device according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
At present, common remote sensing image retrieval methods are divided into two types, namely remote sensing image retrieval based on labels and remote sensing image retrieval based on contents. The method is very mature in technology, but along with the growth of mass data, the labeling mode is increasingly complicated, and the fixed label content cannot meet the diversified retrieval requirements of people.
The remote sensing image retrieval method based on the content uses a feature extraction model to respectively establish the image and the feature vector of the text, and the retrieval from the text to the image is realized by calculating the distance between the image and the feature vector, namely the similarity. However, content-based remote sensing image retrieval still faces three challenges: firstly, compared with a common image, the remote sensing image has higher spectrum complexity, and the phenomenon of homonymy foreign matters and homonymy foreign matters is ubiquitous, so that higher requirements are put on the feature extraction capacity and stability of the model; secondly, the scenes and the ground objects of the remote sensing images are complex and various in types, obvious scale differences exist in different scenes, and how to adapt to the differences plays a vital role in improving the retrieval precision of the remote sensing images; finally, along with the explosive growth of remote sensing images, the storage management and high-performance retrieval of massive remote sensing image feature vectors become the problems which must be solved in actual production.
In summary, the feature extraction capability, multi-scale adaptation, and feature vector storage and retrieval have become bottleneck problems limiting content-based remote sensing image retrieval.
Based on the above, the remote sensing image content retrieval method, the remote sensing image content retrieval device, the electronic equipment and the medium provided by the embodiment of the invention improve the speed and the precision of remote sensing image content retrieval.
For the sake of understanding the present embodiment, a detailed description will be given of a remote sensing image content retrieval method disclosed in the present embodiment, where the method may be executed by an electronic device, such as a smart phone, a computer, a tablet computer, etc. Referring to the flowchart of a remote sensing image content retrieval method shown in fig. 1, the method mainly includes the following steps S101 to S106:
step S101: and performing migration learning on the multi-modal large model based on the remote sensing image description data set to obtain the cross-modal large model.
In one embodiment, a multi-modal large model (Contrastive Language-Image Pre-training, CLIP) large model is subjected to migration learning by using a remote sensing Image description dataset, and a cross-modal large model CLIP-RS applicable to remote sensing images is obtained.
Step S102: and acquiring a remote sensing large-scale image, and preprocessing the remote sensing large-scale image to obtain a block image and metadata information of the block image.
In one embodiment, the remote sensing large-scale image is subjected to preprocessing such as bit conversion, noise elimination, multi-scale sampling, blocking and the like to obtain a block image, and metadata information of the block image is obtained.
Step S103: and encoding the segmented image based on the cross-modal large model to obtain an image feature vector, and storing the image feature vector of the segmented image and corresponding metadata information into a vector library.
In one embodiment, the CLIP-RS model is used to encode the segmented image into 512-length image feature vectors, and store the image feature vectors and corresponding metadata information into a vector library and index.
Step S104: obtaining an image content description text and a geographic range to be searched, encoding the image content description text based on a cross-modal large model to obtain text feature vectors, and filtering the image feature vectors in a vector library based on the geographic range to obtain filtered image feature vectors.
In one embodiment, editing a description text of image content to be retrieved and a geographic range, encoding the description text of the image content by using a CLIP-RS model, generating text feature vectors, and filtering the image feature vectors in a vector library by using metadata information according to the geographic range to obtain filtered image feature vectors.
Step S105: and calculating cosine similarity of the text feature vector and the filtered image feature vector, and acquiring a preset number of image feature vectors and metadata information based on the cosine similarity.
In one embodiment, the cosine similarity is calculated between the text feature vector and the filtered image feature vector, and the image feature vector and the metadata information are returned according to the descending order of the cosine similarity; then selecting the most similar image feature vectors with preset quantity, wherein the corresponding segmented image range in the metadata information meets the image range of the search requirement.
Step S106: and carrying out multi-scale space fusion on the segmented images corresponding to the preset number of image feature vectors and the metadata information to obtain remote sensing image retrieval results.
In one embodiment, the selected segmented images are subjected to multi-scale spatial fusion, and the fusion result is the final result of image retrieval.
According to the remote sensing image content retrieval method provided by the embodiment of the invention, parameter fine adjustment is carried out on the multi-modal large model through a migration learning strategy to obtain a cross-modal large model suitable for the remote sensing image, so that the extraction capability of the characteristics of the remote sensing image is enhanced; secondly, carrying out multi-scale sampling on the remote sensing large-scale image, extracting image feature vectors in a blocking mode and storing the image feature vectors, and carrying out space fusion on search results at the same time, so that the accuracy of remote sensing image content search is improved; in addition, the method filters the image feature vectors in the vector library by utilizing the geographic range, so that not only can the images meeting the conditions in the specific geographic range be queried, but also the retrieval efficiency can be greatly improved, and the high-performance image content retrieval can be realized.
In one embodiment, for the foregoing step S101, that is, when performing migration learning on the multi-modal large model based on the remote sensing image description dataset to obtain the cross-modal large model, the following manners may be adopted, including but not limited to: firstly, acquiring original pre-training weights of a multi-mode large model; then, inputting the image data and the text data of the remote sensing image description data set into a multi-mode large model for coding to obtain an image vector and a text vector; and finally, performing iterative optimization on the original pre-training weight of the multi-modal large model by adopting a contrast loss function based on the distance between the matched image vector and the text vector and the distance between the unmatched image vector and the text vector to obtain the cross-modal large model.
In specific implementation, referring to fig. 2, first, the CLIP-RS model reads the original pre-training weights; then, inputting image data and text data of a remote sensing image description data set (RSICD) into a CLIP model for encoding, encoding the image data and the text data into 512-dimensional image vectors and text vectors, reducing the distance between matched image vector-text vector pairs by using a contrast loss function, increasing the distance between unmatched image vector-text vector pairs, and continuously iterating and optimizing model parameters to finally obtain the cross-mode large model CLIP-RS applicable to the remote sensing images.
In one embodiment, for the step S102, that is, when preprocessing the remote sensing large-scale image to obtain the segmented image and the metadata information of the segmented image, the following methods may be adopted, but are not limited to: firstly, performing image bit number conversion on a remote sensing large-scale image by adopting a percentage cut-off method, and performing noise reduction operation on the remote sensing large-scale image; then, performing downsampling operation on the remote sensing large-scale image after noise reduction to obtain a multi-scale image; and finally, carrying out block cutting on the multi-scale image to obtain a block image, and obtaining metadata information of the block image.
In specific implementation, referring to fig. 3, first, a percentage truncation method is used to convert a 16-bit remote sensing large-scale image into an 8-bit large-scale image synthesized by RGB so as to align with the input requirement of the CLIP model; meanwhile, noise reduction operation is carried out on the remote sensing large-scale image by using a mean value filtering algorithm so as to reduce noise caused by image overexposure; then, carrying out downsampling operation of 2 times of each level on the 8-bit large-scale image subjected to image bit number conversion and denoising, wherein 7 levels are sampled in the embodiment so as to meet the scale difference of different retrieval scenes; finally, the sampled multi-scale image is cut into a block image with 256 multiplied by 256 pixels, and the block image is used as an input image of the CLIP-RS model.
In one embodiment, for the step S103, that is, when the segmented image is encoded based on the cross-modal large model to obtain the image feature vector, and the image feature vector of the segmented image and the corresponding metadata information are stored in the vector library, the following ways may be adopted, but are not limited to: firstly, inputting a segmented image into a cross-modal large model for feature coding to obtain an image feature vector of the segmented image, and storing the image feature vector into a vector library; and then, acquiring metadata information of the segmented image, storing the metadata information of the segmented image into a vector library, and establishing a spatial index based on longitude and latitude coordinates of a central point of the metadata information.
In specific implementation, referring to fig. 4, the block image of 256×256 pixels obtained in step S102 is input into a CLIP-RS model for feature encoding to obtain an image feature vector of 512 length, and then the image feature vector is inserted into a vector library for storage; and metadata information of the segmented image at the same time at least comprises: the number id_tiles, the longitude and latitude coordinate range b_box, the longitude and latitude coordinates lon and lat of the center point, the sampling level boom, the number id of the large-scale image, the path name and the like of the image block are stored in a vector library, and a spatial index is established for the longitude and latitude coordinates lon and lat of the center point and used as a filtering condition of image retrieval, so that the retrieval speed is greatly improved.
In one embodiment, for the step S104, that is, when the image content description text and the geographic range to be retrieved are obtained, the text feature vector is obtained by encoding the image content description text based on the cross-modal large model, and the image feature vector in the vector library is filtered based on the geographic range, the following manners may be adopted to obtain the filtered image feature vector, which include but are not limited to: firstly, encoding an image content description text based on a cross-modal large model to obtain a text feature vector, and then filtering the image feature vector with the longitude and latitude coordinates of a central point in the geographic range in a vector library based on a spatial index by taking the geographic range as a filtering condition to obtain a filtered image feature vector.
In the implementation, referring to fig. 5, firstly, the descriptive text of the image content (such as "adjacent lake building") and the geographic range (such as [112.70,114.23,34.26,34.97 ]) to be retrieved are obtained, then, the descriptive text of the image content is input into a CLIP-RS model, and the descriptive text is encoded to obtain 512-dimensional text feature vectors. And simultaneously, taking the geographic range as a filtering condition, and filtering in a vector library to obtain an image feature vector with the longitude and latitude of the central point in the corresponding geographic range. The spatial filtering condition is added in the method, so that not only can the images meeting the condition in a specific geographic range be queried, but also the retrieval efficiency can be greatly improved, and the high-performance image content retrieval is realized.
In one embodiment, for the step S105, that is, when calculating the cosine similarity between the text feature vector and the filtered image feature vector and obtaining the preset number of image feature vectors and metadata information based on the cosine similarity, the following ways may be adopted, but are not limited to: firstly, calculating cosine similarity of a text feature vector and a filtered image feature vector; and then, selecting a preset number of image feature vectors and corresponding metadata information from large to small according to the ordering result of the cosine similarity descending order.
In the implementation, firstly, the cosine similarity is calculated between the text feature vector and the filtered image feature vector, and the calculation method is shown in formula (1).
（1）
Wherein A is a text feature vector; b is an image feature vector;representing the modular length of the text feature vector a; />The modulo length of the image feature vector B is shown. Wherein, the closer S is to 1, the more similar the A and B vectors are; the closer S is to-1, the more dissimilar the A and B vectors are.
Then, referring to fig. 6, the image feature vectors and the metadata information are returned according to the descending order of cosine similarity, and a preset number of most similar image feature vectors and corresponding metadata information are selected according to the descending order of cosine similarity.
In one embodiment, for the step S106, that is, when the segmented images corresponding to the preset number of image feature vectors and metadata information are subjected to multi-scale spatial fusion to obtain the remote sensing image search result, the following methods may be adopted, but are not limited to: firstly, judging whether the ranges of a plurality of segmented images are intersected or not based on the longitude and latitude coordinate ranges of the segmented images in the metadata information; if the plurality of segmented images have the intersecting ranges and the scale levels of the segmented images with the intersecting ranges are the same, performing space fusion on the plurality of segmented images with the intersecting ranges, and taking the fused images as remote sensing image retrieval results; if the multiple segmented images are intersected in scope and the segmented images with the intersected scope are different in scale level, the segmented image with the highest scale level is used as a remote sensing image retrieval result.
In the implementation, referring to fig. 7, it is assumed that 5 image feature vectors with highest similarity are selected, and the latitude and longitude coordinate range b_box of the segmented image in the corresponding metadata information is regarded as the searched image range. If the edge intersection exists in the plurality of segmented image ranges and the segmented images intersected by the edge are in the same scale level, carrying out space fusion on the segmented images, and combining the segmented images into a polygon to serve as a remote sensing image search result, so that the search result is ensured to cover a scene meeting the search condition as far as possible; if the multiple segmented image ranges intersect (edges intersect or contain) and the segmented images with the intersecting ranges belong to different scale levels, the segmented image range with the highest level (highest resolution) is taken as a search result, so that the accuracy and the visualization degree of the search result are ensured to be better.
The embodiment of the invention provides a remote sensing image content retrieval method, which comprises the steps of firstly, introducing a multi-mode large model CLIP based on a transducer, and carrying out full-parameter fine adjustment by using a migration learning strategy, so that the extraction capability of the model to the remote sensing image characteristics is enhanced; then, a multi-scale image blocking-fusing retrieval strategy is provided, a large-scale remote sensing image is subjected to multi-scale sampling, feature vectors are extracted in a blocking mode and stored, and meanwhile, retrieval results are subjected to multi-scale space fusion, so that retrieval accuracy of the remote sensing image and scene content is improved; finally, based on the distributed deployment and load filtering modes of the vector library, the distributed storage and high-performance retrieval of mass remote sensing data are realized.
In order to facilitate understanding, the embodiment of the present invention further provides a specific method for searching high-performance cross-mode remote sensing image content based on a multi-scale feature vector library, referring to a flowchart shown in fig. 8, the method mainly includes: training a remote sensing cross-mode model, warehousing a remote sensing image feature vector and checking text content of the remote sensing image. Firstly, performing transfer learning on a CLIP model to obtain a CLIP-RS model; then, carrying out multi-scale pretreatment on the remote sensing large-scale image, and encoding the remote sensing large-scale image by utilizing a CLIP-RS model to obtain an image feature vector and storing the image feature vector into a vector library; then, coding the image content description text by using the CLIP-RS model to generate a text feature vector; then, calculating cosine similarity of the image feature vector and the text feature vector, and filtering the image feature vector in the vector library by taking the geographic range as a filtering condition; and finally, selecting n image feature vectors with highest similarity, and carrying out multi-scale space fusion to obtain a final search result.
The method provided by the embodiment of the invention realizes the remote sensing image content retrieval with high precision and high performance, and specifically, the precision verification of the remote sensing image content retrieval is as follows:
The RSICD data set is used for verification, the data set is a multi-resolution data set from multiple data sources such as google maps, hundred-degree maps and the like, and consists of three subsets of training data, verification data and test data, the data amount is 8734, 1094 and 1027, and each data contains descriptive texts of 1 image and 5 images. The invention uses the training set of RSICD to train the CLIP model, uses the verification set to evaluate and select the model, and obtains the optimal CLIP-RS model, and then uses the test set to test the model precision. For comprehensively expressing the accuracy of the model, TOP-1, TOP-3, TOP-5 and TOP-10 were used as evaluation indexes. Wherein TOP-1 indicates that the highest scoring predictor is the correct duty cycle, TOP-3 indicates that there is the correct duty cycle in the first three highest scoring predictors, TOP-5, TOP-10, and so on. The accuracy verification results are shown in table 1.
TABLE 1 CLIP-RS model test set accuracy verification
The results show that firstly, the precision of the trimmed model is obviously improved, the baseline is an original CLIP model, and four indexes are respectively 0.574, 0.752, 0.836 and 0.939, which are obviously lower than the precision of the trimmed CLIP-RS test; secondly, the invention explores the precision of the CLIP-RS model under different learning rates, and the result shows that the learning rate is 5 multiplied by 10 -5 When the model is used, the model has the highest test precision; finally, the invention explores the influence of image data enhancement imgauge and text data enhancement textresults on model precision, and the result shows that the CLIP-RS model using image data enhancement and text data enhancement training has higher test precision. In combination with the above analysis, the present invention uses a learning rate of 5×10 -5 The CLIP-RS models for image data enhancement and text data enhancement training have the highest accuracies of 0.848, 0.965, 0.981 and 0.996, respectively. The specific image content retrieval effect is shown in fig. 9.
Further, the efficiency of remote sensing image content retrieval is verified as follows:
the remote sensing image content retrieval service is deployed on a Tesla V100 display card server, 2000 remote sensing image content retrieval tests are performed, and time consumption of each retrieval is recorded, wherein the time consumption mainly comprises time consumption of CLIP-RS model text coding, time consumption of vector library query, time consumption of data transmission and the like, as shown in fig. 10. 2000 complete remote sensing image content searches take 0.114 seconds on average. The result shows that the remote sensing image content retrieval method provided by the invention has higher efficiency and can meet the requirement of high retrieval efficiency in actual production.
For the remote sensing image content retrieval method provided in the foregoing embodiment, the embodiment of the present invention further provides a remote sensing image content retrieval device, referring to a schematic structural diagram of the remote sensing image content retrieval device shown in fig. 11, which illustrates that the device mainly includes the following parts:
The model training module 1101 is configured to perform migration learning on the multi-modal large model based on the remote sensing image description dataset to obtain a cross-modal large model;
the preprocessing module 1102 is configured to acquire a remote sensing large-scale image, and preprocess the remote sensing large-scale image to obtain a block image and metadata information of the block image;
the image encoding module 1103 is configured to encode the segmented image based on the cross-modal large model to obtain an image feature vector, and store the image feature vector of the segmented image and corresponding metadata information into a vector library;
the text encoding module 1104 is configured to obtain an image content description text to be retrieved and a geographic range, encode the image content description text based on the cross-modal large model to obtain a text feature vector, and filter the image feature vector in the vector library based on the geographic range to obtain a filtered image feature vector;
the similarity calculating module 1105 is configured to calculate cosine similarity between the text feature vector and the filtered image feature vector, and obtain a preset number of image feature vectors and metadata information based on the cosine similarity;
the fusion module 1106 is configured to perform multi-scale spatial fusion on the segmented images corresponding to the preset number of image feature vectors and the metadata information to obtain a remote sensing image retrieval result.
According to the remote sensing image content retrieval device provided by the embodiment of the invention, parameter fine adjustment is carried out on the multi-modal large model through a migration learning strategy to obtain the cross-modal large model suitable for the remote sensing image, so that the extraction capability of the characteristics of the remote sensing image is enhanced; secondly, carrying out multi-scale sampling on the remote sensing large-scale image, extracting image feature vectors in a blocking mode and storing the image feature vectors, and carrying out space fusion on search results at the same time, so that the accuracy of remote sensing image content search is improved; in addition, the method filters the image feature vectors in the vector library by utilizing the geographic range, so that not only can the images meeting the conditions in the specific geographic range be queried, but also the retrieval efficiency can be greatly improved, and the high-performance image content retrieval can be realized.
It should be noted that, for the sake of brevity, reference may be made to the corresponding contents of the foregoing method embodiments for the description of the device embodiment, where the principles and technical effects of the device provided in the embodiment are the same as those of the foregoing method embodiments. The particular values provided in the practice of the present invention are exemplary only and are not limiting herein.
The embodiment of the invention also provides electronic equipment, which comprises a processor and a storage device; the storage means has stored thereon a computer program which, when run by a processor, performs the method according to any of the above embodiments.
Fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, where the electronic device 100 includes: a processor 120, a memory 121, a bus 122 and a communication interface 123, the processor 120, the communication interface 123 and the memory 121 being connected by the bus 122; the processor 120 is configured to execute executable modules, such as computer programs, stored in the memory 121.
The memory 121 may include a high-speed random access memory (RAM, random Access Memory), and may further include a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory. The communication connection between the system network element and at least one other network element is implemented via at least one communication interface 123 (which may be wired or wireless), which may use the internet, a wide area network, a local network, a metropolitan area network, etc.
Bus 122 may be an ISA bus, a PCI bus, an EISA bus, or the like. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 12, but not only one bus or type of bus.
The memory 121 is configured to store a program, and the processor 120 executes the program after receiving an execution instruction, where the method executed by the apparatus for flow defining disclosed in any of the foregoing embodiments of the present invention may be applied to the processor 120 or implemented by the processor 120.
The processor 120 may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuitry in hardware or instructions in software in the processor 120. The processor 120 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a digital signal processor (Digital Signal Processing, DSP for short), application specific integrated circuit (Application Specific Integrated Circuit, ASIC for short), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA for short), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in the memory 121, and the processor 120 reads the information in the memory 121, and in combination with its hardware, performs the steps of the above method.
The computer program product of the readable storage medium provided by the embodiment of the present invention includes a computer readable storage medium storing a program code, where the program code includes instructions for executing the method described in the foregoing method embodiment, and the specific implementation may refer to the foregoing method embodiment and will not be described herein.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Finally, it should be noted that: the above examples are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention, but it should be understood by those skilled in the art that the present invention is not limited thereto, and that the present invention is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (8)
1. The remote sensing image content retrieval method is characterized by comprising the following steps of:
performing migration learning on the multi-modal large model based on the remote sensing image description dataset to obtain a cross-modal large model;
acquiring a remote sensing large-scale image, and preprocessing the remote sensing large-scale image to obtain a block image and metadata information of the block image;
Encoding the segmented image based on the cross-modal large model to obtain an image feature vector, and storing the image feature vector of the segmented image and corresponding metadata information into a vector library;
acquiring an image content description text to be retrieved and a geographic range, encoding the image content description text based on the cross-modal large model to obtain a text feature vector, and filtering the image feature vector in the vector library based on the geographic range to obtain a filtered image feature vector;
calculating cosine similarity of the text feature vector and the filtered image feature vector, and acquiring a preset number of image feature vectors and metadata information based on the cosine similarity;
performing multi-scale space fusion on the segmented images corresponding to the preset number of image feature vectors and the metadata information to obtain remote sensing image retrieval results;
performing migration learning on the multi-modal large model based on the remote sensing image description dataset to obtain a cross-modal large model, including: acquiring original pre-training weights of the multi-mode large model; inputting image data and text data of the remote sensing image description dataset into the multi-mode large model for encoding to obtain an image vector and a text vector; based on the distance between the matched image vector and the text vector and the distance between the unmatched image vector and the text vector, performing iterative optimization on the original pre-training weight of the multi-modal large model by adopting a contrast loss function to obtain a cross-modal large model;
Performing multi-scale spatial fusion on the segmented images corresponding to the preset number of image feature vectors and the metadata information to obtain remote sensing image retrieval results, wherein the remote sensing image retrieval results comprise: judging whether the ranges of the plurality of block images are intersected or not based on the latitude and longitude coordinate ranges of the block images in the metadata information; if the plurality of segmented images have the range intersection and the scale levels of the segmented images with the range intersection are the same, performing space fusion on the plurality of segmented images with the range intersection, and taking the fused images as remote sensing image retrieval results; and if the plurality of segmented image ranges are intersected and the scale levels of the segmented images with the intersected ranges are different, taking the segmented image with the highest scale level as a remote sensing image retrieval result.
2. The method of claim 1, wherein preprocessing the remote sensing macro image to obtain a segmented image and metadata information of the segmented image, comprises:
performing image bit number conversion on the remote sensing large-scale image by adopting a percentage cut-off method, and performing noise reduction operation on the remote sensing large-scale image;
performing downsampling operation on the remote sensing large-scale image after noise reduction to obtain a multi-scale image;
And carrying out block cutting on the multi-scale image to obtain a block image, and acquiring metadata information of the block image.
3. The method of claim 1, wherein encoding the segmented image based on the cross-modal large model to obtain an image feature vector and storing the image feature vector of the segmented image and corresponding metadata information to a vector library comprises:
inputting the segmented image into the cross-modal large model for feature coding to obtain an image feature vector of the segmented image, and storing the image feature vector into a vector library;
and acquiring metadata information of the segmented image, storing the metadata information of the segmented image into a vector library, and establishing a spatial index based on longitude and latitude coordinates of a central point of the metadata information.
4. The method of claim 3, wherein filtering the image feature vectors in the vector library based on the geographic scope to obtain filtered image feature vectors comprises:
and taking the geographic range as a filtering condition, and filtering the image feature vector of the longitude and latitude coordinates of the central point in the geographic range based on the spatial index in the vector library to obtain a filtered image feature vector.
5. The method of claim 1, wherein obtaining a predetermined number of image feature vectors and metadata information based on the cosine similarity comprises:
and selecting a preset number of image feature vectors and corresponding metadata information from large to small according to the ordering result of the cosine similarity descending order.
6. A remote sensing image content retrieval device, comprising:
the model training module is used for performing migration learning on the multi-modal large model based on the remote sensing image description data set to obtain a cross-modal large model;
the preprocessing module is used for acquiring a remote sensing large-scale image, preprocessing the remote sensing large-scale image to obtain a block image and metadata information of the block image;
the image coding module is used for coding the block images based on the cross-modal large model to obtain image feature vectors, and storing the image feature vectors of the block images and corresponding metadata information into a vector library;
the text coding module is used for acquiring an image content description text to be searched and a geographic range, coding the image content description text based on the cross-modal large model to obtain a text feature vector, and filtering the image feature vector in the vector library based on the geographic range to obtain a filtered image feature vector;
The similarity calculation module is used for calculating cosine similarity of the text feature vector and the filtered image feature vector and acquiring a preset number of image feature vectors and metadata information based on the cosine similarity;
the fusion module is used for carrying out multi-scale space fusion on the segmented images corresponding to the preset number of image feature vectors and the metadata information to obtain remote sensing image retrieval results;
the model training module is further configured to: acquiring original pre-training weights of the multi-mode large model; inputting image data and text data of the remote sensing image description dataset into the multi-mode large model for encoding to obtain an image vector and a text vector; based on the distance between the matched image vector and the text vector and the distance between the unmatched image vector and the text vector, performing iterative optimization on the original pre-training weight of the multi-modal large model by adopting a contrast loss function to obtain a cross-modal large model;
the fusion module is also used for: judging whether the ranges of the plurality of block images are intersected or not based on the latitude and longitude coordinate ranges of the block images in the metadata information; if the plurality of segmented images have the range intersection and the scale levels of the segmented images with the range intersection are the same, performing space fusion on the plurality of segmented images with the range intersection, and taking the fused images as remote sensing image retrieval results; and if the plurality of segmented image ranges are intersected and the scale levels of the segmented images with the intersected ranges are different, taking the segmented image with the highest scale level as a remote sensing image retrieval result.
7. An electronic device comprising a processor and a memory, the memory storing computer executable instructions executable by the processor, the processor executing the computer executable instructions to implement the steps of the method of any one of claims 1 to 5.
8. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program when executed by a processor performs the steps of the method of any of the preceding claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410044899.0A CN117556079B (en) | 2024-01-12 | 2024-01-12 | Remote sensing image content retrieval method, remote sensing image content retrieval device, electronic equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410044899.0A CN117556079B (en) | 2024-01-12 | 2024-01-12 | Remote sensing image content retrieval method, remote sensing image content retrieval device, electronic equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117556079A CN117556079A (en) | 2024-02-13 |
| CN117556079B true CN117556079B (en) | 2024-04-16 |
Family
ID=89811480
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410044899.0A Active CN117556079B (en) | 2024-01-12 | 2024-01-12 | Remote sensing image content retrieval method, remote sensing image content retrieval device, electronic equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117556079B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118410200B (en) * | 2024-06-17 | 2024-09-24 | 中国科学院空天信息创新研究院 | Remote sensing image retrieval method and device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116775922A (en) * | 2023-05-16 | 2023-09-19 | 中国航空综合技术研究所 | Remote sensing image cross-modal retrieval method based on fusion of language and visual detail characteristics |
| CN116796038A (en) * | 2023-05-17 | 2023-09-22 | 清华大学深圳国际研究生院 | Remote sensing data retrieval method, remote sensing data retrieval device, edge processing equipment and storage medium |
| CN117056543A (en) * | 2023-08-21 | 2023-11-14 | 数据空间研究院 | Multi-mode patent retrieval method based on images |
| CN117079048A (en) * | 2023-08-29 | 2023-11-17 | 贵州电网有限责任公司 | Geological disaster image recognition method and system based on CLIP model |
| CN117349784A (en) * | 2023-08-22 | 2024-01-05 | 浙江时空道宇科技有限公司 | Remote sensing data processing method, device and equipment |
-
2024
- 2024-01-12 CN CN202410044899.0A patent/CN117556079B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116775922A (en) * | 2023-05-16 | 2023-09-19 | 中国航空综合技术研究所 | Remote sensing image cross-modal retrieval method based on fusion of language and visual detail characteristics |
| CN116796038A (en) * | 2023-05-17 | 2023-09-22 | 清华大学深圳国际研究生院 | Remote sensing data retrieval method, remote sensing data retrieval device, edge processing equipment and storage medium |
| CN117056543A (en) * | 2023-08-21 | 2023-11-14 | 数据空间研究院 | Multi-mode patent retrieval method based on images |
| CN117349784A (en) * | 2023-08-22 | 2024-01-05 | 浙江时空道宇科技有限公司 | Remote sensing data processing method, device and equipment |
| CN117079048A (en) * | 2023-08-29 | 2023-11-17 | 贵州电网有限责任公司 | Geological disaster image recognition method and system based on CLIP model |
Non-Patent Citations (2)
| Title |
|---|
| CLIP-RS: A Cross-modal Remote Sensing Image Retrieval Based on CLIP, a Northern Virginia Case Study;Djoufack Basso L;Virginia Tech;20221231;全文 * |
| Contrasting dual transformer architectures for multi-modal remote sensing image retrieval;Rahhal M M A等;Applied Sciences;20221231;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117556079A (en) | 2024-02-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111814794B (en) | Text detection method and device, electronic equipment and storage medium | |
| Aly et al. | Indexing in large scale image collections: Scaling properties and benchmark | |
| CN102262618B (en) | Method and device for identifying page information | |
| CN117556079B (en) | Remote sensing image content retrieval method, remote sensing image content retrieval device, electronic equipment and medium | |
| US9208401B2 (en) | System and method for using an image to provide search results | |
| CN111680176A (en) | Remote sensing image retrieval method and system based on attention and bidirectional feature fusion | |
| US8266542B2 (en) | Rendering and sorting book search results as three dimensional icons | |
| CN106096028A (en) | Historical relic indexing means based on image recognition and device | |
| KR20130057465A (en) | Object recognition using incremental feature extraction | |
| Iakovidou et al. | Localizing global descriptors for content-based image retrieval | |
| KR102468309B1 (en) | Method for searching building based on image and apparatus for the same | |
| CN104463177A (en) | Similar face image obtaining method and device | |
| CN103353881A (en) | Method and device for searching application | |
| CN105740808A (en) | Human face identification method and device | |
| CN111651674B (en) | Bidirectional searching method and device and electronic equipment | |
| CN114581710A (en) | Image recognition method, device, equipment, readable storage medium and program product | |
| CN111373393A (en) | Image retrieval method and device and image library generation method and device | |
| CN114972947A (en) | Depth scene text detection method and device based on fuzzy semantic modeling | |
| CN113139110B (en) | Regional characteristic processing method, regional characteristic processing device, regional characteristic processing equipment, storage medium and program product | |
| CN115129915A (en) | Repeated image retrieval method, device, equipment and storage medium | |
| Liu et al. | Automatic peak recognition for mountain images | |
| CN111382220A (en) | POI data dividing method and device | |
| CN115358777A (en) | Advertisement putting processing method and device of virtual world | |
| CN106776654B (en) | Data searching method and device | |
| CN114782720A (en) | Method, device, electronic device, medium, and program product for determining matching of document |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |