CN106570125B - Remote sensing image retrieval method and device for rotational scaling and translation invariance - Google Patents
Remote sensing image retrieval method and device for rotational scaling and translation invariance Download PDFInfo
- Publication number
- CN106570125B CN106570125B CN201610950697.8A CN201610950697A CN106570125B CN 106570125 B CN106570125 B CN 106570125B CN 201610950697 A CN201610950697 A CN 201610950697A CN 106570125 B CN106570125 B CN 106570125B
- Authority
- CN
- China
- Prior art keywords
- image
- similarity
- remote sensing
- transformation
- retrieval
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
Abstract
The invention relates to the technical field of remote sensing image retrieval, in particular to a remote sensing image retrieval method and device with rotation, scaling and translation invariance. The retrieval method comprises the following steps: step a: after image transformation processing is carried out on each image in the image library, edge point pixels of each image are extracted, and each image transaction set is constructed; step b: extracting association rules in the images through each image transaction set; step c: and calculating the association rule similarity between the image to be retrieved and the retrieved image in the image library according to the support degree and the confidence index, and performing image retrieval according to the association rule similarity between the image to be retrieved and the retrieved image in the image library. The invention extracts the association rule of the remote sensing image from the constructed transaction set, and then compares the similarity of the association rule of the image to be retrieved and the retrieved image in the image library, thereby realizing the retrieval of the remote sensing image and improving the accuracy of the remote sensing image retrieval.
Description
Technical Field
The invention relates to the technical field of remote sensing image retrieval, in particular to a remote sensing image retrieval method and device with rotation, scaling and translation invariance.
Background
The remote sensing image has the characteristics of large image breadth, more and complex image content, common phenomena of 'same object different spectrum' and 'foreign object same spectrum', and great difficulty is brought to the retrieval of the remote sensing image. Image retrieval, namely searching images with specified characteristics or similar contents in a database, wherein the current mainstream Content-Based image retrieval (CBIR) method can integrate knowledge in various fields such as image processing, information retrieval, machine learning, computer vision, artificial intelligence and the like, and takes the visual characteristics automatically extracted from the images as description of the image contents; currently, content-based image retrieval has achieved a great deal of research effort.
Visual feature extraction plays an important role in image retrieval, and can be divided into two research directions, wherein the extraction of low-level visual features such as spectrum, texture, shape and the like of an image and similarity measurement are researched, the extraction comprises hyperspectral image retrieval based on spectral curve absorption feature extraction, color feature extraction by using color space and color moment, image texture feature description by using methods such as wavelet transformation, Contourlet transformation, Gabor wavelet, generalized Gaussian model, texture spectrum and the like, and remote sensing image shape feature description based on pixel shape index, PHOG (Pyramid of Oriented gradient) shape and wavelet Pyramid. The application of the low-level visual features is mature, but semantic information describing images cannot be described, and the provided retrieval result is quite different from the cognition of human brain on remote sensing images and cannot be completely satisfactory.
Aiming at the problem, the other research direction is to establish a mapping model of low-level visual features and semantics, and improve the accuracy of image retrieval at the semantic level. The main research results comprise semantic retrieval methods based on statistical learning, such as Bayesian network, Bayesian network and EM (maximum expectation) parameter estimation of context of Bayesian classifier model, etc.; a retrieval method based on semantic annotation, such as a language index model, a concept semantic distribution model and the like; a semantic retrieval method based on GIS (Geographic Information System) assistance, such as a method for guiding semantic assignment by using space and attribute Information of vector elements in GIS data; semantic retrieval methods based on ontologies, such as visual object domain ontology based methods, GeoIRIS, and the like. The method can reflect the semantic understanding process of human brain on image retrieval to a certain extent, has higher accuracy and is a development trend of future image retrieval. However, the current semantic retrieval method usually pays too much attention to the construction process of the low-level visual features and the semantic mapping model, neglects the factors of the type of the adopted low-level visual features, the semantic learning method and the like, and finally influences the precision ratio of semantic retrieval.
In recent years, human visual perception characteristics are introduced into the field of image retrieval and are receiving wide attention, but such methods are still in the beginning stage, and many problems are still to be solved: such as physiological processes of the human visual system, a feature description method more conforming to human vision, a bottom-up perception model, salient feature extraction and measurement, a top-down visual attention mechanism, and the like. In addition, aiming at the typical results of remote sensing image data retrieval, the typical results mainly comprise a Switzerland RSIAII + III project, and the description and retrieval of multi-resolution remote sensing image data based on spectrum and texture characteristics are researched; a prototype system Blobworld developed by Berkeley digital library projects takes aerial images, USGS ortho images and topographic maps, SPOT satellite images and the like as data sources, so that a user can intuitively improve a retrieval result; the research content of the (RS)2I project of the Nanyang university of Singapore covers the aspects of remote sensing image feature extraction and description, multidimensional index technology and distributed system structure design; simple of stanford university, using a robust Integrated Region Matching (IRM) method to define the similarity between images, and obtaining good results in satellite based remote sensing image retrieval of association rules; in the iFind of Microsoft Asia institute, the system constructs a semantic network through the labeled information of the image, and combines the labeled information with the visual characteristics of the image in the related feedback, thereby effectively realizing the related feedback on two levels. These systems have achieved significant success, but there is still a need for further intensive research, both in feature extraction and in representative feature selection.
As described above, in most of the pixel-based and object-oriented image search methods, attention is paid to statistical information of low-level features such as color, texture, and shape of the entire or a part of an image or a target region. The retrieval method directly based on the low-level features cannot extract interested targets, lacks the capability of describing image space information, and has the defects of overhigh feature dimension, incomplete description, poor accuracy, lack of regularity, semantic difference between feature description and human cognition and the like. Meanwhile, remote sensing image retrieval based on high-level semantic information lacks of mature theory and method. The semantic gap between the low-level features and the high-level semantic information hinders the development and application of remote sensing image retrieval.
Disclosure of Invention
The invention provides a remote sensing image retrieval method and a remote sensing image retrieval device with rotational scaling and translational invariance, and aims to solve at least one of the technical problems in the prior art to a certain extent.
In order to solve the above problems, the present invention provides the following technical solutions:
a remote sensing image retrieval method of rotation scaling translation invariance comprises the following steps:
step a: after image transformation processing is carried out on each image in the image library, edge point pixels of each image are extracted, and each image transaction set is constructed;
step b: extracting association rules in each image through the image transaction set;
step c: and calculating the association rule similarity between the image to be retrieved and each image in the image library according to the support degree and the confidence index, and retrieving the images according to the association rule similarity between the image to be retrieved and each image in the image library.
The technical scheme adopted by the embodiment of the invention also comprises the following steps: in the step a, the image transformation processing of each image in the image library specifically includes: carrying out Radon transformation and Fourier-Mellin transformation on each image; the Radon transform formula is:
P(r，θ)＝R(r，θ)f(x，y)＝∫∫f(x，y)(r-x cosθ-y sinθ)dxdy
in the above formula, | r | represents the distance from a circular point to a straight line, θ ∈ [0, π ] represents the angle between the straight line and the y-axis, (r) is the Dirac function;
the Fourier-Mellin transformation formula is as follows:
in the above formula, u is a real variable and σ is a real constant greater than 0.
The technical scheme adopted by the embodiment of the invention also comprises the following steps: in the step a, after each image is subjected to Radon transform and Fourier-Mellin transform, the rotation of the image is converted into phase, and the phase is converted into amplitude by scaling, and the function after the Radon transform and the Fourier-Mellin transform is:
the function Z (u, k) has the same width and height as the original image, and the function has rotation, scaling invariance.
The technical scheme adopted by the embodiment of the invention also comprises the following steps: the step a further comprises: and carrying out pixel gray level compression on each image after Radon transformation and Fourier-Mellin transformation.
The technical scheme adopted by the embodiment of the invention also comprises the following steps: in the step c, the calculation method for calculating the similarity of the association rule is as follows: for all association rules extracted from each image, combining the products of the support degree and the confidence degree of each association rule to form a rule vector; the image retrieval is realized by comparing the similarity of the regular vector of the image to be retrieved and each image in the image library; the measurement formula of the regular vector similarity is as follows:
in the above formula, N is the number of associated rules of the image, and r1 and r2 are two rule vectors, μ1And mu2The average value of the two images is obtained, and if the two regular vectors are closer and the average values of the two images are closer, the smaller the value of D is, and the higher the similarity between the two images is.
The embodiment of the invention adopts another technical scheme that: a remote sensing image retrieval device of rotation scaling translation invariance comprises:
the image transformation module: the image conversion device is used for carrying out image conversion processing on each image in the image library;
the transaction set building module: the system is used for extracting edge point pixels of each image and constructing a transaction set of each image;
an association rule extraction module: the association rule is used for extracting the association rule in each image through the transaction set of each image;
a similarity calculation module: the image retrieval system is used for calculating the association rule similarity between the image to be retrieved and each image in the image library according to the support degree and the confidence index;
the image retrieval module: and the image retrieval module is used for retrieving the images according to the similarity of the association rule of the images to be retrieved and each image in the image library.
The technical scheme adopted by the embodiment of the invention also comprises the following steps: the image transformation module carries out image transformation processing on each image in the image library, and specifically comprises the following steps: carrying out Radon transformation and Fourier-Mellin transformation on each image; the Radon transform formula is:
P(r，θ)＝R(r，θ)f(x，y)＝∫∫f(x，y)(r-x cosθ-y sinθ)dxdy
in the above formula, | r | represents the distance from a circular point to a straight line, θ ∈ [0, π ] represents the angle between the straight line and the y-axis, (r) is the Dirac function;
the Fourier-Mellin transformation formula is as follows:
in the above formula, u is a real variable and σ is a real constant greater than 0.
The technical scheme adopted by the embodiment of the invention also comprises the following steps: after each image is subjected to Radon transformation and Fourier-Mellin transformation, the rotation of the image is converted into phase, and the phase is converted into amplitude by scaling, and the function after the Radon transformation and the Fourier-Mellin transformation is as follows:
the function Z (u, k) has the same width and height as the original image, and the function has rotation, scaling invariance.
The technical scheme adopted by the embodiment of the invention further comprises an image compression module, wherein the image compression module is used for carrying out pixel gray level compression on each image after Radon transformation and Fourier-Mellin transformation.
The technical scheme adopted by the embodiment of the invention also comprises the following steps: the similarity calculation module calculates the similarity of the association rule in a calculation mode that: for all association rules extracted from each image in the image library, combining the products of the support degree and the confidence coefficient of each association rule to form a rule vector; the image retrieval is realized by comparing the similarity of the regular vector of the image to be retrieved and each image in the image library; the measurement formula of the regular vector similarity is as follows:
in the above formula, N is the number of associated rules of the image, and r1 and r2 are two rule vectors, μ1And mu2The average value of the two images is obtained, and if the two regular vectors are closer and the average values of the two images are closer, the smaller the value of D is, and the higher the similarity between the two images is.
Compared with the prior art, the embodiment of the invention has the following beneficial effects: the remote sensing image retrieval method and the remote sensing image retrieval device of the rotation scaling translation invariance firstly carry out Radon transformation and Fourier-Mellin transformation on a remote sensing image, and eliminate the influence of image rotation, scaling and translation; pixel gray level compression is carried out on the image after Radon and Fourier-Mellin transformation, and the calculation amount of association rule mining is reduced; by constructing a transaction set, extracting association rules of the remote sensing image through the transaction set, and then comparing similarity between the association rules, the retrieval of the remote sensing image is realized; the invention provides a new feasible way for realizing remote sensing image retrieval based on association rules from low-level visual features to high-level semantic information, and improves the accuracy of remote sensing image retrieval.
Drawings
FIG. 1 is a flow chart of a remote sensing image retrieval method of rotational zoom translation invariance according to an embodiment of the present invention;
FIG. 2 is a diagram of the gray scale values of pixels in 4 directions for each edge point pixel;
FIG. 3(a) is an original remote sensing image, FIG. 3(b) is a compressed remote sensing image, and FIG. 3(c) is an edge detection image;
FIG. 4 is a schematic structural diagram of a remote sensing image retrieval apparatus with rotational scaling and translational invariance according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of 23 remote sensing images;
FIG. 6 is the average precision ratio of the top 5 of the ranking of various retrieval methods;
FIG. 7 is the average precision ratio of top 10 of the various search methods;
FIG. 8 is the average precision ratio of the top 15 ranks of various search methods;
FIG. 9 shows the average precision ratio 20 before ranking for each type of search method;
FIG. 10 is a graph showing the average precision ratio of all search results of various search methods;
FIG. 11 shows the overall average precision of all search methods.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Please refer to fig. 1, which is a flowchart of a remote sensing image retrieval method with rotational scaling and translational invariance according to an embodiment of the present invention. The remote sensing image retrieval method of the rotation scaling translation invariance comprises the following steps:
step 100: performing Radon and Fourier-Mellin transformation on each image in the image library;
in step 100, the embodiment of the present invention transforms the image with rotation, scaling and translation into another domain to eliminate the influence of the rotation, scaling and translation, and then extracts the association rule of the image. Firstly, Radon transformation is carried out on an image, and a transformation formula is as follows:
P(r，θ)＝R(r，θ)f(x，y)＝∫∫f(x，y)(r-x cosθ-y sinθ)dxdy (1)
in equation (1), | r | represents the distance from a dot to a line, θ ∈ [0, π ] represents the angle between the line and the y-axis, and (r) is the Dirac function. The Radon transform is to integrate f (x, y) along a line r-x cos θ -y sin θ 0 to obtain a sum of f (x, y) along the line at an arbitrary position (r, θ).
And then carrying out Fourier-Mellin transformation on the image subjected to Radon transformation, wherein the Fourier-Mellin transformation formula is as follows:
in equation (2), u is a real variable, and σ is a real constant greater than 0, and the value thereof is generally 0.5.
The image is subjected to Radon transform and Fourier-Mellin transform, the rotation of the image is converted into phase, and the scaling is converted into amplitude. Finally, the following function is defined:
the function has the same width and height as the original image, so its value can be considered as one image. The function has rotation, scaling invariance. Combining with the association rule, the method has the invariance of rotation, scaling and translation, and performs image retrieval on the basis.
Step 200: compressing the pixel gray level of each image after Radon and Fourier-Mellin transformation;
in step 200, since the gray levels are too many, the support degree of the frequent item set is very small, which is not favorable for extracting the association rule with the support degree and the confidence degree both being sufficiently large, so that before the association rule mining is performed, pixel gray level compression needs to be performed on the image to be retrieved, and the image to be retrieved is compressed to a few gray levels, so as to reduce the calculation amount of the association rule mining.
In the embodiment of the present invention, the method for compressing the gray level of each image pixel specifically comprises: regarding the transformed function Z (u, k) as an image, regarding the value as a gray value, compressing the image by using the neighborhood mean and variance, calculating the mean mu and standard deviation sigma of each pixel in 3 × 3 neighborhood on the remote sensing image, and then calculating the gray level of the neighborhood center pixel after compression by using the following formula:
in formula (4), g is the original gray level of the central pixel, g' is the gray level of the compressed central pixel, and c is the scale factor, and the value range is [0.1, 0.5 ]. By the method, the image to be retrieved can be compressed to 3 gray levels of 0,1 and 2.
In another embodiment of the present invention, each gray level is quantized to the range of [1, G ] by adopting a uniform segmentation mode according to the gray level, specifically: the average compression method is adopted, all gray levels are evenly distributed into a plurality of gray levels,
wherein maxG is the original maximum gray level, G is the compressed maximum gray level, G is 8, ceil () is an rounding-up function, and G +1 is to compress the gray level of the image to 1-8.
Quantizing each gray level to the range of [1, G ] by adopting a uniform segmentation mode according to the gray levels, specifically, compressing by adopting a linear segmentation method, firstly calculating the maximum gray level gMax and the minimum gray level gMin of the image, and then calculating the compressed gray level by utilizing the following formula:
where G is the maximum gray level, and G is 8.
The more the gray levels are compressed, the larger the calculation amount of association rule mining is, but the closer the reflected relationship between the pixels is to the reality; conversely, the fewer the gray levels after compression, the smaller the difference between the compressed pixels, and the more unfavorable the mining of the meaningful association rules, so that it is very important to select an appropriate gray level number. The maximum gray level number in the embodiment of the invention is selected to be 8, the adopted compression mode is average compression, and the compression formula is as follows:
in the formula (7), maxG is the original maximum gray level, G is the maximum number of compressed gray levels, G is 8, ceil () is an upward rounding function, and G +1 is to compress the gray level of the pixel of the image to be retrieved to 1-8.
Step 300: extracting edge point pixels of each transformed image and constructing each image transaction set;
in step 300, the original remote sensing image is taken as an example to describe the construction process of the transaction set, and the construction process of the transformed image is similar to that of the original remote sensing image. The remote sensing image is based on pixels, so in the data mining of the remote sensing image, a transaction set needs to be constructed on the basis of the pixels. The method for constructing the transaction set comprises the following steps: taking a neighborhood as a unit, taking the arrangement of the gray values of all pixels in the neighborhood as a certain transaction in the transaction set, for example, the gray values of 9 pixels in a 3 × 3 neighborhood can form a transaction. Then for 100 x 100 pixel size images, a transaction set consisting of 98 x 98-9604 transactions, each containing 9 entries, may be constructed. The larger the remote sensing image is, the more transactions are formed, and the larger the transaction set is; the more items a transaction contains, the more frequent item sets that need to be calculated, the larger the calculation amount, so it is necessary to limit the number of items each transaction contains. Considering that the edges of the remote sensing image contain a large amount of useful information and have directionality, the invention firstly extracts the edges of the remote sensing image by using a canny operator, then extracts 4 directions of edge point pixels, and takes a gray value of 3 pixels in each direction as an element of one item. For an edge point pixel, the gray-scale values of the pixels in the 4 directions are specifically shown in fig. 2, which is a schematic diagram of the gray-scale values of the pixels in the 4 directions of an edge point pixel.
Because the proportion of the edge point pixels on the remote sensing image in the whole image is small, and each transaction only contains 3 items, the calculation amount can be obviously reduced. Referring to fig. 3(a), fig. 3(b) and fig. 3(c), fig. 3(a) is an original remote sensing image with a size of 128 × 128 pixels; fig. 3(b) is a compressed remote sensing image, each gray level is multiplied by 256/G for display convenience, and as can be seen from the compressed image, although the number of gray levels is reduced to 8, the content of the remote sensing image is not changed much after compression; fig. 3(c) shows an edge detection image, and the number of detected edge points is 1964, so that the size of the transaction set is 1964 × 4 — 7856, and each transaction includes 3 entries. If the terrain in the image is rich, the detected edge points will be more, and the final constructed transaction set will be larger. Table 1 shows a portion of the transactions in the set.
TABLE 1 partial entries in a transaction set
Step 400: extracting association rules in each image through each image transaction set;
in step 400, after each image transaction set is constructed, since each transaction in the transaction set only contains 3 items, a three-dimensional data cube can be constructed from the transaction set, and on the basis, association rules in the image to be retrieved are extracted. The method for extracting the association rule specifically comprises the following steps:
firstly, a frequent item set needs to be calculated; the calculation of the frequent item set is a very critical step in association rule mining, and the calculation amount directly influences the calculation amount of the whole association rule mining. For an item consisting of 3 elements, set as [ a b c ], if the support is not considered, the extracted association rule has 12 pieces shown in the following table 2:
association rule generated by 23 items of table
| a＝>b | a＝>c | b＝>c |
| b＝>a | c＝>a | c＝>b |
| ab＝>c | ac＝>b | bc＝>a |
| a＝>bc | b＝>ac | c＝>ab |
Because the first 6 association rules only relate to the relationship between two elements, but are not enough to express the content of the remote sensing image, and the computation amount of association rule mining and subsequent similarity calculation can be increased, the embodiment of the invention extracts 6 association rules meeting the specified confidence coefficient and support degree from the constructed transaction set by using a frequent item set algorithm for mining association rules such as Apriori or FP-Growth.
For simplicity, the embodiment of the present invention only selects three pixels near the edge point, and the selection can be specifically performed according to the search requirement. Generally speaking, the more pixels are selected, the more items each transaction contains, the more the amount of calculation is, but the richer the physical meaning each association rule shows, the higher the retrieval precision is.
Step 500: calculating the association rule similarity between the image to be retrieved and each image in the image library according to the support degree and the confidence index, and retrieving the images according to the association rule similarity between the image to be retrieved and each image in the image library;
in step 500, the support degree reflects the distribution frequency of the association rule in the remote sensing image, and the confidence degree reflects the credibility of the association rule, so the product of the support degree and the confidence degree can be used to represent the true proportion of the association rule in the remote sensing image.
The similarity calculation method of the association rule comprises the following steps: for a plurality of association rules obtained by a remote sensing image, the product of the support degree and the confidence coefficient of each association rule is combined to form a rule vector, and the rule vector is used for describing the content of the remote sensing image. For remote sensing images with the same or similar contents, the regular vectors of the remote sensing images should be similar.
Meanwhile, the visual characteristics of the human visual system HVS can be described by Weber's law. According to Weber's law, the sensitivity of HVS to relative brightness change is higher than that of absolute brightness change, the average value of remote sensing image reflects the whole feeling of human eyes to the image, and the average value of remote sensing image before brightness change is set as mu1R is the brightness change relative to the background brightness, and the average value of the remote sensing image after the brightness change can be expressed as mu2＝(1+R)μ1Then expression
The overall feeling of the remote sensing image change can be measured as follows:
therefore, the temperature of the molten metal is controlled,
only a function of R, indicating that it is consistent with weber's law, and is able to express the human eye's response to image brightness. When mu is1And mu2The closer the distance is to each other,
the closer to 1, the higher the similarity of the two remote sensing images in brightness.
If the association rules of the images to be retrieved are N, and the association rules of any retrieved image in the image library are M, matching the association rules of the retrieved images in the image library with the N association rules of the images to be retrieved as a reference, wherein the condition that the two association rules are successfully matched is that the front piece and the back piece of the two association rules are respectively the same. If the two association rules are successfully matched, keeping the product of the support degree and the confidence degree of the association rule; if the match is not successful, the product of the support and confidence of the association rule is set to 0. Therefore, the retrieval image can also generate an N-dimensional rule vector, and the image retrieval can be realized by comparing the similarity of the image to be retrieved and the N-dimensional rule vectors of all images in the image library.
In the embodiment of the invention, Kullback-Leibler divergence first-order approximate distance is used as the measurement for measuring the similarity of two regular vectors, and the expression is as follows:
in formula (6), r1 and r2 are two regular vectors respectively, and the final similarity measure index considering the visual characteristics of human eyes is:
if the two regular vectors are closer, and the mean values of the two remote sensing images are closer, the smaller the value of D is, and the higher the similarity is.
Fig. 4 is a schematic structural diagram of a remote sensing image retrieval device with rotation, zoom and translation invariance according to an embodiment of the present invention. The remote sensing image retrieval device for the rotation scaling translation invariance comprises an image transformation module, an image compression module, a transaction set construction module, an association rule extraction module, a similarity calculation module and an image retrieval module.
The image transformation module is used for carrying out Radon and Fourier-Mellin transformation on each image in the image library; the embodiment of the invention eliminates the influence of rotation, scaling and translation of the image by transforming the image with rotation, scaling and translation into another domain, and then extracts the association rule of the image. Firstly, Radon transformation is carried out on an image, and a transformation formula is as follows:
P(r，θ)＝R(r，θ)f(x，y)＝∫∫f(x，y)(r-x cosθ-y sinθ)dxdy (1)
in equation (1), | r | represents the distance from a dot to a line, θ ∈ [0, π ] represents the angle between the line and the y-axis, and (r) is the Dirac function. The Radon transform is to integrate f (x, y) along a line r-x cos θ -y sin θ 0 to obtain a sum of f (x, y) along the line at an arbitrary position (r, θ).
And then carrying out Fourier-Mellin transformation on the image subjected to Radon transformation, wherein the Fourier-Mellin transformation formula is as follows:
in equation (2), u is a real variable, and σ is a real constant greater than 0, and the value thereof is generally 0.5.
The image is subjected to Radon transform and Fourier-Mellin transform, the rotation of the image is converted into phase, and the scaling is converted into amplitude. Finally, the following function is defined:
the function has the same width and height as the original image, so its value can be considered as one image. The function has rotation, scaling invariance. Combining with the association rule, the method has the invariance of rotation, scaling and translation, and performs image retrieval on the basis.
The image compression module is used for carrying out pixel gray level compression on each image after Radon and Fourier-Mellin transformation; because the gray levels are too many, the support degree of the frequent item set is very small, and the extraction of the association rule with the support degree and the confidence degree both being large enough is not facilitated, so before the association rule mining is performed, pixel gray level compression needs to be performed on each image, and each converted image is compressed to a few gray levels, so that the calculation amount of the association rule mining is reduced.
In the embodiment of the present invention, the method for compressing the gray level of the pixel of each transformed image specifically comprises: regarding the transformed function Z (u, k) as an image, regarding the value as a gray value, compressing the image by using the neighborhood mean and variance, calculating the mean μ and standard deviation σ of each pixel in 3 × 3 neighborhood on the image, and then calculating the gray level of the pixel in the neighborhood after compression by using the following formula:
in formula (4), g is the original gray level of the central pixel, g' is the gray level of the compressed central pixel, and c is the scale factor, and the value range is [0.1, 0.5 ]. By the method, the original remote sensing image can be compressed to 3 gray levels of 0,1 and 2.
In another embodiment of the present invention, the gray scale is quantized to the range of [1, G ] by adopting a uniform segmentation manner according to the gray scale, specifically: the average compression method is adopted, all gray levels are evenly distributed into a plurality of gray levels,
wherein maxG is the original maximum gray level, G is the compressed maximum gray level, G is 8, ceil () is an rounding-up function, and G +1 is to compress the gray level of the image to 1-8.
Quantizing each gray level to the range of [1, G ] by adopting a uniform segmentation mode according to the gray levels, specifically, compressing by adopting a linear segmentation method, firstly calculating the maximum gray level gMax and the minimum gray level gMin of the image, and then calculating the compressed gray level by utilizing the following formula:
where G is the maximum gray level, and G is 8.
The more the gray levels are compressed, the larger the calculation amount of association rule mining is, but the closer the reflected relationship between the pixels is to the reality; conversely, the fewer the gray levels after compression, the smaller the difference between the compressed pixels, and the more unfavorable the mining of the meaningful association rules, so that it is very important to select an appropriate gray level number. The maximum gray level number in the embodiment of the invention is selected to be 8, the adopted compression mode is average compression, and the compression formula is as follows:
in the formula (7), maxG is the original maximum gray level, G is the compressed maximum gray level, G is 8, ceil () is an upward rounding function, and G +1 is to compress the pixel gray level of the remote sensing image to 1 to 8.
The transaction set construction module is used for extracting edge point pixels of each transformed image and constructing an image transaction set; the method for constructing the transaction set comprises the following steps: taking a neighborhood as a unit, taking the arrangement of the gray values of all pixels in the neighborhood as a certain transaction in the transaction set, for example, the gray values of 9 pixels in a 3 × 3 neighborhood can form a transaction. Then for 100 x 100 pixel size images, a transaction set consisting of 98 x 98-9604 transactions, each containing 9 entries, may be constructed. The larger the remote sensing image is, the more transactions are formed, and the larger the transaction set is; the more items a transaction contains, the more frequent item sets that need to be calculated, the larger the calculation amount, so it is necessary to limit the number of items each transaction contains. Considering that the edges of the remote sensing image contain a large amount of useful information and have directionality, the invention firstly extracts the edges of the remote sensing image by using a canny operator, then extracts 4 directions of edge point pixels, and takes a gray value of 3 pixels in each direction as an element of one item. For an edge point pixel, the gray-scale values of the pixels in the 4 directions are specifically shown in fig. 2, which is a schematic diagram of the gray-scale values of the pixels in the 4 directions of an edge point pixel.
Because the proportion of the edge point pixels on the remote sensing image in the whole image is small, and each transaction only contains 3 items, the calculation amount can be obviously reduced. Referring to fig. 3(a), fig. 3(b) and fig. 3(c), fig. 3(a) is an original remote sensing image with a size of 128 × 128 pixels; fig. 3(b) is a compressed remote sensing image, each gray level is multiplied by 256/G for display convenience, and as can be seen from the compressed image, although the number of gray levels is reduced to 8, the content of the remote sensing image is not changed much after compression; fig. 3(c) shows an edge detection image, and the number of detected edge points is 1964, so that the size of the transaction set is 1964 × 4 — 7856, and each transaction includes 3 entries. If the terrain in the image is rich, the detected edge points will be more, and the final constructed transaction set will be larger.
The association rule extraction module is used for extracting the association rule in the image through each converted image transaction set; after the image transaction set is constructed, because each transaction in the transaction set only contains 3 items, a three-dimensional data cube can be constructed from the transaction set, and on the basis, association rules in the image are extracted. The method for extracting the association rule specifically comprises the following steps:
firstly, a frequent item set needs to be calculated; the calculation of the frequent item set is a very critical step in association rule mining, and the calculation amount directly influences the calculation amount of the whole association rule mining. For an item consisting of 3 elements, set to [ a bc ], if the support is not considered, the extracted association rule has 12 pieces shown in the following table 2:
association rule generated by 23 items of table
| a＝>b | a＝>c | b＝>c |
| b＝>a | c＝>a | c＝>b |
| ab＝>c | ac＝>b | bc＝>a |
| a＝>bc | b＝>ac | c＝>ab |
Because the first 6 association rules only relate to the relationship between two elements, but are not enough to express the content of the remote sensing image, and the computation amount of association rule mining and subsequent similarity calculation can be increased, the embodiment of the invention extracts 6 association rules meeting the specified confidence coefficient and support degree from the constructed transaction set by using a frequent item set algorithm for mining association rules such as Apriori or FP-Growth.
For simplicity, the embodiment of the present invention only selects three pixels near the edge point, and the selection can be specifically performed according to the search requirement. Generally speaking, the more pixels are selected, the more items each transaction contains, the more the amount of calculation is, but the richer the physical meaning each association rule shows, the higher the retrieval precision is.
The similarity calculation module is used for calculating the similarity of the association rule between the image to be retrieved and the retrieved image in the image library according to the support degree and the confidence index; the support degree reflects the distribution frequency of the association rule in the remote sensing image, and the confidence degree reflects the credibility degree of the association rule, so that the true proportion of the association rule in the remote sensing image can be represented by the product of the support degree and the confidence degree.
The similarity calculation method of the association rule comprises the following steps: for a plurality of association rules obtained by a remote sensing image, the product of the support degree and the confidence coefficient of each association rule is combined to form a rule vector, and the rule vector is used for describing the content of the remote sensing image. For remote sensing images with the same or similar contents, the regular vectors of the remote sensing images should be similar.
Of the simultaneous human visual system HVSVisual characteristics may be described by weber's law. According to Weber's law, the sensitivity of HVS to relative brightness change is higher than that of absolute brightness change, the average value of remote sensing image reflects the whole feeling of human eyes to the image, and the average value of remote sensing image before brightness change is set as mu1R is the brightness change relative to the background brightness, and the average value of the remote sensing image after the brightness change can be expressed as mu2＝(1+R)μ1Then expression
The overall feeling of the remote sensing image change can be measured as follows:
therefore, the temperature of the molten metal is controlled,
only a function of R, indicating that it is consistent with weber's law, and is able to express the human eye's response to image brightness. When mu is1And mu2The closer the distance is to each other,
the closer to 1, the higher the similarity of the two remote sensing images in brightness.
The image retrieval module is used for retrieving images according to the similarity of the association rules of the images to be retrieved and the retrieved images in the image library; if the association rules of the images to be retrieved are N, and the association rules of any retrieved image in the image library are M, matching the association rules of the retrieved images in the image library with the N association rules of the images to be retrieved as a reference, wherein the condition that the two association rules are successfully matched is that the front piece and the back piece of the two association rules are respectively the same. If the two association rules are successfully matched, keeping the product of the support degree and the confidence degree of the association rule; if the match is not successful, the product of the support and confidence of the association rule is set to 0. Therefore, the retrieval image can also generate an N-dimensional rule vector, and the image retrieval can be realized by comparing the similarity of the image to be retrieved and the N-dimensional rule vectors of all images in the image library.
In the embodiment of the invention, Kullback-Leibler divergence first-order approximate distance is used as the measurement for measuring the similarity of two regular vectors, and the expression is as follows:
in equation (9), r1 and r2 are two regular vectors respectively, and the final similarity measure index considering the visual characteristics of human eyes is:
if the two regular vectors are closer, and the mean values of the two remote sensing images are closer, the smaller the value of D is, and the higher the similarity is.
In order to verify the effectiveness of the invention, the following remote sensing image retrieval experiment was carried out:
a total of 23 remote sensing images (with the size of 300 x 300) are selected from the Quickbird and WorldView-2 image libraries as reference images, and specifically, as shown in FIG. 5, a schematic diagram of 23 remote sensing images is provided. Wherein Q1-Q8 represent QuickBird images, W1-W15 represent WorldView-2 images, and the image contents include roads, sparse forest lands, residential lands, dense forest lands, open lands, water areas, grasslands, farmlands, and the like.
Rotary remote sensing image retrieval experiment
The 23 reference images are rotated clockwise and rotated every 15 degrees to obtain 552 images, and then sub-images with the size of 128 × 128 pixels are cut from the center of the 552 images to serve as a final image library. The remote sensing image retrieval method of the rotation scaling translation invariance is compared with retrieval results of the existing autocorrelation function method, Gabor Wavelet Transform, DT-CWT (Dual-Tree Complex Wavelet Transform) and NSCT retrieval methods. Setting the support degree to be 0.015 and the confidence degree to be 0.3, extracting association rules of all images, then inputting an image to be retrieved, calculating the similarity between the image to be retrieved and the association rules of all images in the image library, and taking the first 24 images as retrieval results according to the similarity. The average precision ratios of the top 5, top 10, top 15, top 20 images and all the retrieval results in the returned results of the images of the various retrieval methods and the overall average precision ratio of all the images are respectively counted, specifically, as shown in fig. 6 to 11, fig. 6 is the average precision ratio of the top 5 of the ranking of the various retrieval methods, fig. 7 is the average precision ratio of the top 10 of the ranking of the various retrieval methods, fig. 8 is the average precision ratio of the top 15 of the ranking of the various retrieval methods, fig. 9 is the average precision ratio of the top 20 of the ranking of the various retrieval methods, fig. 10 is the average precision ratio of all the retrieval results of the various retrieval methods, and fig. 11 is the overall average precision ratio of all the retrieval methods. As can be seen from FIG. 11, the remote sensing image retrieval method with rotational scaling and translational invariance has an average precision ratio of 99.77%, while the autocorrelation function method, Gabor wavelet transform, DT-CWT and NSCT have average precision ratios of only 32.71%, 83.38%, 66.94% and 71.74%. The remote sensing image retrieval method for the rotational scaling translational invariance has very high retrieval precision.
The remote sensing image retrieval method and the remote sensing image retrieval device of the rotation scaling translation invariance firstly carry out Radon transformation and Fourier-Mellin transformation on a remote sensing image, and eliminate the influence of image rotation, scaling and translation; pixel gray level compression is carried out on the image after Radon and Fourier-Mellin transformation, and the calculation amount of association rule mining is reduced; by constructing a transaction set, extracting association rules of the remote sensing image through the transaction set, and then comparing similarity between the association rules, the retrieval of the remote sensing image is realized; the invention provides a new feasible way for realizing remote sensing image retrieval based on association rules from low-level visual features to high-level semantic information, and improves the accuracy of remote sensing image retrieval.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (8)
1. A remote sensing image retrieval method of rotation scaling translation invariance is characterized by comprising the following steps:
step a: after image transformation processing is carried out on each image in the image library, edge point pixels of each image are extracted, and each image transaction set is constructed;
step b: extracting association rules in each image through the image transaction set;
step c: calculating the association rule similarity between the image to be retrieved and the retrieved image in the image library according to the support degree and the confidence index, and performing image retrieval according to the association rule similarity between the image to be retrieved and each image in the image library:
in the step c, the association rule similarity is calculated in the following manner: for all association rules extracted from each image, combining the products of the support degree and the confidence degree of each association rule to form a rule vector; the image retrieval is realized by comparing the similarity of the regular vector of the image to be retrieved and each image in the image library; the measurement formula of the similarity of the regular vectors is as follows:
wherein N is the number of associated rules of the image, r1 and r2 are two rule vectors, μ1And mu2The average value of the two images is obtained, and if the two regular vectors are closer and the average values of the two images are closer, the smaller the value of D is, and the higher the similarity between the two images is.
2. The method for retrieving a remotely sensed image with rotational zoom translational invariance as claimed in claim 1, wherein in said step a, said image transformation processing of each image in the image library specifically comprises: carrying out Radon transformation and Fourier-Mellin transformation on each image, wherein the Radon transformation formula is as follows:
P(r，θ)＝R(r，θ)f(x，y)＝∫∫f(x，y)(r-xcosθ-ysinθ)dxdy
wherein, f (x, y) represents the image to be transformed, r represents the distance from a circular point to a straight line, theta belongs to [0, pi ] represents the included angle between the straight line and the y axis, and (r) is a Dirac function;
the Fourier-Mellin transformation formula is as follows:
where u is a real variable, σ is a real constant greater than 0, i represents an imaginary unit, and k represents a circular frequency.
3. The method for retrieving a remotely sensed image with rotational scaling translational invariance as claimed in claim 2, wherein in said step a, after each image is subjected to Radon transform and Fourier-Mellin transform, the rotation of the image is converted into phase, scaling is converted into amplitude, and the function after Radon transform and Fourier-Mellin transform is:
wherein the function Z (u, k) has the same width and height as the original image, and the function has rotation and scale invariance, where i represents the imaginary unit and k represents the circular frequency.
4. The method for retrieving remotely sensed images of rotational zoom translational invariance as set forth in claim 3, wherein said step a further comprises: and carrying out pixel gray level compression on each image after Radon transformation and Fourier-Mellin transformation.
5. A remote sensing image retrieval apparatus for rotational zoom translational invariance, comprising:
the image transformation module: the image conversion device is used for carrying out image conversion processing on each image in the image library;
the transaction set building module: the system is used for extracting edge point pixels of each image and constructing a transaction set of each image;
an association rule extraction module: the association rule is used for extracting the association rule in each image through the transaction set of each image;
a similarity calculation module: the image retrieval system is used for calculating the association rule similarity between the image to be retrieved and each image in the image library according to the support degree and the confidence index;
the image retrieval module: the image retrieval system is used for retrieving images according to the association rule similarity between the images to be retrieved and each image in the image library;
the similarity calculation module calculates the similarity of the association rule in a calculation mode that: for each association rule extracted from each image in the image library, combining the products of the support degree and the confidence coefficient of each association rule to form a rule vector; the image retrieval is realized by comparing the similarity of the regular vector of the image to be retrieved and each image in the image library; the measurement formula of the similarity of the regular vectors is as follows:
wherein N is the number of associated rules of the image, r1 and r2 are two rule vectors, μ1And mu2The average value of the two images is obtained, and if the two regular vectors are closer and the average values of the two images are closer, the smaller the value of D is, and the higher the similarity between the two images is.
6. The remote sensing image retrieval device for rotational scaling translational invariance according to claim 5, wherein the image transformation module performs image transformation processing on each image in the image library specifically as follows: carrying out Radon transformation and Fourier-Mellin transformation on each image; the Radon transform formula is:
P(r，θ)＝R(r，θ)f(x，y)＝∫∫f(x，y)(r-xcosθ-ysinθ)dxdy
wherein f (x, y) represents an image to be transformed, r represents the distance from a circular point to a straight line, (x, y) represents pixel coordinates, theta epsilon [0, pi ] represents an included angle between the straight line and a y axis, and (r) is a Dirac function;
the Fourier-Mellin transformation formula is as follows:
where u is a real variable, σ is a real constant greater than 0, i represents an imaginary unit, and k represents a circular frequency.
7. The apparatus of claim 6, wherein after each image is subjected to Radon transformation and Fourier-Mellin transformation, the rotation of the image is transformed into phase, and the scaling is transformed into amplitude, and the function of the Radon transformation and the Fourier-Mellin transformation is:
wherein the function Z (u, k) has the same width and height as the original image, and the function has rotation and scale invariance, where i represents the imaginary unit and k represents the circular frequency.
8. A remote sensing image retrieval apparatus for rotational scaling translational invariance as recited in claim 7, further comprising an image compression module for performing pixel grayscale compression on each image after Radon transform and Fourier-Mellin transform.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610950697.8A CN106570125B (en) | 2016-11-02 | 2016-11-02 | Remote sensing image retrieval method and device for rotational scaling and translation invariance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610950697.8A CN106570125B (en) | 2016-11-02 | 2016-11-02 | Remote sensing image retrieval method and device for rotational scaling and translation invariance |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106570125A CN106570125A (en) | 2017-04-19 |
| CN106570125B true CN106570125B (en) | 2020-08-25 |
Family
ID=58535131
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610950697.8A Active CN106570125B (en) | 2016-11-02 | 2016-11-02 | Remote sensing image retrieval method and device for rotational scaling and translation invariance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106570125B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110276240B (en) * | 2019-03-28 | 2021-05-28 | 北京市遥感信息研究所 | SAR image building wall window information extraction method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101930608A (en) * | 2010-08-26 | 2010-12-29 | 北京交通大学 | Method and system for blindly detecting tampered image |
| CN102073748A (en) * | 2011-03-08 | 2011-05-25 | 武汉大学 | Visual keyword based remote sensing image semantic searching method |
| CN104463200A (en) * | 2014-11-27 | 2015-03-25 | 西安空间无线电技术研究所 | Satellite remote sensing image sorting method based on rule mining |
-
2016
- 2016-11-02 CN CN201610950697.8A patent/CN106570125B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101930608A (en) * | 2010-08-26 | 2010-12-29 | 北京交通大学 | Method and system for blindly detecting tampered image |
| CN102073748A (en) * | 2011-03-08 | 2011-05-25 | 武汉大学 | Visual keyword based remote sensing image semantic searching method |
| CN104463200A (en) * | 2014-11-27 | 2015-03-25 | 西安空间无线电技术研究所 | Satellite remote sensing image sorting method based on rule mining |
Non-Patent Citations (2)
| Title |
|---|
| 基于关联规则挖掘的图像检索;周易;《软件》;20120430;第33卷(第4期);第I138-1206页 * |
| 基于语义的图像检索相关问题研究;袁本刚;《中国优秀硕士学位论文全文数据库 信息科技辑》;20090815;第2009卷(第8期);第28-30页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106570125A (en) | 2017-04-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Elaiwat et al. | A curvelet-based approach for textured 3D face recognition | |
| US8686992B1 (en) | Methods and systems for 3D shape matching and retrieval | |
| CN101937549B (en) | Picture guidance system for network shopping guidance | |
| CN103927511B (en) | image identification method based on difference feature description | |
| Yang et al. | An improved Bag-of-Words framework for remote sensing image retrieval in large-scale image databases | |
| CN101950400B (en) | Picture retrieving method of network shopping guiding method | |
| CN103077512A (en) | Feature extraction and matching method and device for digital image based on PCA (principal component analysis) | |
| WO2015096676A1 (en) | System and method for retrieval based on perspective image content and security check device | |
| Wang et al. | Image retrieval based on exponent moments descriptor and localized angular phase histogram | |
| CN106844739A (en) | A kind of Remote Sensing Imagery Change information retrieval method based on neutral net coorinated training | |
| CN105183795B (en) | Remote Sensing Imagery Change Detection information retrieval method based on content | |
| Jin et al. | Content-based image retrieval based on shape similarity calculation | |
| Etemad et al. | Color texture image retrieval based on Copula multivariate modeling in the Shearlet domain | |
| Liu et al. | RGB-D action recognition using linear coding | |
| Akram et al. | A comparative study on face sketch synthesis | |
| Li et al. | Texture analysis of remote sensing imagery with clustering and Bayesian inference | |
| Dandotiya et al. | Image retrieval using edge detection, RLBP, color moment method for YCbCr and HSV color space | |
| Hajati et al. | Surface geodesic pattern for 3D deformable texture matching | |
| CN106570125B (en) | Remote sensing image retrieval method and device for rotational scaling and translation invariance | |
| Xi et al. | Finger vein recognition based on the hyperinformation feature | |
| Gupta et al. | Combination of local, global and k-mean using wavelet transform for content base image retrieval | |
| CN106570127B (en) | Remote sensing image retrieval method and system based on object attribute association rule | |
| CN105955473A (en) | Computer-based static gesture image recognition interactive system | |
| CN106570124B (en) | Remote sensing images semantic retrieving method and system based on object level correlation rule | |
| CN107871110B (en) | A kind of pedestrian target recognition methods based on Multi-scale model correlation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |