CN117011147B - Infrared remote sensing image feature detection and splicing method and device - Google Patents
Infrared remote sensing image feature detection and splicing method and device Download PDFInfo
- Publication number
- CN117011147B CN117011147B CN202311285977.8A CN202311285977A CN117011147B CN 117011147 B CN117011147 B CN 117011147B CN 202311285977 A CN202311285977 A CN 202311285977A CN 117011147 B CN117011147 B CN 117011147B
- Authority
- CN
- China
- Prior art keywords
- image
- scale
- images
- spliced
- log
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000001514 detection method Methods 0.000 title claims description 12
- 239000011159 matrix material Substances 0.000 claims abstract description 72
- 230000009466 transformation Effects 0.000 claims abstract description 23
- 230000010354 integration Effects 0.000 claims description 37
- 238000004590 computer program Methods 0.000 claims description 17
- 238000003860 storage Methods 0.000 claims description 17
- 238000001914 filtration Methods 0.000 claims description 15
- 238000000605 extraction Methods 0.000 claims description 12
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000005520 cutting process Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 11
- 230000006872 improvement Effects 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000032669 eclosion Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 229920001296 polysiloxane Polymers 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012887 quadratic function Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 239000010979 ruby Substances 0.000 description 1
- 229910001750 ruby Inorganic materials 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
Abstract
The specification discloses a method and a device for detecting and splicing characteristics of infrared remote sensing images, which can acquire images to be spliced, wherein the images to be spliced are infrared images, the images to be spliced are subjected to Fourier transform, two-dimensional Log-Gabor filters with a plurality of scales and directions are constructed, the transformed images are filtered through the two-dimensional Log-Gabor filters to obtain filtered images, so as to obtain a scale space, the scale space is divided into scale subspaces, and each characteristic point in the images to be spliced is determined according to a scale integral space corresponding to each scale subspace. And then, based on each characteristic point in each image to be spliced, carrying out characteristic point matching, generating a similar transformation matrix, determining a splicing path, and splicing each image to be spliced to obtain an infrared remote sensing image splicing result.
Description
Technical Field
The present disclosure relates to the field of image stitching technologies, and in particular, to a method and an apparatus for detecting and stitching features of an infrared remote sensing image.
Background
The temperature information contained in the infrared remote sensing image can sense the difference of heat radiation energy emitted by the ground object, and has important application value. Image stitching can combine multiple overlapping images together to form a large image with a wide field of view. By splicing the infrared remote sensing images, panoramic images of a research area can be obtained, and panoramic images with larger visual fields can sense more comprehensive temperature information so as to promote further research and application, such as urban thermal environment and geological drawing.
However, the characteristics are difficult to extract due to low contrast ratio and poor signal-to-noise ratio of the infrared image, so that the splicing effect is affected. Most of the infrared stitching technologies at present are based on feature point extraction and matching, and align images by searching for common feature points in adjacent images, and the process of the technology generally comprises feature point extraction, feature point matching and image transformation. However, the traditional SIFT, SURF, ORB, BRISK-based feature extraction method has poor effect on the infrared low-contrast remote sensing image, has few extracted feature point data, and has a large number of mismatching invalid feature points, so that the infrared image is seriously spliced and deformed or fails to splice, and the method is not suitable for the infrared image.
Therefore, how to generate more accurate infrared stitched images is a problem to be solved. Therefore, the specification provides an effective infrared remote sensing image feature detection and splicing method.
Disclosure of Invention
The specification provides a method and a device for detecting and splicing characteristics of infrared remote sensing images, which are used for partially solving the problems existing in the prior art.
The technical scheme adopted in the specification is as follows:
the specification provides an infrared remote sensing image feature detection and splicing method, which comprises the following steps:
acquiring images to be spliced, wherein the images to be spliced are infrared images;
performing Fourier transform on each image to be spliced to obtain a transformed image, constructing a plurality of two-dimensional Log-Gabor filters, and filtering the transformed image through the two-dimensional Log-Gabor filters to obtain each filtered image, wherein the scales or directions corresponding to different two-dimensional Log-Gabor filters are different;
obtaining a Log-Gabor scale space according to each filtered image, dividing the Log-Gabor scale space into scale subspaces, determining a scale integration space corresponding to each scale subspace according to each scale subspace, and determining each characteristic point in the image to be spliced according to the scale integration space corresponding to each scale subspace;
Based on each characteristic point in each image to be spliced, carrying out characteristic point matching between the images to be spliced to obtain a characteristic point matching result, generating a similar transformation matrix, and determining a splicing path according to the characteristic point matching result;
and determining an optimal suture line, and splicing the images to be spliced according to the optimal suture line through the splicing path and the similar transformation matrix to obtain an infrared remote sensing image splicing result.
Optionally, acquiring each image to be spliced specifically includes:
preprocessing each original infrared image to obtain each image to be spliced, wherein the preprocessing comprises the following steps: performing histogram stretching on the original infrared image; the image after the histogram stretching is rotated to the horizontal direction by the center of the image; and cutting the rotated image to obtain an image to be spliced.
Optionally, constructing a plurality of two-dimensional Log-Gabor filters with preset scales and directions, which specifically includes:
selecting 7 scales and 4 directions to form 28 two-dimensional Log-Gabor filters;
filtering the transformed image through the plurality of two-dimensional Log-Gabor filters to obtain each filtered image, wherein the method specifically comprises the following steps of:
And filtering the transformed images through the 28 two-dimensional Log-Gabor filters to obtain 28 filtered images.
Optionally, dividing the Log-Gabor scale space into scale subspaces specifically includes:
and grouping all the filtered images contained in the Log-Gabor scale space according to the scale to obtain all the scale subspaces, wherein one scale subspace contains the filtered images with the same scale.
Optionally, for each scale subspace, determining a scale integration space corresponding to the scale subspace specifically includes:
and adding pixel values of the filtered images contained in each scale subspace to obtain a scale integration space corresponding to the scale subspace.
Optionally, determining each feature point in the image to be spliced according to a scale integration space corresponding to each scale subspace specifically includes:
obtaining a hessian matrix through a scale integration space corresponding to each scale subspace;
determining a comparison result of each point and an adjacent point in a three-dimensional matrix formed by the Heisen matrixes, and determining whether the point is a candidate feature point according to the comparison result;
And determining a local maximum value corresponding to the local matrix by the local matrix of each candidate feature point to obtain the feature point.
Optionally, determining the splicing path according to the feature point matching result specifically includes:
setting the total number of the images to be spliced as N, constructing an N multiplied by N matrix S, if the number of the feature points matched with the image i and the image j is greater than or equal to a preset value, considering that the matching is successful, and setting the element S (i, j) in the matrix as the reciprocal of the number of the matching points; if the number of the matching points of the image i and the image j is smaller than the preset value, setting S (i, j) as infinity;
and determining a splicing path according to the matrix S.
The specification provides an infrared remote sensing image feature detection and splicing apparatus, including:
the acquisition module is used for acquiring each image to be spliced, wherein each image to be spliced is an infrared image;
the filtering module is used for carrying out Fourier transform on each image to be spliced to obtain a transformed image, constructing a plurality of two-dimensional Log-Gabor filters, and filtering the transformed image through the two-dimensional Log-Gabor filters to obtain each filtered image, wherein the scales or directions corresponding to different two-dimensional Log-Gabor filters are different;
Obtaining a Log-Gabor scale space according to each filtered image, dividing the Log-Gabor scale space into scale subspaces, determining a scale integration space corresponding to each scale subspace according to each scale subspace, and determining each characteristic point in the image to be spliced according to the scale integration space corresponding to each scale subspace;
based on each characteristic point in each image to be spliced, carrying out characteristic point matching between the images to be spliced to obtain a characteristic point matching result, generating a similar transformation matrix, and determining a splicing path according to the characteristic point matching result;
and determining an optimal suture line, and splicing the images to be spliced according to the optimal suture line through the splicing path and the similar transformation matrix to obtain an infrared remote sensing image splicing result.
The present disclosure provides a computer readable storage medium storing a computer program which, when executed by a processor, implements the above-described infrared remote sensing image feature detection and stitching method.
The present disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the method for detecting and stitching features of an infrared remote sensing image when executing the program.
The above-mentioned at least one technical scheme that this specification adopted can reach following beneficial effect:
according to the method, each image to be spliced is an infrared image, fourier transformation is carried out on each image to be spliced to obtain a transformed image, a plurality of two-dimensional Log-Gabor filters are constructed, the transformed image is filtered through the plurality of two-dimensional Log-Gabor filters to obtain each filtered image, scales or directions corresponding to different two-dimensional Log-Gabor filters are different, a Log-Gabor scale space is obtained according to each filtered image, the Log-Gabor scale space is divided into scale subspaces, a scale integral space corresponding to the scale subspace is determined for each scale subspace, and feature points in the image to be spliced are determined according to the scale integral space corresponding to each scale subspace. And then, based on each characteristic point in each image to be spliced, carrying out characteristic point matching between the images to be spliced to obtain a characteristic point matching result, generating a similar transformation matrix, determining a splicing path through the characteristic point matching result, finally, determining an optimal suture line, and splicing the images to be spliced according to the optimal suture line through the splicing path and the similar transformation matrix to obtain an infrared remote sensing image splicing result.
From the above, it can be seen that, in the method, images expressing texture features under multiple scales and directions are obtained through multiple Log-Gabor filters, images under the same scale are combined to obtain a scale integration space, and feature points in an original image to be spliced are determined through each scale integration space, so that feature matching and subsequent image splicing are performed through the extracted feature points.
The contrast ratio of the infrared image is poor, the traditional method is difficult for extracting the characteristics of the infrared image, but the method using the Log-Gabor filter in the method can construct a plurality of scaled and oriented Log-Gabor scale spaces by using a Log-Gabor function without depending on the image intensity during characteristic extraction, so that the image characteristics of the infrared image can be effectively extracted.
Drawings
The accompanying drawings, which are included to provide a further understanding of the specification, illustrate and explain the exemplary embodiments of the present specification and their description, are not intended to limit the specification unduly. In the drawings:
fig. 1 is a schematic flow chart of an infrared remote sensing image feature detection and stitching method provided in the present specification;
FIG. 2 is a schematic illustration of one of the various dimensions of the subspace provided herein;
FIG. 3 is a schematic diagram of a stitching path between images to be stitched provided in the present disclosure;
fig. 4 is a schematic diagram of an infrared remote sensing image feature detection and stitching device provided in the present specification;
fig. 5 is a schematic view of the electronic device corresponding to fig. 1 provided in the present specification.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the present specification more apparent, the technical solutions of the present specification will be clearly and completely described below with reference to specific embodiments of the present specification and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present specification. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present disclosure.
The following describes in detail the technical solutions provided by the embodiments of the present specification with reference to the accompanying drawings.
Fig. 1 is a schematic flow chart of an infrared remote sensing image feature detection and stitching method provided in the present specification, which specifically includes the following steps:
s100: and acquiring each image to be spliced, wherein each image to be spliced is an infrared image.
S102: and carrying out Fourier transform on each image to be spliced to obtain a transformed image, constructing a plurality of two-dimensional Log-Gabor filters, and filtering the transformed image through the two-dimensional Log-Gabor filters to obtain each filtered image, wherein the scales or directions corresponding to different two-dimensional Log-Gabor filters are different.
In the present specification, a plurality of infrared images acquired for a certain area need to be spliced to obtain a panoramic image of the area, where the area may be referred to as a region, and the obtained panoramic image may be applied to multiple scenes such as geological drawing, urban thermal environment research, and the like.
Based on the above, the server can obtain each image to be spliced, wherein each image to be spliced is an infrared image.
It should be noted that, each image to be spliced may be an infrared image subjected to pretreatment, and the original infrared image may be subjected to histogram stretching; the image after the histogram stretching is rotated to the horizontal direction by the center of the image; and cutting the rotated images to obtain the images to be spliced. The pretreatment is to strengthen the infrared image, adjust the angle of the infrared image and the like, so as to facilitate the subsequent feature extraction.
Then, for each image to be spliced, fourier transformation can be performed on the image to be spliced to obtain a transformed image, a plurality of two-dimensional Log-Gabor filters are constructed, the transformed image is filtered through the two-dimensional Log-Gabor filters, and each filtered image is obtained, and scales or directions corresponding to different two-dimensional Log-Gabor filters are different.
The method comprises the steps of carrying out Fourier transform on an image to be spliced to convert the image into a frequency domain, and then filtering the transformed image through a plurality of Log-Gabor filters, namely, convolving the transformed image with a two-dimensional Log-Gabor filter to obtain a frequency image with characteristics transformed, and taking the frequency image as each filtered image.
The above-mentioned plurality of Log-Gabor filters may be constructed by presetting a plurality of scales and directions, for example, 7 scales and 4 directions may be selected to form 28 two-dimensional Log-Gabor filters in total, that is, 7×4= 28,7 scales and 4 directions may be preset manually, and it should be noted that the scales and directions are parameters required by the two-dimensional Log-Gabor filters. The transformed images were filtered by 28 two-dimensional Log-Gabor filters to obtain 28 filtered images.
The scale parameter may adjust the frequency range of the filter by changing the value of the center frequency. Smaller dimensions will result in a higher center frequency and vice versa. Thus, the scale is used to control how sensitive the filter is to textures of different frequencies in the image. The direction is similar, and the sensitivity degree of the filter to textures in different directions in the image can be adjusted by the direction parameters, so that the feature extraction method used by the method is irrelevant to the intensity of the image, and features in more aspects of the image can be extracted.
S104: according to each filtered image, a Log-Gabor scale space is obtained, the Log-Gabor scale space is divided into scale subspaces, a scale integration space corresponding to each scale subspace is determined according to each scale subspace, and each feature point in the image to be spliced is determined according to the scale integration space corresponding to each scale subspace.
And then, according to each filtered image, a Log-Gabor scale space is obtained, the Log-Gabor scale space is divided into scale subspaces, a scale integration space corresponding to each scale subspace is determined according to each scale subspace, and according to the scale integration space corresponding to each scale subspace, each characteristic point in the image to be spliced is determined.
The filtered images can be subjected to inverse Fourier transform to obtain a Log-Gabor scale space. If 7 kinds of images exist according to the scales and 4 kinds of images exist in the directions, 28 images are contained in the Log-Gabor scale space, and the images corresponding to the 7 kinds of scales coexist.
The scale subspace mentioned above may refer to a space corresponding to each scale included in the Log-Gabor scale space, that is, the images under each scale included in the Log-Gabor scale space may form a scale subspace, and the scale integration space corresponding to one scale subspace may refer to the integral feature of images in different directions under one scale obtained by overlapping the images in the scale subspace in a preset manner.
FIG. 2 is a schematic diagram of a subspace of various dimensions provided herein.
It can be seen in fig. 2 that every fourth image constitutes a scale subspace, i.e. four images in parallel in fig. 2 constitute a scale subspace.
Specifically, each image included in the Log-Gabor scale space may be grouped according to the scale to obtain each scale subspace, one scale subspace includes images corresponding to the Log-Gabor filters with the same scale, and then, for each scale subspace, pixel values of the images included in the scale subspace may be added to obtain a scale integral space corresponding to the scale subspace.
It can be understood that the final scale integration space is an image, and still illustrated according to the 4 directions of 7 scales, the obtained filtered images are 28, then, the obtained Log-Gabor scale space is subjected to inverse fourier transform, and then, the images are grouped into scale subspaces, 7 scale subspaces are total, and 4 images exist in each scale subspace.
The scale integration space obtained in the above manner may be obtained by superimposing these 4 images, and thus, it can be understood that the scale integration space is one image.
After determining each scale integration space, each feature point in the image to be spliced can be determined through each scale integration space, and specifically, each hessian matrix can be obtained through the scale integration space corresponding to each scale subspace (namely, the hessian matrix corresponding to each scale integration space is determined).
When determining the hessian matrix corresponding to each scale integration space, the gray values of all pixels in the rectangle of the diagonal line formed by each pixel value and the origin of coordinates in the scale integration space can be added by a box filter approximate to laplace (Laplacian of Gaussian, loG) to obtain the hessian matrix.
And then, determining a comparison result of each point and the adjacent points according to each point in the three-dimensional matrix formed by the Hessen matrixes, determining whether the point is a candidate feature point according to the comparison result, and determining a local maximum value corresponding to the local matrix according to the local matrix of each candidate feature point to obtain the feature point.
The feature points that are not candidates are not directly selected as feature points, and are more accurate points in a local area centered on the candidate feature points are selected as feature points.
In determining candidate feature points, for each point in the matrix (points on the matrix boundary may be disregarded), a 3 x 3 local matrix centered on that point may be determined, and compares the point with each neighboring point (26 neighboring points) in the 3 x 3 local matrix, if the point is larger than all the adjacent points, the point can be used as a candidate feature point.
Then, a 3 x 3 local matrix centered on the point can be fitted to a three-dimensional quadratic function, thereby obtaining the local maximum corresponding to the local matrix, and the sub-pixel where the local maximum value is located is used as the characteristic point, compared with the mode that the candidate characteristic point is directly used as the characteristic point, the mode has higher precision.
Further, it is assumed that the image to be stitched is 256×256, the log-Gabor scale space includes 28 images 256×256, the scale subspace includes 4 images 256×256, and the scale integration space corresponding to one scale subspace can be understood as a matrix 256×256, and then, the hessian matrix corresponding to each scale integration space can be obtained, and the hessian matrix corresponding to each scale integration space can be combined to obtain a three-dimensional matrix 256×256×7.
Then, for each point in the 256X 7 three-dimensional matrix (points on the matrix side may not be considered), a 3 x 3 local matrix centered on the point can be determined and, thereafter, and comparing the local matrix with the other 26 (3 multiplied by 3-1) adjacent points to determine whether the point is a candidate characteristic point, and then fitting the local maximum value through the local matrix to obtain the characteristic point with sub-pixel precision and position.
S106: and carrying out characteristic point matching between the images to be spliced based on the characteristic points in each image to be spliced to obtain characteristic point matching results, generating a similar transformation matrix, and determining a splicing path according to the characteristic point matching results.
S108: and determining an optimal suture line, and splicing the images to be spliced according to the optimal suture line through the splicing path and the similar transformation matrix to obtain an infrared remote sensing image splicing result.
After determining the characteristic points in each image to be spliced, the server can perform characteristic point matching between the images to be spliced based on the characteristic points in each image to be spliced to obtain a characteristic point matching result, generate a similar transformation matrix, determine a splicing path through the characteristic point matching result, further determine an optimal suture line, splice the images to be spliced according to the optimal suture line, the determined splicing path and the similar transformation matrix, and obtain an infrared remote sensing image splicing result.
The server may determine, for each image to be spliced, a feature point descriptor of each feature point in the image to be spliced, specifically may perform feature description on the extracted feature points based on a KAZE description method, calculate a main direction of the feature points, calculate a descriptor for one feature point, and obtain a 64-dimensional feature descriptor as a feature point descriptor of the feature point.
And then, feature descriptors based on the feature points can be used for matching the feature points of the images to be spliced, so that a feature point matching result is obtained.
And obtaining a similar transformation matrix through the characteristic point matching result, wherein the similar transformation matrix is used for adjusting the same visual field among the images to be spliced.
Then, a stitching path between images may be determined, as shown in fig. 3.
Fig. 3 is a schematic diagram of a stitching path between images to be stitched provided in the present disclosure.
In fig. 3, each circle represents an image, black circles represent images closest to the center of the measurement area, and it can be seen that the connection lines between circles represent paths between images, and in fig. 3, all the connection lines between circles can represent an overall stitching path, and when the connection lines between circles exist, the connection lines between the circles represent that the images corresponding to the circles are stitched preferentially.
If the number of feature points matched with the image i and the image j is greater than or equal to a preset value (such as 10), the matching is considered successful, and the element S (i, j) of the S in the matrix is set as the reciprocal of the number of matching points; if the number of matching points between the image i and the image j is smaller than a preset value (e.g. 10), setting S (i, j) to infinity, and determining a splicing path according to the matrix S.
Through the matrix S, the feature matching relation between every two images can be obtained. The image closest to the physical center of the area can be selected from the images to be spliced as the optimal reference image, and then, the splicing path among the images to be spliced can be searched out through the matrix S from the optimal reference image.
Specifically, if there are more matched feature points between two images, the path between the two images can be preferentially generated, and according to this principle, the overall splicing path between the images to be spliced can be generated through the matrix S.
And finally, when the images are spliced, splicing the images to be spliced through the splicing path, the similar transformation matrix and the optimal suture line by using an eclosion fusion algorithm to generate an infrared remote sensing image splicing result.
For convenience of description, the execution body for executing the method will be described as a server, and the execution body may be a desktop computer, a server, a large-sized service platform, or the like, which is not limited herein.
From the above, it can be seen that, in the method, images expressing texture features under multiple scales and directions are obtained through multiple Log-Gabor filters, images under the same scale are combined to obtain a scale integration space, and feature points in an original image to be spliced are determined through each scale integration space, so that feature matching and subsequent image splicing are performed through the extracted feature points.
The contrast ratio of the infrared image is poor, the traditional method is difficult for extracting the characteristics of the infrared image, but the method using the Log-Gabor filter in the specification can construct a plurality of scaled and oriented Log-Gabor scale spaces by using a Log-Gabor function without depending on the image intensity during characteristic extraction, so that the image characteristics of the infrared image can be effectively extracted.
The above provides the method for detecting and splicing the characteristics of the infrared remote sensing image for one or more embodiments of the present disclosure, and based on the same concept, the present disclosure also provides the device for detecting and splicing the characteristics of the infrared remote sensing image, as shown in fig. 4.
Fig. 4 is a schematic diagram of an infrared remote sensing image feature detection and stitching device provided in the present specification, including;
the acquisition module 401 is configured to acquire each image to be stitched, where each image to be stitched is an infrared image;
the filtering module 402 is configured to perform fourier transform on each image to be spliced to obtain a transformed image, construct a plurality of two-dimensional Log-Gabor filters, and filter the transformed image through the plurality of two-dimensional Log-Gabor filters to obtain each filtered image, where the scales or directions corresponding to the different two-dimensional Log-Gabor filters are different;
The feature extraction module 403 is configured to obtain Log-Gabor scale space according to each filtered image, divide the Log-Gabor scale space into scale subspaces, determine a scale integration space corresponding to the scale subspace for each scale subspace, and determine each feature point in the image to be spliced according to the scale integration space corresponding to each scale subspace;
the matching module 404 is configured to perform feature point matching between the images to be spliced based on the feature points in each image to be spliced, obtain a feature point matching result, generate a similar transformation matrix, and determine a splicing path according to the feature point matching result;
and the stitching module 405 is configured to determine an optimal stitching line, and stitch each image to be stitched through the stitching path and the similar transformation matrix according to the optimal stitching line, so as to obtain an infrared remote sensing image stitching result.
Optionally, the obtaining module 401 is specifically configured to perform preprocessing on each original infrared image to obtain each image to be spliced, where the preprocessing includes: performing histogram stretching on the original infrared image; the image after the histogram stretching is rotated to the horizontal direction by the center of the image; and cutting the rotated image to obtain an image to be spliced.
Optionally, the filtering module 402 is specifically configured to select 7 scales and 4 directions to form 28 two-dimensional Log-Gabor filters in total; and filtering the transformed images through the 28 two-dimensional Log-Gabor filters to obtain 28 filtered images.
Optionally, the feature extraction module 403 is specifically configured to group each image included in the Log-Gabor scale space according to a scale, so as to obtain each scale subspace, where one scale subspace includes images corresponding to Log-Gabor filters with the same scale.
Optionally, the feature extraction module 403 is specifically configured to add, for each scale subspace, pixel values of an image included in the scale subspace to obtain a scale integration space corresponding to the scale subspace.
Optionally, the feature extraction module 403 is specifically configured to obtain each hessian matrix through a scale integration space corresponding to each scale subspace; determining a comparison result of each point and an adjacent point in a three-dimensional matrix formed by the Heisen matrixes, and determining whether the point is a candidate feature point according to the comparison result; and determining a local maximum value corresponding to the local matrix by the local matrix of each candidate feature point to obtain the feature point.
Optionally, the matching module 404 is specifically configured to set the total number of the images to be spliced as N, construct an n×n matrix S, and if the number of feature points that the image i matches the image j is greater than or equal to a preset value, consider that the matching is successful, and set an element S (i, j) of S in the matrix as the reciprocal of the number of matching points; if the number of the matching points of the image i and the image j is smaller than the preset value, setting S (i, j) as infinity; and determining a splicing path according to the matrix S.
The present disclosure also provides a computer readable storage medium storing a computer program, where the computer program is configured to perform the above-described method for detecting and stitching features of an infrared remote sensing image.
The present specification also provides a schematic structural diagram of the electronic device shown in fig. 5. At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile storage, as illustrated in fig. 5, although other hardware required by other services may be included. The processor reads the corresponding computer program from the nonvolatile memory to the memory and then operates the computer program to realize the infrared remote sensing image characteristic detection and splicing method.
Of course, other implementations, such as logic devices or combinations of hardware and software, are not excluded from the present description, that is, the execution subject of the following processing flows is not limited to each logic unit, but may be hardware or logic devices.
In the 90 s of the 20 th century, improvements to one technology could clearly be distinguished as improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (Programmable Logic Device, PLD) (e.g., field programmable gate array (Field Programmable Gate Array, FPGA)) is an integrated circuit whose logic function is determined by the programming of the device by a user. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented by using "logic compiler" software, which is similar to the software compiler used in program development and writing, and the original code before the compiling is also written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), but not just one of the hdds, but a plurality of kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), lava, lola, myHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog are currently most commonly used. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.
The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable logic controllers, and embedded microcontrollers, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in one or more software and/or hardware elements when implemented in the present specification.
It will be appreciated by those skilled in the art that embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The present description is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
It will be appreciated by those skilled in the art that embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing is merely exemplary of the present disclosure and is not intended to limit the disclosure. Various modifications and alterations to this specification will become apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, or the like, which are within the spirit and principles of the present description, are intended to be included within the scope of the claims of the present description.
Claims (7)
1. The infrared remote sensing image characteristic detection and splicing method is characterized by comprising the following steps of:
acquiring images to be spliced, wherein the images to be spliced are infrared images;
performing Fourier transform on each image to be spliced to obtain a transformed image, constructing a plurality of two-dimensional Log-Gabor filters, and filtering the transformed image through the two-dimensional Log-Gabor filters to obtain each filtered image, wherein the scales or directions corresponding to different two-dimensional Log-Gabor filters are different;
obtaining a Log-Gabor scale space according to each filtered image, grouping each image contained in the Log-Gabor scale space according to the scale to obtain each scale subspace, wherein one scale subspace contains images corresponding to the same scale through a Log-Gabor filter;
adding pixel values of images contained in each scale subspace to obtain a scale integration space corresponding to the scale subspace, obtaining each hessian matrix through the scale integration space corresponding to each scale subspace, determining a comparison result of each point and an adjacent point in a three-dimensional matrix formed by each hessian matrix, determining whether the point is a candidate feature point according to the comparison result, and determining a local maximum value corresponding to the local matrix through the local matrix of each candidate feature point to obtain the feature point;
Based on each characteristic point in each image to be spliced, carrying out characteristic point matching between the images to be spliced to obtain a characteristic point matching result, generating a similar transformation matrix, and determining a splicing path according to the characteristic point matching result;
and determining an optimal suture line, and splicing the images to be spliced according to the optimal suture line through the splicing path and the similar transformation matrix to obtain an infrared remote sensing image splicing result.
2. The method of claim 1, wherein obtaining each image to be stitched specifically comprises:
preprocessing each original infrared image to obtain each image to be spliced, wherein the preprocessing comprises the following steps: performing histogram stretching on the original infrared image; the image after the histogram stretching is rotated to the horizontal direction by the center of the image; and cutting the rotated image to obtain an image to be spliced.
3. The method of claim 1, wherein constructing a plurality of two-dimensional Log-Gabor filters of a predetermined scale and direction specifically comprises:
7 scales and 4 directions are selected to form 28 two-dimensional Log-Gabor filters in total;
filtering the transformed image through the plurality of two-dimensional Log-Gabor filters to obtain each filtered image, wherein the method specifically comprises the following steps of:
And filtering the transformed images through the 28 two-dimensional Log-Gabor filters to obtain 28 filtered images.
4. The method of claim 1, wherein determining a splice path from the feature point matching result specifically comprises:
setting the total number of the images to be spliced as N, constructing an N multiplied by N matrix S, if the number of the feature points matched with the image i and the image j is greater than or equal to a preset value, considering that the matching is successful, and setting the element S (i, j) of the S in the matrix as the reciprocal of the number of the matching points; if the number of the matching points of the image i and the image j is smaller than the preset value, setting S (i, j) as infinity;
and determining a splicing path according to the matrix S.
5. The utility model provides an infrared remote sensing image feature detects and splicing apparatus which characterized in that includes:
the acquisition module is used for acquiring each image to be spliced, wherein each image to be spliced is an infrared image;
the filtering module is used for carrying out Fourier transform on each image to be spliced to obtain a transformed image, constructing a plurality of two-dimensional Log-Gabor filters, and filtering the transformed image through the two-dimensional Log-Gabor filters to obtain each filtered image, wherein the scales or directions corresponding to different two-dimensional Log-Gabor filters are different;
The feature extraction module is used for obtaining Log-Gabor scale spaces according to the filtered images, grouping the images contained in the Log-Gabor scale spaces according to scales to obtain scale subspaces, wherein one scale subspace contains images corresponding to the same scale through Log-Gabor filters, adding pixel values of the images contained in the scale subspaces for each scale subspace to obtain a scale integration space corresponding to the scale subspace, obtaining hessian matrixes through the scale integration spaces corresponding to the scale subspaces, determining a comparison result of each point and adjacent points in a three-dimensional matrix formed by the hessian matrixes, determining whether the point is a candidate feature point according to the comparison result, and determining a local maximum value corresponding to the local matrix according to the local matrix of each candidate feature point to obtain the feature point;
the matching module is used for carrying out characteristic point matching between the images to be spliced based on the characteristic points in each image to be spliced to obtain characteristic point matching results, generating a similar transformation matrix and determining a splicing path according to the characteristic point matching results;
And the splicing module is used for determining an optimal suture line, and splicing the images to be spliced according to the optimal suture line through the splicing path and the similar transformation matrix to obtain an infrared remote sensing image splicing result.
6. A computer readable storage medium, characterized in that the storage medium stores a computer program which, when executed by a processor, implements the method of any of the preceding claims 1-4.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of the preceding claims 1-4 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311285977.8A CN117011147B (en) | 2023-10-07 | 2023-10-07 | Infrared remote sensing image feature detection and splicing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311285977.8A CN117011147B (en) | 2023-10-07 | 2023-10-07 | Infrared remote sensing image feature detection and splicing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117011147A CN117011147A (en) | 2023-11-07 |
| CN117011147B true CN117011147B (en) | 2024-01-12 |
Family
ID=88573073
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311285977.8A Active CN117011147B (en) | 2023-10-07 | 2023-10-07 | Infrared remote sensing image feature detection and splicing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117011147B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104881671A (en) * | 2015-05-21 | 2015-09-02 | 电子科技大学 | High resolution remote sensing image local feature extraction method based on 2D-Gabor |
| WO2019047284A1 (en) * | 2017-09-05 | 2019-03-14 | 平安科技（深圳）有限公司 | Methods for feature extraction and panoramic stitching, and apparatus thereof, device, readable storage medium |
| CN111044570A (en) * | 2019-12-28 | 2020-04-21 | 广东电网有限责任公司 | Defect identification and early warning method and device for power equipment and computer equipment |
| CN111985502A (en) * | 2020-08-03 | 2020-11-24 | 武汉大学 | Multi-mode image feature matching method with scale invariance and rotation invariance |
| CN113112403A (en) * | 2021-03-31 | 2021-07-13 | 国网山东省电力公司枣庄供电公司 | Infrared image splicing method, system, medium and electronic equipment |
| CN114529681A (en) * | 2022-02-10 | 2022-05-24 | 中国人民解放军战略支援部队航天工程大学士官学校 | Hand-held double-camera building temperature field three-dimensional model construction method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2677464B1 (en) * | 2012-05-16 | 2018-05-02 | IMEC vzw | Feature detection in numeric data |
-
2023
- 2023-10-07 CN CN202311285977.8A patent/CN117011147B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104881671A (en) * | 2015-05-21 | 2015-09-02 | 电子科技大学 | High resolution remote sensing image local feature extraction method based on 2D-Gabor |
| WO2019047284A1 (en) * | 2017-09-05 | 2019-03-14 | 平安科技（深圳）有限公司 | Methods for feature extraction and panoramic stitching, and apparatus thereof, device, readable storage medium |
| CN111044570A (en) * | 2019-12-28 | 2020-04-21 | 广东电网有限责任公司 | Defect identification and early warning method and device for power equipment and computer equipment |
| CN111985502A (en) * | 2020-08-03 | 2020-11-24 | 武汉大学 | Multi-mode image feature matching method with scale invariance and rotation invariance |
| CN113112403A (en) * | 2021-03-31 | 2021-07-13 | 国网山东省电力公司枣庄供电公司 | Infrared image splicing method, system, medium and electronic equipment |
| CN114529681A (en) * | 2022-02-10 | 2022-05-24 | 中国人民解放军战略支援部队航天工程大学士官学校 | Hand-held double-camera building temperature field three-dimensional model construction method and system |
Non-Patent Citations (2)
| Title |
|---|
| Large-area seamless mosaic for UAV thermal infrared sequence images based on prior coordinates and overlap area;He, JN et al.;INFRARED, MILLIMETER-WAVE, AND TERAHERTZ TECHNOLOGIES IX;第12324卷;全文 * |
| 基于点线特征结合的大视场机载摆扫红外影像配准研究;钟佩珂;中国优秀硕士学位论文库信息科技辑;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117011147A (en) | 2023-11-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chaudhuri et al. | Depth from defocus: a real aperture imaging approach | |
| CN107274442B (en) | Image identification method and device | |
| KR102316230B1 (en) | Image processing method and device | |
| CN111612834B (en) | Method, device and equipment for generating target image | |
| WO2022179002A1 (en) | Image matching method and apparatus, electronic device, and storage medium | |
| CN110348453B (en) | Object detection method and system based on cascade connection, storage medium and terminal | |
| CN117011147B (en) | Infrared remote sensing image feature detection and splicing method and device | |
| CN116805393A (en) | Hyperspectral image classification method and system based on 3DUnet spectrum-space information fusion | |
| CN115830633B (en) | Pedestrian re-recognition method and system based on multi-task learning residual neural network | |
| CN111967365B (en) | Image connection point extraction method and device | |
| CN114359935A (en) | Model training and form recognition method and device | |
| CN114863206A (en) | Model training method, target detection method and device | |
| CN106488118A (en) | Image adjusting method and photographic system | |
| CN116152274B (en) | Image segmentation method, device, equipment and storage medium | |
| CN116363390B (en) | Infrared dim target detection method and device, storage medium and electronic equipment | |
| CN111862098A (en) | Individual matching method, device, equipment and medium based on light field semantics | |
| CN116740114B (en) | Object boundary fitting method and device based on convex hull detection | |
| CN116817754B (en) | Soybean plant phenotype extraction method and system based on sparse reconstruction | |
| CN117197781B (en) | Traffic sign recognition method and device, storage medium and electronic equipment | |
| CN116188919A (en) | Test method and device, readable storage medium and electronic equipment | |
| CN117392688A (en) | Image classification method, device, storage medium and electronic equipment | |
| CN117575941A (en) | Image denoising method based on self-adaptive value range filtering, electronic equipment and medium | |
| CN117325171A (en) | Target identification and positioning method and device for live working robot and storage medium | |
| CN116894819A (en) | Concrete structure surface pixel level defect detection method, device and storage medium | |
| CN115019062A (en) | Method, device, equipment and medium for acquiring salient region |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |