CN114937067B - Image registration method of split-aperture polarization camera - Google Patents
Image registration method of split-aperture polarization camera Download PDFInfo
- Publication number
- CN114937067B CN114937067B CN202210670279.9A CN202210670279A CN114937067B CN 114937067 B CN114937067 B CN 114937067B CN 202210670279 A CN202210670279 A CN 202210670279A CN 114937067 B CN114937067 B CN 114937067B
- Authority
- CN
- China
- Prior art keywords
- image
- sub
- polarization
- polarized
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000010287 polarization Effects 0.000 title claims abstract description 169
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000006073 displacement reaction Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 claims description 3
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 3
- 230000021615 conjugation Effects 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000011426 transformation method Methods 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 abstract description 12
- 230000006870 function Effects 0.000 description 5
- 238000005314 correlation function Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
- G06T3/147—Transformations for image registration, e.g. adjusting or mapping for alignment of images using affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/10—Image enhancement or restoration using non-spatial domain filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Studio Devices (AREA)
- Exposure Control For Cameras (AREA)
- Image Analysis (AREA)
Abstract
An image registration method of a split-aperture polarization camera comprises the steps of obtaining a polarization subimage, roughly registering the polarization subimage, extracting characteristic points of the polarization subimage, determining the number of points in the polarization subimage, obtaining the optimal registration coefficient of the polarization subimage and finishing the registration of the polarization subimage. According to the invention, the polarization sub-images are roughly registered, so that the calculated amount when the characteristic points of the polarization sub-images are extracted is reduced, the optimal registration coefficient when the number of the points in the polarization sub-images is the largest is determined by traversing the characteristic points, the high-precision, high-speed and full-automatic registration between the polarization sub-images is completed, the technical problem of position deviation between the sub-aperture polarization sub-images is solved, the image quality when the sub-aperture polarization camera is used for polarization imaging is improved, and the method can be popularized and used in the technical field of polarization imaging of the sub-aperture polarization camera.
Description
Technical Field
The invention belongs to the technical field of image registration, and particularly relates to an image registration method of a split-aperture polarization camera.
Background
The polarization imaging technology can highlight the detail information of scene targets from a complex background, can remarkably improve the image quality under severe weather conditions, and has wide application in the aspects of camouflage removal, smoke penetration and defogging imaging and the like. To meet different requirements, various types of polarization imaging systems have been developed. Compared with a time-sharing polarization imaging system, an amplitude-dividing polarization imaging system and a focal plane-dividing polarization imaging system, the aperture-dividing polarization imaging system has the advantages of small and compact structure, easiness in implementation, lower cost and capability of simultaneously acquiring full-polarization information of a scene. However, even if the sub-aperture polarization camera can achieve a better use state after being manually adjusted and assembled, the obtained polarized sub-images still inevitably have position deviation.
In the technical field of polarization imaging, the technical problem to be solved by the wide application of the aperture-dividing polarization camera is to provide a high-precision, high-speed and full-automatic image registration method of the aperture-dividing polarization camera.
Disclosure of Invention
The technical problem to be solved by the present invention is to overcome the drawbacks of the prior art, and to provide a method for image registration of a full-automatic aperture-dividing polarization camera with high precision and high speed.
The technical scheme adopted for solving the technical problems comprises the following steps:
(1) Obtaining a polarized sub-image
The method comprises the steps of shooting a target by using a sub-aperture polarization camera, obtaining an original frame image, cutting the original frame image into four polarization sub-images, selecting any one polarization sub-image in the four polarization sub-images as a reference polarization sub-image, and using the other three polarization sub-images as polarization sub-images to be registered.
(2) Coarse registration of polarization subimages
Determining a Fourier transform F (ω) of a reference polarized subimage according to equation (1) x ,ω y )：
F(ω x ,ω y )＝F{f(x,y)} (1)
Where F (x, y) is a reference polarization sub-image of coordinates (x, y), and F { } denotes taking a Fourier transform.
Determining the Fourier transform G (omega) of the polarization subimage to be registered according to equation (2) x ,ω y )：
G(ω x ,ω y (＝F{g(x,y)} (2)
Wherein g (x, y) is the polarization subimage to be registered at coordinates (x, y).
The impulse function δ (x-dx, y-dy) is determined as in equation (3):
wherein dx is the displacement of the polarization sub-image to be registered in the horizontal direction, dy is the displacement of the polarization sub-image to be registered in the vertical direction, and F -1 Is an inverse Fourier transform, F * (ω x ,ω y ) Is F (omega) x ,ω y ) Complex conjugation of (a).
The coordinates (x ', y') of the coarse registration polarization sub-image are determined as in equation (4):
wherein (x) 0 ,y 0 ) The coordinates of the polarization sub-image are registered.
Determination of a coarse registration polarization subimage I according to equation (5) rough (x 0 ,y 0 )：
I rough (x 0 ,y 0 )＝I(x′,y′) (5)
Where I (x ', y') is the intensity of the sub-image of the polarization to be registered at the coordinates (x ', y').
(3) Extracting characteristic points of polarized sub-images
Respectively extracting the characteristic points of the reference polarized sub-image and the rough registration polarized sub-image, selecting two characteristic points with the minimum Euclidean distance and the minimum Euclidean distance in the rough registration polarized sub-image for any one characteristic point in the reference polarized sub-image, forming a characteristic point pair by the selected characteristic point in the reference polarized sub-image and the characteristic point with the minimum Euclidean distance in the rough registration polarized sub-image according to the condition that the ratio of the minimum Euclidean distance to the minimum Euclidean distance is not larger than the set threshold value, and forming the characteristic point pair by other characteristic points.
(4) Determining the number of dots within a polarized sub-image
Determining an affine transformation matrix a between the coarsely registered polarization sub-image and the reference polarization sub-image according to equation (6):
wherein,
and
coordinates of a first characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image respectively;
and
coordinates of a second feature point pair of the reference polarized sub-image and the coarse registration polarized sub-image respectively;
and
coordinates of a third pair of feature points of the reference and coarse registration polarization sub-images, respectively.
Determining transformed coordinates (x) of remaining feature points of the coarse registration polarization subimage according to equation (7) cal ,y cal )：
Wherein (x) rough ,y rough ) The coordinates of the remaining feature points in the polarization sub-image are coarsely registered.
And the absolute value of the difference between the feature point transformation coordinate of the rough registration polarization subimage and the corresponding feature point coordinate of the reference polarization subimage is less than 0.5, the feature point is marked as an interior point, and the number of the interior points in the polarization subimage is determined.
(5) Obtaining optimal registration coefficients for polarization sub-images
And (4) selecting any other 3 characteristic point pairs in the characteristic point pairs, and repeating the step (4) until all the characteristic point pairs are traversed.
An optimal registration coefficient a' is determined according to equation (8):
wherein,
and
respectively obtaining the coordinates of a first characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image when the number of the interior points is the maximum;
and
respectively the coordinates of the second characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image when the number of the inner points is the maximum;
and
and coordinates of the third characteristic point pair of the reference polarized sub-image and the coarse registration polarized sub-image when the number of the inner points is the maximum are respectively obtained.
(6) Completing a polarized image registration
Determining coordinates (x) of the registered polarization sub-images according to equation (9) reg ,y reg )：
Obtaining a registered polarization image I according to the formula (10) reg (x′,y′)：
I reg (x′,y′)＝I rough (x reg ,y reg ) (10) wherein I rough (x reg ,y reg ) Registering the polarization sub-image in coordinates (x) for coarse reg ,y reg ) The intensity of the spot.
In the step of (1) acquiring the polarized sub-image, the acquiring original frame image includes a frame of sub-aperture polarized image containing a 0 ° linear polarized image, a 45 ° linear polarized image, a 90 ° linear polarized image, and a circular polarized image; the original frame image is cut into four polarized sub-images, wherein the four polarized sub-images comprise 0-degree linear polarized sub-images, 45-degree linear polarized sub-images, 90-degree linear polarized sub-images and circular polarized sub-images which are the same in size.
In the step (3) of extracting the characteristic points of the polarized sub-images, the method for respectively extracting the characteristic points of the reference polarized sub-images and the rough registration polarized sub-images is a scale-invariant characteristic transformation method or an accelerated robust characteristic method.
In the step (3) of extracting the characteristic point of the polarized sub-image, the ratio of the minimum Euclidean distance to the next minimum Euclidean distance is not more than the set threshold condition, and the value of the threshold is 0.5-0.8.
According to the invention, the polarized sub-images are subjected to rough registration, the calculated amount when the characteristic points of the polarized sub-images are extracted is reduced, the optimal registration coefficient when the number of the points in the polarized sub-images is the maximum is determined by traversing the characteristic points, the high-precision, fast-speed and full-automatic registration between the polarized sub-images is completed, the technical problem of position deviation between the sub-images of the aperture-dividing polarization is solved, the image quality when the aperture-dividing polarization camera is used for polarization imaging is improved, and the method can be used for polarization imaging of the aperture-dividing polarization camera.
Drawings
FIG. 1 is a flowchart of example 1 of the present invention.
Fig. 2 is a 0 ° linearly polarized subimage of embodiment 1 of the invention.
Fig. 3 is the registration result of the 45 ° linearly polarized sub-images in embodiment 1 of the present invention.
Detailed Description
The present invention will be described in further detail below with reference to the drawings and examples, but the present invention is not limited to the embodiments described below.
Example 1
The image registration method of the split-aperture polarization camera of the present embodiment is composed of the following steps (see fig. 1):
(1) Obtaining a polarized sub-image
The method comprises the steps of using a sub-aperture polarization camera to shoot a target, obtaining an original frame image, cutting the original frame image into four polarization sub-images, selecting any one polarization sub-image in the four polarization sub-images as a reference polarization sub-image, and using the other three polarization sub-images as polarization sub-images to be registered.
The acquisition original frame image of the embodiment includes a frame of sub-aperture polarization image containing a 0 ° linear polarization image, a 45 ° linear polarization image, a 90 ° linear polarization image, and a circular polarization image; the original frame image is cut into four polarized sub-images, wherein the four polarized sub-images comprise 0-degree linear polarized sub-images, 45-degree linear polarized sub-images, 90-degree linear polarized sub-images and circular polarized sub-images which are the same in size. The 0 ° linearly polarized sub-image serves as a reference polarized sub-image (see fig. 2).
(2) Coarse registration of polarized subimages
Determining a Fourier transform F (ω) of a reference polarized subimage according to equation (1) x ,ω y )：
F(ω x ,ω y )＝F{f(x,y)} (1)
Where F (x, y) is a reference polarization sub-image of coordinates (x, y), and F { } denotes taking a Fourier transform.
Determining the Fourier transform G (omega) of the polarization subimage to be registered according to equation (2) x ,ω y )：
G(ω x ,ω y )＝F{g(x,t)} (2)
Wherein g (x, y) is the polarization subimage to be registered at coordinates (x, y).
An impulse function δ (x-dx, y-dy) is determined as equation (3):
wherein dx is the displacement of the polarization sub-image to be registered in the horizontal direction, dy is the displacement of the polarization sub-image to be registered in the vertical direction, and F -1 Is an inverse Fourier transform, F * (ω x ,ω y ) Is F (omega) x ,ω y ) Complex conjugation of (a).
The coordinates (x ', y') of the coarsely registered polarization subimages are determined as in equation (4):
wherein (x) 0 ,y 0 ) The coordinates of the polarization subimages are to be registered.
Determining a coarse registration polarization subimage I according to equation (5) rough (x 0 ,y 0 )：
I rough (x 0 ,y 0 )＝I(x′,y′) (5)
Where I (x ', y') is the intensity of the polarization sub-image to be registered at the coordinates (x ', y').
(3) Extracting characteristic points of polarized sub-images
The method comprises the steps of respectively extracting characteristic points of a reference polarized sub-image and a rough registration polarized sub-image, wherein a scale invariant feature transformation method is adopted in the embodiment, and is disclosed in the Distingtive image features from scale-invariant keys [ J ]. International Journal of Computer Vision,2004,60 (2): 91-110. For any feature point in the reference polarization sub-image, two feature points with the minimum Euclidean distance and the second minimum Euclidean distance in the rough registration polarization sub-image are selected, and according to the condition that the ratio of the minimum Euclidean distance to the second minimum Euclidean distance is not greater than the set threshold value, the threshold value is 0.5-0.8, and the threshold value of the embodiment is 0.6.
The judgment method for forming the characteristic point pair by the other characteristic points is the same as the judgment method for forming the characteristic point pair by the selected characteristic point in the reference polarization sub-image and the characteristic point with the minimum European distance in the rough registration polarization sub-image.
(4) Determining the number of dots within a polarized sub-image
Determining an affine transformation matrix a between the coarsely registered polarization sub-image and the reference polarization sub-image according to equation (6):
wherein,
and
coordinates of a first characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image respectively;
and
coordinates of a second feature point pair of the reference polarized sub-image and the coarse registration polarized sub-image respectively;
and
the coordinates of the third pair of feature points of the reference polarized sub-image and the coarsely registered polarized sub-image, respectively.
Determining transformed coordinates (x) of remaining feature points of the coarse registration polarization subimage according to equation (7) cal ,y cal )：
Wherein (x) rough ,y rough ) The coordinates of the remaining feature points in the polarization sub-image are coarsely registered.
And the absolute value of the difference between the feature point transformation coordinate of the rough registration polarization sub-image and the corresponding feature point coordinate of the reference polarization sub-image is less than 0.5, the feature point is marked as an inner point, and the number of the inner points in the polarization sub-image is determined.
(5) Obtaining optimal registration coefficients for polarization sub-images
And (4) selecting any other 3 characteristic point pairs in the characteristic point pairs, and repeating the step (4) until all the characteristic point pairs are traversed.
An optimal registration coefficient a' is determined according to equation (8):
wherein,
and
respectively obtaining the coordinates of a first characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image when the number of the inner points is the maximum;
and
respectively the coordinates of the second characteristic point pair of the reference polarized sub-image and the coarse registration polarized sub-image when the number of the interior points is the maximum;
and
and coordinates of the third characteristic point pair of the reference polarized sub-image and the coarse registration polarized sub-image when the number of the inner points is the maximum are respectively obtained.
(6) Completing a polarized image registration
Determining coordinates of the registered polarization sub-image according to equation (9)(x reg ,y reg )：
Obtaining a registered polarization image I according to the formula (10) reg (x′,y′)：
I reg (x′,y′)＝I rough (x reg ,y reg ) (10)
Wherein, I rough (x reg ,y reg ) Registering the polarization sub-image in coordinates (x) for coarse reg ,y reg ) The intensity of the spot.
And finishing the image registration method of the sub-aperture polarization camera. And obtaining a registered 45-degree linear polarization sub-image (see fig. 3), and as can be seen from fig. 2 and fig. 3, the invention realizes the image registration of the sub-aperture polarization sub-image.
Example 2
The image registration method of the split-aperture polarization camera of the embodiment comprises the following steps:
(1) Obtaining a polarized sub-image
This procedure is the same as in example 1.
(2) Coarse registration of polarization subimages
This procedure is the same as in example 1.
(3) Extracting characteristic points of polariton images
Respectively extracting feature points of a reference polarized sub-image and a rough registration polarized sub-image, selecting two feature points with the minimum Euclidean distance and the second minimum Euclidean distance in the rough registration polarized sub-image for any one feature point in the reference polarized sub-image, and taking the threshold value as 0.5-0.8 according to the condition that the ratio of the minimum Euclidean distance to the second minimum Euclidean distance is not more than the set threshold value, wherein the threshold value of the embodiment is 0.5.
The judgment method for forming the characteristic point pair by the other characteristic points is the same as the judgment method for forming the characteristic point pair by the selected characteristic point in the reference polarization sub-image and the characteristic point with the minimum European distance in the rough registration polarization sub-image.
The other steps were the same as in example 1.
And finishing the image registration method of the sub-aperture polarization camera.
Example 3
The image registration method of the split-aperture polarization camera of the embodiment comprises the following steps:
(1) Obtaining a polarized sub-image
This procedure is the same as in example 1.
(2) Coarse registration of polarization subimages
This procedure is the same as in example 1.
(3) Extracting characteristic points of polariton images
Respectively extracting feature points of a reference polarized sub-image and a rough registration polarized sub-image, selecting two feature points with the minimum Euclidean distance and the second minimum Euclidean distance in the rough registration polarized sub-image for any one feature point in the reference polarized sub-image, and taking the threshold value as 0.5-0.8 according to the condition that the ratio of the minimum Euclidean distance to the second minimum Euclidean distance is not more than the set threshold value, wherein the threshold value of the embodiment is 0.8.
The judgment method for forming the characteristic point pair by the other characteristic points is the same as the judgment method for forming the characteristic point pair by the selected characteristic point in the reference polarization sub-image and the characteristic point with the minimum European distance in the rough registration polarization sub-image.
The other steps were the same as in example 1.
And finishing the image registration method of the sub-aperture polarization camera.
Example 4
In the above embodiments 1 to 3, the image registration method of the split-aperture polarization camera of the present embodiment is composed of the following steps:
(1) Obtaining a polarized sub-image
This procedure is the same as in example 1.
(2) Coarse registration of polarized subimages
This procedure is the same as in example 1.
(3) Extracting characteristic points of polarized sub-images
The method comprises the steps of respectively extracting feature points of a reference polarization sub-image and a rough registration polarization sub-image, adopting an accelerated robust feature method in the embodiment, selecting two feature points with the minimum Euclidean distance and the second minimum Euclidean distance in the rough registration polarization sub-image for any feature point in the reference polarization sub-image, and setting a threshold value to be 0.5-0.8 according to the condition that the ratio of the minimum Euclidean distance to the second minimum Euclidean distance is not larger than a set threshold value condition, wherein the threshold value of the embodiment is the same as that of the corresponding embodiment.
And forming a characteristic point pair by the selected characteristic point in the reference polarization sub-image and the characteristic point with the minimum European distance in the rough registration polarization sub-image, wherein the judgment method for forming the characteristic point pair by other characteristic points is the same as the judgment method.
The other steps were the same as in example 1.
And finishing the image registration method of the sub-aperture polarization camera.
In order to verify the beneficial effects of the present invention, the inventor performed experiments on the images of the split-aperture polarization camera by using the image registration method of the split-aperture polarization camera in embodiment 1 of the present invention, and the experimental conditions are as follows.
1. Conditions of the experiment
The experimental test environment is a Hewlett packard computer of a Windows l0 (64-bit) operating system, which is configured as an Intercore i3-10105F and 16GB memory, and experimental operation is carried out on a MATLAB R2019b platform.
2. Introduction to Experimental data
The original frame image shot by the aperture-dividing polarization camera is shot by the aperture-dividing polarization camera of the advanced optical imaging laboratory of university of Shanxi.
3. Evaluation index
Structural Similarity Index (SSIM) and Normalized Mutual Information (NMI) were used as evaluation indices. Determining the structural similarity index according to equation (11):
SSIM(R,T)＝L(R,T)C(R,T)S(R,T) (11)
wherein, R and T represent two images with the same resolution, L (R, T) represents the brightness correlation function of the two images, C (R, T) represents the contrast correlation function of the two images, S (R, T) represents the structure correlation function of the two images, and mu R And mu T Respectively representing the mean value of the gray levels, sigma, of the two images R And σ T Respectively representing the respective gray scale standard deviations, sigma, of the two images RT Representing the gray-scale covariance, C, between image R and image T 1 、C 2 And C 3 All are small normal numbers, and are intended to avoid instability caused by denominator of 0 or close to 0.
Normalized mutual information is determined as in equation (12):
wherein M and N represent two images with the same resolution and the same gray scale, H (M) represents the average information content of the image M, H (N) represents the average information content of the image N, H (M, N) represents the related average information content between the image M and the image N, M and N represent the gray scale value of any pixel point in the image M and the image N respectively, P is M (M) represents the edge probability density function, P, of the image M N (N) edges of the image NEdge probability density function, P MN (M, N) represents the joint probability density function between image M and image N.
And quantitatively evaluating the similarity degree of the images of the split-aperture polarization camera before registration with the images which are not registered.
The test was performed according to the method of example 1, and the structural similarity index between the sub-aperture polarized images is shown in table 1, and the normalized mutual information between the sub-aperture polarized images is shown in table 2.
TABLE 1 Structural Similarity Index (SSIM) of two per sub-aperture polarization images
TABLE 2 Normalized Mutual Information (NMI) of every two sub-aperture polarization images
In table 1, SSIM _0 \45denotes a structural similarity index of a 0 ° linearly polarized sub-image and a 45 ° linearly polarized sub-image; SSIM _0 \90denotes the structural similarity index of the 0 ° linearly polarized subimage with the 90 ° linearly polarized subimage; SSIM _0_C represents the structural similarity index of the 0 ° linear polarization sub-image and the circular polarization sub-image; SSIM _45 \90denotes the structural similarity index of a 45 ° linearly polarized subimage with a 90 ° linearly polarized subimage; SSIM — 45 \ c represents the structural similarity index of a 45 ° linearly polarized subimage to a circularly polarized subimage; SSIM _90 \ c represents the structural similarity index of a 90 ° linearly polarized subimage to a circularly polarized subimage.
In table 2, NMI _0 \45represents normalized mutual information of the 0 ° linearly polarized sub-image and the 45 ° linearly polarized sub-image; NMI _0 _90represents the normalized mutual information of the 0 ° linearly polarized subimage and the 90 ° linearly polarized subimage; NMI _0_C represents the normalized mutual information of the 0 ° linear polarization sub-image and the circular polarization sub-image; NMI _45 \90represents the normalized mutual information of the 45 ° linearly polarized subimage and the 90 ° linearly polarized subimage; NMI _45 \ c represents the normalized mutual information of the 45 ° linear polarization subimage and the circular polarization subimage; NMI _90 \ u c represents the normalized mutual information of the 90 ° linearly polarized subimage and the circularly polarized subimage.
As can be seen from tables 1 and 2, compared with the four unregistered polarization images, the four polarization images after registration in the invention have the structure similarity index improved by 72.08% on average and the normalized mutual information improved by 18.5% on average, which indicates that the invention can well realize the image registration of the aperture-dividing polarization camera.
Claims (4)
1. An image registration method of a split-aperture polarization camera is characterized by comprising the following steps:
(1) Obtaining a polarized sub-image
Shooting a target by using a sub-aperture polarization camera, acquiring an original frame image, cutting the original frame image into four polarization sub-images, selecting any one polarization sub-image in the four polarization sub-images as a reference polarization sub-image, and taking the other three polarization sub-images as polarization sub-images to be registered;
(2) Coarse registration of polarization subimages
Determining a Fourier transform F (ω) of a reference polarized subimage according to equation (1) x ，ω y )：
F(ω x ，ω y )＝F{f(x，y)} (1)
Wherein F (x, y) is a reference polarization sub-image of coordinates (x, y), and F { } represents taking a Fourier transform;
determining the Fourier transform G (omega) of the polarization subimage to be registered according to equation (2) x ，ω y )：
G(ω x ，ω y )＝F{g(x，y)} (2)
Wherein g (x, y) is a polarization subimage to be registered of coordinates (x, y);
the impulse function δ (x-dx, y-dy) is determined as in equation (3):
wherein dx is the displacement of the polarization sub-image to be registered in the horizontal direction, dy is the displacement of the polarization sub-image to be registered in the vertical direction, and F -1 Is an inverse Fourier transform, F * (ω x ，ω y ) Is F (omega) x ，ω y ) Complex conjugation of (a);
the coordinates (x ', y') of the coarse registration polarization sub-image are determined as in equation (4):
wherein (x) 0 ，y 0 ) Coordinates of a polarization subimage to be registered are obtained;
determining a coarse registration polarization subimage I according to equation (5) rough (x 0 ，y 0 )：
I rough (x 0 ，y 0 )＝I(x′，y′) (5)
Wherein I (x ', y') is the intensity of the polarization subimage to be registered at the coordinates (x ', y');
(3) Extracting characteristic points of polarized sub-images
Respectively extracting feature points of the reference polarized sub-image and the rough registration polarized sub-image, selecting two feature points with the minimum Euclidean distance and the second minimum Euclidean distance in the rough registration polarized sub-image for any one feature point in the reference polarized sub-image, forming a feature point pair by the selected feature point in the reference polarized sub-image and the feature point with the minimum Euclidean distance in the rough registration polarized sub-image according to the condition that the ratio of the minimum Euclidean distance to the second minimum Euclidean distance is not larger than a set threshold value, and forming the feature point pair by other feature points by the same judging method as the above;
(4) Determining the number of dots within a polarized sub-image
Determining an affine transformation matrix a between the coarsely registered polarization sub-image and the reference polarization sub-image according to equation (6):
wherein,
and
coordinates of a first characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image respectively;
and
coordinates of a second feature point pair of the reference polarized sub-image and the coarse registration polarized sub-image respectively;
and
coordinates of a third feature point pair of the reference polarized sub-image and the coarse registration polarized sub-image respectively;
determining transformed coordinates (x) of remaining feature points of the coarse registration polarization subimage according to equation (7) cal ，y cal )：
Wherein (x) rough ，y rough ) The coordinates of the rest feature points in the polarized sub-image are roughly registered;
the absolute value of the difference between the feature point transformation coordinate of the rough registration polarization subimage and the corresponding feature point coordinate of the reference polarization subimage is less than 0.5, the feature point is marked as an interior point, and the number of the interior points in the polarization subimage is determined;
(5) Obtaining optimal registration coefficients for polarization sub-images
Selecting any other 3 characteristic point pairs in the characteristic point pairs, and repeating the step (4) until all the characteristic point pairs are traversed;
an optimal registration coefficient a' is determined according to equation (8):
wherein,
and
respectively obtaining the coordinates of a first characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image when the number of the interior points is the maximum;
and
respectively the coordinates of the second characteristic point pair of the reference polarized sub-image and the coarse registration polarized sub-image when the number of the interior points is the maximum;
and
respectively obtaining the coordinates of the third characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image when the number of the inner points is the maximum;
(6) Completing a polarized image registration
Determining coordinates (x) of the registered polarization sub-images according to equation (9) reg ，y reg )：
Obtaining a registered polarization image I according to the formula (10) reg (x′，y′)：
I reg (x′，y′)＝I rough (x reg ，y reg ) (10)
Wherein, I rough (x reg ，y reg ) Registering the polarization sub-image in coordinates (x) for coarse reg ，y reg ) The intensity of the spot.
2. The image registration method of a split-aperture polarization camera according to claim 1, wherein: in (1) the step of acquiring polarized sub-images, the acquiring original frame images comprise a frame of sub-aperture polarized images containing a 0 degree linear polarized image, a 45 degree linear polarized image, a 90 degree linear polarized image and a circular polarized image; the original frame image is cut into four polarized sub-images, wherein the four polarized sub-images comprise 0-degree linear polarized sub-images, 45-degree linear polarized sub-images, 90-degree linear polarized sub-images and circular polarized sub-images which are the same in size.
3. The image registration method of a split-aperture polarization camera according to claim 1, wherein: in the step (3) of extracting the characteristic points of the polarized sub-images, the method for respectively extracting the characteristic points of the reference polarized sub-images and the rough registration polarized sub-images is a scale-invariant characteristic transformation method or an accelerated robust characteristic method.
4. The image registration method of a split-aperture polarization camera according to claim 1, wherein: in the step (3) of extracting the characteristic point of the polarized sub-image, the ratio of the minimum Euclidean distance to the second minimum Euclidean distance is not more than the set threshold condition, and the value of the threshold is 0.5-0.8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210670279.9A CN114937067B (en) | 2022-06-14 | 2022-06-14 | Image registration method of split-aperture polarization camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210670279.9A CN114937067B (en) | 2022-06-14 | 2022-06-14 | Image registration method of split-aperture polarization camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114937067A CN114937067A (en) | 2022-08-23 |
| CN114937067B true CN114937067B (en) | 2023-01-31 |
Family
ID=82866432
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210670279.9A Active CN114937067B (en) | 2022-06-14 | 2022-06-14 | Image registration method of split-aperture polarization camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114937067B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107392948A (en) * | 2017-07-10 | 2017-11-24 | 大连理工大学 | A kind of method for registering images of point of amplitude real-time polarization imaging system |
| CN108765476A (en) * | 2018-06-05 | 2018-11-06 | 安徽大学 | Polarized image registration method |
| CN109886883A (en) * | 2019-01-21 | 2019-06-14 | 吉林大学 | Real-time polarization fog-penetrating imaging image enhancement processing method |
| CN111882594A (en) * | 2020-07-27 | 2020-11-03 | 北京环境特性研究所 | ORB feature point-based polarization image rapid registration method and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015115041A (en) * | 2013-12-16 | 2015-06-22 | ソニー株式会社 | Image processor, and image processing method |
| CN111487201B (en) * | 2020-04-27 | 2021-02-09 | 华中科技大学 | Characterization method of polarization image information and calculation method of characterization parameters |
-
2022
- 2022-06-14 CN CN202210670279.9A patent/CN114937067B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107392948A (en) * | 2017-07-10 | 2017-11-24 | 大连理工大学 | A kind of method for registering images of point of amplitude real-time polarization imaging system |
| CN108765476A (en) * | 2018-06-05 | 2018-11-06 | 安徽大学 | Polarized image registration method |
| CN109886883A (en) * | 2019-01-21 | 2019-06-14 | 吉林大学 | Real-time polarization fog-penetrating imaging image enhancement processing method |
| CN111882594A (en) * | 2020-07-27 | 2020-11-03 | 北京环境特性研究所 | ORB feature point-based polarization image rapid registration method and device |
Non-Patent Citations (4)
| Title |
|---|
| 一种基于相位相关与子图像的偏振图像配准方法;万钇良等;《液晶与显示》;20190515(第05期);全文 * |
| 偏振探测系统的图像配准;褚金奎等;《光学精密工程》;20180515(第05期);全文 * |
| 全偏振态同时探测实时彩色偏振成像技术;巨海娟等;《红外与毫米波学报》;20171215(第06期);全文 * |
| 同时偏振成像探测系统的偏振图像配准研究;杨长久等;《红外与激光工程》;20130125(第01期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114937067A (en) | 2022-08-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110097093B (en) | Method for accurately matching heterogeneous images | |
| KR101117837B1 (en) | Multi-image feature matching using multi-scale oriented patches | |
| CN111340701B (en) | Circuit board image splicing method for screening matching points based on clustering method | |
| CN110992263B (en) | Image stitching method and system | |
| CN110956661B (en) | Method for calculating dynamic pose of visible light and infrared camera based on bidirectional homography matrix | |
| US9767383B2 (en) | Method and apparatus for detecting incorrect associations between keypoints of a first image and keypoints of a second image | |
| CN110909750B (en) | Image difference detection method and device, storage medium and terminal | |
| CN114299275A (en) | Hough transform-based license plate inclination correction method | |
| CN109118473A (en) | Angular-point detection method, storage medium and image processing system neural network based | |
| CN112017223A (en) | Heterologous image registration method based on improved SIFT-Delaunay | |
| CN114897705A (en) | Unmanned aerial vehicle remote sensing image splicing method based on feature optimization | |
| CN115272403A (en) | Fragment scattering characteristic testing method based on image processing technology | |
| CN110060208B (en) | Method for improving reconstruction performance of super-resolution algorithm | |
| CN114937067B (en) | Image registration method of split-aperture polarization camera | |
| CN112184533B (en) | Watermark synchronization method based on SIFT feature point matching | |
| CN113096016A (en) | Low-altitude aerial image splicing method and system | |
| CN113642397A (en) | Object length measuring method based on mobile phone video | |
| CN110322476B (en) | Target tracking method for improving STC and SURF feature joint optimization | |
| CN115497114B (en) | Structured information extraction method for cigarette logistics receiving bill | |
| CN111833384A (en) | Method and device for quickly registering visible light and infrared images | |
| CN109359513B (en) | Anomaly detection method based on edge detection and color matching | |
| CN114565653B (en) | Heterologous remote sensing image matching method with rotation change and scale difference | |
| CN116258983A (en) | Bullet target surface image acquisition and processing method and device based on multi-rotor unmanned aerial vehicle | |
| CN111402281B (en) | Book edge detection method and device | |
| CN112669360B (en) | Multi-source image registration method based on non-closed multi-dimensional contour feature sequence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |