CN114937067A - Image registration method of sub-aperture polarization camera - Google Patents
Image registration method of sub-aperture polarization camera Download PDFInfo
- Publication number
- CN114937067A CN114937067A CN202210670279.9A CN202210670279A CN114937067A CN 114937067 A CN114937067 A CN 114937067A CN 202210670279 A CN202210670279 A CN 202210670279A CN 114937067 A CN114937067 A CN 114937067A
- Authority
- CN
- China
- Prior art keywords
- image
- sub
- polarization
- polarized
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000010287 polarization Effects 0.000 title claims abstract description 174
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000006073 displacement reaction Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 claims description 3
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 3
- 230000021615 conjugation Effects 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000011426 transformation method Methods 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 abstract description 12
- 230000006870 function Effects 0.000 description 5
- 238000005314 correlation function Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
- G06T3/147—Transformations for image registration, e.g. adjusting or mapping for alignment of images using affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/10—Image enhancement or restoration using non-spatial domain filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Studio Devices (AREA)
- Exposure Control For Cameras (AREA)
- Image Analysis (AREA)
Abstract
An image registration method of a split-aperture polarization camera comprises the steps of obtaining a polarization subimage, roughly registering the polarization subimage, extracting characteristic points of the polarization subimage, determining the number of points in the polarization subimage, obtaining the optimal registration coefficient of the polarization subimage and finishing the registration of the polarization subimage. According to the invention, the polarization sub-images are roughly registered, so that the calculated amount when the characteristic points of the polarization sub-images are extracted is reduced, the optimal registration coefficient when the number of the points in the polarization sub-images is the largest is determined by traversing the characteristic points, the high-precision, high-speed and full-automatic registration between the polarization sub-images is completed, the technical problem of position deviation between the sub-aperture polarization sub-images is solved, the image quality when the sub-aperture polarization camera is used for polarization imaging is improved, and the method can be popularized and used in the technical field of polarization imaging of the sub-aperture polarization camera.
Description
Technical Field
The invention belongs to the technical field of image registration, and particularly relates to an image registration method of a split-aperture polarization camera.
Background
The polarization imaging technology can highlight the detail information of scene targets from a complex background, can obviously improve the image quality under severe weather conditions, and has wide application in the aspects of camouflage removal, smoke penetration and defogging imaging and the like. To meet different requirements, various types of polarization imaging systems have been developed. Compared with a time-sharing polarization imaging system, an amplitude-dividing polarization imaging system and a focal plane-dividing polarization imaging system, the aperture-dividing polarization imaging system has the advantages of small and compact structure, easiness in implementation, lower cost and capability of simultaneously acquiring full-polarization information of a scene. However, even if the sub-aperture polarization camera can achieve a better use state after being manually adjusted and assembled, the obtained polarized sub-images still inevitably have position deviation.
In the technical field of polarization imaging, the technical problem to be solved by the wide application of the aperture-dividing polarization camera is to provide a high-precision, high-speed and full-automatic image registration method of the aperture-dividing polarization camera.
Disclosure of Invention
The technical problem to be solved by the present invention is to overcome the drawbacks of the prior art, and to provide a method for image registration of a full-automatic aperture-dividing polarization camera with high precision and high speed.
The technical scheme adopted for solving the technical problems comprises the following steps:
(1) obtaining a polarized sub-image
The method comprises the steps of shooting a target by using a sub-aperture polarization camera, obtaining an original frame image, cutting the original frame image into four polarization sub-images, selecting any one polarization sub-image in the four polarization sub-images as a reference polarization sub-image, and using the other three polarization sub-images as polarization sub-images to be registered.
(2) Coarse registration of polarization subimages
Determining the Fourier transform F (omega) of a reference polarization subimage by equation (1) x ,ω y )：
F(ω x ,ω y )＝F{f(x,y)} (1)
Where F (x, y) is a reference polarization subimage of coordinates (x, y), and F { } denotes taking a fourier transform.
Determining the Fourier transform G (omega) of the polarization subimage to be registered according to equation (2) x ,ω y )：
G(ω x ,ω y (＝F{g(x,y)} (2)
Wherein g (x, y) is a polarization subimage to be registered of the coordinates (x, y).
The impulse function δ (x-dx, y-dy) is determined as in equation (3):
wherein dx is the displacement of the polarization sub-image to be registered in the horizontal direction, dy is the displacement of the polarization sub-image to be registered in the vertical direction, and F -1 Is an inverse Fourier transform, F * (ω x ,ω y ) Is F (omega) x ,ω y ) Complex conjugation of (a).
The coordinates (x ', y') of the coarse registration polarization sub-image are determined as in equation (4):
wherein (x) 0 ,y 0 ) The coordinates of the polarization sub-image are registered.
Determining a coarse registration polarization subimage I according to equation (5) rough (x 0 ,y 0 )：
I rough (x 0 ,y 0 )＝I(x′,y′) (5)
Where I (x ', y') is the intensity of the polarization sub-image to be registered at the coordinates (x ', y').
(3) Extracting characteristic points of polarized sub-images
Respectively extracting feature points of a reference polarized sub-image and a rough registration polarized sub-image, selecting two feature points with the minimum Euclidean distance and the minimum Euclidean distance from the rough registration polarized sub-image to any feature point in the reference polarized sub-image, forming a feature point pair by the selected feature point in the reference polarized sub-image and the feature point with the minimum Euclidean distance in the rough registration polarized sub-image according to the condition that the ratio of the minimum Euclidean distance to the minimum Euclidean distance is not larger than a set threshold value, and forming the feature point pair by other feature points by the same judging method.
(4) Determining the number of dots within a polarized sub-image
An affine transformation matrix a between the coarsely registered polarization subimage and the reference polarization subimage is determined as follows:
wherein,
and
coordinates of a first characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image respectively;
and
coordinates of a second feature point pair of the reference polarized sub-image and the coarse registration polarized sub-image respectively;
and
the coordinates of the third pair of feature points of the reference polarized sub-image and the coarsely registered polarized sub-image, respectively.
Determining transformed coordinates (x) of remaining feature points of the coarse registration polarization subimage according to equation (7) cal ,y cal )：
Wherein (x) rough ,y rough ) Is coarseThe coordinates of the remaining feature points in the polarized sub-image are registered.
And the absolute value of the difference between the feature point transformation coordinate of the rough registration polarization subimage and the corresponding feature point coordinate of the reference polarization subimage is less than 0.5, the feature point is marked as an interior point, and the number of the interior points in the polarization subimage is determined.
(5) Obtaining optimal registration coefficients for polarization sub-images
And (5) selecting any other 3 characteristic point pairs in the characteristic point pairs, and repeating the step (4) until all the characteristic point pairs are traversed.
The optimal registration coefficient a' is determined according to equation (8):
wherein,
and
respectively obtaining the coordinates of a first characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image when the number of the inner points is the maximum;
and
respectively the coordinates of the second characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image when the number of the inner points is the maximum;
and
and coordinates of a third characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image when the number of the inner points is the maximum are respectively obtained.
(6) Completing a polarized image registration
Determining coordinates (x) of the registered polarized subimages according to equation (9) reg ,y reg )：
Obtaining a registered polarization image I according to the formula (10) reg (x′,y′)：
I reg (x′,y′)＝I rough (x reg ,y reg ) (10) in which I rough (x reg ,y reg ) Registering the polarization sub-image in coordinates (x) for coarse reg ,y reg ) The intensity of the spot.
In the step of (1) acquiring the polarized sub-image, the acquiring original frame image includes a frame of sub-aperture polarized image containing a 0 ° linear polarized image, a 45 ° linear polarized image, a 90 ° linear polarized image, and a circular polarized image; the original frame image is cut into four polarized sub-images, wherein the four polarized sub-images comprise 0-degree linear polarized sub-images, 45-degree linear polarized sub-images, 90-degree linear polarized sub-images and circular polarized sub-images which are the same in size.
In the step (3) of extracting the characteristic points of the polarized sub-images, the method for respectively extracting the characteristic points of the reference polarized sub-images and the rough registration polarized sub-images is a scale-invariant characteristic transformation method or an accelerated robust characteristic method.
In the step (3) of extracting the characteristic point of the polarized sub-image, the ratio of the minimum Euclidean distance to the next minimum Euclidean distance is not more than the set threshold condition, and the value of the threshold is 0.5-0.8.
According to the invention, the polarized sub-images are subjected to rough registration, the calculated amount when the characteristic points of the polarized sub-images are extracted is reduced, the optimal registration coefficient when the number of the points in the polarized sub-images is the maximum is determined by traversing the characteristic points, the high-precision, fast-speed and full-automatic registration between the polarized sub-images is completed, the technical problem of position deviation between the sub-images of the aperture-dividing polarization is solved, the image quality when the aperture-dividing polarization camera is used for polarization imaging is improved, and the method can be used for polarization imaging of the aperture-dividing polarization camera.
Drawings
FIG. 1 is a flowchart of example 1 of the present invention.
Fig. 2 is a 0 ° linearly polarized subimage of example 1 of the present invention.
Fig. 3 is the registration result of the 45 ° linearly polarized sub-images in embodiment 1 of the present invention.
Detailed Description
The present invention will be described in further detail below with reference to the drawings and examples, but the present invention is not limited to the embodiments described below.
Example 1
The image registration method of the split-aperture polarization camera of the embodiment comprises the following steps (see fig. 1):
(1) obtaining a polarized sub-image
The method comprises the steps of using a sub-aperture polarization camera to shoot a target, obtaining an original frame image, cutting the original frame image into four polarization sub-images, selecting any one polarization sub-image in the four polarization sub-images as a reference polarization sub-image, and using the other three polarization sub-images as polarization sub-images to be registered.
The acquiring original frame image of the embodiment includes a frame of sub-aperture polarization image containing a 0 ° linear polarization image, a 45 ° linear polarization image, a 90 ° linear polarization image, and a circular polarization image; the original frame image is cut into four polarized sub-images, wherein the four polarized sub-images comprise 0-degree linear polarized sub-images, 45-degree linear polarized sub-images, 90-degree linear polarized sub-images and circular polarized sub-images which are the same in size. The 0 ° linearly polarized subimage serves as a reference polarized subimage (see fig. 2).
(2) Coarse registration of polarization subimages
Determining the Fourier transform F (omega) of a reference polarization subimage by equation (1) x ,ω y )：
F(ω x ,ω y )＝F{f(x,y)} (1)
Where F (x, y) is a reference polarized subimage of coordinates (x, y), and F { } represents taking a fourier transform.
Determining the Fourier transform G (omega) of the polarization subimage to be registered according to equation (2) x ,ω y )：
G(ω x ,ω y )＝F{g(x,t)} (2)
Wherein g (x, y) is the polarization subimage to be registered at coordinates (x, y).
The impulse function δ (x-dx, y-dy) is determined as in equation (3):
wherein dx is the displacement of the polarization sub-image to be registered in the horizontal direction, dy is the displacement of the polarization sub-image to be registered in the vertical direction, and F -1 Is an inverse Fourier transform, F * (ω x ,ω y ) Is F (omega) x ,ω y ) Complex conjugation of (a).
The coordinates (x ', y') of the coarsely registered polarization subimages are determined as in equation (4):
wherein (x) 0 ,y 0 ) The coordinates of the polarization sub-image are registered.
Determination of a coarse registration polarization subimage I according to equation (5) rough (x 0 ,y 0 )：
I rough (x 0 ,y 0 )＝I(x′,y′) (5)
Where I (x ', y') is the intensity of the polarization sub-image to be registered at the coordinates (x ', y').
(3) Extracting characteristic points of polarized sub-images
Respectively extracting the feature points of the reference polarized sub-image and the rough registration polarized sub-image, wherein the embodiment adopts a scale-invariant feature transformation method, which is disclosed in the national Journal of Computer Vision,2004,60(2):91-110. For any feature point in the reference polarization sub-image, two feature points with the minimum Euclidean distance and the second minimum Euclidean distance in the rough registration polarization sub-image are selected, and according to the condition that the ratio of the minimum Euclidean distance to the second minimum Euclidean distance is not greater than a set threshold value, the threshold value is 0.5-0.8, and the threshold value of the embodiment is 0.6.
The judgment method for forming the characteristic point pair by the other characteristic points is the same as the judgment method for forming the characteristic point pair by the selected characteristic point in the reference polarization sub-image and the characteristic point with the minimum European distance in the rough registration polarization sub-image.
(4) Determining the number of points within a polarized subimage
Determining an affine transformation matrix a between the coarsely registered polarization sub-image and the reference polarization sub-image according to equation (6):
wherein,
and
coordinates of a first characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image respectively;
and
coordinates of a second feature point pair of the reference polarized sub-image and the coarse registration polarized sub-image respectively;
and
the coordinates of the third pair of feature points of the reference polarized sub-image and the coarsely registered polarized sub-image, respectively.
Determining transformed coordinates (x) of remaining feature points of the coarsely registered polarized subimages as in equation (7) cal ,y cal )：
Wherein (x) rough ,y rough ) The coordinates of the remaining feature points in the polarization sub-image are coarsely registered.
And the absolute value of the difference between the feature point transformation coordinate of the rough registration polarization sub-image and the corresponding feature point coordinate of the reference polarization sub-image is less than 0.5, the feature point is marked as an inner point, and the number of the inner points in the polarization sub-image is determined.
(5) Obtaining optimal registration coefficients for polarization sub-images
And (4) selecting any other 3 characteristic point pairs in the characteristic point pairs, and repeating the step (4) until all the characteristic point pairs are traversed.
The optimal registration coefficient a' is determined according to equation (8):
wherein,
and
respectively obtaining the coordinates of a first characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image when the number of the inner points is the maximum;
and
respectively the coordinates of the second characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image when the number of the inner points is the maximum;
and
and coordinates of a third characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image when the number of the inner points is the maximum are respectively obtained.
(6) Completing a polarized image registration
Determining coordinates (x) of the registered polarization sub-images according to equation (9) reg ,y reg )：
Obtaining a registered polarization image I according to the formula (10) reg (x′,y′)：
I reg (x′,y′)＝I rough (x reg ,y reg ) (10)
Wherein, I rough (x reg ,y reg ) For coarse registration of polarization sub-images in coordinates (x) reg ,y reg ) The intensity of the spot.
And finishing the image registration method of the sub-aperture polarization camera. And obtaining a registered 45-degree linear polarization sub-image (see fig. 3), and as can be seen from fig. 2 and 3, the invention realizes image registration of the sub-aperture polarization sub-image.
Example 2
The image registration method of the split-aperture polarization camera of the embodiment comprises the following steps:
(1) obtaining a polarized sub-image
This procedure is the same as in example 1.
(2) Coarse registration of polarization subimages
This procedure is the same as in example 1.
(3) Extracting characteristic points of polarized sub-images
Respectively extracting feature points of a reference polarized sub-image and a rough registration polarized sub-image, selecting two feature points with the minimum Euclidean distance and the second minimum Euclidean distance in the rough registration polarized sub-image for any feature point in the reference polarized sub-image, and taking the threshold value as 0.5-0.8 according to the condition that the ratio of the minimum Euclidean distance to the second minimum Euclidean distance is not greater than the set threshold value condition, wherein the threshold value of the embodiment is 0.5.
And forming a characteristic point pair by the selected characteristic point in the reference polarization sub-image and the characteristic point with the minimum European distance in the rough registration polarization sub-image, wherein the judgment method for forming the characteristic point pair by other characteristic points is the same as the judgment method.
The other steps were the same as in example 1.
And finishing the image registration method of the sub-aperture polarization camera.
Example 3
The image registration method of the split-aperture polarization camera of the embodiment comprises the following steps:
(1) obtaining a polarized sub-image
This procedure is the same as in example 1.
(2) Coarse registration of polarization subimages
This procedure is the same as in example 1.
(3) Extracting characteristic points of polarized sub-images
Respectively extracting feature points of a reference polarized sub-image and a rough registration polarized sub-image, selecting two feature points with the minimum Euclidean distance and the second minimum Euclidean distance in the rough registration polarized sub-image for any feature point in the reference polarized sub-image, and taking the threshold value as 0.5-0.8 according to the condition that the ratio of the minimum Euclidean distance to the second minimum Euclidean distance is not greater than the set threshold value condition, wherein the threshold value of the embodiment is 0.8.
The judgment method for forming the characteristic point pair by the other characteristic points is the same as the judgment method for forming the characteristic point pair by the selected characteristic point in the reference polarization sub-image and the characteristic point with the minimum European distance in the rough registration polarization sub-image.
The other steps were the same as in example 1.
And finishing the image registration method of the sub-aperture polarization camera.
Example 4
In the above embodiments 1 to 3, the image registration method of the split-aperture polarization camera of the present embodiment includes the following steps:
(1) obtaining a polarized sub-image
This procedure is the same as in example 1.
(2) Coarse registration of polarization subimages
This procedure is the same as in example 1.
(3) Extracting characteristic points of polarized sub-images
Respectively extracting feature points of a reference polarization sub-image and a rough registration polarization sub-image, adopting an accelerated robust feature method in the embodiment, selecting two feature points with the minimum Euclidean distance and the second minimum Euclidean distance in the rough registration polarization sub-image for any feature point in the reference polarization sub-image, and taking a threshold value of 0.5-0.8 according to the condition that the ratio of the minimum Euclidean distance to the second minimum Euclidean distance is not greater than a set threshold value, wherein the threshold value of the embodiment is the same as that of the corresponding embodiment.
The judgment method for forming the characteristic point pair by the other characteristic points is the same as the judgment method for forming the characteristic point pair by the selected characteristic point in the reference polarization sub-image and the characteristic point with the minimum European distance in the rough registration polarization sub-image.
The other steps were the same as in example 1.
And finishing the image registration method of the sub-aperture polarization camera.
In order to verify the beneficial effects of the present invention, the inventor performed experiments on the images of the split-aperture polarization camera by using the image registration method of the split-aperture polarization camera in embodiment 1 of the present invention, and the experimental conditions are as follows.
1. Conditions of the experiment
The experimental test environment is a Hewlett packard computer of a Windows l0 (64-bit) operating system, which is configured as an Intercore i3-10105F and 16GB memory, and experimental operation is carried out on a MATLAB R2019b platform.
2. Introduction to Experimental data
The original frame image shot by the aperture-dividing polarization camera is shot by the aperture-dividing polarization camera of the advanced optical imaging laboratory of university of Shanxi.
3. Evaluation index
The Structural Similarity Index (SSIM) and Normalized Mutual Information (NMI) were used as evaluation indices. Determining the structural similarity index according to equation (11):
SSIM(R,T)＝L(R,T)C(R,T)S(R,T) (11)
wherein, R and T represent two images with the same resolution, L (R, T) represents the brightness correlation function of the two images, C (R, T) represents the contrast correlation function of the two images, S (R, T) represents the structure correlation function of the two images, and mu R And mu T Respectively representing the respective mean values of the gray levels, σ, of the two images R And σ T Respectively representing the respective gray scale standard deviations, sigma, of the two images RT Representing the gray-scale covariance, C, between image R and image T 1 、C 2 And C 3 All are small normal numbers, and are intended to avoid instability caused by denominator of 0 or close to 0.
The normalized mutual information is determined as follows (12):
wherein M and N represent two images with the same resolution and the same gray scale, H (M) represents the average information content of the image M, H (N) represents the average information content of the image N, H (M, N) represents the related average information content between the image M and the image N, M and N represent the gray scale value of any pixel point in the image M and the image N respectively, P M (M) an edge probability density function, P, of the image M N (N) an edge probability density function, P, for the image N MN (M, N) represents the joint probability density function between image M and image N.
And quantitatively evaluating the similarity degree of the images of the split-aperture polarization camera before registration with the images which are not registered.
The test was performed according to the method of example 1, and the structural similarity index between the sub-aperture polarized images is shown in table 1, and the normalized mutual information between the sub-aperture polarized images is shown in table 2.
TABLE 1 Structural Similarity Index (SSIM) of each two sub-aperture polarization images
TABLE 2 Normalized Mutual Information (NMI) of every two sub-aperture polarization images
In table 1, SSIM _0_45 represents a structural similarity index of a 0 ° linearly polarized sub-image and a 45 ° linearly polarized sub-image; SSIM — 0 — 90 represents the structural similarity index of the 0 ° linearly polarized sub-image and the 90 ° linearly polarized sub-image; SSIM _0_ C represents the structural similarity index of the 0 ° linearly polarized sub-image and the circularly polarized sub-image; SSIM _45_90 represents the structural similarity index of the 45 ° linearly polarized sub-image and the 90 ° linearly polarized sub-image; SSIM _45_ C represents the structural similarity index of the 45 ° linearly polarized sub-image and the circularly polarized sub-image; SSIM _90_ C represents the structural similarity index of a 90 ° linearly polarized sub-image to a circularly polarized sub-image.
In table 2, NMI _0_45 represents normalized mutual information of the 0 ° linearly polarized sub-image and the 45 ° linearly polarized sub-image; NMI _0_90 represents normalized mutual information of the 0-degree linear polarization sub-image and the 90-degree linear polarization sub-image; NMI _0_ C represents the normalized mutual information of the linear polarization sub-image and the circular polarization sub-image of 0 degree; NMI _45_90 represents normalized mutual information of the 45-degree linear polarization sub-image and the 90-degree linear polarization sub-image; NMI _45_ C represents normalized mutual information of the 45-degree linear polarization sub-image and the circular polarization sub-image; NMI _90_ C represents the normalized mutual information of the 90 ° linearly polarized sub-image and the circularly polarized sub-image.
As can be seen from tables 1 and 2, compared with the four unregistered polarization images, the four polarization images after registration in the invention have the structure similarity index improved by 72.08% on average and the normalized mutual information improved by 18.5% on average, which indicates that the invention can well realize the image registration of the aperture-dividing polarization camera.
Claims (4)
1. An image registration method of a split-aperture polarization camera is characterized by comprising the following steps:
(1) obtaining a polarized sub-image
Shooting a target by using a sub-aperture polarization camera, acquiring an original frame image, cutting the original frame image into four polarization sub-images, selecting any one polarization sub-image in the four polarization sub-images as a reference polarization sub-image, and taking the other three polarization sub-images as polarization sub-images to be registered;
(2) coarse registration of polarized subimages
Determining the Fourier transform F (ω) of a reference polarization sub-image as in equation (1) x ，ω y )：
F(ω x ，ω y )＝F{f(x，y)} (1)
Wherein F (x, y) is a reference polarized sub-image of coordinates (x, y), and F { } represents taking Fourier transform;
determining the Fourier transform G (omega) of the polarization subimage to be registered according to equation (2) x ，ω y )：
G(ω x ，ω y )＝F{g(x，y)} (2)
Wherein g (x, y) is a polarization subimage to be registered of coordinates (x, y);
the impulse function δ (x-dx, y-dy) is determined as in equation (3):
wherein dx is the displacement of the polarization sub-image to be registered in the horizontal direction, dy is the displacement of the polarization sub-image to be registered in the vertical direction, and F -1 Is an inverse Fourier transform, F * (ω x ，ω y ) Is F (omega) x ，ω y ) Complex conjugation of (a);
the coordinates (x ', y') of the coarsely registered polarization subimages are determined as in equation (4):
wherein (x) 0 ，y 0 ) Coordinates of the polarization subimage to be registered are obtained;
determining a coarse registration polarization subimage I according to equation (5) rough (x 0 ，y 0 )：
I rough (x 0 ，y 0 )＝I(x′，y′) (5)
Wherein I (x ', y') is the intensity of the polarization subimage to be registered at the coordinates (x ', y');
(3) extracting characteristic points of polarized sub-images
Respectively extracting feature points of the reference polarized sub-image and the rough registration polarized sub-image, selecting two feature points with the minimum Euclidean distance and the second minimum Euclidean distance in the rough registration polarized sub-image for any one feature point in the reference polarized sub-image, forming a feature point pair by the selected feature point in the reference polarized sub-image and the feature point with the minimum Euclidean distance in the rough registration polarized sub-image according to the condition that the ratio of the minimum Euclidean distance to the second minimum Euclidean distance is not larger than a set threshold value, and forming the feature point pair by other feature points by the same judging method as the above;
(4) determining the number of dots within a polarized sub-image
Determining an affine transformation matrix a between the coarsely registered polarization sub-image and the reference polarization sub-image according to equation (6):
wherein,
and
coordinates of a first characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image respectively;
and
coordinates of a second feature point pair of the reference polarized sub-image and the coarse registration polarized sub-image respectively;
and
coordinates of a third feature point pair of the reference polarized sub-image and the rough registration polarized sub-image respectively;
determining transformed coordinates (x) of remaining feature points of the coarsely registered polarized subimages as in equation (7) cal ，y cal )：
Wherein (x) rough ，y rough ) Coordinates of other feature points in the polarized sub-image are roughly registered;
the absolute value of the difference between the feature point transformation coordinate of the rough registration polarization subimage and the corresponding feature point coordinate of the reference polarization subimage is less than 0.5, the feature point is marked as an interior point, and the number of the interior points in the polarization subimage is determined;
(5) obtaining optimal registration coefficients for polarization sub-images
Selecting any other 3 characteristic point pairs in the characteristic point pairs, and repeating the step (4) until all the characteristic point pairs are traversed;
the optimal registration coefficient a' is determined according to equation (8):
wherein,
and
respectively obtaining the coordinates of a first characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image when the number of the inner points is the maximum;
and
respectively the coordinates of the second characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image when the number of the inner points is the maximum;
and
respectively obtaining the coordinates of the third characteristic point pair of the reference polarized sub-image and the rough registration polarized sub-image when the number of the inner points is the maximum;
(6) completing the polarized image registration
Determining coordinates (x) of the registered polarization sub-images according to equation (9) reg ，y reg )：
Obtaining a registered polarization image I according to the formula (10) reg (x′，y′)：
I reg (x′，y′)＝I rough (x reg ，y reg ) (10)
Wherein, I rough (x reg ，y reg ) Registering the polarization sub-image in coordinates (x) for coarse reg ，y reg ) The intensity of the spot.
2. The image registration method of a split-aperture polarization camera according to claim 1, wherein: in (1) the acquiring polarization sub-image step, the acquiring original frame image includes a frame of sub-aperture polarization image containing 0 ° linear polarization image, 45 ° linear polarization image, 90 ° linear polarization image, circular polarization image; the original frame image is cut into four polarized sub-images, wherein the four polarized sub-images comprise 0-degree linear polarized sub-images, 45-degree linear polarized sub-images, 90-degree linear polarized sub-images and circular polarized sub-images which are the same in size.
3. The image registration method of a split-aperture polarization camera according to claim 1, wherein: in the step (3) of extracting the characteristic points of the polarized sub-images, the method for respectively extracting the characteristic points of the reference polarized sub-images and the rough registration polarized sub-images is a scale-invariant characteristic transformation method or an accelerated robust characteristic method.
4. The image registration method of a split-aperture polarization camera according to claim 1, wherein: in the step (3) of extracting the characteristic point of the polarized sub-image, the ratio of the minimum Euclidean distance to the second minimum Euclidean distance is not more than the set threshold condition, and the value of the threshold is 0.5-0.8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210670279.9A CN114937067B (en) | 2022-06-14 | 2022-06-14 | Image registration method of split-aperture polarization camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210670279.9A CN114937067B (en) | 2022-06-14 | 2022-06-14 | Image registration method of split-aperture polarization camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114937067A true CN114937067A (en) | 2022-08-23 |
| CN114937067B CN114937067B (en) | 2023-01-31 |
Family
ID=82866432
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210670279.9A Active CN114937067B (en) | 2022-06-14 | 2022-06-14 | Image registration method of split-aperture polarization camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114937067B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160267348A1 (en) * | 2013-12-16 | 2016-09-15 | Sony Corporation | Image processing device and image processing method |
| CN107392948A (en) * | 2017-07-10 | 2017-11-24 | 大连理工大学 | A kind of method for registering images of point of amplitude real-time polarization imaging system |
| CN108765476A (en) * | 2018-06-05 | 2018-11-06 | 安徽大学 | Polarized image registration method |
| CN109886883A (en) * | 2019-01-21 | 2019-06-14 | 吉林大学 | Real-time polarization fog-penetrating imaging image enhancement processing method |
| CN111882594A (en) * | 2020-07-27 | 2020-11-03 | 北京环境特性研究所 | ORB feature point-based polarization image rapid registration method and device |
| US20210335013A1 (en) * | 2020-04-27 | 2021-10-28 | Huazhong University Of Science And Technology | Method for characterizing polarization image information and method for computing characterization parameters |
-
2022
- 2022-06-14 CN CN202210670279.9A patent/CN114937067B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160267348A1 (en) * | 2013-12-16 | 2016-09-15 | Sony Corporation | Image processing device and image processing method |
| CN107392948A (en) * | 2017-07-10 | 2017-11-24 | 大连理工大学 | A kind of method for registering images of point of amplitude real-time polarization imaging system |
| CN108765476A (en) * | 2018-06-05 | 2018-11-06 | 安徽大学 | Polarized image registration method |
| CN109886883A (en) * | 2019-01-21 | 2019-06-14 | 吉林大学 | Real-time polarization fog-penetrating imaging image enhancement processing method |
| US20210335013A1 (en) * | 2020-04-27 | 2021-10-28 | Huazhong University Of Science And Technology | Method for characterizing polarization image information and method for computing characterization parameters |
| CN111882594A (en) * | 2020-07-27 | 2020-11-03 | 北京环境特性研究所 | ORB feature point-based polarization image rapid registration method and device |
Non-Patent Citations (4)
| Title |
|---|
| 万钇良等: "一种基于相位相关与子图像的偏振图像配准方法", 《液晶与显示》 * |
| 巨海娟等: "全偏振态同时探测实时彩色偏振成像技术", 《红外与毫米波学报》 * |
| 杨长久等: "同时偏振成像探测系统的偏振图像配准研究", 《红外与激光工程》 * |
| 褚金奎等: "偏振探测系统的图像配准", 《光学精密工程》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114937067B (en) | 2023-01-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110097093B (en) | Method for accurately matching heterogeneous images | |
| KR101117837B1 (en) | Multi-image feature matching using multi-scale oriented patches | |
| CN111340701B (en) | Circuit board image splicing method for screening matching points based on clustering method | |
| CN110969670B (en) | Multispectral camera dynamic three-dimensional calibration method based on significant features | |
| CN110992263B (en) | Image stitching method and system | |
| CN110956661B (en) | Method for calculating dynamic pose of visible light and infrared camera based on bidirectional homography matrix | |
| US9767383B2 (en) | Method and apparatus for detecting incorrect associations between keypoints of a first image and keypoints of a second image | |
| CN110969667A (en) | Multi-spectrum camera external parameter self-correction algorithm based on edge features | |
| CN110909750B (en) | Image difference detection method and device, storage medium and terminal | |
| CN113012234B (en) | High-precision camera calibration method based on plane transformation | |
| CN110969669A (en) | Visible light and infrared camera combined calibration method based on mutual information registration | |
| CN112017223A (en) | Heterologous image registration method based on improved SIFT-Delaunay | |
| CN114897705A (en) | Unmanned aerial vehicle remote sensing image splicing method based on feature optimization | |
| CN115272403A (en) | Fragment scattering characteristic testing method based on image processing technology | |
| CN114863384A (en) | Traffic sign detection method based on YOLO v4 algorithm | |
| CN114937067B (en) | Image registration method of split-aperture polarization camera | |
| CN112184533B (en) | Watermark synchronization method based on SIFT feature point matching | |
| CN113642397A (en) | Object length measuring method based on mobile phone video | |
| CN113096016A (en) | Low-altitude aerial image splicing method and system | |
| CN110322476B (en) | Target tracking method for improving STC and SURF feature joint optimization | |
| CN111833384A (en) | Method and device for quickly registering visible light and infrared images | |
| CN114565653B (en) | Heterologous remote sensing image matching method with rotation change and scale difference | |
| CN111402281B (en) | Book edge detection method and device | |
| CN108389219B (en) | Weak and small target tracking loss re-detection method based on multi-peak judgment | |
| CN113963174B (en) | Bogie identification image feature extraction method for multi-view intensity domain and frequency domain fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |