CN109389630A - Visible images and the determination of Infrared Image Features point set, method for registering and device - Google Patents
Visible images and the determination of Infrared Image Features point set, method for registering and device Download PDFInfo
- Publication number
- CN109389630A CN109389630A CN201811162538.7A CN201811162538A CN109389630A CN 109389630 A CN109389630 A CN 109389630A CN 201811162538 A CN201811162538 A CN 201811162538A CN 109389630 A CN109389630 A CN 109389630A
- Authority
- CN
- China
- Prior art keywords
- infrared image
- visible images
- point set
- feature point
- infrared
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a kind of visible images and the determination of Infrared Image Features point set, method for registering and device, belong to monitoring technical field.The described method includes: obtaining visible images and infrared image, and identify each characteristic point in the visible images, obtains fisrt feature point set, identify each characteristic point in the infrared image, obtain second feature point set；According to the image transformation matrix of Visible Light Camera and infrared camera, determine that the fisrt feature point set and second feature point concentrate the characteristic point pair being mutually matched；According to the characteristic point pair being mutually matched, visible images feature point set and Infrared Image Features point set are formed.Visible images provided in an embodiment of the present invention and Infrared Image Features point set determine method, the characteristic point pair to match each other is determined using the image transformation matrix of Visible Light Camera and infrared camera, with clock synchronization, without carrying out matching primitives one by one to each characteristic point, calculation amount is small, calculating speed is fast, registration is high-efficient.
Description
Technical field
The present invention relates to a kind of visible images and the determination of Infrared Image Features point set, method for registering and device, belong to prison
Survey technology field.
Background technique
Target identification technology mostly uses Visible Light Camera at present.Visible images include spectral information abundant, have line
The features such as reason, shade also cause its operation complicated, simply and effectively can not quickly be identified to human body, animal etc.；And it is infrared
Camera can people in fast Acquisition environment with higher temperature, animal, but general resolution ratio is lower, details missing, Limited information.
Furthermore infrared camera is very sensitive to sunlight, and target information is easy to be submerged, this will limit its use significantly.
Currently, in target detection and field of target recognition, it usually needs at the same it is same using Visible Light Camera and infrared camera
When obtain the image information of same target, to improve target detection rate and discrimination using two different images.
Due to the difference of image-forming mechanism and camera position, parameter, visible images and the infrared image of Same Scene are not
It just matches, it is therefore desirable to two images are registrated, Chang Liyong SURF algorithm extracts characteristic point and matched to calculate
Transformation matrix, this method need first to identify characteristic point, then determine matching relationship according to the vector distance between each characteristic point
And transformation matrix calculating is carried out, this method needs to carry out complicated feature point pair matching calculating and registration transformation matrix and calculates, and leads
Cause calculating speed is slow, is registrated low efficiency.
Summary of the invention
The purpose of the invention is to provide determining a kind of visible images and Infrared Image Features point set, method for registering and
Device determines the characteristic point pair to match each other using the image transformation matrix of Visible Light Camera and infrared camera, matches clock synchronization, is not necessarily to
Matching primitives are carried out one by one to each characteristic point, calculation amount is small, calculating speed is fast, registration is high-efficient.
The purpose of the present invention is what is be achieved through the following technical solutions:
A kind of visible images and Infrared Image Features point set determine method, comprising:
Visible images and infrared image are obtained, and identifies each characteristic point in the visible images, obtains the first spy
Point set is levied, each characteristic point in the infrared image is identified, obtains second feature point set；
According to the image transformation matrix of Visible Light Camera and infrared camera, the fisrt feature point set and second feature are determined
Point concentrates the characteristic point pair being mutually matched；
According to the characteristic point pair being mutually matched, visible images feature point set and Infrared Image Features point set are formed.
In an alternative embodiment, further includes:
According to visible light uncalibrated image and infrared calibration image, the image transformation square of Visible Light Camera and infrared camera is determined
Battle array.
In an alternative embodiment, each characteristic point identified in the visible images obtains fisrt feature point set,
Identify each characteristic point in the infrared image, obtaining second feature point set includes:
According to visible light uncalibrated image and infrared calibration image, determines and participate in infrared camera internal reference in Visible Light Camera；
According to the determining internal reference, the visible images and infrared image of acquisition are corrected；
Each characteristic point in the visible images after identification correction obtains fisrt feature point set, after identification correction
Each characteristic point in the infrared image, obtains second feature point set.
A kind of visible images and infrared image method for registering, comprising:
Visible images and infrared image are obtained, and identifies each characteristic point in the visible images, obtains the first spy
Point set is levied, each characteristic point in the infrared image is identified, obtains second feature point set；
According to the image transformation matrix of Visible Light Camera and infrared camera, the fisrt feature point set and second feature are determined
Point concentrates the characteristic point pair being mutually matched；
According to the characteristic point pair being mutually matched, visible images feature point set and Infrared Image Features point set are formed；
According to the visible images feature point set and Infrared Image Features point set, to visible images and infrared image into
Row registration.
In an alternative embodiment, further includes:
According to visible light uncalibrated image and infrared calibration image, the image transformation square of Visible Light Camera and infrared camera is determined
Battle array.
In an alternative embodiment, each characteristic point identified in the visible images obtains fisrt feature point set,
Identify each characteristic point in the infrared image, obtaining second feature point set includes:
According to visible light uncalibrated image and infrared calibration image, determines and participate in infrared camera internal reference in Visible Light Camera；
According to the determining internal reference, the visible images and infrared image of acquisition are corrected；
Each characteristic point in the visible images after identification correction obtains fisrt feature point set, after identification correction
Each characteristic point in the infrared image, obtains second feature point set.
It is described according to the visible images feature point set and Infrared Image Features point set in an alternative embodiment, it is right
Visible images are registrated with infrared image, comprising:
Determine that the visible images feature point set and Infrared Image Features point concentrate the vector distance of each characteristic point pair；
The vector distance of each characteristic point pair is compared with preset standard phasor distance, it is visible after being screened
Infrared Image Features point set after light image feature point set and screening；
According to the visible images feature point set after the screening and the Infrared Image Features point set after screening to visible light
Image is registrated with infrared image.
It is described according to the visible images feature point set and Infrared Image Features point set in an alternative embodiment, it is right
Visible images are registrated with infrared image, comprising:
Whether the image transformation matrix for judging the Visible Light Camera and infrared camera according to the following formula is transformation matrix of coordinates:
Crad (C) >=η n, C={ (Ai,Bi)|||TAi-Bi||2< ε }
Wherein, Crad (C) is the element number of C, and T is the image transformation matrix of Visible Light Camera and infrared camera, AiFor
The ith feature point of visible images, BiFor with AiThe characteristic point of matched infrared image, the positive integer of i=1~n, n are described
The total quantity of characteristic point in visible images feature point set after screening, ε and η are judgment threshold；
If so, according to the image transformation matrix of the Visible Light Camera and infrared camera to visible images and red
Each pixel is coordinately transformed to realize registration in outer image.
In an alternative embodiment, further includes:
If it is not, determining that the visible images feature point set and infrared image are special then according to random consistency sampling algorithm
Levy the corresponding transformation matrix of point set；
According to determining transformation matrix to each pixel in the visible images and infrared image be coordinately transformed with
Realize registration.
In an alternative embodiment, the random consistency sampling algorithm of basis determines the visible images characteristic point
Collect transformation matrix corresponding with Infrared Image Features point set, comprising:
(1) several groups of characteristic points pair are randomly selected from the visible images feature point set and Infrared Image Features point concentration,
Intra-office point is formed to collection；
(2) determine intra-office point to collection corresponding transformation matrix collection according to the intra-office point；
(3) judge that the visible images feature point set and Infrared Image Features point concentrate remaining characteristic point to whether full
The foot intra-office point is to the corresponding transformation matrix of collection；
(4) if so, by the remaining characteristic point to being included in the intra-office point in collection；
(5) when the intra-office point reaches preset value to the quantity of the characteristic point pair in collection, according to the intra-office point to collection
Corresponding transformation matrix determines the visible images feature point set and the corresponding transformation matrix of Infrared Image Features point set.
In an alternative embodiment, institute is determined to corresponding transformation matrix is collected according to the intra-office point described in step (5)
State visible images feature point set and the corresponding transformation matrix of Infrared Image Features point set, comprising:
It repeats step (1)-(4) and obtains multiple intra-office points to the corresponding transformation matrix of collection；
Q when according to the following formula, by k value minimumjAs the visible images feature point set and Infrared Image Features point set
Corresponding transformation matrix:
K=∑ | | QjAi-Bi||2
Wherein, QjIt is intra-office point to collection to corresponding transformation matrix, the positive integer of j=1~m, m is collected for j-th of intra-office point
Total quantity, AiFor the ith feature point of visible images, BiFor with AiThe characteristic point of matched infrared image, i=1-n, n are
The total quantity of characteristic point in visible images feature point set after the screening.
It is described that each pixel in the visible images and infrared image is coordinately transformed in an alternative embodiment
To realize registration, comprising:
The coordinate of each pixel of the infrared image is converted, rounding, the infrared image after obtaining coordinate transform；
By the pixel value of pixel each in the infrared image after the coordinate transform and the corresponding visible images picture
The pixel value of vegetarian refreshments is merged, and registration is completed.
In an alternative embodiment, by the pixel value of pixel each in the infrared image after the coordinate transform with it is corresponding
The pixel value of the visible images pixel is merged, comprising:
The adjacent pixels point of each pixel in the absent region in infrared image after determining the coordinate transform；
The pixel value that each pixel in the absent region is determined according to the pixel value of the adjacent pixels point, obtains interpolation
Infrared image afterwards；
According to the pixel value of pixel each in the infrared image after the interpolation and the corresponding visible images pixel
The pixel value of point is merged.
A kind of visible images and Infrared Image Features point set determining device, comprising:
Identification module is obtained, for obtaining visible images and infrared image, and is identified each in the visible images
Characteristic point obtains fisrt feature point set, identifies each characteristic point in the infrared image, obtains second feature point set；
Determining module determines the fisrt feature for the image transformation matrix according to Visible Light Camera and infrared camera
Point set and second feature point concentrate the characteristic point pair being mutually matched；
Form module, the characteristic point pair for being mutually matched according to, formation visible images feature point set and infrared
Characteristics of image point set.
A kind of visible images and infrared image registration apparatus, comprising:
Identification module is obtained, for obtaining visible images and infrared image, and is identified each in the visible images
Characteristic point obtains fisrt feature point set, identifies each characteristic point in the infrared image, obtains second feature point set；
Determining module determines the fisrt feature for the image transformation matrix according to Visible Light Camera and infrared camera
Point set and second feature point concentrate the characteristic point pair being mutually matched；
Form module, the characteristic point pair for being mutually matched according to, formation visible images feature point set and infrared
Characteristics of image point set；
Registration module is used for according to the visible images feature point set and Infrared Image Features point set, to visible light figure
As being registrated with infrared image.
The present invention is possessed compared with prior art to be had the beneficial effect that:
Visible images provided in an embodiment of the present invention and Infrared Image Features point set determine method, make full use of known
The relative position of infrared camera and Visible Light Camera is determined mutual using the image transformation matrix of Visible Light Camera and infrared camera
Matched characteristic point pair matches clock synchronization, and without carrying out matching primitives one by one to each characteristic point, calculation amount is small, calculating speed is fast, registration
It is high-efficient；With punctual, without carrying out transformation matrix calculating every time, it is registrated high-efficient.
Detailed description of the invention
Fig. 1 is that the embodiment of the invention provides a kind of visible images and Infrared Image Features point set to determine method flow
Figure；
Fig. 2 is a kind of visible images and infrared image method for registering flow chart provided in an embodiment of the present invention；
Fig. 3 is a kind of visible images and infrared image method for registering flow chart that a specific embodiment of the invention provides；
Fig. 4 is that the embodiment of the invention provides a kind of visible images and Infrared Image Features point set determining device to illustrate
Figure；
Fig. 5 is that the embodiment of the invention provides a kind of visible images and infrared image registration apparatus schematic diagram.
Specific embodiment
Below in conjunction with drawings and examples, the invention will be further described.
Referring to Fig. 1, the embodiment of the invention provides a kind of visible images and Infrared Image Features point set to determine method, wraps
It includes:
Step 101: obtaining visible images and infrared image, and identify each characteristic point in the visible images, obtain
To fisrt feature point set, identifies each characteristic point in the infrared image, obtain second feature point set；
Step 102: according to the image transformation matrix T of Visible Light Camera and infrared camera, determining the fisrt feature point set
And second feature point concentrates the characteristic point pair being mutually matched；
Specifically, when shooting due to Visible Light Camera and infrared camera, installation site has differences, and the two is caused to be clapped
It takes the photograph gained each pixel coordinate of image to have differences, shooting gained image according to the two can be obtained image transformation matrix；First
Each characteristic point in feature point set obtains changing the characteristic point after coordinate, if second after image transformation matrix coordinate transforming
There is the characteristic point consistent or almost the same with the characteristic point coordinate after variation coordinate in feature point set, then it is assumed that fisrt feature point
Collection and second feature point concentrate corresponding two Feature Points Matchings；The characteristic point centering includes what a fisrt feature point was concentrated
Characteristic point and second feature point concentrate matching characteristic point；
Step 103: according to the characteristic point pair being mutually matched, forming visible images feature point set and infrared image is special
Levy point set.
Specifically, fisrt feature point set and second feature point are concentrated and are mismatched on the basis of step 102 by step 103
Characteristic point removal, obtained visible images feature point set and Infrared Image Features point set.
Visible images provided in an embodiment of the present invention and Infrared Image Features point set determine method, utilize Visible Light Camera
The characteristic point pair to match each other is determined with the image transformation matrix of infrared camera, matches clock synchronization, without carrying out one by one to each characteristic point
Matching primitives, calculation amount is small, calculating speed is fast, registration is high-efficient.
In an alternative embodiment, the visible images and Infrared Image Features point set determine method, further includes:
According to visible light uncalibrated image and infrared calibration image, the image transformation square of Visible Light Camera and infrared camera is determined
Battle array.Image transformation matrix is obtained using visible light uncalibrated image and infrared calibration image calibration, it is similar in environment such as illumination, distances
In the case where can rapid registering visible images and infrared image, accelerate registration efficiency.
Referring to Fig. 3, in an alternative embodiment, each characteristic point identified in the visible images obtains first
Feature point set identifies each characteristic point in the infrared image, obtains second feature point set, comprising:
According to visible light uncalibrated image and infrared calibration image, determines and participate in infrared camera internal reference M in Visible Light Camerai；
According to the determining internal reference Mi, the visible images and infrared image of acquisition are corrected；
Each characteristic point in the visible images after identification correction obtains fisrt feature point set, after identification correction
Each characteristic point in the infrared image, obtains second feature point set.It is matched using characteristic point, reduces operand, improved
System robustness.
Referring to fig. 2, the embodiment of the invention also provides a kind of visible images and infrared image method for registering, comprising:
Step 101: obtaining visible images and infrared image, and identify each characteristic point in the visible images, obtain
To fisrt feature point set, identifies each characteristic point in the infrared image, obtain second feature point set；
Step 102: according to the image transformation matrix T of Visible Light Camera and infrared camera, determining the fisrt feature point set
And second feature point concentrates the characteristic point pair being mutually matched；
Step 103: according to the characteristic point pair being mutually matched, forming visible images feature point set and infrared image is special
Levy point set；
Step 104: according to the visible images feature point set and Infrared Image Features point set, to visible images with it is red
Outer image is registrated.
Specifically, step 101-104 is identical as above-mentioned determining embodiment of the method in the embodiment of the present invention, specific descriptions and function
Can be referring to above-described embodiment, details are not described herein.
It is described according to the visible images feature point set and Infrared Image Features point set in an alternative embodiment, it is right
Visible images are registrated with infrared image, comprising:
Determine that the visible images feature point set and Infrared Image Features point concentrate the vector distance of each characteristic point pair；
The vector distance of each characteristic point pair is compared with preset standard phasor distance, it is visible after being screened
Infrared Image Features point set (the feature point set after screening after light image feature point set (the feature point set A after screening) and screening
B)；Specifically, when the vector distance of characteristic point pair is less than preset standard distance vector, then otherwise characteristic point is abandoned to effective
Characteristic point pair forms the visible images feature point set after screening and the Infrared Image Features after screening according to effective characteristic point
Point set；
According to the visible images feature point set after described screened and the Infrared Image Features point set after screening to can
Light-exposed image is registrated with infrared image.Further the characteristic point pair of matching difference is deleted in screening, is improved registration accuracy and is matched
The robustness of quasi- process.
It is described according to the visible images feature point set and Infrared Image Features point set in an alternative embodiment, it is right
Visible images are registrated with infrared image, comprising:
Whether the image transformation matrix for judging the Visible Light Camera and infrared camera according to the following formula is transformation matrix of coordinates:
Crad (C) >=η n, C={ (Ai,Bi)|||TAi-Bi||2< ε } wherein, Crad (C) is the element number of C, and T is can
The image transformation matrix of light-exposed camera and infrared camera, AiFor the ith feature point of visible images, BiFor with AiIt is matched red
The characteristic point of outer image, the positive integer of i=1~n, n be the screening after visible images feature point set in characteristic point it is total
Quantity, ε and η are judgment threshold.
If so, according to the image transformation matrix of the Visible Light Camera and infrared camera to visible images and red
Each pixel is coordinately transformed to realize registration in outer image.Referring to Fig. 3, in an alternative embodiment, the visible light
Image and infrared image method for registering, which is characterized in that further include:
If it is not, then according to random consistency sampling algorithm (RANSAC), the visible images feature point set and red is determined
The corresponding transformation matrix N of outer characteristics of image point set；
According to determining transformation matrix N to each pixel in the visible images and infrared image be coordinately transformed with
Realize registration.
In an alternative embodiment, the random consistency sampling algorithm of basis determines the visible images characteristic point
Collect transformation matrix corresponding with Infrared Image Features point set, comprising:
(1) several groups of characteristic points pair are chosen immediately from the visible images feature point set and Infrared Image Features point concentration,
Intra-office point is formed to collection；
(2) determine intra-office point to collection corresponding transformation matrix collection according to the intra-office point；
(3) judge that the visible images feature point set and Infrared Image Features point concentrate remaining characteristic point to whether full
The foot intra-office point is to the corresponding transformation matrix of collection；
(4) if so, by the remaining characteristic point to being included in the intra-office point in collection；
(5) when the intra-office point reaches preset value to the quantity of the characteristic point pair in collection, according to the intra-office point to collection
Corresponding transformation matrix determines the visible images feature point set and the corresponding transformation matrix of Infrared Image Features point set.Setting
Intra-office point can avoid the characteristic point of part matching difference to influence registration accuracy to collection quantity threshold.
In an alternative embodiment, institute is determined to corresponding transformation matrix is collected according to the intra-office point described in step (5)
State visible images feature point set and the corresponding transformation matrix of Infrared Image Features point set, comprising:
It repeats step (1)-(4) and obtains multiple intra-office points to the corresponding transformation matrix of collection；
Q when according to the following formula, using k value minimum is as the visible images feature point set and Infrared Image Features point set pair
The transformation matrix answered:
K=∑ | | QjAi-Bi||2
Wherein, QjIt is intra-office point to collection to corresponding transformation matrix, the positive integer of j=1~m, m is collected for j-th of intra-office point
Total quantity, AiFor the ith feature point of visible images, BiFor with AiThe characteristic point of matched infrared image, i=1-n, n are
The total quantity of characteristic point in visible images feature point set after the screening.
It is described that each pixel in the visible images and infrared image is coordinately transformed in an alternative embodiment
To realize registration, comprising:
The coordinate of each pixel of the infrared image is converted, rounding, the infrared image after obtaining coordinate transform；
By the pixel value of pixel each in the infrared image after the coordinate transform and the corresponding visible images picture
The pixel value of vegetarian refreshments is merged, and registration is completed.
In an alternative embodiment, as shown in figure 3, each pixel in the infrared image by after the coordinate transform
Pixel value is merged with the pixel value of the corresponding visible images pixel, comprising:
The adjacent pixels point of each pixel in the absent region in infrared image after determining the coordinate transform；
The pixel value that each pixel in the absent region is determined according to the pixel value of the adjacent pixels point, obtains interpolation
Infrared image afterwards；
According to the pixel value of pixel each in the infrared image after the interpolation and the corresponding visible images pixel
The pixel value of point is merged.
In one embodiment, each pixel in the absent region in infrared image is determined as follows infrared
Pixel value:
Select the pixel point set around the missing point with normal infrared image element value；Take the infrared image element of above-mentioned pixel point set
Infrared image element value of the average value as the missing point.
In one embodiment, it is merged by following formula:
vi=k1·ai+k2·bi
Wherein, ViFor the pixel value of pixel after fusion, aiFor the pixel value of the pixel in visible images, biIt is slotting
The pixel value of the pixel, k in infrared image after value1、k2For fusion parameters.
Referring to fig. 4, the embodiment of the invention also provides a kind of visible images and Infrared Image Features point set determining device,
Include:
Identification module 10 is obtained, for obtaining visible images and infrared image, and is identified in the visible images
Each characteristic point obtains fisrt feature point set, identifies each characteristic point in the infrared image, obtains second feature point set；
Determining module 20 determines that described first is special for the image transformation matrix according to Visible Light Camera and infrared camera
Sign point set and second feature point concentrate the characteristic point pair being mutually matched；
Form module 30, the characteristic point pair for being mutually matched according to, formation visible images feature point set and red
Outer characteristics of image point set.
Referring to Fig. 5, the embodiment of the invention also provides a kind of visible images and infrared image registration apparatus, comprising:
Identification module 10 is obtained, for obtaining visible images and infrared image, and is identified in the visible images
Each characteristic point obtains fisrt feature point set, identifies each characteristic point in the infrared image, obtains second feature point set；
Determining module 20 determines that described first is special for the image transformation matrix according to Visible Light Camera and infrared camera
Sign point set and second feature point concentrate the characteristic point pair being mutually matched；
Form module 30, the characteristic point pair for being mutually matched according to, formation visible images feature point set and red
Outer characteristics of image point set；
Registration module 40 is used for according to the visible images feature point set and Infrared Image Features point set, to visible light
Image is registrated with infrared image.
Apparatus of the present invention embodiment and embodiment of the method correspond, and specifically describe and effect is referring to embodiment of the method,
This is repeated no more.
Above-mentioned is only section Example of the invention.It should be pointed out that for those of ordinary skill in the art, not
Under the premise of being detached from the principle of the invention, several modifications and improvements can also be made, also should be regarded as belonging to protection category of the invention.
The content being not described in detail in description of the invention is known to the skilled person technology.
Claims (15)
1. a kind of visible images and Infrared Image Features point set determine method characterized by comprising
Visible images and infrared image are obtained, and identifies each characteristic point in the visible images, obtains fisrt feature point
Collection, identifies each characteristic point in the infrared image, obtains second feature point set；
According to the image transformation matrix of Visible Light Camera and infrared camera, the fisrt feature point set and second feature point set are determined
In the characteristic point pair that is mutually matched；
According to the characteristic point pair being mutually matched, visible images feature point set and Infrared Image Features point set are formed.
2. visible images according to claim 1 and Infrared Image Features point set determine method, which is characterized in that also wrap
It includes:
According to visible light uncalibrated image and infrared calibration image, the image transformation matrix of Visible Light Camera and infrared camera is determined.
3. visible images according to claim 1 and Infrared Image Features point set determine method, which is characterized in that described
It identifies each characteristic point in the visible images, obtains fisrt feature point set, identify each characteristic point in the infrared image,
Obtaining second feature point set includes:
According to visible light uncalibrated image and infrared calibration image, determines and participate in infrared camera internal reference in Visible Light Camera；
According to the determining internal reference, the visible images and infrared image of acquisition are corrected；
Each characteristic point in the visible images after identification correction obtains fisrt feature point set, described after identification correction
Each characteristic point in infrared image, obtains second feature point set.
4. a kind of visible images and infrared image method for registering characterized by comprising
Visible images and infrared image are obtained, and identifies each characteristic point in the visible images, obtains fisrt feature point
Collection, identifies each characteristic point in the infrared image, obtains second feature point set；
According to the image transformation matrix of Visible Light Camera and infrared camera, the fisrt feature point set and second feature point set are determined
In the characteristic point pair that is mutually matched；
According to the characteristic point pair being mutually matched, visible images feature point set and Infrared Image Features point set are formed；
According to the visible images feature point set and Infrared Image Features point set, visible images and infrared image are matched
It is quasi-.
5. visible images according to claim 4 and infrared image method for registering, which is characterized in that further include:
According to visible light uncalibrated image and infrared calibration image, the image transformation matrix of Visible Light Camera and infrared camera is determined.
6. visible images according to claim 4 and infrared image method for registering, which is characterized in that described in the identification
Each characteristic point in visible images, obtains fisrt feature point set, identifies each characteristic point in the infrared image, obtains second
Feature point set includes:
According to visible light uncalibrated image and infrared calibration image, determines and participate in infrared camera internal reference in Visible Light Camera；
According to the determining internal reference, the visible images and infrared image of acquisition are corrected；
Each characteristic point in the visible images after identification correction obtains fisrt feature point set, described after identification correction
Each characteristic point in infrared image, obtains second feature point set.
7. visible images according to claim 4 and infrared image method for registering, which is characterized in that described according to
Visible images feature point set and Infrared Image Features point set are registrated visible images with infrared image, comprising:
Determine that the visible images feature point set and Infrared Image Features point concentrate the vector distance of each characteristic point pair；
The vector distance of each characteristic point pair is compared with preset standard phasor distance, the visible light figure after being screened
As the Infrared Image Features point set after feature point set and screening；
According to the visible images feature point set after the screening and the Infrared Image Features point set after screening to visible images
It is registrated with infrared image.
8. visible images according to claim 7 and infrared image method for registering, which is characterized in that described according to
Visible images feature point set and Infrared Image Features point set are registrated visible images with infrared image, comprising:
Whether the image transformation matrix for judging the Visible Light Camera and infrared camera according to the following formula is transformation matrix of coordinates:
Crad (C) >=η n, C={ (Ai,Bi)|||TAi-Bi||2< ε }
Wherein, Crad (C) is the element number of C, and T is the image transformation matrix of Visible Light Camera and infrared camera, AiFor visible light
The ith feature point of image, BiFor with AiThe characteristic point of matched infrared image, the positive integer of i=1~n, n is after the screening
Visible images feature point set in characteristic point total quantity, ε and η are judgment threshold；
If so, according to the image transformation matrix of the Visible Light Camera and infrared camera to the visible images and infrared figure
Each pixel is coordinately transformed to realize registration as in.
9. visible images according to claim 8 and infrared image method for registering, which is characterized in that further include:
If it is not, then determining the visible images feature point set and Infrared Image Features point according to random consistency sampling algorithm
Collect corresponding transformation matrix；
Each pixel in the visible images and infrared image is coordinately transformed to realize according to determining transformation matrix
Registration.
10. visible images according to claim 9 and infrared image method for registering, which is characterized in that the basis with
Machine consistency sampling algorithm determines the visible images feature point set and the corresponding transformation matrix of Infrared Image Features point set,
Include:
(1) several groups of characteristic points pair are randomly selected from the visible images feature point set and Infrared Image Features point concentration, is formed
Intra-office point is to collection；
(2) determine intra-office point to collection corresponding transformation matrix collection according to the intra-office point；
(3) judge that the visible images feature point set and Infrared Image Features point concentrate remaining characteristic point to whether meeting institute
Intra-office point is stated to the corresponding transformation matrix of collection；
(4) if so, by the remaining characteristic point to being included in the intra-office point in collection；
(5) when the intra-office point reaches preset value to the quantity of the characteristic point pair in collection, collection is corresponded to according to the intra-office point
Transformation matrix determine the visible images feature point set and the corresponding transformation matrix of Infrared Image Features point set.
11. visible images according to claim 10 and infrared image method for registering, which is characterized in that step (5) institute
That states determines the visible images feature point set and Infrared Image Features to corresponding transformation matrix is collected according to the intra-office point
The corresponding transformation matrix of point set, comprising:
It repeats step (1)-(4) and obtains multiple intra-office points to the corresponding transformation matrix of collection；
Q when according to the following formula, by k value minimumjIt is corresponding as the visible images feature point set and Infrared Image Features point set
Transformation matrix:
K=∑ | | QjAi-Bi||2
Wherein, QjIt is sum of the intra-office point to collection to corresponding transformation matrix, the positive integer of j=1~m, m is collected for j-th of intra-office point
Amount, AiFor the ith feature point of visible images, BiFor with AiThe characteristic point of matched infrared image, i=1-n, n are the sieve
The total quantity of characteristic point in visible images feature point set after choosing.
12. being existed according to a kind of described in any item visible images of claim 8-11 and infrared image method for registering, feature
In described to be coordinately transformed to each pixel in the visible images and infrared image to realize registration, comprising:
The coordinate of each pixel of the infrared image is converted, rounding, the infrared image after obtaining coordinate transform；
By the pixel value of pixel each in the infrared image after the coordinate transform and the corresponding visible images pixel
Pixel value merged, complete registration.
13. a kind of visible images according to claim 12 and infrared image method for registering, which is characterized in that will be described
In infrared image after coordinate transform the pixel value of each pixel and the pixel value of the corresponding visible images pixel into
Row fusion, comprising:
The adjacent pixels point of each pixel in the absent region in infrared image after determining the coordinate transform；
The pixel value that each pixel in the absent region is determined according to the pixel value of the adjacent pixels point, after obtaining interpolation
Infrared image；
According to the pixel value of pixel each in the infrared image after the interpolation and the corresponding visible images pixel
Pixel value is merged.
14. a kind of visible images and Infrared Image Features point set determining device characterized by comprising
Identification module is obtained, for obtaining visible images and infrared image, and identifies each feature in the visible images
Point obtains fisrt feature point set, identifies each characteristic point in the infrared image, obtains second feature point set；
Determining module determines the fisrt feature point set for the image transformation matrix according to Visible Light Camera and infrared camera
And second feature point concentrates the characteristic point pair being mutually matched；
Module is formed, the characteristic point pair for being mutually matched according to forms visible images feature point set and infrared image
Feature point set.
15. a kind of visible images and infrared image registration apparatus characterized by comprising
Identification module is obtained, for obtaining visible images and infrared image, and identifies each feature in the visible images
Point obtains fisrt feature point set, identifies each characteristic point in the infrared image, obtains second feature point set；
Determining module determines the fisrt feature point set for the image transformation matrix according to Visible Light Camera and infrared camera
And second feature point concentrates the characteristic point pair being mutually matched；
Module is formed, the characteristic point pair for being mutually matched according to forms visible images feature point set and infrared image
Feature point set；
Registration module, for according to the visible images feature point set and Infrared Image Features point set, to visible images with
Infrared image is registrated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811162538.7A CN109389630B (en) | 2018-09-30 | 2018-09-30 | Method and device for determining and registering feature point set of visible light image and infrared image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811162538.7A CN109389630B (en) | 2018-09-30 | 2018-09-30 | Method and device for determining and registering feature point set of visible light image and infrared image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109389630A true CN109389630A (en) | 2019-02-26 |
| CN109389630B CN109389630B (en) | 2020-10-23 |
Family
ID=65419116
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811162538.7A Active CN109389630B (en) | 2018-09-30 | 2018-09-30 | Method and device for determining and registering feature point set of visible light image and infrared image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109389630B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111242991A (en) * | 2020-01-10 | 2020-06-05 | 大连理工大学 | Method for quickly registering visible light and infrared camera |
| CN111327788A (en) * | 2020-02-28 | 2020-06-23 | 北京迈格威科技有限公司 | Synchronization method, temperature measurement method and device of camera set and electronic system |
| CN112577439A (en) * | 2020-12-03 | 2021-03-30 | 华中科技大学 | Microelectronic substrate warpage measurement method and system based on infrared and optical images |
| WO2021098080A1 (en) * | 2019-11-22 | 2021-05-27 | 大连理工大学 | Multi-spectral camera extrinsic parameter self-calibration algorithm based on edge features |
| WO2022104817A1 (en) * | 2020-11-23 | 2022-05-27 | 江苏镭博智能科技有限公司 | Non-contact body temperature measurement method and system |
| CN115965843A (en) * | 2023-01-04 | 2023-04-14 | 长沙观谱红外科技有限公司 | Visible light and infrared image fusion method |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104167003A (en) * | 2014-08-29 | 2014-11-26 | 福州大学 | Method for fast registering remote-sensing image |
| CN105205818A (en) * | 2015-09-18 | 2015-12-30 | 国网上海市电力公司 | Method for registering infrared image and visible light image of electrical equipment |
| US20160117821A1 (en) * | 2014-10-23 | 2016-04-28 | Hanwha Techwin Co., Ltd. | Apparatus and method for registering images |
| CN107063228A (en) * | 2016-12-21 | 2017-08-18 | 上海交通大学 | Targeted attitude calculation method based on binocular vision |
| CN107230199A (en) * | 2017-06-23 | 2017-10-03 | 歌尔科技有限公司 | Image processing method, device and augmented reality equipment |
| CN107680112A (en) * | 2017-10-16 | 2018-02-09 | 北京邮电大学 | Method for registering images |
| CN107993258A (en) * | 2017-11-23 | 2018-05-04 | 浙江大华技术股份有限公司 | A kind of method for registering images and device |
| CN108010085A (en) * | 2017-11-30 | 2018-05-08 | 西南科技大学 | Target identification method based on binocular Visible Light Camera Yu thermal infrared camera |
-
2018
- 2018-09-30 CN CN201811162538.7A patent/CN109389630B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104167003A (en) * | 2014-08-29 | 2014-11-26 | 福州大学 | Method for fast registering remote-sensing image |
| US20160117821A1 (en) * | 2014-10-23 | 2016-04-28 | Hanwha Techwin Co., Ltd. | Apparatus and method for registering images |
| CN105205818A (en) * | 2015-09-18 | 2015-12-30 | 国网上海市电力公司 | Method for registering infrared image and visible light image of electrical equipment |
| CN107063228A (en) * | 2016-12-21 | 2017-08-18 | 上海交通大学 | Targeted attitude calculation method based on binocular vision |
| CN107230199A (en) * | 2017-06-23 | 2017-10-03 | 歌尔科技有限公司 | Image processing method, device and augmented reality equipment |
| CN107680112A (en) * | 2017-10-16 | 2018-02-09 | 北京邮电大学 | Method for registering images |
| CN107993258A (en) * | 2017-11-23 | 2018-05-04 | 浙江大华技术股份有限公司 | A kind of method for registering images and device |
| CN108010085A (en) * | 2017-11-30 | 2018-05-08 | 西南科技大学 | Target identification method based on binocular Visible Light Camera Yu thermal infrared camera |
Non-Patent Citations (3)
| Title |
|---|
| BINGJIAN WANG ET AL: "A new image registration method for infrared images and visible images", 《2010 3RD INTERNATIONAL CONGRESS ON IMAGE AND SIGNAL PROCESSING》 * |
| 彭逸月 等: "基于特征点的红外与可见光图像配准研究", 《激光与红外》 * |
| 马丽丽 等: "基于RANSAC的特征点匹配算法", 《计算机工程与设计》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021098080A1 (en) * | 2019-11-22 | 2021-05-27 | 大连理工大学 | Multi-spectral camera extrinsic parameter self-calibration algorithm based on edge features |
| US11398053B2 (en) | 2019-11-22 | 2022-07-26 | Dalian University Of Technology | Multispectral camera external parameter self-calibration algorithm based on edge features |
| CN111242991A (en) * | 2020-01-10 | 2020-06-05 | 大连理工大学 | Method for quickly registering visible light and infrared camera |
| CN111242991B (en) * | 2020-01-10 | 2022-09-20 | 大连理工大学 | Method for quickly registering visible light and infrared camera |
| CN111327788A (en) * | 2020-02-28 | 2020-06-23 | 北京迈格威科技有限公司 | Synchronization method, temperature measurement method and device of camera set and electronic system |
| WO2022104817A1 (en) * | 2020-11-23 | 2022-05-27 | 江苏镭博智能科技有限公司 | Non-contact body temperature measurement method and system |
| CN112577439A (en) * | 2020-12-03 | 2021-03-30 | 华中科技大学 | Microelectronic substrate warpage measurement method and system based on infrared and optical images |
| CN115965843A (en) * | 2023-01-04 | 2023-04-14 | 长沙观谱红外科技有限公司 | Visible light and infrared image fusion method |
| CN115965843B (en) * | 2023-01-04 | 2023-09-29 | 长沙观谱红外科技有限公司 | Visible light and infrared image fusion method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109389630B (en) | 2020-10-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109389630A (en) | Visible images and the determination of Infrared Image Features point set, method for registering and device | |
| EP3525134A1 (en) | Device and method with image matching | |
| CN107357286A (en) | Vision positioning guider and its method | |
| CN107103589B (en) | A kind of highlight area restorative procedure based on light field image | |
| CN110097093A (en) | A kind of heterologous accurate matching of image method | |
| TW201118804A (en) | Method and system for object detection | |
| CN111582092B (en) | Pedestrian abnormal behavior detection method based on human skeleton | |
| CN104751465A (en) | ORB (oriented brief) image feature registration method based on LK (Lucas-Kanade) optical flow constraint | |
| CN104685513A (en) | Feature based high resolution motion estimation from low resolution images captured using an array source | |
| CN110991266A (en) | Binocular face living body detection method and device | |
| CN108280411A (en) | A kind of pedestrian's searching method with spatial alternation ability | |
| CN110263768A (en) | A kind of face identification method based on depth residual error network | |
| CN107560592A (en) | A kind of precision ranging method for optronic tracker linkage target | |
| CN109461186A (en) | Image processing method, device, computer readable storage medium and electronic equipment | |
| WO2021217764A1 (en) | Human face liveness detection method based on polarization imaging | |
| CN103281513B (en) | Pedestrian recognition method in the supervisory control system of a kind of zero lap territory | |
| Wehrwein et al. | Shadow detection and sun direction in photo collections | |
| CN109977882B (en) | A kind of half coupling dictionary is to the pedestrian of study again recognition methods and system | |
| CN106295657A (en) | A kind of method extracting human height's feature during video data structure | |
| CN102881160A (en) | Outdoor traffic sign identification method under low-illumination scene | |
| Kagarlitsky et al. | Piecewise-consistent color mappings of images acquired under various conditions | |
| CN108737785B (en) | Indoor automatic detection system that tumbles based on TOF 3D camera | |
| CN110378995B (en) | Method for three-dimensional space modeling by using projection characteristics | |
| JP2011209896A (en) | Obstacle detecting apparatus, obstacle detecting method, and obstacle detecting program | |
| CN110090425A (en) | A kind of clocking method of middle-distance race campaign |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |