CN111311658B - Image registration method and related device for dual-light imaging system - Google Patents
Image registration method and related device for dual-light imaging system Download PDFInfo
- Publication number
- CN111311658B CN111311658B CN202010255440.7A CN202010255440A CN111311658B CN 111311658 B CN111311658 B CN 111311658B CN 202010255440 A CN202010255440 A CN 202010255440A CN 111311658 B CN111311658 B CN 111311658B
- Authority
- CN
- China
- Prior art keywords
- image
- band image
- imaging system
- light imaging
- dual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 113
- 238000000034 method Methods 0.000 title claims abstract description 47
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims abstract description 44
- 230000009466 transformation Effects 0.000 claims abstract description 43
- 230000009977 dual effect Effects 0.000 claims description 22
- 239000013598 vector Substances 0.000 claims description 17
- 238000010586 diagram Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 abstract description 8
- 230000007547 defect Effects 0.000 abstract description 7
- 239000011159 matrix material Substances 0.000 description 5
- 230000004927 fusion Effects 0.000 description 4
- 230000006978 adaptation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a method and a device for registering images of a double-light imaging system, wherein the double-light imaging system is used for acquiring images of a target object, acquiring a first wave band image and a second wave band image of the target object, and acquiring the distance between the double-light imaging system and the target object; cutting out corresponding image areas from a first wave band image and a second wave band image of the target object according to affine transformation parameters corresponding to the distances; and registering the first band image and the second band image of the target object according to edge features contained in the image area cut out from the first band image and the image area cut out from the second band image of the target object. The image registration method and device of the dual-light imaging system can improve registration accuracy and overcome the defects of high requirements on the stability of an imaging device framework and poor registration effect of a close-range target in the prior art. The invention also discloses a dual-light imaging system and a computer readable storage medium.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an image registration method and device of a dual-light imaging system. The invention also relates to a dual light imaging system and also to a computer readable storage medium.
Background
The double-light image fusion technology can acquire imaging information of light from two different wave bands, and the imaging information of the two can complement each other, for example, in a scheme of fusing infrared light imaging and visible light imaging, the detection target is realized, and meanwhile, the visible light image can be utilized to contain rich information content, so that the spatial resolution is improved, and the image details are enhanced. The image registration is a basic link of double-light image fusion and is also an important part of the image fusion.
In the prior art, the image registration scheme used is to calculate a correction matrix based on the geometrical optics principle, and calibrate the image or imaging device by using the correction matrix. The method is simple in implementation algorithm, has no special requirement on a scene, is stable in correction result, has higher requirement on the stability of an imaging device framework, and is poor in registration effect of a close-range target.
Disclosure of Invention
The invention aims to provide a method and a device for registering images of a double-light imaging system, which can improve registration accuracy and overcome the defects of high requirements on the stability of an imaging device framework and poor registration effect of a close-range target in the prior art. The invention also provides a dual light imaging system and also provides a computer readable storage medium.
In order to achieve the above purpose, the present invention provides the following technical solutions:
a dual light imaging system image registration method comprising:
acquiring an image of a target object by using a dual-light imaging system, acquiring a first wave band image and a second wave band image of the target object, and acquiring a first distance from the dual-light imaging system to the target object;
cutting out corresponding image areas from a first wave band image and a second wave band image of the target object according to affine transformation parameters corresponding to the distance;
and registering the first band image and the second band image of the target object according to edge features contained in an image area cut out from the first band image of the target object and an image area cut out from the second band image of the target object.
Preferably, obtaining the affine transformation parameters includes:
acquiring an image of a preset calibration target by using the dual-light imaging system, acquiring a first wave band image and a second wave band image of the preset calibration target, and acquiring the distance from the dual-light imaging system to the preset calibration target;
according to the angular point characteristics contained in the first wave band image and the second wave band image of the preset calibration target, selecting N groups of characteristic points with matching degree meeting preset requirements from the first wave band image and the second wave band image of the preset calibration target, and determining coordinates of N characteristic points selected from the first wave band image of the preset calibration target and coordinates of N characteristic points selected from the second wave band image of the preset calibration target, wherein N is a positive integer greater than or equal to 3;
and obtaining the affine transformation parameters corresponding to the distance from the dual-light imaging system to the preset calibration target according to the N characteristic points selected from the first wave band image of the preset calibration target and the N characteristic points selected from the second wave band image of the preset calibration target.
Preferably, the method specifically comprises the following steps:
n feature points selected from the first band image of the preset calibration target are expressed as v1= [ x1 x 2..xn, y1 y2...yn, 11 … 1], and N feature points selected from the second band image of the preset calibration target are expressed as v2= [ x1'x 2'..xn ', y1' y2 '..yn', 11 … 1];
establishing a function f (H) = (H.times.V 1-V2)/(2) according to a least square method, deriving the function f (H), and calculating and obtaining the affine transformation parameter H corresponding to the distance from the dual-light imaging system to the preset calibration target according to the following formula: h= (V1) T ×V1) -1 ×V1 T ×V2。
Preferably, registering the first band image and the second band image of the target object according to edge features contained in an image area cut out from the first band image of the target object and an image area cut out from the second band image of the target object includes:
extracting edge features from an image region cut out of a first band image of the target object and edge features from an image region cut out of a second band image of the target object, respectively obtaining a first image and a second image;
correspondingly dividing the first image and the second image into a plurality of blocks, acquiring the feature data sum of each block of the first image, and selecting the block with the largest feature data sum;
and selecting a block corresponding to the block selected from the first image from each block of the second image, and registering the first band image and the second band image of the target object according to edge features contained in the two selected blocks.
Preferably, extracting edge features from an image region cut out from the first band image of the object and extracting edge features from an image region cut out from the second band image of the object, respectively, to obtain the first image and the second image includes:
using sobel operatorAnd +.>Convolving an image area cut out from the first band image of the target object, respectively extracting a vertical gradient V1_Iy and a horizontal gradient V1_Ix, and obtaining a first gradient diagram V1_sobel=V 1_Iy+V1_Ix;
using sobel operatorAnd +.>Convolving an image area cut out from the second band image of the target object, respectively extracting a vertical gradient V2_Iy and a horizontal gradient V2_Ix, and obtaining a second gradient diagram V2_sobel=V 2_Iy+V2_Ix;
and respectively carrying out binarization processing on the first gradient map and the second gradient map, and correspondingly obtaining the first image and the second image.
Preferably, performing binarization processing on the first gradient map and the second gradient map respectively, and correspondingly obtaining the first image and the second image includes:
carrying out histogram statistics on the first gradient map, calculating a mean and a mean square error sigma, setting 1 to the pixel with the pixel value larger than (mean+3 x sigma) in the first gradient map, and setting 0 to the rest pixels to obtain the first image;
and carrying out histogram statistics on the second gradient map, calculating a mean and a mean square error sigma, setting 1 to the pixel with the pixel value larger than (mean+3 x sigma) in the second gradient map, and setting 0 to the rest pixels to obtain the second image.
Preferably, registering the first band image and the second band image of the target object according to the edge features contained in the two selected blocks includes:
registering the two selected blocks by using offset vectors, obtaining the offset vector which maximizes the overlapping area of the two selected blocks according to the edge characteristics contained in the two selected blocks, and registering the first band image and the second band image of the target object according to the obtained offset vector.
A dual light imaging system image registration apparatus for performing the dual light imaging system image registration method described above.
A dual light imaging system comprising the dual light imaging system image registration apparatus described above.
A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of a dual light imaging system image registration method as described above.
According to the technical scheme, the image registration method and the image registration device for the dual-light imaging system, provided by the invention, are used for acquiring the image of the target object, acquiring the first band image and the second band image of the target object, acquiring the distance between the dual-light imaging system and the target object, then cutting out the image areas corresponding to each other from the first band image and the second band image of the target object according to affine transformation parameters corresponding to the distance, and further registering the first band image and the second band image of the target object according to edge characteristics contained in the image areas cut out from the first band image and the second band image of the target object.
According to the image registration method and device of the double-light imaging system, firstly, the obtained first wave band image and the obtained second wave band image are roughly registered according to the affine transformation parameter pair corresponding to the distance from the double-light imaging system to the target object, the image areas corresponding to each other are cut out from the two images, then, the first wave band image and the second wave band image are further finely registered according to the edge characteristics contained in the image areas cut out from the two images, the registration precision can be improved, and the defects that the stability of the imaging device architecture is high and the registration effect of a close-range target is poor in the prior art are overcome.
The dual-light imaging system provided by the invention can achieve the beneficial effects.
The computer readable storage medium provided by the invention can achieve the beneficial effects.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of an image registration method of a dual-light imaging system according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method for obtaining affine transformation parameters in an embodiment of the invention;
FIG. 3 is a schematic diagram of a pre-set calibration target in an embodiment of the invention;
FIG. 4 is a flowchart of a method for registering a first band image and a second band image of a target object according to edge features included in a cropped image region according to an embodiment of the present invention;
FIG. 5 is a first image corresponding to a first band image of a preset calibration target in an embodiment of the present invention;
FIG. 6 is a second image corresponding to a second band image of a preset calibration target in an embodiment of the present invention.
Detailed Description
In order to make the technical solution of the present invention better understood by those skilled in the art, the technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart of an image registration method of a dual-light imaging system according to the present embodiment, and as can be seen from the figure, the method includes the following steps:
s10: and acquiring an image of the target object by using a dual-light imaging system, acquiring a first wave band image and a second wave band image of the target object, and acquiring the distance from the dual-light imaging system to the target object.
The object is an object to be detected and imaged by using the dual-light imaging system, and can be a scene or an object in the scene. The first band image of the target object obtained by the dual-light imaging system is an image formed by capturing the first band light emitted by the target object by the dual-light imaging system, and the second band image is an image formed by capturing the second band light emitted by the target object by the dual-light imaging system, wherein the wavelength range of the first band light is different from the wavelength range of the second band light. In this embodiment, the dual-light imaging system may be a system having both visible light imaging and infrared light imaging, and correspondingly, the first band image may be a visible light image, and the second band image may be an infrared light image.
The acquired distance refers to the distance from the dual-light imaging system to the target object when the dual-light imaging system acquires an image of the target object.
S11: and cutting out corresponding image areas from the first wave band image and the second wave band image of the target object according to affine transformation parameters corresponding to the distance.
Corresponding affine transformation parameters are respectively arranged corresponding to the distances from different double-light imaging systems to the target object. The affine transformation parameter refers to a parameter that affine transforms a vector space in which a first band image and a vector space in which a second band image acquired by a two-light imaging system are located. In the step, according to affine transformation parameters corresponding to the distance from a dual-light imaging system to a target object, registering a first wave band image and a second wave band image obtained by the dual-light imaging system, and cutting out mutually corresponding image areas from the first wave band image and the second wave band image respectively.
S12: and registering the first band image and the second band image of the target object according to edge features contained in an image area cut out from the first band image of the target object and an image area cut out from the second band image of the target object.
The edge feature included in the image means a feature reflecting edge information included in the image. And registering the first band image and the second band image of the target object according to the edge characteristics contained in the image area cut out from the first band image of the target object and the edge characteristics contained in the image area cut out from the second band image of the target object.
According to the image registration method of the dual-light imaging system, firstly, the first band image and the second band image which are obtained according to affine transformation parameter pairs corresponding to the distance from the dual-light imaging system to the target object are subjected to rough registration, mutually corresponding image areas are cut out from the two images, then the first band image and the second band image are subjected to fine registration further according to edge features contained in the image areas cut out from the two images, registration accuracy can be improved, and the defects that in the prior art, the stability of an imaging device architecture is high and the registration effect of a close-range target is poor are overcome.
The following describes the image registration method of the dual light imaging system in detail with reference to specific embodiments. In the image registration method of the dual-light imaging system of the present embodiment, affine transformation parameters can be obtained by the following method, please refer to fig. 2, fig. 2 is a flowchart of a method for obtaining affine transformation parameters in the present embodiment, which specifically includes the following steps:
s100: and acquiring an image of a preset calibration target by using the dual-light imaging system, acquiring a first wave band image and a second wave band image of the preset calibration target, and acquiring the distance from the dual-light imaging system to the preset calibration target.
The used preset calibration targets have square patterns, so that the first wave band image and the second wave band image acquired by the preset calibration targets by using the double-light imaging system have bright and easily-extracted strong features. Preferably, the preset calibration target has a hollowed-out checkerboard and a black surface, wherein the hollowed-out checkerboard is placed in front of the black surface, please refer to fig. 3, fig. 3 is a schematic diagram of the preset calibration target in this embodiment, and the use of the preset calibration target can enable clear and easily extracted strong feature points in the formed first band image and second band image, which is helpful for improving the matching rate of the feature points and the accuracy of image registration.
S101: according to the angular point characteristics contained in the first wave band image and the second wave band image of the preset calibration target, selecting N groups of characteristic points with matching degree meeting preset requirements from the first wave band image and the second wave band image of the preset calibration target, determining coordinates of N characteristic points selected from the first wave band image of the preset calibration target and coordinates of N characteristic points selected from the second wave band image of the preset calibration target, wherein N is a positive integer greater than or equal to 3.
In specific implementation, angular point features can be extracted from a first band image and a second band image of a preset calibration target respectively, then N groups of characteristic points with matching degree meeting preset requirements are selected from the two images according to the angular point features contained in the two images, and particularly N groups of characteristic points with high matching degree can be manually selected according to the angular point features contained in the two images to obtain coordinates of each group of characteristic points. Alternatively, the harris algorithm may be used to extract the corner features from the image, but not limited thereto, and other algorithms may be used to extract the corner features from the image.
Specifically, N feature points selected from the first band image of the preset calibration target are expressed as V1= [ x1 x 2..xN, y1 y2...yN, 11 … 1], the N feature points selected from the second band image of the preset calibration target are expressed as v2= [ x1'x2'. XN ', y1' y2'. YN', 11 … 1].
S102: and obtaining the affine transformation parameters corresponding to the distance from the dual-light imaging system to the preset calibration target according to the N characteristic points selected from the first wave band image of the preset calibration target and the N characteristic points selected from the second wave band image of the preset calibration target.
According to N characteristic points selected from a first wave band image of a preset calibration target and N characteristic points selected from a second wave band image of the preset calibration target, parameters which correspond to the distance from a current dual-light imaging system to the preset calibration target and can carry out affine transformation on a vector space where the currently acquired first wave band image is located and a vector space where the currently acquired second wave band image is located are obtained.
Preferably, using the N feature points selected from the first band image of the preset calibration target and the N feature points selected from the second band image of the preset calibration target, a function f (H) = (h×v1—v2)/(2) may be established according to a least square method, and the function f (H) may be derived, and affine transformation parameters H corresponding to the distance from the dual-light imaging system to the preset calibration target may be calculated according to the following formula: h= (V1) T ×V1) -1 ×V1 T ×V2。
Deriving the function f (H), and making the derivative f (H) be zero to obtain H= (V1) T ×V1) -1 ×V1 T X V2. According to the formula, an affine transformation matrix, namely affine transformation parameters, are obtained, and the obtained affine transformation matrix is expressed as:
correspondingly, the pixel coordinates of the first band image after coarse registration by using the corresponding affine transformation matrix are as follows: x '=h1×x+h2×y+h3, y' =h4×x+h5×y+h6, where (x, y) represents the pixel coordinates in the first band image before registration and (x ', y') represents the corresponding pixel coordinates after registration.
And acquiring M groups of images of a preset calibration target by using a dual-light imaging system, wherein the M groups of images are images of the preset calibration target acquired by the dual-light imaging system at M different distances, and the M groups of images are positive integers which are larger than or equal to 1, and the affine transformation parameters corresponding to the M different distances are acquired according to the M groups of images. According to the method for obtaining affine transformation parameters, affine transformation parameters corresponding to M different distances of the dual-light imaging system are obtained respectively, namely M affine transformation parameters corresponding to the M different distances are obtained. After obtaining affine transformation parameters of M different distances of the dual-light imaging system according to the method, the obtained M affine transformation parameters can be stored so as to be called when the images obtained by the dual-light imaging system are subjected to coarse registration. In practical application, if no affine transformation parameter corresponding to the distance from the current dual-light imaging system to the target object is completely consistent in the prestored affine transformation parameters, the affine transformation parameter corresponding to the distance from the dual-light imaging system to the target object closest to the distance can be called to perform coarse registration on the first band image and the second band image of the target object.
In the image registration method of the dual-light imaging system of the embodiment, for acquiring a first band image and a second band image of a target object by using the dual-light imaging system, affine transformation parameters corresponding to the distance from the dual-light imaging system to the target object are called, and the first band image and the second band image of the target object are roughly registered, including translating, rotating or zooming the images, and respectively cutting out mutually corresponding image areas from the first band image and the second band image of the target object.
Further, in the image registration method of the dual-optical imaging system of the present embodiment, step S12 registers the first band image and the second band image of the target object according to the edge features included in the image region cut out from the first band image and the image region cut out from the second band image of the target object, by referring to fig. 4, fig. 4 is a flowchart of a method for registering the first band image and the second band image of the target object according to the edge features included in the cut-out image region in the present embodiment, and specifically includes the following steps:
s120: and extracting edge features from an image area cut out of the first band image of the target object and extracting edge features from an image area cut out of the second band image of the target object, so as to obtain a first image and a second image respectively.
Alternatively, the sobel operator can be used to extract edge features from the image, which specifically includes the following procedures:
using sobel operatorAnd +.>And convolving an image area cut out from the first band image of the target object, respectively extracting a vertical gradient V1_Iy and a horizontal gradient V1_Ix, and obtaining a first gradient map V1_sobel=V 1_Iy+V1_Ix. Similarly, the sobel operator is used +.>And +.>And convolving an image area cut out from the second band image of the target object, respectively extracting a vertical gradient V2-Iy and a horizontal gradient V2-Ix, and obtaining a second gradient diagram V2-sobel=V 2-Iy+V 2-Ix.
And then, respectively carrying out binarization processing on the first gradient map and the second gradient map, and correspondingly obtaining the first image and the second image. Specifically, histogram statistics is performed on the first gradient map, mean and mean square error sigma are calculated, pixels with pixel values greater than (mean+3 x sigma) in the first gradient map are set to 1, and the rest pixels are set to 0, so that the first image is obtained. And carrying out histogram statistics on the second gradient map, calculating a mean and a mean square error sigma, setting 1 to the pixel with the pixel value larger than (mean+3 x sigma) in the second gradient map, and setting 0 to the rest pixels to obtain the second image. Referring to fig. 5 and fig. 6, fig. 5 is a first image, which is a strong edge binarized image corresponding to a first band image of a target object in the present embodiment, and fig. 6 is a second image, which is a strong edge binarized image corresponding to a second band image of the target object in the present embodiment.
S121: and correspondingly dividing the first image and the second image into a plurality of blocks, acquiring the characteristic data sum of each block of the first image, and selecting the block with the largest characteristic data sum.
The first image and the second image including the edge feature information are subjected to the blocking processing, and the first image is preferably divided into a plurality of blocks on average, and the second image is preferably divided into a plurality of blocks on average in correspondence. For example, please refer to fig. 5, the first image is divided into blocks with a size of (1/3×h1,1/3×w1) averagely, H1 and W1 respectively represent the height and width of the first image, and the feature data sum of each block is obtained, where the feature data sum of each block is as follows:
158 158 294
299 152 406
158 161 211
as can be seen from the above table, the block with the feature data sum 406 is selected as v1_aim' with the largest feature data sum.
S122: and selecting a block corresponding to the block selected from the first image from each block of the second image, and registering the first band image and the second band image of the target object according to edge features contained in the two selected blocks.
From the respective blocks of the second image, the block v2_aim ' corresponding to the block v1_aim ' selected from the first image is selected based on the block v1_aim ' having the largest sum of the feature data selected from the respective blocks of the first image. The first and second band images of the object are registered by superimposing and registering according to the blocks v1_aim 'and v2_aim'.
Preferably, in the implementation, the offset vector is used for registering the two selected blocks, the offset vector which makes the overlapping area of the two selected blocks maximum is obtained according to the edge characteristics contained in the two selected blocks, and the first band image and the second band image of the target object are registered according to the obtained offset vector. For example, the two blocks are overlapped and judged in the offset vectors of [ ±5, ±5] to obtain the offset vector which maximizes the overlapping area of the two selected blocks.
According to the image registration method of the dual-light imaging system, firstly, the first band image and the second band image which are obtained according to affine transformation parameter pairs corresponding to the distance from the dual-light imaging system to a target object are roughly registered, mutually corresponding image areas are cut out from the two images, then the first band image and the second band image are further finely registered according to edge features contained in the image areas cut out from the two images, registration precision can be improved, calibration cost is reduced, operation is relatively convenient, and a physical distance fault-tolerant space is provided while the registration precision is not lost.
Correspondingly, the embodiment of the invention also provides a device for registering images of the double-light imaging system, which is used for executing the method for registering images of the double-light imaging system.
According to the image registration device of the dual-light imaging system, firstly, the first band image and the second band image which are obtained according to the affine transformation parameter pair corresponding to the distance from the dual-light imaging system to the target object are roughly registered, the image areas corresponding to each other are cut out from the two images, then the first band image and the second band image are further finely registered according to the edge characteristics contained in the image areas cut out from the two images, the registration precision can be improved, and the defects that the stability of the architecture of the imaging device is high and the registration effect of a close-range target is poor in the prior art are overcome.
The embodiment of the invention also provides a double-light imaging system, which comprises the image registration device of the double-light imaging system.
After the first band image and the second band image of the target object are obtained, the two-light imaging system of the embodiment firstly coarsely registers the obtained first band image and second band image according to affine transformation parameters corresponding to the distance from the two-light imaging system to the target object, cuts out mutually corresponding image areas from the two images, and then finely registers the first band image and the second band image further according to edge features contained in the image areas cut out from the two images, so that the registration precision can be improved, and the defects that the prior art has higher requirements on the stability of the imaging device architecture and has poor registration effect on the short-distance target object are overcome.
Specifically, the dual light imaging system of the present embodiment may include:
the image acquisition device is used for acquiring an image of a target object and acquiring a first wave band image and a second wave band image of the target object;
the distance acquisition device is used for acquiring the distance from the dual-light imaging system to the target object;
the first registration device is used for cutting out mutually corresponding image areas from the first wave band image and the second wave band image of the target object according to affine transformation parameters corresponding to the distance;
the second registration device is used for registering the first band image and the second band image of the target object according to edge features contained in an image area cut out from the first band image of the target object and an image area cut out from the second band image of the target object;
and the fusion device is used for superposing and fusing the registered first wave band image and the registered second wave band image of the target object.
Accordingly, embodiments of the present invention also provide a computer readable storage medium having a computer program stored thereon, which when executed by a processor, implements the steps of the dual light imaging system image registration method as described above.
The computer readable storage medium of the embodiment is applied, and when the stored computer program is executed by a processor, the method can realize coarse registration of the obtained first band image and the second band image according to affine transformation parameters corresponding to the distance from a dual-light imaging system to a target object, cut out mutually corresponding image areas from the two images, and then further fine register the first band image and the second band image according to edge features contained in the image areas cut out from the two images, thereby improving the registration precision, and overcoming the defects of higher requirements on the stability of an imaging device framework and poor registration effect of a close-range target in the prior art.
The image registration method and device for the dual-light imaging system, the dual-light imaging system and the computer readable storage medium provided by the invention are described in detail above. The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to facilitate an understanding of the method of the present invention and its core ideas. It should be noted that it will be apparent to those skilled in the art that various modifications and adaptations of the invention can be made without departing from the principles of the invention and these modifications and adaptations are intended to be within the scope of the invention as defined in the following claims.
Claims (10)
1. A method for registering images of a dual light imaging system, comprising:
acquiring an image of a target object by using a dual-light imaging system, acquiring a first wave band image and a second wave band image of the target object, and acquiring the distance from the dual-light imaging system to the target object;
cutting out corresponding image areas from a first band image and a second band image of the target object according to affine transformation parameters corresponding to the distance, wherein the affine transformation parameters are parameters for carrying out affine transformation on a vector space in which the first band image and a vector space in which the second band image are positioned, which are acquired by the dual-light imaging system;
and registering the first band image and the second band image of the target object according to edge features contained in an image area cut out from the first band image of the target object and an image area cut out from the second band image of the target object.
2. The method of image registration of a dual light imaging system according to claim 1, wherein obtaining the affine transformation parameters comprises:
acquiring an image of a preset calibration target by using the dual-light imaging system, acquiring a first wave band image and a second wave band image of the preset calibration target, and acquiring the distance from the dual-light imaging system to the preset calibration target;
according to the angular point characteristics contained in the first wave band image and the second wave band image of the preset calibration target, selecting N groups of characteristic points with matching degree meeting preset requirements from the first wave band image and the second wave band image of the preset calibration target, and determining coordinates of N characteristic points selected from the first wave band image of the preset calibration target and coordinates of N characteristic points selected from the second wave band image of the preset calibration target, wherein N is a positive integer greater than or equal to 3;
and obtaining the affine transformation parameters corresponding to the distance from the dual-light imaging system to the preset calibration target according to the N characteristic points selected from the first wave band image of the preset calibration target and the N characteristic points selected from the second wave band image of the preset calibration target.
3. The dual light imaging system image registration method of claim 2, comprising:
n feature points selected from the first band image of the preset calibration target are expressed as v1= [ x1 x 2..xn, y1 y2...yn, 11 … 1], and N feature points selected from the second band image of the preset calibration target are expressed as v2= [ x1'x 2'..xn ', y1' y2 '..yn', 11 … 1];
establishing a function f (H) = (H.times.V 1-V2)/(2) according to a least square method, deriving the function f (H), and calculating and obtaining the affine transformation parameter H corresponding to the distance from the dual-light imaging system to the preset calibration target according to the following formula: h= (V1) T ×V1) -1 ×V1 T ×V2。
4. The method of image registration of a dual light imaging system of claim 1, wherein registering the first band image and the second band image of the object based on edge features contained in an image region cropped from the first band image and an image region cropped from the second band image of the object comprises:
extracting edge features from an image region cut out of a first band image of the target object and edge features from an image region cut out of a second band image of the target object, respectively obtaining a first image and a second image;
correspondingly dividing the first image and the second image into a plurality of blocks, acquiring the feature data sum of each block of the first image, and selecting the block with the largest feature data sum;
and selecting a block corresponding to the block selected from the first image from each block of the second image, and registering the first band image and the second band image of the target object according to edge features contained in the two selected blocks.
5. The method of image registration of a dual light imaging system of claim 4, wherein extracting edge features for an image region cropped from a first band image of the object and extracting edge features for an image region cropped from a second band image of the object, respectively, obtaining a first image and a second image comprises:
using sobel operatorAnd +.>Convolving an image area cut out from the first band image of the target object, respectively extracting a vertical gradient V1_Iy and a horizontal gradient V1_Ix, and obtaining a first gradient diagram V1_sobel=V 1_Iy+V1_Ix;
using sobel operatorAnd +.>Convolving an image area cut out from the second band image of the target object, respectively extracting a vertical gradient V2_Iy and a horizontal gradient V2_Ix, and obtaining a second gradient diagram V2_sobel=V 2_Iy+V2_Ix;
and respectively carrying out binarization processing on the first gradient map and the second gradient map, and correspondingly obtaining the first image and the second image.
6. The method of image registration of a dual light imaging system of claim 5, wherein binarizing the first and second gradient maps, respectively, the obtaining the first and second images includes:
carrying out histogram statistics on the first gradient map, calculating a mean and a mean square error sigma, setting 1 to the pixel with the pixel value larger than (mean+3 x sigma) in the first gradient map, and setting 0 to the rest pixels to obtain the first image;
and carrying out histogram statistics on the second gradient map, calculating a mean and a mean square error sigma, setting 1 to the pixel with the pixel value larger than (mean+3 x sigma) in the second gradient map, and setting 0 to the rest pixels to obtain the second image.
7. The method of image registration for a dual-light imaging system of claim 4, wherein registering the first band image and the second band image of the object based on edge features contained in the two selected blocks comprises:
registering the two selected blocks by using offset vectors, obtaining the offset vector which maximizes the overlapping area of the two selected blocks according to the edge characteristics contained in the two selected blocks, and registering the first band image and the second band image of the target object according to the obtained offset vector.
8. A dual light imaging system image registration apparatus for performing the dual light imaging system image registration method of any one of claims 1-7.
9. A dual light imaging system comprising the dual light imaging system image registration apparatus of claim 8.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the dual light imaging system image registration method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010255440.7A CN111311658B (en) | 2020-04-02 | 2020-04-02 | Image registration method and related device for dual-light imaging system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010255440.7A CN111311658B (en) | 2020-04-02 | 2020-04-02 | Image registration method and related device for dual-light imaging system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111311658A CN111311658A (en) | 2020-06-19 |
| CN111311658B true CN111311658B (en) | 2023-11-07 |
Family
ID=71147394
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010255440.7A Active CN111311658B (en) | 2020-04-02 | 2020-04-02 | Image registration method and related device for dual-light imaging system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111311658B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111879412B (en) * | 2020-08-03 | 2021-12-21 | 烟台艾睿光电科技有限公司 | Image generation method and device for refrigeration type infrared detector and readable storage medium |
| CN112561837A (en) * | 2020-11-25 | 2021-03-26 | 广州紫川电子科技有限公司 | Target object display method and system based on double-light fusion, electronic equipment and medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107169921A (en) * | 2017-04-26 | 2017-09-15 | 国网上海市电力公司 | The figure registration system and method for a kind of pair of spectrum |
| CN107301661A (en) * | 2017-07-10 | 2017-10-27 | 中国科学院遥感与数字地球研究所 | High-resolution remote sensing image method for registering based on edge point feature |
| CN107680112A (en) * | 2017-10-16 | 2018-02-09 | 北京邮电大学 | Method for registering images |
| CN107689058A (en) * | 2017-09-01 | 2018-02-13 | 哈尔滨理工大学 | A kind of image registration algorithm based on SURF feature extractions |
| CN107862708A (en) * | 2017-11-08 | 2018-03-30 | 合肥工业大学 | A kind of SAR and visible light image registration method |
| CN107977924A (en) * | 2016-10-21 | 2018-05-01 | 杭州海康威视数字技术股份有限公司 | A kind of image processing method based on dual sensor imaging, system |
| CN108198157A (en) * | 2017-12-22 | 2018-06-22 | 湖南源信光电科技股份有限公司 | Heterologous image interfusion method based on well-marked target extracted region and NSST |
| CN109118525A (en) * | 2017-06-23 | 2019-01-01 | 北京遥感设备研究所 | A kind of dual-band infrared image airspace method for registering |
| CN109741376A (en) * | 2018-11-19 | 2019-05-10 | 天津津航技术物理研究所 | It is a kind of based on improve RANSAC algorithm in, LONG WAVE INFRARED method for registering images |
| CN114217313A (en) * | 2021-12-24 | 2022-03-22 | 中国科学院空天信息创新研究院 | Multi-band angle feature map generation method, device, equipment and storage medium |
-
2020
- 2020-04-02 CN CN202010255440.7A patent/CN111311658B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107977924A (en) * | 2016-10-21 | 2018-05-01 | 杭州海康威视数字技术股份有限公司 | A kind of image processing method based on dual sensor imaging, system |
| CN107169921A (en) * | 2017-04-26 | 2017-09-15 | 国网上海市电力公司 | The figure registration system and method for a kind of pair of spectrum |
| CN109118525A (en) * | 2017-06-23 | 2019-01-01 | 北京遥感设备研究所 | A kind of dual-band infrared image airspace method for registering |
| CN107301661A (en) * | 2017-07-10 | 2017-10-27 | 中国科学院遥感与数字地球研究所 | High-resolution remote sensing image method for registering based on edge point feature |
| CN107689058A (en) * | 2017-09-01 | 2018-02-13 | 哈尔滨理工大学 | A kind of image registration algorithm based on SURF feature extractions |
| CN107680112A (en) * | 2017-10-16 | 2018-02-09 | 北京邮电大学 | Method for registering images |
| CN107862708A (en) * | 2017-11-08 | 2018-03-30 | 合肥工业大学 | A kind of SAR and visible light image registration method |
| CN108198157A (en) * | 2017-12-22 | 2018-06-22 | 湖南源信光电科技股份有限公司 | Heterologous image interfusion method based on well-marked target extracted region and NSST |
| CN109741376A (en) * | 2018-11-19 | 2019-05-10 | 天津津航技术物理研究所 | It is a kind of based on improve RANSAC algorithm in, LONG WAVE INFRARED method for registering images |
| CN114217313A (en) * | 2021-12-24 | 2022-03-22 | 中国科学院空天信息创新研究院 | Multi-band angle feature map generation method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111311658A (en) | 2020-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111145238B (en) | Three-dimensional reconstruction method and device for monocular endoscopic image and terminal equipment | |
| CN111179358B (en) | Calibration method, device, equipment and storage medium | |
| CN108805934B (en) | External parameter calibration method and device for vehicle-mounted camera | |
| US9070042B2 (en) | Image processing apparatus, image processing method, and program thereof | |
| CN109784250B (en) | Positioning method and device of automatic guide trolley | |
| CN109961417B (en) | Image processing method, image processing apparatus, and mobile apparatus control method | |
| CN109447908A (en) | A kind of coil of strip recognition positioning method based on stereoscopic vision | |
| CN111950426A (en) | Target detection method and device and delivery vehicle | |
| CN110033407B (en) | Shield tunnel surface image calibration method, splicing method and splicing system | |
| CN111311658B (en) | Image registration method and related device for dual-light imaging system | |
| CN110956661A (en) | Method for calculating dynamic pose of visible light and infrared camera based on bidirectional homography matrix | |
| JP6188052B2 (en) | Information system and server | |
| CN112927306B (en) | Calibration method and device of shooting device and terminal equipment | |
| KR101589167B1 (en) | System and Method for Correcting Perspective Distortion Image Using Depth Information | |
| CN115953483A (en) | Parameter calibration method and device, computer equipment and storage medium | |
| CN113012197A (en) | Binocular vision odometer positioning method suitable for dynamic traffic scene | |
| CN111383264B (en) | Positioning method, positioning device, terminal and computer storage medium | |
| CN114998773A (en) | Characteristic mismatching elimination method and system suitable for aerial image of unmanned aerial vehicle system | |
| CN111382591A (en) | Binocular camera ranging correction method and vehicle-mounted equipment | |
| CN114445583A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN110245674B (en) | Template matching method, device, equipment and computer storage medium | |
| CN112102378A (en) | Image registration method and device, terminal equipment and computer readable storage medium | |
| CN115841517A (en) | Structural light calibration method and device based on DIC double-circle cross ratio | |
| KR101574195B1 (en) | Auto Calibration Method for Virtual Camera based on Mobile Platform | |
| CN113723432B (en) | Intelligent identification and positioning tracking method and system based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |