CN114399540A - Heterogeneous image registration method and system based on two-dimensional iteration closest point - Google Patents
Heterogeneous image registration method and system based on two-dimensional iteration closest point Download PDFInfo
- Publication number
- CN114399540A CN114399540A CN202111512753.7A CN202111512753A CN114399540A CN 114399540 A CN114399540 A CN 114399540A CN 202111512753 A CN202111512753 A CN 202111512753A CN 114399540 A CN114399540 A CN 114399540A
- Authority
- CN
- China
- Prior art keywords
- image
- point
- point set
- pixel
- mapping matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 239000011159 matrix material Substances 0.000 claims abstract description 42
- 238000013507 mapping Methods 0.000 claims abstract description 39
- 238000003708 edge detection Methods 0.000 claims abstract description 13
- 238000004364 calculation method Methods 0.000 claims description 10
- 238000012897 Levenberg–Marquardt algorithm Methods 0.000 claims description 3
- 238000011426 transformation method Methods 0.000 claims description 3
- 238000004422 calculation algorithm Methods 0.000 abstract description 3
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/08—Projecting images onto non-planar surfaces, e.g. geodetic screens
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/20—Linear translation of whole images or parts thereof, e.g. panning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The invention belongs to the technical field of image processing and computer vision, and discloses a heterogeneous image registration method and system based on a two-dimensional iteration nearest point. Firstly, carrying out edge detection on images output by different sensors to obtain a binary image and a corresponding pixel point set; then, carrying out mapping projection on a pixel point set on the binary image by using the initialized mapping matrix, and finding out the closest point of the projected pixel; then, updating the mapping matrix by using the closest point, continuing projection, circularly and iteratively updating, and recording the sum of projection errors of the mapping matrix after each updating; and finally, judging, and if the sum of the adjacent reprojection errors is smaller than a set threshold, outputting a corresponding mapping matrix as a final result. The invention fully considers the characteristics of similar overall outline but different brightness details of the image to be registered, designs the technical scheme of directly registering the extracted image outline, can effectively solve the problems of low precision and large error in the registration of different images, and improves the robustness of the algorithm to the image difference.
Description
Technical Field
The invention belongs to the technical field of image processing and computer vision, and particularly relates to a heterogeneous image registration method and system based on a two-dimensional iteration nearest point.
Background
The heterogeneous image registration is to find the corresponding relation of two or more image pixel points in the same scene so as to achieve the purpose of accurate image processing. However, in an actual task, due to the difference between the acquisition equipment and the acquisition environment, the difference between the resolution and the detail of the image to be matched is easily caused, so that the common image registration method is difficult to obtain an accurate result.
Disclosure of Invention
The method aims at the problems of low precision and large error of the common image registration method. The invention fully considers the characteristics of similar overall outline but different brightness details of the image to be registered and the like, and provides a technical scheme for directly registering the extracted image outline so as to improve the robustness of the algorithm to the image difference.
In order to achieve the purpose, the invention is realized by adopting the following technical scheme.
A heterogeneous image registration method and system based on two-dimensional iteration closest points comprises the following steps:
step S1, collecting an image of a set object by a first type image collecting device to obtain an image A, and collecting an image of the set object by a second type image collecting device to obtain an image B;
specifically, the first type of image acquisition device is an infrared camera; the second type of image capture device is an RBG camera,
step S2, performing edge detection on the image A by using a Sobel operator to obtain a binary image A and a corresponding pixel point set P; performing edge detection on the image B by using a Sobel operator to obtain a binary image B and a corresponding pixel point set Q;
specifically, step S2 includes the steps of:
step S201, an image A is output by an infrared camera, and an image B is output by an RBG camera and is used as an input image for edge detection;
step S202, performing convolution and operation on two groups of matrixes 3x3 and the input image respectively in the transverse direction and the longitudinal direction to obtain brightness difference approximate values in the transverse direction and the longitudinal direction;
step S203, solving the gradient amplitude M of the input image;
step S204, comparing M with a set edge node threshold, and if M is greater than the edge node threshold, taking M as an edge node;
step S205, obtaining a binary image A and a binary image B from the result after edge detection, wherein all pixel points with the value of 1 on the binary image A form a point set P, and all pixel points with the value of 1 on the binary image B form a point set Q;
in step S3, the number of initialization iterations k is 0, and the mapping matrix H is assumed in advancekAnd all the points in the point set P are mapped by using a mapping matrix HkProjecting the image to a binary image B to obtain a point set Pk+1。
Step S4, searching point set P in point set Qk+1Closest set of pixel points Rk+1。
Specifically, step S4 includes the steps of:
step S401, for piPixel point coordinate (x)i’,yi') is rounded to give round (x)i’,yi’)；
Step S402, round pixel coordinate round (x)i’,yi') select x image block of x for the centre, and traverse and search to the image matrix, and judge whether the picture contains the pixel point in the point set Q;
step S403, on the basis of the step S402, continuing to expand the image block, and searching for the pixel point containing the point set Q in the image block until the current image block contains the pixel point containing the point set Q;
step S404, calculating coordinates and p of pixel points in the point set QiPixel point coordinate (x)i’,yi') and taking the smallest distance as qiAll of qiForm a set of points Rk+1。
Step S5, calculating point set P to point set Rk+1Mapping matrix H ofk+1And recording the sum of the minimum errors of the reprojection of all the pointsdk+1。
Specifically, step S5 includes the steps of:
step S501, according to the point set P and the point set Rk+1Calculating the initial solution of the mapping matrix by a direct linear transformation method (DLT) according to the pixel coordinate value in the mapping matrix;
step S502, taking the initial solution as a starting point, taking the sum of the reprojection errors as a target function, and calculating by using a Levenberg-Marquardt algorithm to obtain the optimal solution of the mapping matrix as Hk+1；
Step S503, recording the sum d of the minimum errors of the corresponding re-projectionk+1。
Step S6, using mapping matrix Hk+1Projecting all points in the point set P to the binary image B to obtain a point set Pk+2；
Step S7, searching point set P in point set Qk+2Closest set of pixel points Rk+2；
Step S8, calculating point set P to point set Rk+2Mapping matrix H ofk+2And recording the sum d of the minimum errors of the reprojection of all the pointsk+2；
Step S9, a convergence threshold T is set, the number of iterations k is updated to k +1, and a relation d is obtainedk+1-dk+2<When T is satisfied, outputting the corresponding mapping matrix Hk+2As an optimal mapping matrix for registration of image a and image B.
The invention provides a heterogeneous image registration method and a heterogeneous image registration system based on a two-dimensional iteration closest point, and compared with the prior art, the heterogeneous image registration method and the heterogeneous image registration system have the following beneficial effects:
the invention fully considers the characteristics of similar overall outline but different brightness details of the images to be registered and the like, adopts the calculation method of mapping matrix projection and direct linear transformation to register the heterogeneous images, can effectively reduce the influence of illumination conditions and an imaging mechanism on a registration algorithm, and better solves the matching problem between the heterogeneous images.
Drawings
Fig. 1 is a flow chart of an implementation in an embodiment of the invention.
Detailed Description
In order to make the objects and features of the present invention more apparent and understandable, the present invention will be described in detail below with reference to embodiments and the accompanying drawings.
A method and a system for registering heterogeneous images based on two-dimensional iterative closest point are disclosed, as shown in FIG. 1, and comprise the following steps:
in step S1, an image of a set object is acquired by a first type image acquisition means to obtain an image a, and an image of the set object is acquired by a second type image acquisition means to obtain an image B.
In the present embodiment, the first type of image pickup device is an infrared camera; the second type of image capture device is an RBG camera.
Step S2, performing edge detection on the image A by using a Sobel operator to obtain a binary image A and a corresponding pixel point set P; performing edge detection on the image B by using a Sobel operator to obtain a binary image B and a corresponding pixel point set Q, wherein the method comprises the following specific steps:
first, an image a is output from the infrared camera, and an image B is output from the RBG camera as an input image for edge detection.
Then, in the horizontal direction and the vertical direction, the two groups of 3 × 3 matrixes are convolved with the input image to obtain the brightness difference approximate values in the horizontal direction and the vertical direction:
then, the gradient amplitude M of the input image is obtained, and the calculation formula is:
wherein G isxAnd GyRespectively representing the luminance difference approximate value in the horizontal direction and the luminance difference approximate value in the vertical direction, and representing convolution operation.
And secondly, comparing M with a set edge node threshold, and if M is greater than the edge node threshold, taking M as an edge node.
And finally, obtaining a binary image A and a binary image B from the result after the edge detection, wherein all the pixel points with the value of 1 on the binary image A form a point set P, and all the pixel points with the value of 1 on the binary image B form a point set Q.
In step S3, the number of initialization iterations k is 0, and the mapping matrix is assumed in advance
Further, all points in the point set P are utilized with the mapping matrix HkProjecting the image to a binary image B to obtain a point set Pk+1The projection formula is:
wherein (x)i,yi) Coordinate values i ═ 1, …, N indicating the number of pixels in the point set P, (x'i,y'i) The pixel coordinates of the projected point are shown.
Step S4, searching point set P in point set Qk+1Closest set of pixel points Rk+1The method comprises the following specific steps:
first, for piPixel point coordinate (x)i’,yi') is rounded to give round (x)i’,yi’)。
Preferably, a round () function is chosen as the rounding function.
Then, the integer pixel coordinate round (x)i’,yi') selects the image block of x as the center, and makes traversal search for the image matrix, and judges whether the image contains the pixel point in the point set Q.
Then, in step S402, the image blocks continue to be enlarged.
Preferably, the calculation formula for enlarging the image block is:
x＝2n+1
where n denotes the number of times the image block is enlarged.
Further, the pixel point containing the point set Q in the image block is searched until the pixel point containing the point set Q in the current image block.
Finally, calculating the coordinates and p of the pixel points in the point set QiPixel point coordinate (x)i’,yi') and taking the smallest distance as qiAll of qiForm a set of points Rk+1。
Step S5, calculating point set P to point set Rk+1Mapping matrix H ofk+1And recording the sum d of the minimum errors of the reprojection of all the pointsk+1The method comprises the following specific steps:
first, according to the point set P and the point set Rk+1The initial solution of the mapping matrix is calculated by a direct linear transformation method (DLT) for the pixel coordinate values in (1).
Then, taking the initial solution as a starting point, taking the sum of the reprojection errors as an objective function, and calculating by utilizing a Levenberg-Marquardt algorithm to obtain an optimal solution of a mapping matrix as Hk+1。
Finally, the sum d of the minimum errors of the corresponding reprojection is recordedk+1Sum of minimum errors dk+1The calculation method is as follows:
wherein (u)i,vi) Is a set of points Rk+1Pixel point q iniThe coordinates of the pixels of (a) and (b),
is that the pixel points in the point set P pass through the mapping matrix Hk+1Pixel coordinates of a point projected onto the binary image B.
Step S6, using mapping matrix Hk+1Place in point set PProjecting the points on a binary image B to obtain a point set Pk+2。
Step S7, searching point set P in point set Qk+2Closest set of pixel points Rk+2。
Step S8, calculating point set P to point set Rk+2Mapping matrix H ofk+2And recording the sum d of the minimum errors of the reprojection of all the pointsk+2。
In step S9, a convergence threshold T is set, and the calculation formula of the convergence threshold T is preferably:
T＝0.1*N
wherein N is the number of pixel points.
Further, the number of iterations k is continuously updated to k +1, when the relation dk+1-dk+2<When T is satisfied, outputting the corresponding mapping matrix Hk+2As an optimal mapping matrix for registration of image a and image B.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (10)
1. A heterogeneous image registration method and system based on two-dimensional iteration closest point is characterized by comprising the following steps:
step S1, collecting an image of a set object by a first type image collecting device to obtain an image A, and collecting an image of the set object by a second type image collecting device to obtain an image B;
step S2, performing edge detection on the image A by using a Sobel operator to obtain a binary image A and a corresponding pixel point set P; performing edge detection on the image B by using a Sobel operator to obtain a binary image B and a corresponding pixel point set Q;
step S3, initialAssuming that the iteration number k is 0, the mapping matrix H is assumed in advancekAnd all the points in the point set P are mapped by using a mapping matrix HkProjecting the image to a binary image B to obtain a point set Pk+1；
Step S4, searching point set P in point set Qk+1Closest set of pixel points Rk+1；
Step S5, calculating point set P to point set Rk+1Mapping matrix H ofk+1And recording the sum d of the minimum errors of the reprojection of all the pointsk+1；
Step S6, using mapping matrix Hk+1Projecting all points in the point set P to the binary image B to obtain a point set Pk+2；
Step S7, searching point set P in point set Qk+2Closest set of pixel points Rk+2；
Step S8, calculating point set P to point set Rk+2Mapping matrix H ofk+2And recording the sum d of the minimum errors of the reprojection of all the pointsk+2；
Step S9, a convergence threshold T is set, the number of iterations k is updated to k +1, and a relation d is obtainedk+1-dk+2<When T is satisfied, outputting the corresponding mapping matrix Hk+2As an optimal mapping matrix for registration of image a and image B.
2. The method and system for registration of heterogeneous images based on two-dimensional iterative closest points according to claim 1, wherein the first type of image acquisition device is an infrared camera; the second type of image capture device is an RBG camera.
3. The method and system for registration of heterogeneous images based on two-dimensional iterative closest point according to claim 1, wherein step S2 includes the steps of:
step S201, an image A is output by an infrared camera, and an image B is output by an RBG camera and is used as an input image for edge detection;
step S202, convolving and operating two sets of matrices 3 × 3 with the input image in the horizontal and vertical directions, respectively, to obtain luminance difference approximate values in the horizontal and vertical directions:
step S203, the gradient amplitude M of the input image is obtained, and the calculation formula is:
wherein G isxAnd GyRespectively representing a luminance difference approximation value in the horizontal direction and a luminance difference approximation value in the vertical direction, representing a convolution operation,
step S204, comparing M with a set edge node threshold, and if M is greater than the edge node threshold, taking M as an edge node;
step S205, obtaining a binary image A and a binary image B from the result after the edge detection, wherein all the pixel points with the value of 1 on the binary image A form a point set P, and all the pixel points with the value of 1 on the binary image B form a point set Q.
4. The method and system for registration of heterogeneous images based on two-dimensional iterative closest point according to claim 1, wherein the mapping matrix H in step S3kEquation of matrix
5. The method and system for registration of heterogeneous images based on two-dimensional iterative closest point according to claim 1, wherein all points in the point set P in step S3 use mapping matrix HkProjection to binaryOn the image B, the specific calculation formula is:
wherein (x)i,yi) Coordinate values i ═ 1, …, N indicating the number of pixels in the point set P, (x'i,y'i) The pixel coordinates of the projected point are shown.
6. The method and system for registration of heterogeneous images based on two-dimensional iterative closest point according to claim 1, wherein step S4 includes the steps of:
step S401, for piPixel point coordinate (x)i’,yi') is rounded to give round (x)i’,yi’)；
Step S402, round pixel coordinate round (x)i’,yi') select x image block of x for the centre, and traverse and search to the image matrix, and judge whether the picture contains the pixel point in the point set Q;
step S403, on the basis of the step S402, continuing to expand the image block, and searching for the pixel point containing the point set Q in the image block until the current image block contains the pixel point containing the point set Q;
step S404, calculating coordinates and p of pixel points in the point set QiPixel point coordinate (x)i’,yi') and taking the smallest distance as qiAll of qiForm a set of points Rk+1。
7. The method and system for registration of different source images based on two-dimensional iterative closest point according to claim 6, wherein the specific calculation formula of the method for enlarging image blocks is as follows:
x＝2n+1
where n denotes the number of times the image block is enlarged.
8. The method and system for registration of heterogeneous images based on two-dimensional iterative closest point according to claim 1, wherein step S5 includes the steps of:
step S501, according to the point set P and the point set Rk+1Calculating the initial solution of the mapping matrix by a direct linear transformation method (DLT) according to the pixel coordinate value in the mapping matrix;
step S502, taking the initial solution as a starting point, taking the sum of the reprojection errors as a target function, and calculating by using a Levenberg-Marquardt algorithm to obtain the optimal solution of the mapping matrix as Hk+1；
Step S503, recording the sum d of the minimum errors of the corresponding re-projectionk+1。
9. The method and system for registration of heterogeneous images based on two-dimensional iterative closest point according to claim 1, wherein the step S5 records the sum d of minimum error of reprojection of all pointsk+1The specific calculation formula is as follows:
wherein (u)i,vi) Is a set of points Rk+1Pixel point q iniThe coordinates of the pixels of (a) and (b),
is that the pixel points in the point set P pass through the mapping matrix Hk+1Pixel coordinates of a point projected onto the binary image B.
10. The method and system for registration of different source images based on two-dimensional iterative closest point according to claim 1, wherein the convergence threshold T is set in step S8, and the specific calculation formula is:
T＝0.1*N
wherein N is the number of pixel points.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111512753.7A CN114399540A (en) | 2021-12-10 | 2021-12-10 | Heterogeneous image registration method and system based on two-dimensional iteration closest point |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111512753.7A CN114399540A (en) | 2021-12-10 | 2021-12-10 | Heterogeneous image registration method and system based on two-dimensional iteration closest point |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114399540A true CN114399540A (en) | 2022-04-26 |
Family
ID=81227433
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111512753.7A Pending CN114399540A (en) | 2021-12-10 | 2021-12-10 | Heterogeneous image registration method and system based on two-dimensional iteration closest point |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114399540A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116863861A (en) * | 2023-09-05 | 2023-10-10 | 欣瑞华微电子(上海)有限公司 | Image processing method and device based on non-explicit point judgment and readable storage medium |
-
2021
- 2021-12-10 CN CN202111512753.7A patent/CN114399540A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116863861A (en) * | 2023-09-05 | 2023-10-10 | 欣瑞华微电子(上海)有限公司 | Image processing method and device based on non-explicit point judgment and readable storage medium |
| CN116863861B (en) * | 2023-09-05 | 2023-11-24 | 欣瑞华微电子(上海)有限公司 | Image processing method and device based on non-explicit point judgment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021115071A1 (en) | Three-dimensional reconstruction method and apparatus for monocular endoscope image, and terminal device | |
| TWI729995B (en) | Generating a merged, fused three-dimensional point cloud based on captured images of a scene | |
| JP4371130B2 (en) | Method for generating a composite image from a set of images | |
| CN104685513A (en) | Feature based high resolution motion estimation from low resolution images captured using an array source | |
| CN113920205B (en) | Calibration method of non-coaxial camera | |
| CN109118544B (en) | Synthetic aperture imaging method based on perspective transformation | |
| CN110909750B (en) | Image difference detection method and device, storage medium and terminal | |
| KR100951309B1 (en) | New Calibration Method of Multi-view Camera for a Optical Motion Capture System | |
| CN112200203A (en) | Matching method of weak correlation speckle images in oblique field of view | |
| Cvišić et al. | Recalibrating the KITTI dataset camera setup for improved odometry accuracy | |
| CN116912333B (en) | Camera attitude self-calibration method based on operation fence calibration rod | |
| CN112946679B (en) | Unmanned aerial vehicle mapping jelly effect detection method and system based on artificial intelligence | |
| CN110598795A (en) | Image difference detection method and device, storage medium and terminal | |
| WO2021035627A1 (en) | Depth map acquisition method and device, and computer storage medium | |
| JP6956051B2 (en) | Image processing equipment, driving support system, image processing method and program | |
| CN111553845A (en) | Rapid image splicing method based on optimized three-dimensional reconstruction | |
| CN112929626A (en) | Three-dimensional information extraction method based on smartphone image | |
| KR20210133472A (en) | Method of merging images and data processing device performing the same | |
| CN114399540A (en) | Heterogeneous image registration method and system based on two-dimensional iteration closest point | |
| CN113808103A (en) | Automatic road surface depression detection method and device based on image processing and storage medium | |
| CN113793266A (en) | Multi-view machine vision image splicing method, system and storage medium | |
| CN112017259B (en) | Indoor positioning and image building method based on depth camera and thermal imager | |
| CN113723432B (en) | Intelligent identification and positioning tracking method and system based on deep learning | |
| CN115375843A (en) | Three-dimensional reconstruction method of uncalibrated monocular vision system | |
| JP2002352231A (en) | Method and device for detecting end point of human body |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |