CN111968035B - Image relative rotation angle calculation method based on loss function - Google Patents
Image relative rotation angle calculation method based on loss function Download PDFInfo
- Publication number
- CN111968035B CN111968035B CN202010779144.7A CN202010779144A CN111968035B CN 111968035 B CN111968035 B CN 111968035B CN 202010779144 A CN202010779144 A CN 202010779144A CN 111968035 B CN111968035 B CN 111968035B
- Authority
- CN
- China
- Prior art keywords
- images
- rotation angle
- spliced
- matching point
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention discloses a loss function-based image relative rotation angle calculation method, which comprises the following steps: acquiring any two adjacent images to be spliced, and adopting a surf algorithm to match and detect characteristic points to obtain N pairs of characteristic matching point pairs; screening feature points by adopting a RANSAC algorithm to obtain M pairs of feature matching point pairs; taking one of the images as a reference to obtain a translation vector; obtaining original course angles of two adjacent images to be spliced; establishing an objective function for searching the optimal rotation angle, obtaining a rotation angle theta corresponding to the minimum value of the objective function, and obtaining the relative rotation angles of the two images to be spliced
Through the scheme, the method has the advantages of simple logic, accurate calculation, no image deformation and the like, and has high practical value and popularization value in the technical field of image splicing.
Description
Technical Field
The invention relates to the technical field of image stitching, in particular to a loss function-based image relative rotation angle calculation method.
Background
The image stitching technology is a technology for stitching a plurality of images with overlapped parts (possibly obtained by different time, different visual angles or different sensors) into a large seamless high-resolution image. In image stitching, a popular way is a homography matrix, which mainly comprises the following steps: loading images to be spliced; creating an AKAZE symptom extractor; extracting key point and descriptor features; descriptor matching and extracting key points with better matching; aligning homography matrix images; creating a fusion mask layer, and preparing to start fusion; performing image perspective transformation and fusion operation; outputting the panorama after stitching. However, homography matrices also have the following problems:
firstly, in image stitching, if a mode of searching a homography matrix is adopted, image deformation transmission can be caused; if large area image stitching is performed, stitching by way of a homography matrix is not appropriate.
Secondly, in order to achieve accurate positioning after splicing, the images are not allowed to be deformed greatly.
In addition, the invention is disclosed in China with the patent application number of 201711132452.5 and the name of 'an image stitching method based on combination of unmanned aerial vehicle POS information and image SURF characteristics'. The global stitching is adopted to transform four corners, and the technology also has the problem of image deformation. And as a Chinese patent with the patent application number of 201210197720.2 and the name of a real-time splicing method of multi-beam side-scan sonar images based on course angle rotation, the method directly uses the course angle for calculation and does not correct the rotation angle; the spliced images have extremely large deviation.
Therefore, it is highly desirable to provide a method for calculating the relative rotation angle of an image based on a loss function, which is simple in logic, less in calculation effort, and free from image distortion.
Disclosure of Invention
In view of the above problems, the present invention aims to provide a method for calculating an image relative rotation angle based on a loss function, which adopts the following technical scheme:
the image relative rotation angle calculating method based on the loss function comprises the following steps:
acquiring any two adjacent images to be spliced, adopting surf algorithm to match and detect characteristic points to obtain N pairs of characteristic matching point pairs, and respectively marking characteristic matching point sets Kpts0A and Kpts0B of the two images to be spliced; the N is a natural number greater than 1;
screening feature points by adopting a RANSAC algorithm to obtain M pairs of feature matching point pairs; marking the feature matching point sets KptsA and KptsB after screening the two images to be spliced respectively; m is a natural number less than or equal to N;
taking one of the two images to be spliced as a reference, and converting the distance between the i feature matching point pairs after screening into a translation vector T i And translating another image to be stitched;
obtaining original course angles Angle of two adjacent images to be spliced;
establishing an objective function for searching the optimal rotation angle, wherein the expression is as follows:
R i the expression of (2) is:
wherein θ represents a rotation angle, kptsA i Represents the ith feature matching point in the A image after screening, kptsB i Representing the ith feature matching point, T in the B image after screening i A translation vector representing an ith feature matching point pair;
obtaining a rotation angle theta corresponding to the minimum value of the objective function L, and obtaining the relative rotation angles of the two images to be spliced
The expression is
Preferably, one of the two images to be spliced is taken as a reference, and a previous image is extracted along the photographing time sequence to be taken as a reference.
Further, the original heading Angle is the difference of the heading angles of the two images to be spliced.
Compared with the prior art, the invention has the following beneficial effects:
(1) The invention skillfully utilizes surf algorithm to match and detect characteristic points, and utilizes RANSAC algorithm to screen the characteristic points so as to obtain effective characteristic matching point pairs; the accuracy and reliability of parallel vectors between the images to be spliced are ensured;
(2) The invention skillfully establishes a loss function for searching the optimal rotation angle, and the smaller the value is, the more accurate the rotation angle is; according to the method, the minimum value of the objective function is obtained, so that the calculation accuracy of the rotation angle is improved;
(3) According to the invention, through correction calculation of the rotation angle, the splicing angle is ensured to be more reliable; the invention takes images on a time axis as a reference in sequence to realize large-area large-data lower splicing, and only carries out translation and rotation operations when pictures are spliced, so that the shapes of the pictures are not changed;
in conclusion, the method has the advantages of simple logic, accurate calculation, no image deformation and the like, and has high practical value and popularization value in the technical field of image stitching.
Drawings
For a clearer description of the technical solutions of the embodiments of the present invention, the drawings to be used in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and should not be considered as limiting the scope of protection, and other related drawings may be obtained according to these drawings without the need of inventive effort for a person skilled in the art.
FIG. 1 is a logic flow diagram of the present invention.
Fig. 2 shows two images to be stitched of a scene according to the present invention.
Fig. 3 is the stitched image of fig. 2.
Detailed Description
For the purposes, technical solutions and advantages of the present application, the present invention will be further described with reference to the accompanying drawings and examples, and embodiments of the present invention include, but are not limited to, the following examples. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present application based on the embodiments herein.
Examples
As shown in fig. 1 to 3, the present embodiment provides a loss function-based image relative rotation angle calculation method, which includes the steps of:
the method comprises the steps of firstly, randomly extracting two adjacent images to be spliced along a shooting sequence; and matching and detecting characteristic points by adopting a surf algorithm to obtain N pairs of characteristic matching point pairs, and respectively marking characteristic matching point sets Kpts0A and Kpts0B of two images to be spliced. Wherein, the ith pair of characteristic points is marked as Kpts0A i ，Kpts0B i ，0＜i＜N。
Secondly, screening feature points by adopting a RANSAC algorithm to obtain M pairs of feature matching point pairs; and marking the feature matching point sets KptsA and KptsB after screening the two images to be spliced respectively. Wherein, the ith pair of characteristic points is marked as KptsA i ，KptsB i ，0＜i＜M。
Third, taking the A diagram as the reference, taking any one characteristic point of the A diagram, such as taking the first point KptsA in KptsA 1 And KptsB 1 Calculating translation vector T i (T x ，T y )
T x ＝KptsB 1 x-KptsA 1 x
T y ＝KptsB 1 y-KptsA 1 y
Wherein KptsA 1 x，KptsA 1 y represents the characteristic point KptsA of the A diagram 1 X, y coordinates of KptsB 1 x，KptsB 1 y represents the characteristic point KptsB of the B diagram 1 X, y coordinates of (c).
Fourth, initial rotation angle: the original course Angle recorded by the A and B pictures is used for obtaining the course Angle of the A picture as angle=angle B -Angle A Wherein, the heading Angle of the B picture is Angle B Then, initial heading angle=angle B -Angle A 。
Fifthly, establishing an objective function for searching the optimal rotation angle, wherein the expression is as follows:
R i the expression of (2) is:
wherein θ represents a rotation angle, kptsA i Represents the ith feature matching point in the A image after screening, kptsB i Representing the ith feature matching point, T in the B image after screening i A translation vector representing an ith feature matching point pair;
step six, obtaining a rotation angle theta corresponding to the minimum value of the objective function L, and obtaining the relative rotation angles of the two images to be spliced
The expression is
In summary, the invention can realize accurate calculation of the relative rotation angle without changing the shape of the image. Compared with the prior art, the invention has the specific outstanding substantive characteristics and remarkable progress, and has high practical value and popularization value in the technical field of image splicing.
The above embodiments are only preferred embodiments of the present invention and are not intended to limit the scope of the present invention, but all changes made by adopting the design principle of the present invention and performing non-creative work on the basis thereof shall fall within the scope of the present invention.
Claims (3)
1. The image relative rotation angle calculating method based on the loss function is characterized by comprising the following steps of:
acquiring any two adjacent images to be spliced, adopting surf algorithm to match and detect characteristic points to obtain N pairs of characteristic matching point pairs, and respectively marking characteristic matching point sets Kpts0A and Kpts0B of the two images to be spliced; the N is a natural number greater than 1;
screening feature points by adopting a RANSAC algorithm to obtain M pairs of feature matching point pairs; marking the feature matching point sets KptsA and KptsB after screening the two images to be spliced respectively; m is a natural number less than or equal to N;
taking one of the two images to be spliced as a reference, and converting the distance between the i feature matching point pairs after screening into a translation vector T i And translating another image to be stitched;
obtaining original course angles Angle of two adjacent images to be spliced;
establishing an objective function for searching the optimal rotation angle, wherein the expression is as follows:
R i the expression of (2) is:
wherein θ represents a rotation angle, kptsA i Represents the ith feature matching point in the A image after screening, kptsB i Representing the ith feature matching point, T in the B image after screening i A translation vector representing an ith feature matching point pair;
obtaining the most valued objective function LThe rotation angle theta corresponding to the hour is used for obtaining the relative rotation angle of the two images to be spliced
The expression is
2. The method for calculating the relative rotation angle of the images based on the loss function according to claim 1, wherein one of the two images to be spliced is used as a reference, and the previous image is extracted along the photographing time sequence as a reference.
3. The loss function-based image relative rotation Angle calculation method according to claim 2, wherein the original heading Angle is a difference of heading angles of two images to be spliced.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010779144.7A CN111968035B (en) | 2020-08-05 | 2020-08-05 | Image relative rotation angle calculation method based on loss function |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010779144.7A CN111968035B (en) | 2020-08-05 | 2020-08-05 | Image relative rotation angle calculation method based on loss function |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111968035A CN111968035A (en) | 2020-11-20 |
| CN111968035B true CN111968035B (en) | 2023-06-20 |
Family
ID=73364558
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010779144.7A Active CN111968035B (en) | 2020-08-05 | 2020-08-05 | Image relative rotation angle calculation method based on loss function |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111968035B (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103149939A (en) * | 2013-02-26 | 2013-06-12 | 北京航空航天大学 | Dynamic target tracking and positioning method of unmanned plane based on vision |
| CN103292804A (en) * | 2013-05-27 | 2013-09-11 | 浙江大学 | Monocular natural vision landmark assisted mobile robot positioning method |
| CN103489176A (en) * | 2012-06-13 | 2014-01-01 | 中国科学院电子学研究所 | Method for extracting TPs from SAR image of serious geometric distortion |
| CN105931185A (en) * | 2016-04-20 | 2016-09-07 | 中国矿业大学 | Automatic splicing method of multiple view angle image |
| CN106708066A (en) * | 2015-12-20 | 2017-05-24 | 中国电子科技集团公司第二十研究所 | Autonomous landing method of unmanned aerial vehicle based on vision/inertial navigation |
| CN108932737A (en) * | 2018-06-15 | 2018-12-04 | 深圳地平线机器人科技有限公司 | In-vehicle camera pitch angle scaling method and device, electronic equipment and vehicle |
| CN109459046A (en) * | 2018-11-23 | 2019-03-12 | 杭州电子科技大学 | The positioning and air navigation aid of suspending underwater autonomous navigation device |
| CN110211043A (en) * | 2019-05-11 | 2019-09-06 | 复旦大学 | A kind of method for registering based on grid optimization for Panorama Mosaic |
| CN110458845A (en) * | 2019-06-25 | 2019-11-15 | 上海圭目机器人有限公司 | Unmanned plane image difference analysis method based on image similarity |
| CN110490045A (en) * | 2019-06-25 | 2019-11-22 | 上海市城市排水有限公司 | A kind of unmanned plane image difference analysis method |
| CN110636263A (en) * | 2019-09-20 | 2019-12-31 | 黑芝麻智能科技(上海)有限公司 | Panoramic annular view generation method, vehicle-mounted equipment and vehicle-mounted system |
| CN110827337A (en) * | 2018-08-08 | 2020-02-21 | 深圳地平线机器人科技有限公司 | Method and device for determining posture of vehicle-mounted camera and electronic equipment |
| CN111080529A (en) * | 2019-12-23 | 2020-04-28 | 大连理工大学 | Unmanned aerial vehicle aerial image splicing method for enhancing robustness |
| CN111161143A (en) * | 2019-12-16 | 2020-05-15 | 首都医科大学 | Optical positioning technology-assisted operation visual field panoramic stitching method |
-
2020
- 2020-08-05 CN CN202010779144.7A patent/CN111968035B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103489176A (en) * | 2012-06-13 | 2014-01-01 | 中国科学院电子学研究所 | Method for extracting TPs from SAR image of serious geometric distortion |
| CN103149939A (en) * | 2013-02-26 | 2013-06-12 | 北京航空航天大学 | Dynamic target tracking and positioning method of unmanned plane based on vision |
| CN103292804A (en) * | 2013-05-27 | 2013-09-11 | 浙江大学 | Monocular natural vision landmark assisted mobile robot positioning method |
| CN106708066A (en) * | 2015-12-20 | 2017-05-24 | 中国电子科技集团公司第二十研究所 | Autonomous landing method of unmanned aerial vehicle based on vision/inertial navigation |
| CN105931185A (en) * | 2016-04-20 | 2016-09-07 | 中国矿业大学 | Automatic splicing method of multiple view angle image |
| CN108932737A (en) * | 2018-06-15 | 2018-12-04 | 深圳地平线机器人科技有限公司 | In-vehicle camera pitch angle scaling method and device, electronic equipment and vehicle |
| CN110827337A (en) * | 2018-08-08 | 2020-02-21 | 深圳地平线机器人科技有限公司 | Method and device for determining posture of vehicle-mounted camera and electronic equipment |
| CN109459046A (en) * | 2018-11-23 | 2019-03-12 | 杭州电子科技大学 | The positioning and air navigation aid of suspending underwater autonomous navigation device |
| CN110211043A (en) * | 2019-05-11 | 2019-09-06 | 复旦大学 | A kind of method for registering based on grid optimization for Panorama Mosaic |
| CN110458845A (en) * | 2019-06-25 | 2019-11-15 | 上海圭目机器人有限公司 | Unmanned plane image difference analysis method based on image similarity |
| CN110490045A (en) * | 2019-06-25 | 2019-11-22 | 上海市城市排水有限公司 | A kind of unmanned plane image difference analysis method |
| CN110636263A (en) * | 2019-09-20 | 2019-12-31 | 黑芝麻智能科技(上海)有限公司 | Panoramic annular view generation method, vehicle-mounted equipment and vehicle-mounted system |
| CN111161143A (en) * | 2019-12-16 | 2020-05-15 | 首都医科大学 | Optical positioning technology-assisted operation visual field panoramic stitching method |
| CN111080529A (en) * | 2019-12-23 | 2020-04-28 | 大连理工大学 | Unmanned aerial vehicle aerial image splicing method for enhancing robustness |
Non-Patent Citations (2)
| Title |
|---|
| Vision-based Road-Following Using Proportional Navigation;Ryan S.Holt;《Selected papers from the 2nd International Symposium on UAVs》;193-216 * |
| 机载图像拼接关键技术研究;董强;《中国博士学位论文全文数据库 信息科技辑》(第10期);I138-49 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111968035A (en) | 2020-11-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111145238B (en) | Three-dimensional reconstruction method and device for monocular endoscopic image and terminal equipment | |
| CN106886979B (en) | Image splicing device and image splicing method | |
| JP2021533507A (en) | Image stitching methods and devices, in-vehicle image processing devices, electronic devices, storage media | |
| CN109829853A (en) | A kind of unmanned plane image split-joint method | |
| CN108171735B (en) | Billion pixel video alignment method and system based on deep learning | |
| CN108257089A (en) | A kind of method of the big visual field video panorama splicing based on iteration closest approach | |
| CN106447608A (en) | Video image splicing method and device | |
| CN113724135A (en) | Image splicing method, device, equipment and storage medium | |
| Shen et al. | Distortion-tolerant monocular depth estimation on omnidirectional images using dual-cubemap | |
| Jin | A three-point minimal solution for panoramic stitching with lens distortion | |
| CN103167247A (en) | Video sequence color image stitching method | |
| CN111968035B (en) | Image relative rotation angle calculation method based on loss function | |
| CN110956664B (en) | Real-time repositioning method for camera position of handheld three-dimensional scanning system | |
| CN109598675B (en) | Splicing method of multiple repeated texture images | |
| CN111047513A (en) | Robust image alignment method and device for cylindrical panoramic stitching | |
| CN107330856A (en) | A kind of method for panoramic imaging based on projective transformation and thin plate spline | |
| JP6931267B2 (en) | A program, device and method for generating a display image obtained by transforming the original image based on the target image. | |
| CN107146195A (en) | Sphere image split-joint method and device | |
| CN116579923A (en) | Image stitching method, device and storage medium | |
| CN116109484A (en) | Image splicing method, device and equipment for retaining foreground information and storage medium | |
| CN115456870A (en) | Multi-image splicing method based on external parameter estimation | |
| CN114463170A (en) | Large scene image splicing method for AGV application | |
| CN106709942A (en) | Panoramic image mistaken matching elimination method based on characteristic azimuth | |
| CN113112404A (en) | Image splicing method and device based on sliding window | |
| JP3993672B2 (en) | Coordinate conversion method, image composition method, and information recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |