CN111814792B - Feature point extraction and matching method based on RGB-D image - Google Patents
Feature point extraction and matching method based on RGB-D image Download PDFInfo
- Publication number
- CN111814792B CN111814792B CN202010923532.8A CN202010923532A CN111814792B CN 111814792 B CN111814792 B CN 111814792B CN 202010923532 A CN202010923532 A CN 202010923532A CN 111814792 B CN111814792 B CN 111814792B
- Authority
- CN
- China
- Prior art keywords
- point
- points
- feature
- attribute
- rgb
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a feature point extracting and matching method based on RGB-D images, which comprises the steps of firstly calibrating an RGB-D camera to obtain internal and external parameters of the RGB-D camera, then collecting a picture, correcting the picture according to the internal and external parameters, extracting feature points on the corrected RGB image through a local feature extraction method, obtaining depth information of the feature points from a depth map, calculating an interested region near the feature points in the depth map according to the depth information of the feature points, converting pixels in the interested region into three-dimensional point clouds, and selecting n points closest to the feature points to obtain the adjacent three-dimensional point clouds; finally, a covariance matrix of the adjacent three-dimensional point clouds is obtained through calculation, singular value decomposition is carried out, and three eigenvalues lambda arranged from large to small are obtained1、λ2、λ3And judging the space geometric characteristic attribute of the characteristic points according to the university relation of the characteristic values, and performing characteristic matching on the characteristic points belonging to the same attribute. The method has simple principle and high accuracy of image matching.
Description
Technical Field
The invention relates to the field of image feature extraction, in particular to a feature point extraction and matching method based on RGB-D images.
Background
There are many feature extraction and matching methods for 2-dimensional RGB images, such as SIFT, SURF, FAST, BRIEF, ORB, etc., and these features have been widely used in practical algorithms. However, in a scene with high similarity of features, such as the gobi, desert, or extraterrestrial (e.g., moon, mars), the feature extraction is prone to be inaccurate or mismatching.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a feature point extraction and matching method based on an RGB-D image, and the specific technical scheme is as follows:
a feature point extracting and matching method based on RGB-D images specifically comprises the following steps:
s1: calibrating the RGB-D camera to obtain internal and external parameters of the RGB-D camera;
s2: collecting a picture, and correcting the picture according to the internal and external parameters obtained in S1 to obtain a corrected RGB image;
s3: extracting characteristic points on the corrected RGB image by a local characteristic extraction method, and acquiring depth information of the characteristic points from the depth mapd_depth;
S4: depth information from feature pointsd_depthCalculating an interested area near the feature point in the depth map, converting pixels in the interested area into three-dimensional point cloud, and selecting n points closest to the feature point to obtain adjacent three-dimensional point cloud;
s5: calculating to obtain a covariance matrix of the adjacent three-dimensional point cloud, and performing singular value decomposition to obtain three eigenvalues lambda arranged from large to small1、λ2、λ3When the characteristic value satisfies λ1/λ2<1.5、λ2/λ3>10、λ1/λ3If the attribute is more than 10, setting the attribute of the feature point as a surface point; when the characteristic value satisfies λ1/λ2>10、λ1/λ3>10、λ2/λ3If the attribute is less than 1.5, the attribute of the feature point is set as a line point; when the three characteristic values satisfy lambda1/λ2<1.5、λ1/λ3<1.5、λ2/λ3If the attribute is less than 1.5, the attribute of the feature point is set as a cluster point;
s6: and carrying out feature matching on the feature points belonging to the same attribute.
Further, the step S4 is implemented by the following sub-steps:
s4.1: determining an r multiplied by r interested area with the feature point as the center, and filtering out pixel points with depth values smaller than 1m and larger than 7 m; wherein r is the side length of the region of interest, expressed in number of pixels, determined by calculating k-d_ depthAnd rounding up to obtain; k represents a proportionality coefficient and controls the size of the region of interest;
s4.2: will feelTwo-dimensional image coordinates of region of interest pixels (u i,v i) Point cloud converted into world coordinate system (x i,y i, z i) I =1,2 · · wherein the three-dimensional points corresponding to the feature points are represented by (a) ((b))x 0,y 0, z 0);
S4.3: comparing all the point clouds: (x i,y i, z i) Three-dimensional points corresponding to feature points: (x 0,y 0, z 0) And selecting n three-dimensional points closest to the three-dimensional points corresponding to the feature points to form an adjacent three-dimensional point cloud set { (x 1,y 1, z 1), (x 2,y 2, z 2),···,(x n,y n, z n) }, i.e. a great faceX i}={(x i,y i, z i)}, i=1,2···n。
Further, the step S5 is implemented by the following sub-steps:
S5.2: the covariance matrix sigma of the point set formed by the characteristic point and the adjacent points is obtained by calculation
S5.3: SVD singular value decomposition of covariance matrix
Wherein R is an orthogonal matrix; lambda [ alpha ]1、λ2、λ3For three eigenvalues arranged from large to small, when the eigenvalue satisfies lambda1/λ2<1.5、λ2/λ3>10、λ1/λ3If the attribute is more than 10, setting the attribute of the feature point as a surface point; when the characteristic value satisfies λ1/λ2>10、λ1/λ3>10、λ2/λ3If the attribute is less than 1.5, the attribute of the feature point is set as a line point; when the three characteristic values satisfy lambda1/λ2<1.5、λ1/λ3<1.5、λ2/λ3If < 1.5, the feature point attribute is set as a cluster point.
Furthermore, the value range of the side length r of the region of interest is more than or equal to 4 and less than or equal to 10.
The invention has the following beneficial effects:
the feature point extraction method is simple in principle, can improve the accuracy of image matching of RGB-D equipment under the condition of similar texture, and can be applied to various image feature extraction containing RGB information and depth information.
Drawings
FIG. 1 is a flow chart of a feature point extraction and matching method based on RGB-D images according to the present invention;
FIG. 2 is a diagram of the effect of ORB feature based extraction and matching;
FIG. 3 is a diagram illustrating the effect of the method of the present invention.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments, and the objects and effects of the present invention will become more apparent, it being understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.
The feature point extraction and matching method based on the RGB-D image is specially designed for RGB-D camera equipment, wherein the RGB-D camera equipment mainly refers to motion sensing equipment which can simultaneously acquire image information and object depth information, such as Kinect and Xtion. Before starting, a camera to be calibrated is used for shooting a plurality of chessboard pictures under different visual angles, and then internal parameters and external parameters of the RGB camera and the depth camera are respectively calculated by using a GML Calibration Toolbox.
As shown in fig. 1, the method for extracting and matching feature points based on RGB-D images of the present invention includes the following steps:
s1: calibrating the RGB-D camera to obtain internal and external parameters of the RGB-D camera;
s2: collecting a picture, and correcting the picture according to the internal and external parameters obtained in S1 to obtain a corrected RGB image;
s3: extracting characteristic points on the corrected RGB image by a local characteristic extraction method, and acquiring depth information of the characteristic points from the depth mapd_depth;
S4: depth information from feature pointsd_depthCalculating an interested area near the feature point in the depth map, converting pixels in the interested area into three-dimensional point cloud, and selecting n points closest to the feature point to obtain adjacent three-dimensional point cloud;
s4.1: determining an r multiplied by r interested area with the feature point as the center, and filtering out pixel points with depth values smaller than 1m and larger than 7 m;
wherein r is the side length of the region of interest, expressed in number of pixels, determined by calculating k-d_depthAnd rounding up to obtain; k represents a proportionality coefficient and controls the size of the region of interest; the farther the three-dimensional point represented by the feature point is from the camera, the smaller the selected area is, so as to ensure that the three-dimensional point corresponding to the pixel of the selected region of interest is near the three-dimensional point represented by the feature point. The value range of r is preferably 4-10.
S4.2: two-dimensional image coordinates of region-of-interest pixels (u i,v i) Point cloud converted into world coordinate system (x i,y i, z i) I =1,2 · · wherein the three-dimensional points corresponding to the feature points are represented by (a) ((b))x 0,y 0, z 0);
S4.3: comparing all the point clouds: (x i,y i, z i) And characteristic pointCorresponding three-dimensional points (x 0,y 0, z 0) And selecting the 15 three-dimensional points closest to the three-dimensional points corresponding to the feature points to form a neighboring three-dimensional point cloud set { (x 1,y 1, z 1), (x 2,y 2, z 2),···,(x n,y n, z n) }, i.e. a great faceX i}={(x i,y i, z i)}, i=1,2···15;
S5: calculating to obtain a covariance matrix of the adjacent three-dimensional point cloud, and performing singular value decomposition to obtain three eigenvalues lambda arranged from large to small1、λ2、λ3And classify it as follows:
S5.2: the covariance matrix sigma of the point set formed by the characteristic point and the adjacent points is obtained by calculation
S5.3: SVD singular value decomposition of covariance matrix
Wherein R is an orthogonal matrix; lambda [ alpha ]1、λ2、λ3For three eigenvalues arranged from large to small, when the eigenvalue satisfies lambda1/λ2<1.5、λ2/λ3>10、λ1/λ3When the attribute is more than 10, the attribute of the feature point is setDefining as a surface point; when the characteristic value satisfies λ1/λ2>10、λ1/λ3>10、λ2/λ3If the attribute is less than 1.5, the attribute of the feature point is set as a line point; when the three characteristic values satisfy lambda1/λ2<1.5、λ1/λ3<1.5、λ2/λ3If < 1.5, the feature point attribute is set as a cluster point.
S6: and carrying out feature matching on the feature points belonging to the same attribute.
Fig. 2 is a result of feature extraction and matching based on ORB, and fig. 3 is a result of feature extraction and matching of the method of the present invention. As can be seen from fig. 2, by using the conventional ORB feature extraction method, mismatching is likely to occur in some visual feature approximation environments, the method provided by the present invention fuses spatial information of features, and feature points having the same spatial geometric features can be registered, thereby improving the accuracy of feature point matching, as shown in fig. 3.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and although the invention has been described in detail with reference to the foregoing examples, it will be apparent to those skilled in the art that various changes in the form and details of the embodiments may be made and equivalents may be substituted for elements thereof. All modifications, equivalents and the like which come within the spirit and principle of the invention are intended to be included within the scope of the invention.
Claims (3)
1. A feature point extraction and matching method based on RGB-D images is characterized by comprising the following steps:
s1: calibrating the RGB-D camera to obtain internal and external parameters of the RGB-D camera;
s2: collecting a picture, and correcting the picture according to the internal and external parameters obtained in S1 to obtain a corrected RGB image;
s3: extracting characteristic points on the corrected RGB image by a local characteristic extraction method, and acquiring depth information of the characteristic points from the depth mapd_depth;
S4: depth information from feature pointsd_depthCalculating an interested area near the feature point in the depth map, converting pixels in the interested area into three-dimensional point cloud, and selecting n points closest to the feature point to obtain adjacent three-dimensional point cloud; the S4 is realized by the following substeps:
s4.1: determining an r multiplied by r interested area with the feature point as the center, and filtering out pixel points with depth values smaller than 1m and larger than 7 m; wherein r is the side length of the region of interest, expressed in number of pixels, determined by calculating k-d_depthAnd rounding up to obtain; k represents a proportionality coefficient and controls the size of the region of interest;
s4.2: two-dimensional image coordinates of region-of-interest pixels (u i,v i) Point cloud converted into world coordinate system (x i,y i, z i) I =1,2 · · wherein the three-dimensional points corresponding to the feature points are represented by (a) ((b))x 0,y 0, z 0);
S4.3: comparing all the point clouds: (x i,y i, z i) Three-dimensional points corresponding to feature points: (x 0,y 0, z 0) And selecting n three-dimensional points closest to the three-dimensional points corresponding to the feature points to form an adjacent three-dimensional point cloud set { (x 1,y 1, z 1), (x 2,y 2, z 2),···,(x n,y n, z n) }, i.e. a great faceX i}={(x i,y i, z i)}, i=1,2···n;
S5: calculating to obtain a covariance matrix of the adjacent three-dimensional point cloud, and performing singular value decomposition to obtain three eigenvalues lambda arranged from large to small1、λ2、λ3When the characteristic value satisfies λ1/λ2<1.5、λ2/λ3>10、λ1/λ3If the attribute is more than 10, setting the attribute of the feature point as a surface point; when the characteristic value satisfies λ1/λ2>10、λ1/λ3>10、λ2/λ3If the attribute is less than 1.5, the attribute of the feature point is set as a line point; when the three characteristic values satisfy lambda1/λ2<1.5、λ1/λ3<1.5、λ2/λ3If the attribute is less than 1.5, the attribute of the feature point is set as a cluster point;
s6: and carrying out feature matching on the feature points belonging to the same attribute.
2. The RGB-D image-based feature point extracting and matching method according to claim 1, wherein the S5 is implemented by the following sub-steps:
S5.2: the covariance matrix sigma of the point set formed by the characteristic point and the adjacent points is obtained by calculation
S5.3: SVD singular value decomposition of covariance matrix
Wherein R is an orthogonal matrix; lambda [ alpha ]1、λ2、λ3For three eigenvalues arranged from large to small, when the eigenvalue satisfies lambda1/λ2<1.5、λ2/λ3>10、λ1/λ3If the attribute is more than 10, setting the attribute of the feature point as a surface point; when the characteristic value satisfies λ1/λ2>10、λ1/λ3>10、λ2/λ3If the attribute is less than 1.5, the attribute of the feature point is set as a line point; when the three characteristic values satisfy lambda1/λ2<1.5、λ1/λ3<1.5、λ2/λ3If < 1.5, the feature point attribute is set as a cluster point.
3. The method for extracting and matching feature points based on RGB-D images as claimed in claim 1, wherein the side length r of the region of interest is in a range of 4 ≤ r ≤ 10.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010923532.8A CN111814792B (en) | 2020-09-04 | 2020-09-04 | Feature point extraction and matching method based on RGB-D image |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010923532.8A CN111814792B (en) | 2020-09-04 | 2020-09-04 | Feature point extraction and matching method based on RGB-D image |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111814792A CN111814792A (en) | 2020-10-23 |
CN111814792B true CN111814792B (en) | 2020-12-29 |
Family
ID=72859930
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010923532.8A Active CN111814792B (en) | 2020-09-04 | 2020-09-04 | Feature point extraction and matching method based on RGB-D image |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111814792B (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112489215B (en) * | 2020-11-27 | 2022-09-16 | 之江实验室 | Single-drawing-batch three-dimensional road parametric modeling method with road surface marks |
CN114720993B (en) * | 2022-03-30 | 2024-08-20 | 上海木蚁机器人科技有限公司 | Robot positioning method, apparatus, electronic device and storage medium |
CN116170601B (en) * | 2023-04-25 | 2023-07-11 | 之江实验室 | Image compression method based on four-column vector block singular value decomposition |
CN117114782B (en) * | 2023-10-24 | 2024-05-28 | 佛山电力设计院有限公司 | Construction engineering cost analysis method |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101490055B1 (en) * | 2013-10-30 | 2015-02-06 | 한국과학기술원 | Method for localization of mobile robot and mapping, and apparatuses operating the same |
CN111311679A (en) * | 2020-01-31 | 2020-06-19 | 武汉大学 | Free floating target pose estimation method based on depth camera |
-
2020
- 2020-09-04 CN CN202010923532.8A patent/CN111814792B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101490055B1 (en) * | 2013-10-30 | 2015-02-06 | 한국과학기술원 | Method for localization of mobile robot and mapping, and apparatuses operating the same |
CN111311679A (en) * | 2020-01-31 | 2020-06-19 | 武汉大学 | Free floating target pose estimation method based on depth camera |
Non-Patent Citations (1)
Title |
---|
基于无源信标的移动机器人室内定位技术研究;李月华;《中国博士学位论文全文数据库信息科技辑》;20190115;第91-112页 * |
Also Published As
Publication number | Publication date |
---|---|
CN111814792A (en) | 2020-10-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111814792B (en) | Feature point extraction and matching method based on RGB-D image | |
CN110443836B (en) | Point cloud data automatic registration method and device based on plane features | |
CN108010036B (en) | Object symmetry axis detection method based on RGB-D camera | |
CN108596961B (en) | Point cloud registration method based on three-dimensional convolutional neural network | |
CN106228507B (en) | A kind of depth image processing method based on light field | |
CN103810744B (en) | It is backfilled a little in cloud | |
CN107977996B (en) | Space target positioning method based on target calibration positioning model | |
CN109472820B (en) | Monocular RGB-D camera real-time face reconstruction method and device | |
CN112200203B (en) | Matching method of weak correlation speckle images in oblique field of view | |
CN110796691B (en) | Heterogeneous image registration method based on shape context and HOG characteristics | |
CN104574347A (en) | On-orbit satellite image geometric positioning accuracy evaluation method on basis of multi-source remote sensing data | |
CN109003307B (en) | Underwater binocular vision measurement-based fishing mesh size design method | |
Cherian et al. | Accurate 3D ground plane estimation from a single image | |
CN108401565B (en) | Remote sensing image registration method based on improved KAZE features and Pseudo-RANSAC algorithms | |
CN114396875A (en) | Rectangular parcel volume measurement method based on vertical shooting of depth camera | |
CN114998773B (en) | Characteristic mismatching elimination method and system suitable for aerial image of unmanned aerial vehicle system | |
CN108462866A (en) | A kind of 3D stereo-picture color calibration methods based on matching and optimization | |
US11475629B2 (en) | Method for 3D reconstruction of an object | |
KR102547333B1 (en) | Depth Image based Real-time ground detection method | |
CN113642397A (en) | Object length measuring method based on mobile phone video | |
CN111369435B (en) | Color image depth up-sampling method and system based on self-adaptive stable model | |
CN111179271B (en) | Object angle information labeling method based on retrieval matching and electronic equipment | |
Shen et al. | A 3D modeling method of indoor objects using Kinect sensor | |
CN108416815B (en) | Method and apparatus for measuring atmospheric light value and computer readable storage medium | |
CN108805896B (en) | Distance image segmentation method applied to urban environment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |