CN112613372A - Outdoor environment visual inertia SLAM method and device - Google Patents

Outdoor environment visual inertia SLAM method and device Download PDF

Info

Publication number
CN112613372A
CN112613372A CN202011489168.5A CN202011489168A CN112613372A CN 112613372 A CN112613372 A CN 112613372A CN 202011489168 A CN202011489168 A CN 202011489168A CN 112613372 A CN112613372 A CN 112613372A
Authority
CN
China
Prior art keywords
image
slam
sky
gradient
optimized
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011489168.5A
Other languages
Chinese (zh)
Other versions
CN112613372B (en
Inventor
陶文宇
戴志强
朱祥维
李芳�
陈伟翔
陈彦莛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sun Yat Sen University
Original Assignee
Sun Yat Sen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sun Yat Sen University filed Critical Sun Yat Sen University
Priority to CN202011489168.5A priority Critical patent/CN112613372B/en
Publication of CN112613372A publication Critical patent/CN112613372A/en
Application granted granted Critical
Publication of CN112613372B publication Critical patent/CN112613372B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components

Abstract

The invention discloses an outdoor environment visual inertia SLAM method and device, wherein the method comprises the following steps: obtaining a gradient image by extracting image gradient information of an original image, and performing threshold segmentation processing on the gradient image to obtain an image to be optimized; calculating to-be-integrated boundary data according to the to-be-optimized image; integrating the boundary data to be integrated by adopting a polynomial fitting algorithm to obtain final boundary data; and segmenting the original image according to the final boundary data to obtain a sky region image, performing SLAM initialization on the inertial navigation data and the non-sky region image to obtain an SLAM frame, and performing SLAM composition positioning according to the SLAM frame. According to the embodiment of the invention, the boundary line is corrected by adopting a polynomial fitting algorithm, so that the operation amount and the operation time can be effectively reduced, the sky area segmentation effect can be quickly and accurately obtained, and the accuracy of positioning and track mapping can be effectively improved.

Description

Outdoor environment visual inertia SLAM method and device
Technical Field
The invention relates to the technical field of image processing, in particular to an outdoor environment visual inertia SLAM method and device.
Background
SLAM (Simultaneous Localization and mapping), the simultaneous Localization and mapping of Chinese translation, is a technology for building an environment model and estimating the self-movement by carrying one or more sensors under the condition of no environment prior information. As an important method for positioning and establishing an environment model, the method has wide application in the fields such as automatic driving and the like. The basic visual SLAM structure is relatively simple, but the problems of inaccurate positioning and track deviation can occur in various scenes. The visual-inertial unit, in which the inertial measurement unit and the camera are combined, is becoming an increasingly popular application framework. The existing outdoor environment visual inertia SLAM method adopts a mature SLAM framework, such as a VINS-Mono, ORB-SLAM3 and other SLAM frameworks, and can obtain ideal experimental results in an outdoor environment. However, the existing outdoor environment visual inertial SLAM method needs to waste a large amount of computing resources in an invalid sky area, so that the efficiency of positioning and trajectory mapping is poor.
Disclosure of Invention
The invention provides an outdoor environment vision inertia SLAM method and device, and aims to solve the technical problem that the existing outdoor environment vision inertia SLAM method wastes a large amount of computing resources in an invalid sky area, so that the positioning and track mapping efficiency is poor.
The first embodiment of the invention provides an outdoor environment visual inertia SLAM method, which comprises the following steps:
obtaining a gradient image by extracting image gradient information of an original image, and performing threshold segmentation processing on the gradient image to obtain an image to be optimized;
defining a sky boundary function according to the parameters of the image to be optimized, and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated;
integrating the boundary data to be integrated by adopting a polynomial fitting algorithm to obtain final boundary data;
and segmenting the original image according to the final boundary data to obtain a sky region image, performing SLAM initialization on inertial navigation data and the non-sky region image to obtain an SLAM frame, and performing SLAM composition positioning according to the SLAM frame.
Further, the gradient image is obtained by extracting image gradient information of the original image, and the image to be optimized is obtained by performing threshold segmentation processing on the gradient image, specifically:
calculating gradient information of the original image based on image space domain convolution by using a Sobel operator, and drawing a gradient image according to the gradient information;
and segmenting the sky area and the non-sky area of the gradient image to obtain an image to be optimized.
Further, the parameters of the image to be optimized include, but are not limited to, the width of the image and the height of the image; the method comprises the following steps of defining a sky boundary function according to parameters of the image to be optimized, and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated, wherein the method specifically comprises the following steps:
respectively calculating a covariance matrix of a sky region and a covariance matrix of a non-sky region in the image to be optimized according to the number of sky region pixels and the number of non-sky region pixels in the image to be optimized, and defining a gradient optimization energy function according to the covariance matrix of the sky region and the covariance matrix of the non-sky region;
defining a sky boundary function according to a width of the image and a height of the image;
and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated.
Further, performing SLAM initialization on the inertial navigation data and the segmentation image to obtain a SLAM framework, and performing SLAM composition positioning according to the SLAM framework, specifically:
initializing the segmentation image and the inertial navigation data by adopting visual inertial odometer, rear-end optimization and loop detection processing to obtain an SLAM frame, and performing SLAM composition positioning according to the SLAM frame.
A second embodiment of the present invention provides an outdoor environment visual inertia SLAM device, including:
the threshold segmentation module is used for obtaining a gradient image by extracting image gradient information of an original image and carrying out threshold segmentation processing on the gradient image to obtain an image to be optimized;
the calculation module is used for defining a sky boundary function according to the parameters of the image to be optimized and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated;
the integration module is used for integrating the boundary data to be integrated by adopting a polynomial fitting algorithm to obtain final boundary data;
and the composition positioning module is used for segmenting the original image according to the final boundary data to obtain a sky area image, carrying out SLAM initialization according to inertial navigation data and segmentation to obtain an SLAM frame, and carrying out SLAM composition positioning according to the SLAM frame.
Further, the threshold segmentation module is specifically configured to:
calculating gradient information of the original image based on image space domain convolution by using a Sobel operator, and drawing a gradient image according to the gradient information;
and segmenting the sky area and the non-sky area of the gradient image to obtain an image to be optimized.
Further, the parameters of the image to be optimized include, but are not limited to, the width of the image and the height of the image; the calculation module is specifically configured to:
respectively calculating a covariance matrix of a sky region and a covariance matrix of a non-sky region in the image to be optimized according to the number of sky region pixels and the number of non-sky region pixels in the image to be optimized, and defining a gradient optimization energy function according to the covariance matrix of the sky region and the covariance matrix of the non-sky region;
defining a sky boundary function according to a width of the image and a height of the image;
and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated.
Further, the composition positioning module is specifically configured to:
initializing the segmentation image and the inertial navigation data by adopting visual inertial odometer, rear-end optimization and loop detection processing to obtain an SLAM frame, and performing SLAM composition positioning according to the SLAM frame.
According to the embodiment of the invention, the boundary line is corrected by adopting a polynomial fitting algorithm, so that the consumption of a large amount of computing resources and computing time is avoided, and the effect of sky region segmentation can be quickly and accurately obtained; after the segmentation of the sky area is completed, the segmented image and inertial navigation data after the sky segmentation are initialized to obtain the SLAM frame, so that the SLAM frame is more reliable, the condition of inaccurate feature matching can be effectively avoided in the positioning and track mapping in the outdoor environment, and the accuracy of the positioning and track mapping can be effectively improved.
Drawings
Fig. 1 is a schematic flow chart of an outdoor environment visual inertia SLAM method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of an outdoor environment visual inertia SLAM method according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the description of the present application, it is to be understood that the terms "first", "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present application, "a plurality" means two or more unless otherwise specified.
In the description of the present application, it is to be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meaning of the above terms in the present application can be understood in a specific case by those of ordinary skill in the art.
Referring to fig. 1, a first embodiment of the present invention provides an outdoor environment visual inertia SLAM method, including:
s1, obtaining a gradient image by extracting image gradient information of the original image, and performing threshold segmentation processing on the gradient image to obtain an image to be optimized;
in the embodiment of the invention, the gradient image has gradient information, and the gradient information is one of the original information of the image and can be used for clearly expressing the trend of the gray scale change of the image, thereby providing important information for the next image processing. Exemplarily, in an original image shot by a SLAM monocular camera in an outdoor scene, a sky region and a non-sky region have obvious visual distinction, and based on the visual distinction, a gradient image with gradient information can accurately reflect the distinction between the sky region and the non-sky region in most cases.
S2, defining a sky boundary function according to the parameters of the image to be optimized, and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated;
s3, integrating the boundary data to be integrated by adopting a polynomial fitting algorithm to obtain final boundary data;
and S4, segmenting the original image according to the final boundary data to obtain a sky region image, performing SLAM initialization on the inertial navigation data and the non-sky region image to obtain an SLAM frame, and performing SLAM composition positioning according to the SLAM frame.
According to the embodiment of the invention, the boundary line is corrected by adopting a polynomial fitting algorithm, so that the consumption of a large amount of computing resources and computing time is avoided, and the effect of sky region segmentation can be quickly and accurately obtained; after the segmentation of the sky area is completed, the segmented image and inertial navigation data after the sky segmentation are initialized to obtain the SLAM frame, so that the SLAM frame is more reliable, the condition of inaccurate feature matching can be effectively avoided in the positioning and track mapping in the outdoor environment, and the accuracy of the positioning and track mapping can be effectively improved.
As a specific implementation manner of the embodiment of the present invention, a gradient image is obtained by extracting image gradient information of an original image, and a threshold segmentation process is performed on the gradient image to obtain an image to be optimized, which specifically includes:
calculating gradient information of the original image based on image space domain convolution by using a Sobel operator, and drawing a gradient image according to the gradient information;
and segmenting the sky area and the non-sky area of the gradient image to obtain an image to be optimized.
In a specific implementation mode, a threshold segmentation method is used for processing the gradient image, and according to the obvious difference between the sky area and the city building digital wood area in the gradient image and the difference between the sky area and other areas in gray level, the gradient image is classified in pixel level by setting a threshold, so that the elimination of fine crushing extraction of the sky area is realized.
As a specific implementation manner of the embodiment of the present invention, the parameters of the image to be optimized include, but are not limited to, the width of the image and the height of the image; defining a sky boundary function according to parameters of an image to be optimized, and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated, wherein the method specifically comprises the following steps:
respectively calculating a covariance matrix of a sky region and a covariance matrix of a non-sky region in the image to be optimized according to the number of pixels of the sky region and the number of pixels of the non-sky region in the image to be optimized, and defining a gradient optimization energy function according to the covariance matrix of the sky region and the covariance matrix of the non-sky region;
defining a sky boundary function according to the width of the image and the height of the image;
and calculating the sky boundary function according to the gradient optimization energy function to obtain boundary data to be integrated.
In an embodiment of the present invention, the gradient optimization energy function is expressed as follows:
Figure BDA0002840232840000061
therein, sigmaSSum ΣgRespectively representing covariance matrices of a sky region and a flying sky region expressed by RGB values, gamma is a parameter of uniformity of the sky region,
Figure BDA0002840232840000062
and
Figure BDA0002840232840000063
(i ═ {1,2,3}) corresponds to two matrices, | · | represents the corresponding determinant, ΣSSum ΣgThe definition is as follows:
Figure BDA0002840232840000064
Figure BDA0002840232840000065
Nsand NgRespectively representing the number of pixels of the sky area and the non-sky area.
In the embodiment of the invention, the gradient optimization energy function can effectively optimize the segmentation result between the sky region and the non-sky region.
In a specific embodiment, a sky boundary function, border (x):
1≤border(x)≤H(1≤x≤W)
where W and H represent the width and height of the gradient image, respectively. The sky area and the non-sky area may be calculated using the following formula:
sky＝{(x,y)|1≤x≤W,1≤y≤border(x)}
ground＝{(x,y)|1≤x≤W,border(x)≤y≤H}
and calculating to obtain an optimal value of the sky region and an optimal value of the non-sky region according to the gradient optimization energy function to obtain boundary data to be integrated.
After the boundary data to be integrated are obtained through calculation, a polynomial fitting method is introduced to further correct the boundary line of the sky area.
In particular, a data point p is giveni(xi,yi) Where i is 1,2, … m, it is required that the deviation of the approximation curve y is f (x) is minimal, and that the approximation curve at point piDeviation of (A) from
Figure BDA0002840232840000066
General form of the polynomial:
y＝p0xn+p1xn-1+p2xn-2+…+pn
the difference of the fit function from the true result is as follows:
Figure BDA0002840232840000071
it is understood that the process of polynomial fitting is the process of finding the minimum loss.
As a specific implementation manner of the embodiment of the present invention, performing SLAM initialization on inertial navigation data and a segmentation image to obtain a SLAM framework, and performing SLAM composition positioning according to the SLAM framework, specifically:
and initializing the segmentation image and inertial navigation data by adopting visual inertial odometer, rear-end optimization and loop detection processing to obtain an SLAM frame, and performing SLAM composition positioning according to the SLAM frame.
In an embodiment of the present invention, a visual inertial odometer is employed, with the visual portion and inertial navigation portion beginning with an initialization portion. Wherein, the visual part uses a characteristic point method to extract characteristic corner points in the image; the inertia part adopts pre-integration to realize optimization of calculated amount; a back end optimization part, which uses a BA (bundle adjustment) method to optimally adjust the camera attitude and the characteristic point null concept position; the loop detection part uses a scheme based on a bag-of-words model, and the words regard each element in the dictionary as a set of adjacent feature points, so that the success rate and the speed of image comparison are optimized. According to the embodiment of the invention, the SLAM composition positioning is carried out through the constructed SLAM framework, so that the waste of computing resources can be effectively reduced, and the accuracy of positioning and track composition can be effectively improved.
The embodiment of the invention has the following beneficial effects:
according to the embodiment of the invention, the boundary line is corrected by adopting a polynomial fitting algorithm, so that the consumption of a large amount of computing resources and computing time is avoided, and the effect of sky region segmentation can be quickly and accurately obtained; after the segmentation of the sky area is completed, the segmented image and inertial navigation data after the sky segmentation are initialized to obtain the SLAM frame, so that the SLAM frame is more reliable, the condition of inaccurate feature matching can be effectively avoided in the positioning and track mapping in the outdoor environment, and the accuracy of the positioning and track mapping can be effectively improved.
Referring to fig. 2, a second embodiment of the present invention provides an outdoor environment visual inertia SLAM device, including:
the threshold segmentation module 10 is configured to obtain a gradient image by extracting image gradient information of an original image, and perform threshold segmentation processing on the gradient image to obtain an image to be optimized;
in the embodiment of the invention, the gradient image has gradient information, and the gradient information is one of the original information of the image and can be used for clearly expressing the trend of the gray scale change of the image, thereby providing important information for the next image processing. Exemplarily, in an original image shot by a SLAM monocular camera in an outdoor scene, a sky region and a non-sky region have obvious visual distinction, and based on the visual distinction, a gradient image with gradient information can accurately reflect the distinction between the sky region and the non-sky region in most cases.
The calculation module 20 is configured to define a sky boundary function according to the parameters of the image to be optimized, and calculate the sky boundary function according to the gradient optimization energy function to obtain boundary data to be integrated;
the integration module 30 is configured to integrate the boundary data to be integrated by using a polynomial fitting algorithm to obtain final boundary data;
and the composition positioning module 40 is used for segmenting the original image according to the final boundary data to obtain a sky area image, performing SLAM initialization according to the inertial navigation data and segmentation to obtain an SLAM frame, and performing SLAM composition positioning according to the SLAM frame.
According to the embodiment of the invention, the boundary line is corrected by adopting a polynomial fitting algorithm, so that the consumption of a large amount of computing resources and computing time is avoided, and the effect of sky region segmentation can be quickly and accurately obtained; after the segmentation of the sky area is completed, the segmented image and inertial navigation data after the sky segmentation are initialized to obtain the SLAM frame, so that the SLAM frame is more reliable, the condition of inaccurate feature matching can be effectively avoided in the positioning and track mapping in the outdoor environment, and the accuracy of the positioning and track mapping can be effectively improved.
As a specific implementation manner of the embodiment of the present invention, the threshold segmentation module 10 is specifically configured to:
calculating gradient information of the original image based on image space domain convolution by using a Sobel operator, and drawing a gradient image according to the gradient information;
and segmenting the sky area and the non-sky area of the gradient image to obtain an image to be optimized.
In a specific implementation mode, a threshold segmentation method is used for processing the gradient image, and according to the obvious difference between the sky area and the city building digital wood area in the gradient image and the difference between the sky area and other areas in gray level, the gradient image is classified in pixel level by setting a threshold, so that the elimination of fine crushing extraction of the sky area is realized.
As a specific implementation manner of the embodiment of the present invention, the parameters of the image to be optimized include, but are not limited to, the width of the image and the height of the image; the calculation module 20 is specifically configured to:
respectively calculating a covariance matrix of a sky region and a covariance matrix of a non-sky region in the image to be optimized according to the number of pixels of the sky region and the number of pixels of the non-sky region in the image to be optimized, and defining a gradient optimization energy function according to the covariance matrix of the sky region and the covariance matrix of the non-sky region;
defining a sky boundary function according to the width of the image and the height of the image;
and calculating the sky boundary function according to the gradient optimization energy function to obtain boundary data to be integrated.
In an embodiment of the present invention, the gradient optimization energy function is expressed as follows:
Figure BDA0002840232840000091
therein, sigmaSSum ΣgRespectively representing covariance matrices of a sky region and a flying sky region expressed by RGB values, gamma is a parameter of uniformity of the sky region,
Figure BDA0002840232840000092
and
Figure BDA0002840232840000093
(i ═ {1,2,3}) corresponds to two matrices, | · | represents the corresponding determinant, ΣSSum ΣgThe definition is as follows:
Figure BDA0002840232840000094
Figure BDA0002840232840000095
Nsand NgRespectively representing the number of pixels of the sky area and the non-sky area.
In the embodiment of the invention, the gradient optimization energy function can effectively optimize the segmentation result between the sky region and the non-sky region.
In a specific embodiment, a sky boundary function, border (x):
1≤border(x)≤H(1≤x≤W)
where W and H represent the width and height of the gradient image, respectively. The sky area and the non-sky area may be calculated using the following formula:
sky＝{(x,y)|1≤x≤W,1≤y≤border(x)}
ground＝{(x,y)|1≤x≤W,border(x)≤y≤H}
and calculating to obtain an optimal value of the sky region and an optimal value of the non-sky region according to the gradient optimization energy function to obtain boundary data to be integrated.
After the boundary data to be integrated are obtained through calculation, a polynomial fitting method is introduced to further correct the boundary line of the sky area.
In particular, a data point p is giveni(xi,yi) Where i is 1,2, … m, it is required that the deviation of the approximation curve y is f (x) is minimal, and that the approximation curve at point piDeviation of (A) from
Figure BDA0002840232840000102
General form of the polynomial:
y＝p0xn+p1xn-1+p2xn-2+…+pn
the difference of the fit function from the true result is as follows:
Figure BDA0002840232840000101
it is understood that the process of polynomial fitting is the process of finding the minimum loss.
As a specific implementation manner of the embodiment of the present invention, the composition positioning module 40 is specifically configured to:
and initializing the segmentation image and inertial navigation data by adopting visual inertial odometer, rear-end optimization and loop detection processing to obtain an SLAM frame, and performing SLAM composition positioning according to the SLAM frame.
In an embodiment of the present invention, a visual inertial odometer is employed, with the visual portion and inertial navigation portion beginning with an initialization portion. Wherein, the visual part uses a characteristic point method to extract characteristic corner points in the image; the inertia part adopts pre-integration to realize optimization of calculated amount; a back end optimization part, which uses a BA (bundle adjustment) method to optimally adjust the camera attitude and the characteristic point null concept position; the loop detection part uses a scheme based on a bag-of-words model, and the words regard each element in the dictionary as a set of adjacent feature points, so that the success rate and the speed of image comparison are optimized. According to the embodiment of the invention, the SLAM composition positioning is carried out through the constructed SLAM framework, so that the waste of computing resources can be effectively reduced, and the accuracy of positioning and track composition can be effectively improved.
The embodiment of the invention has the following beneficial effects:
according to the embodiment of the invention, the boundary line is corrected by adopting a polynomial fitting algorithm, so that the consumption of a large amount of computing resources and computing time is avoided, and the effect of sky region segmentation can be quickly and accurately obtained; after the segmentation of the sky area is completed, the segmented image and inertial navigation data after the sky segmentation are initialized to obtain the SLAM frame, so that the SLAM frame is more reliable, the condition of inaccurate feature matching can be effectively avoided in the positioning and track mapping in the outdoor environment, and the accuracy of the positioning and track mapping can be effectively improved.
The foregoing is a preferred embodiment of the present invention, and it should be noted that it would be apparent to those skilled in the art that various modifications and enhancements can be made without departing from the principles of the invention, and such modifications and enhancements are also considered to be within the scope of the invention.

Claims (8)

1. An outdoor environment visual inertia SLAM method, comprising:
obtaining a gradient image by extracting image gradient information of an original image, and performing threshold segmentation processing on the gradient image to obtain an image to be optimized;
defining a sky boundary function according to the parameters of the image to be optimized, and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated;
integrating the boundary data to be integrated by adopting a polynomial fitting algorithm to obtain final boundary data;
and segmenting the original image according to the final boundary data to obtain a sky region image, performing SLAM initialization on inertial navigation data and the non-sky region image to obtain an SLAM frame, and performing SLAM composition positioning according to the SLAM frame.
2. The outdoor environment visual inertia SLAM method according to claim 1, wherein the gradient image is obtained by extracting image gradient information of an original image, and the image to be optimized is obtained by performing threshold segmentation processing on the gradient image, specifically:
calculating gradient information of the original image based on image space domain convolution by using a Sobel operator, and drawing a gradient image according to the gradient information;
and segmenting the sky area and the non-sky area of the gradient image to obtain an image to be optimized.
3. The outdoor environment visual inertial SLAM method of claim 1, in which the parameters of the image to be optimized include, but are not limited to, the width of the image and the height of the image; the method comprises the following steps of defining a sky boundary function according to parameters of the image to be optimized, and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated, wherein the method specifically comprises the following steps:
respectively calculating a covariance matrix of a sky region and a covariance matrix of a non-sky region in the image to be optimized according to the number of sky region pixels and the number of non-sky region pixels in the image to be optimized, and defining a gradient optimization energy function according to the covariance matrix of the sky region and the covariance matrix of the non-sky region;
defining a sky boundary function according to a width of the image and a height of the image;
and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated.
4. The outdoor environment visual inertial SLAM method according to claim 1, wherein the SLAM initialization of the inertial navigation data and the segmented image is performed to obtain a SLAM frame, and SLAM composition positioning is performed according to the SLAM frame, specifically:
initializing the segmentation image and the inertial navigation data by adopting visual inertial odometer, rear-end optimization and loop detection processing to obtain an SLAM frame, and performing SLAM composition positioning according to the SLAM frame.
5. An outdoor environment visual inertial SLAM device, comprising:
the threshold segmentation module is used for obtaining a gradient image by extracting image gradient information of an original image and carrying out threshold segmentation processing on the gradient image to obtain an image to be optimized;
the calculation module is used for defining a sky boundary function according to the parameters of the image to be optimized and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated;
the integration module is used for integrating the boundary data to be integrated by adopting a polynomial fitting algorithm to obtain final boundary data;
and the composition positioning module is used for segmenting the original image according to the final boundary data to obtain a sky area image, carrying out SLAM initialization according to inertial navigation data and segmentation to obtain an SLAM frame, and carrying out SLAM composition positioning according to the SLAM frame.
6. The outdoor-environment visual-inertial SLAM device of claim 5, wherein the threshold segmentation module is specifically configured to:
calculating gradient information of the original image based on image space domain convolution by using a Sobel operator, and drawing a gradient image according to the gradient information;
and segmenting the sky area and the non-sky area of the gradient image to obtain an image to be optimized.
7. The outdoor environment visual inertial SLAM device of claim 5, in which the parameters of the image to be optimized include, but are not limited to, the width of the image and the height of the image; the calculation module is specifically configured to:
respectively calculating a covariance matrix of a sky region and a covariance matrix of a non-sky region in the image to be optimized according to the number of sky region pixels and the number of non-sky region pixels in the image to be optimized, and defining a gradient optimization energy function according to the covariance matrix of the sky region and the covariance matrix of the non-sky region;
defining a sky boundary function according to a width of the image and a height of the image;
and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated.
8. The outdoor environment visual inertial SLAM device of claim 5, wherein the composition positioning module is specifically configured to:
initializing the segmentation image and the inertial navigation data by adopting visual inertial odometer, rear-end optimization and loop detection processing to obtain an SLAM frame, and performing SLAM composition positioning according to the SLAM frame.
CN202011489168.5A 2020-12-16 2020-12-16 Outdoor environment visual inertia SLAM method and device Active CN112613372B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011489168.5A CN112613372B (en) 2020-12-16 2020-12-16 Outdoor environment visual inertia SLAM method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011489168.5A CN112613372B (en) 2020-12-16 2020-12-16 Outdoor environment visual inertia SLAM method and device

Publications (2)

Publication Number Publication Date
CN112613372A true CN112613372A (en) 2021-04-06
CN112613372B CN112613372B (en) 2024-04-02

Family

ID=75239960

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011489168.5A Active CN112613372B (en) 2020-12-16 2020-12-16 Outdoor environment visual inertia SLAM method and device

Country Status (1)

Country Link
CN (1) CN112613372B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113177971A (en) * 2021-05-07 2021-07-27 中德（珠海）人工智能研究院有限公司 Visual tracking method and device, computer equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090129675A1 (en) * 2006-11-16 2009-05-21 Honda Research Institute Europe Gmbh Method And Device For Continuous Figure-Ground Segregation In Images From Dynamic Visual Scenes
US20110150320A1 (en) * 2009-06-30 2011-06-23 Srikumar Ramalingam Method and System for Localizing in Urban Environments From Omni-Direction Skyline Images
CN105869174A (en) * 2016-04-20 2016-08-17 中国航空工业集团公司洛阳电光设备研究所 Sky scene image segmentation method
CN110097584A (en) * 2019-03-18 2019-08-06 国网浙江省电力有限公司信息通信分公司 The method for registering images of combining target detection and semantic segmentation
CN111210434A (en) * 2019-12-19 2020-05-29 上海艾麒信息科技有限公司 Image replacement method and system based on sky identification

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090129675A1 (en) * 2006-11-16 2009-05-21 Honda Research Institute Europe Gmbh Method And Device For Continuous Figure-Ground Segregation In Images From Dynamic Visual Scenes
US20110150320A1 (en) * 2009-06-30 2011-06-23 Srikumar Ramalingam Method and System for Localizing in Urban Environments From Omni-Direction Skyline Images
CN105869174A (en) * 2016-04-20 2016-08-17 中国航空工业集团公司洛阳电光设备研究所 Sky scene image segmentation method
CN110097584A (en) * 2019-03-18 2019-08-06 国网浙江省电力有限公司信息通信分公司 The method for registering images of combining target detection and semantic segmentation
CN111210434A (en) * 2019-12-19 2020-05-29 上海艾麒信息科技有限公司 Image replacement method and system based on sky identification

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
王联君;王静秋;: "基于GrabCut的磨粒图像分割方法研究", 机械制造与自动化, no. 02, pages 133 - 136 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113177971A (en) * 2021-05-07 2021-07-27 中德（珠海）人工智能研究院有限公司 Visual tracking method and device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN112613372B (en) 2024-04-02

Similar Documents

Publication Publication Date Title
CN107341814B (en) Four-rotor unmanned aerial vehicle monocular vision range measurement method based on sparse direct method
CN101729911B (en) Multi-view image color correction method based on visual perception
CN111402170B (en) Image enhancement method, device, terminal and computer readable storage medium
CN108537782B (en) Building image matching and fusing method based on contour extraction
CN109685045B (en) Moving target video tracking method and system
CN108596961A (en) Point cloud registration method based on Three dimensional convolution neural network
CN109376641B (en) Moving vehicle detection method based on unmanned aerial vehicle aerial video
US11367195B2 (en) Image segmentation method, image segmentation apparatus, image segmentation device
CN110110608B (en) Forklift speed monitoring method and system based on vision under panoramic monitoring
CN102982524B (en) Splicing method for corn ear order images
CN110853064A (en) Image collaborative segmentation method based on minimum fuzzy divergence
CN111415374A (en) KVM system and method for monitoring and managing scenic spot pedestrian flow
CN105606123A (en) Method for automatic correction of digital ground elevation model for low-altitude aerial photogrammetry
CN111414938B (en) Target detection method for bubbles in plate heat exchanger
CN116434088A (en) Lane line detection and lane auxiliary keeping method based on unmanned aerial vehicle aerial image
CN111160107B (en) Dynamic region detection method based on feature matching
CN116309686A (en) Video positioning and speed measuring method, device and equipment for swimmers and storage medium
CN105447869A (en) Particle swarm optimization algorithm based camera self-calibration method and apparatus
CN112613372A (en) Outdoor environment visual inertia SLAM method and device
CN109635649B (en) High-speed detection method and system for unmanned aerial vehicle reconnaissance target
CN109691185B (en) Positioning method, positioning device, terminal and readable storage medium
CN110659593A (en) Urban haze visibility detection method based on improved DiracNet
CN113096016A (en) Low-altitude aerial image splicing method and system
CN108765384B (en) Significance detection method for joint manifold sequencing and improved convex hull
CN115984592A (en) Point-line fusion feature matching method based on SuperPoint + SuperGlue

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant