CN112613372B - Outdoor environment visual inertia SLAM method and device - Google Patents
Outdoor environment visual inertia SLAM method and device Download PDFInfo
- Publication number
- CN112613372B CN112613372B CN202011489168.5A CN202011489168A CN112613372B CN 112613372 B CN112613372 B CN 112613372B CN 202011489168 A CN202011489168 A CN 202011489168A CN 112613372 B CN112613372 B CN 112613372B
- Authority
- CN
- China
- Prior art keywords
- image
- sky
- slam
- gradient
- optimized
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 34
- 238000000034 method Methods 0.000 title claims abstract description 30
- 230000011218 segmentation Effects 0.000 claims abstract description 31
- 239000000203 mixture Substances 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 21
- 238000005457 optimization Methods 0.000 claims description 37
- 239000011159 matrix material Substances 0.000 claims description 24
- 238000001514 detection method Methods 0.000 claims description 8
- 230000010354 integration Effects 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 6
- 238000010276 construction Methods 0.000 abstract 1
- 238000013507 mapping Methods 0.000 description 16
- 238000004364 calculation method Methods 0.000 description 13
- 239000002699 waste material Substances 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008030 elimination Effects 0.000 description 2
- 238000003379 elimination reaction Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 239000002023 wood Substances 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
- G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Abstract
The invention discloses an outdoor environment visual inertia SLAM method and a device, wherein the method comprises the following steps: obtaining a gradient image by extracting image gradient information of an original image, and performing threshold segmentation processing on the gradient image to obtain an image to be optimized; calculating to-be-integrated boundary data according to the to-be-optimized image; integrating the boundary data to be integrated by adopting a polynomial fitting algorithm to obtain final boundary data; and dividing the original image according to the final boundary data to obtain a sky area image, carrying out SLAM initialization on the inertial navigation data and the non-sky area image to obtain an SLAM frame, and carrying out SLAM composition positioning according to the SLAM frame. According to the embodiment of the invention, the boundary line is corrected by adopting a polynomial fitting algorithm, so that the operation amount and operation time can be effectively reduced, the sky region segmentation effect can be rapidly and accurately obtained, and the accuracy of positioning and track map construction can be effectively improved.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an outdoor environment visual inertia SLAM method and device.
Background
SLAM (Simultaneous Localization and Mapping) Chinese translation is a technique for simultaneously locating and mapping, and establishing an environment model and estimating own motion by carrying one or more sensors under the condition of no environment priori information. As an important method for locating and establishing an environmental model, it has a wide range of applications in fields such as automatic driving. The basic visual SLAM structure is relatively simple, but the problems of inaccurate positioning and track deviation can occur in various scenes. The combined inertial measurement unit and camera vision-inertial unit is an increasingly popular application framework. The existing outdoor environment visual inertia SLAM method adopts a mature SLAM framework such as VINS-Mono, ORB-SLAM3 and the like, and can obtain ideal experimental results in the outdoor environment. However, the existing outdoor environment visual inertial SLAM method needs to waste a large amount of computing resources in an ineffective sky area, resulting in poor efficiency of positioning and track mapping.
Disclosure of Invention
The invention provides an outdoor environment visual inertia SLAM method and device, which are used for solving the technical problem that the existing outdoor environment visual inertia SLAM method needs to waste a large amount of computing resources in an invalid sky area, so that the efficiency of positioning and track mapping is poor.
A first embodiment of the present invention provides an outdoor environment visual inertial SLAM method, including:
obtaining a gradient image by extracting image gradient information of an original image, and performing threshold segmentation processing on the gradient image to obtain an image to be optimized;
defining a sky boundary function according to the parameters of the image to be optimized, and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated;
adopting a polynomial fitting algorithm to integrate the boundary data to be integrated to obtain final boundary data;
and dividing the original image according to the final boundary data to obtain a sky area image, carrying out SLAM initialization on the inertial navigation data and the non-sky area image to obtain an SLAM framework, and carrying out SLAM composition positioning according to the SLAM framework.
Further, the step of obtaining a gradient image by extracting image gradient information of an original image, and the step of performing threshold segmentation processing on the gradient image to obtain an image to be optimized specifically includes:
calculating gradient information of the original image based on image space domain convolution by adopting a Sobel operator, and drawing a gradient image according to the gradient information;
and dividing the sky area and the non-sky area of the gradient image to obtain an image to be optimized.
Further, the parameters of the image to be optimized include, but are not limited to, the width of the image and the height of the image; defining a sky boundary function according to the parameters of the image to be optimized, and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated, wherein the boundary data to be integrated is specifically as follows:
respectively calculating a covariance matrix of a sky area and a covariance matrix of a non-sky area in the image to be optimized according to the number of pixels of the sky area and the number of pixels of the non-sky area in the image to be optimized, and defining a gradient optimization energy function according to the covariance matrix of the sky area and the covariance matrix of the non-sky area;
defining a sky boundary function according to the width of the image and the height of the image;
and calculating the sky boundary function according to the gradient optimization energy function to obtain boundary data to be integrated.
Further, the step of performing SLAM initialization on the inertial navigation data and the segmented image to obtain a SLAM frame, and performing SLAM composition positioning according to the SLAM frame specifically includes:
and initializing the segmented image and the inertial navigation data by adopting a visual inertial odometer, rear-end optimization and loop detection processing to obtain an SLAM framework, and carrying out SLAM composition positioning according to the SLAM framework.
A second embodiment of the present invention provides an outdoor environment visual inertial SLAM apparatus, comprising:
the threshold segmentation module is used for obtaining a gradient image by extracting image gradient information of an original image, and performing threshold segmentation processing on the gradient image to obtain an image to be optimized;
the computing module is used for defining a sky boundary function according to the parameters of the image to be optimized, and computing the sky boundary function according to the gradient optimization energy function to obtain boundary data to be integrated;
the integration module is used for integrating the boundary data to be integrated by adopting a polynomial fitting algorithm to obtain final boundary data;
and the composition positioning module is used for dividing the original image according to the final boundary data to obtain a sky area image, carrying out SLAM initialization according to the inertial navigation data and the division to obtain an SLAM framework, and carrying out SLAM composition positioning according to the SLAM framework.
Further, the threshold segmentation module is specifically configured to:
calculating gradient information of the original image based on image space domain convolution by adopting a Sobel operator, and drawing a gradient image according to the gradient information;
and dividing the sky area and the non-sky area of the gradient image to obtain an image to be optimized.
Further, the parameters of the image to be optimized include, but are not limited to, the width of the image and the height of the image; the computing module is specifically configured to:
respectively calculating a covariance matrix of a sky area and a covariance matrix of a non-sky area in the image to be optimized according to the number of pixels of the sky area and the number of pixels of the non-sky area in the image to be optimized, and defining a gradient optimization energy function according to the covariance matrix of the sky area and the covariance matrix of the non-sky area;
defining a sky boundary function according to the width of the image and the height of the image;
and calculating the sky boundary function according to the gradient optimization energy function to obtain boundary data to be integrated.
Further, the composition positioning module is specifically configured to:
and initializing the segmented image and the inertial navigation data by adopting a visual inertial odometer, rear-end optimization and loop detection processing to obtain an SLAM framework, and carrying out SLAM composition positioning according to the SLAM framework.
According to the embodiment of the invention, the boundary line is corrected by adopting a polynomial fitting algorithm, so that a great amount of calculation resources and calculation time are avoided, and the sky area segmentation effect can be rapidly and accurately obtained; after the sky area is segmented, initializing segmented images after sky segmentation and inertial navigation data to obtain an SLAM frame, so that the SLAM frame is more reliable, and in positioning and track mapping in an outdoor environment, the situation of inaccurate feature matching can be effectively avoided, and the accuracy of positioning and track mapping can be effectively improved.
Drawings
FIG. 1 is a schematic flow chart of an outdoor environment visual inertial SLAM method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of an outdoor environment visual inertial SLAM method according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
In the description of the present application, it should be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or an implicit indication of the number of technical features being indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present application, unless otherwise indicated, the meaning of "a plurality" is two or more.
In the description of the present application, it should be noted that, unless explicitly specified and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be either fixedly connected, detachably connected, or integrally connected, for example; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the terms in this application will be understood by those of ordinary skill in the art in a specific context.
Referring to fig. 1, a first embodiment of the present invention provides an outdoor environment visual inertial SLAM method, including:
s1, obtaining a gradient image by extracting image gradient information of an original image, and performing threshold segmentation processing on the gradient image to obtain an image to be optimized;
in the embodiment of the invention, the gradient image has gradient information, and the gradient information is used as one of the original information of the image and can be used for clearly representing the trend of the gray level change of the image, thereby providing important information for the next image processing. For example, in an original image captured by a SLAM monocular camera in an outdoor scene, there is a clear visual distinction between a sky area and a non-sky area, and based on this, a gradient image with gradient information can accurately reflect the distinction between the sky area and the non-sky area in most cases.
S2, defining a sky boundary function according to parameters of the image to be optimized, and calculating the sky boundary function according to the gradient optimization energy function to obtain boundary data to be integrated;
s3, integrating the boundary data to be integrated by adopting a polynomial fitting algorithm to obtain final boundary data;
s4, dividing the original image according to the final boundary data to obtain a sky area image, carrying out SLAM initialization on the inertial navigation data and the non-sky area image to obtain an SLAM frame, and carrying out SLAM composition positioning according to the SLAM frame.
According to the embodiment of the invention, the boundary line is corrected by adopting a polynomial fitting algorithm, so that a great amount of calculation resources and calculation time are avoided, and the sky area segmentation effect can be rapidly and accurately obtained; after the sky area is segmented, initializing segmented images after sky segmentation and inertial navigation data to obtain an SLAM frame, so that the SLAM frame is more reliable, and in positioning and track mapping in an outdoor environment, the situation of inaccurate feature matching can be effectively avoided, and the accuracy of positioning and track mapping can be effectively improved.
As a specific implementation manner of the embodiment of the invention, the gradient image is obtained by extracting the image gradient information of the original image, and the image to be optimized is obtained by performing threshold segmentation processing on the gradient image, which is specifically as follows:
calculating gradient information of an original image based on image space domain convolution by adopting a Sobel operator, and drawing a gradient image according to the gradient information;
and dividing the sky area and the non-sky area of the gradient image to obtain an image to be optimized.
In a specific embodiment, a threshold segmentation method is used for processing the gradient image, and pixel-level classification is carried out on the gradient image by setting a threshold according to the obvious difference between a sky area and a city building wood counting area in the gradient image and the difference of the sky area and other areas in gray scale, so that the elimination of fine crushing extraction of the sky area is realized.
As a specific implementation of the embodiment of the present invention, parameters of the image to be optimized include, but are not limited to, the width of the image and the height of the image; defining a sky boundary function according to parameters of an image to be optimized, and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated, wherein the boundary data to be integrated is specifically as follows:
respectively calculating a covariance matrix of a sky area and a covariance matrix of a non-sky area in the image to be optimized according to the number of pixels of the sky area and the number of pixels of the non-sky area in the image to be optimized, and defining a gradient optimization energy function according to the covariance matrix of the sky area and the covariance matrix of the non-sky area;
defining a sky boundary function according to the width of the image and the height of the image;
and calculating the sky boundary function according to the gradient optimization energy function to obtain boundary data to be integrated.
In an embodiment of the present invention, the expression of the gradient optimized energy function is as follows:
wherein, sigma S Sum sigma g Covariance matrices representing sky regions and non-sky regions, respectively, represented by RGB values, γ being a parameter of sky region uniformity,and->Corresponding to the two matrices, |·| represents the corresponding determinant, Σ S Sum sigma g The definition is as follows:
N s and N g The number of pixels representing the sky region and the non-sky region, respectively.
In the embodiment of the invention, the gradient optimization energy function can effectively optimize the segmentation result between the sky area and the non-sky area.
In a specific embodiment, a sky boundary function boundary (x) is defined:
1≤border(x)≤H(1≤x≤W)
where W and H represent the width and height of the gradient image, respectively. The sky and non-sky regions can be calculated using the following formula:
sky＝{(x,y)|1≤x≤W,1≤y≤border(x)}
ground＝{(x,y)|1≤x≤W,border(x)≤y≤H}
and calculating an optimal value of the sky area and an optimal value of the non-sky area according to the gradient optimization energy function to obtain boundary data to be integrated.
After the boundary data to be integrated is obtained through calculation, a polynomial fitting method is introduced to further correct the boundary line of the sky area.
Specifically, given data point p i (x i ,y i ) Where i=1, 2, … m, the deviation of the approximation curve y=f (x) is required to be minimal, the approximation curve being at point p i Deviation at
General form of polynomial:
y＝p 0 x n +p 1 x n-1 +p 2 x n-2 +...+p n
the difference between the fitting function and the true result is as follows:
it will be appreciated that the process of polynomial fitting is the process of finding the minimum loss.
As a specific implementation manner of the embodiment of the present invention, performing SLAM initialization on inertial navigation data and a split image to obtain a SLAM frame, and performing SLAM composition positioning according to the SLAM frame, specifically:
and initializing the split image and the inertial navigation data by adopting a visual inertial odometer, rear-end optimization and loop detection processing to obtain an SLAM framework, and carrying out SLAM composition positioning according to the SLAM framework.
In an embodiment of the invention, a visual odometer is employed, with the visual portion and the inertial navigation portion starting from the initialization portion. The visual part extracts characteristic angular points in the image by using a characteristic point method; the inertia part adopts pre-integration to realize the optimization of the calculated amount; the rear-end optimization part uses BA (Bundle Adjustment) method to make optimal adjustment for the pose of the camera and the conceptual position of the feature point space; the loop detection part uses a scheme based on a bag-of-words model, and each element in the dictionary is regarded as a set of adjacent characteristic points, so that the success rate and the speed of image comparison are optimized. According to the embodiment of the invention, SLAM composition positioning is performed through the constructed SLAM frame, so that the waste of computing resources can be effectively reduced, and the accuracy of positioning and track mapping can be effectively improved.
The embodiment of the invention has the following beneficial effects:
according to the embodiment of the invention, the boundary line is corrected by adopting a polynomial fitting algorithm, so that a great amount of calculation resources and calculation time are avoided, and the sky area segmentation effect can be rapidly and accurately obtained; after the sky area is segmented, initializing segmented images after sky segmentation and inertial navigation data to obtain an SLAM frame, so that the SLAM frame is more reliable, and in positioning and track mapping in an outdoor environment, the situation of inaccurate feature matching can be effectively avoided, and the accuracy of positioning and track mapping can be effectively improved.
Referring to fig. 2, a second embodiment of the present invention provides an outdoor environment visual inertial SLAM device, comprising:
the threshold segmentation module 10 is used for obtaining a gradient image by extracting image gradient information of an original image, and performing threshold segmentation processing on the gradient image to obtain an image to be optimized;
in the embodiment of the invention, the gradient image has gradient information, and the gradient information is used as one of the original information of the image and can be used for clearly representing the trend of the gray level change of the image, thereby providing important information for the next image processing. For example, in an original image captured by a SLAM monocular camera in an outdoor scene, there is a clear visual distinction between a sky area and a non-sky area, and based on this, a gradient image with gradient information can accurately reflect the distinction between the sky area and the non-sky area in most cases.
The computing module 20 is configured to define a sky boundary function according to parameters of the image to be optimized, and calculate the sky boundary function according to the gradient optimization energy function to obtain boundary data to be integrated;
the integration module 30 is configured to integrate the boundary data to be integrated by using a polynomial fitting algorithm to obtain final boundary data;
the composition positioning module 40 is configured to divide the original image according to the final boundary data to obtain a sky area image, initialize the inertial navigation data and the division to obtain a SLAM frame, and perform SLAM composition positioning according to the SLAM frame.
According to the embodiment of the invention, the boundary line is corrected by adopting a polynomial fitting algorithm, so that a great amount of calculation resources and calculation time are avoided, and the sky area segmentation effect can be rapidly and accurately obtained; after the sky area is segmented, initializing segmented images after sky segmentation and inertial navigation data to obtain an SLAM frame, so that the SLAM frame is more reliable, and in positioning and track mapping in an outdoor environment, the situation of inaccurate feature matching can be effectively avoided, and the accuracy of positioning and track mapping can be effectively improved.
As a specific implementation of the embodiment of the present invention, the threshold segmentation module 10 is specifically configured to:
calculating gradient information of an original image based on image space domain convolution by adopting a Sobel operator, and drawing a gradient image according to the gradient information;
and dividing the sky area and the non-sky area of the gradient image to obtain an image to be optimized.
In a specific embodiment, a threshold segmentation method is used for processing the gradient image, and pixel-level classification is carried out on the gradient image by setting a threshold according to the obvious difference between a sky area and a city building wood counting area in the gradient image and the difference of the sky area and other areas in gray scale, so that the elimination of fine crushing extraction of the sky area is realized.
As a specific implementation of the embodiment of the present invention, parameters of the image to be optimized include, but are not limited to, the width of the image and the height of the image; the calculation module 20 is specifically configured to:
respectively calculating a covariance matrix of a sky area and a covariance matrix of a non-sky area in the image to be optimized according to the number of pixels of the sky area and the number of pixels of the non-sky area in the image to be optimized, and defining a gradient optimization energy function according to the covariance matrix of the sky area and the covariance matrix of the non-sky area;
defining a sky boundary function according to the width of the image and the height of the image;
and calculating the sky boundary function according to the gradient optimization energy function to obtain boundary data to be integrated.
In an embodiment of the present invention, the expression of the gradient optimized energy function is as follows:
wherein, sigma S Sum sigma g Covariance matrices representing sky regions and non-sky regions, respectively, represented by RGB values, γ being a parameter of sky region uniformity,and->Corresponding to the two matrices, |·| represents the corresponding determinant, Σ S Sum sigma g The definition is as follows:
N s and N g The number of pixels representing the sky region and the non-sky region, respectively.
In the embodiment of the invention, the gradient optimization energy function can effectively optimize the segmentation result between the sky area and the non-sky area.
In a specific embodiment, a sky boundary function boundary (x) is defined:
1≤border(x)≤H(1≤x≤W)
where W and H represent the width and height of the gradient image, respectively. The sky and non-sky regions can be calculated using the following formula:
sky＝{(x,y)|1≤x≤W,1≤y≤border(x)}
ground＝{(x,y)|1≤x≤W,border(x)≤y≤H}
and calculating an optimal value of the sky area and an optimal value of the non-sky area according to the gradient optimization energy function to obtain boundary data to be integrated.
After the boundary data to be integrated is obtained through calculation, a polynomial fitting method is introduced to further correct the boundary line of the sky area.
Specifically, given data point p i (x i ,y i ) Where i=1, 2..m, the deviation of the approximation curve y=f (x) is required to be minimal, the approximation curve being at point p i Deviation at
General form of polynomial:
y＝p 0 x n +p 1 x n-1 +p 2 x n-2 +...+p n
the difference between the fitting function and the true result is as follows:
it will be appreciated that the process of polynomial fitting is the process of finding the minimum loss.
As a specific implementation of the embodiment of the present invention, the composition positioning module 40 is specifically configured to:
and initializing the split image and the inertial navigation data by adopting a visual inertial odometer, rear-end optimization and loop detection processing to obtain an SLAM framework, and carrying out SLAM composition positioning according to the SLAM framework.
In an embodiment of the invention, a visual odometer is employed, with the visual portion and the inertial navigation portion starting from the initialization portion. The visual part extracts characteristic angular points in the image by using a characteristic point method; the inertia part adopts pre-integration to realize the optimization of the calculated amount; the rear-end optimization part uses BA (Bundle Adjustment) method to make optimal adjustment for the pose of the camera and the conceptual position of the feature point space; the loop detection part uses a scheme based on a bag-of-words model, and each element in the dictionary is regarded as a set of adjacent characteristic points, so that the success rate and the speed of image comparison are optimized. According to the embodiment of the invention, SLAM composition positioning is performed through the constructed SLAM frame, so that the waste of computing resources can be effectively reduced, and the accuracy of positioning and track mapping can be effectively improved.
The embodiment of the invention has the following beneficial effects:
according to the embodiment of the invention, the boundary line is corrected by adopting a polynomial fitting algorithm, so that a great amount of calculation resources and calculation time are avoided, and the sky area segmentation effect can be rapidly and accurately obtained; after the sky area is segmented, initializing segmented images after sky segmentation and inertial navigation data to obtain an SLAM frame, so that the SLAM frame is more reliable, and in positioning and track mapping in an outdoor environment, the situation of inaccurate feature matching can be effectively avoided, and the accuracy of positioning and track mapping can be effectively improved.
The foregoing is a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention and are intended to be comprehended within the scope of the present invention.
Claims (6)
1. An outdoor environment visual inertial SLAM method, comprising:
obtaining a gradient image by extracting image gradient information of an original image, and performing threshold segmentation processing on the gradient image to obtain an image to be optimized;
defining a sky boundary function according to the parameters of the image to be optimized, and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated; the parameters of the image to be optimized comprise the width of the image and the height of the image; defining a sky boundary function according to the parameters of the image to be optimized, and calculating the sky boundary function according to a gradient optimization energy function to obtain boundary data to be integrated, wherein the boundary data to be integrated is specifically as follows: respectively calculating a covariance matrix of a sky area and a covariance matrix of a non-sky area in the image to be optimized according to the number of pixels of the sky area and the number of pixels of the non-sky area in the image to be optimized, and defining a gradient optimization energy function according to the covariance matrix of the sky area and the covariance matrix of the non-sky area; defining a sky boundary function according to the width of the image and the height of the image; calculating the sky boundary function according to the gradient optimization energy function to obtain boundary data to be integrated;
the expression of the gradient optimization energy function is as follows:
wherein, sigma S Sum sigma g Covariance matrices representing sky regions and non-sky regions, respectively, represented by RGB values, γ being a parameter of sky region uniformity,and->Corresponding to the two matrices, |·| represents the corresponding determinant, Σ S Sum sigma g The definition is as follows:
N s and N g The number of pixels representing the sky region and the non-sky region, respectively;
adopting a polynomial fitting algorithm to integrate the boundary data to be integrated to obtain final boundary data;
and dividing the original image according to the final boundary data to obtain a sky area image, carrying out SLAM initialization on the inertial navigation data and the sky area image to obtain an SLAM framework, and carrying out SLAM composition positioning according to the SLAM framework.
2. The outdoor environment visual inertia SLAM method of claim 1, wherein the gradient image is obtained by extracting image gradient information of an original image, and the image to be optimized is obtained by performing threshold segmentation processing on the gradient image, specifically:
calculating gradient information of the original image based on image space domain convolution by adopting a Sobel operator, and drawing a gradient image according to the gradient information;
and dividing the sky area and the non-sky area of the gradient image to obtain an image to be optimized.
3. The outdoor environment vision inertial SLAM method of claim 1, wherein the step of performing SLAM initialization on the inertial navigation data and the sky area image to obtain a SLAM frame, and performing SLAM composition positioning according to the SLAM frame comprises the following steps:
and initializing the sky area image and the inertial navigation data by adopting a visual inertial odometer, rear-end optimization and loop detection processing to obtain an SLAM framework, and carrying out SLAM composition positioning according to the SLAM framework.
4. An outdoor environment visual inertial SLAM device, comprising:
the threshold segmentation module is used for obtaining a gradient image by extracting image gradient information of an original image, and performing threshold segmentation processing on the gradient image to obtain an image to be optimized;
the computing module is used for defining a sky boundary function according to the parameters of the image to be optimized, and computing the sky boundary function according to the gradient optimization energy function to obtain boundary data to be integrated; the parameters of the image to be optimized comprise the width of the image and the height of the image; the computing module is specifically configured to: respectively calculating a covariance matrix of a sky area and a covariance matrix of a non-sky area in the image to be optimized according to the number of pixels of the sky area and the number of pixels of the non-sky area in the image to be optimized, and defining a gradient optimization energy function according to the covariance matrix of the sky area and the covariance matrix of the non-sky area; defining a sky boundary function according to the width of the image and the height of the image; calculating the sky boundary function according to the gradient optimization energy function to obtain boundary data to be integrated;
the expression of the gradient optimization energy function is as follows:
wherein, sigma S Sum sigma g Covariance matrices representing sky regions and non-sky regions, respectively, represented by RGB values, γ being a parameter of sky region uniformity,and->Corresponding to the two matrices, |·| represents the corresponding determinant, Σ S Sum sigma g The definition is as follows:
N s and N g The number of pixels representing the sky region and the non-sky region, respectively;
the integration module is used for integrating the boundary data to be integrated by adopting a polynomial fitting algorithm to obtain final boundary data;
and the composition positioning module is used for dividing the original image according to the final boundary data to obtain a sky area image, carrying out SLAM initialization on the inertial navigation data and the sky area image to obtain an SLAM framework, and carrying out SLAM composition positioning according to the SLAM framework.
5. The outdoor environment visual inertial SLAM device of claim 4, wherein the threshold segmentation module is specifically configured to:
calculating gradient information of the original image based on image space domain convolution by adopting a Sobel operator, and drawing a gradient image according to the gradient information;
and dividing the sky area and the non-sky area of the gradient image to obtain an image to be optimized.
6. The outdoor environment visual inertial SLAM device of claim 4, wherein the composition positioning module is specifically configured to:
and initializing the sky area image and the inertial navigation data by adopting a visual inertial odometer, rear-end optimization and loop detection processing to obtain an SLAM framework, and carrying out SLAM composition positioning according to the SLAM framework.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011489168.5A CN112613372B (en) | 2020-12-16 | 2020-12-16 | Outdoor environment visual inertia SLAM method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011489168.5A CN112613372B (en) | 2020-12-16 | 2020-12-16 | Outdoor environment visual inertia SLAM method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112613372A CN112613372A (en) | 2021-04-06 |
| CN112613372B true CN112613372B (en) | 2024-04-02 |
Family
ID=75239960
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011489168.5A Active CN112613372B (en) | 2020-12-16 | 2020-12-16 | Outdoor environment visual inertia SLAM method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112613372B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113177971A (en) * | 2021-05-07 | 2021-07-27 | 中德（珠海）人工智能研究院有限公司 | Visual tracking method and device, computer equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105869174A (en) * | 2016-04-20 | 2016-08-17 | 中国航空工业集团公司洛阳电光设备研究所 | Sky scene image segmentation method |
| CN110097584A (en) * | 2019-03-18 | 2019-08-06 | 国网浙江省电力有限公司信息通信分公司 | The method for registering images of combining target detection and semantic segmentation |
| CN111210434A (en) * | 2019-12-19 | 2020-05-29 | 上海艾麒信息科技有限公司 | Image replacement method and system based on sky identification |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE602007012270D1 (en) * | 2007-11-16 | 2011-03-10 | Honda Res Inst Europe Gmbh | Method and apparatus for continuous object-background segmentation in images from dynamic visual scenes |
| US8311285B2 (en) * | 2009-06-30 | 2012-11-13 | Mitsubishi Electric Research Laboratories, Inc. | Method and system for localizing in urban environments from omni-direction skyline images |
-
2020
- 2020-12-16 CN CN202011489168.5A patent/CN112613372B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105869174A (en) * | 2016-04-20 | 2016-08-17 | 中国航空工业集团公司洛阳电光设备研究所 | Sky scene image segmentation method |
| CN110097584A (en) * | 2019-03-18 | 2019-08-06 | 国网浙江省电力有限公司信息通信分公司 | The method for registering images of combining target detection and semantic segmentation |
| CN111210434A (en) * | 2019-12-19 | 2020-05-29 | 上海艾麒信息科技有限公司 | Image replacement method and system based on sky identification |
Non-Patent Citations (1)
| Title |
|---|
| 基于GrabCut的磨粒图像分割方法研究;王联君;王静秋;;机械制造与自动化(02);第133-136页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112613372A (en) | 2021-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115439424B (en) | Intelligent detection method for aerial video images of unmanned aerial vehicle | |
| CN108229475B (en) | Vehicle tracking method, system, computer device and readable storage medium | |
| CN109376641B (en) | Moving vehicle detection method based on unmanned aerial vehicle aerial video | |
| US11367195B2 (en) | Image segmentation method, image segmentation apparatus, image segmentation device | |
| CN110930411B (en) | Human body segmentation method and system based on depth camera | |
| CN109214254B (en) | Method and device for determining displacement of robot | |
| CN111738033B (en) | Vehicle driving information determination method and device based on plane segmentation and vehicle-mounted terminal | |
| CN111046843A (en) | Monocular distance measurement method under intelligent driving environment | |
| CN110245600B (en) | Unmanned aerial vehicle road detection method for self-adaptive initial quick stroke width | |
| CN110570435A (en) | method and device for carrying out damage segmentation on vehicle damage image | |
| CN103841298A (en) | Video image stabilization method based on color constant and geometry invariant features | |
| CN105606123A (en) | Method for automatic correction of digital ground elevation model for low-altitude aerial photogrammetry | |
| CN108021857B (en) | Building detection method based on unmanned aerial vehicle aerial image sequence depth recovery | |
| US9232128B2 (en) | Image capture position and image capture direction estimation device, image capture device, image capture position and image capture direction estimation method and program | |
| CN114419165B (en) | Camera external parameter correction method, camera external parameter correction device, electronic equipment and storage medium | |
| CN112613372B (en) | Outdoor environment visual inertia SLAM method and device | |
| CN102324043B (en) | Image matching method based on DCT (Discrete Cosine Transformation) through feature description operator and optimization space quantization | |
| CN113096016A (en) | Low-altitude aerial image splicing method and system | |
| CN111488762A (en) | Lane-level positioning method and device and positioning equipment | |
| CN114998629A (en) | Satellite map and aerial image template matching method and unmanned aerial vehicle positioning method | |
| CN116188535A (en) | Video tracking method, device, equipment and storage medium based on optical flow estimation | |
| CN115984592A (en) | Point-line fusion feature matching method based on SuperPoint + SuperGlue | |
| Zhang et al. | An enhanced multi-view vertical line locus matching algorithm of object space ground primitives based on positioning consistency for aerial and space images | |
| CN112330660B (en) | Sperm tail detection method and system based on neural network | |
| CN111860084B (en) | Image feature matching and positioning method and device and positioning system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |