CN111161219B - Robust monocular vision SLAM method suitable for shadow environment - Google Patents
Robust monocular vision SLAM method suitable for shadow environment Download PDFInfo
- Publication number
- CN111161219B CN111161219B CN201911259076.5A CN201911259076A CN111161219B CN 111161219 B CN111161219 B CN 111161219B CN 201911259076 A CN201911259076 A CN 201911259076A CN 111161219 B CN111161219 B CN 111161219B
- Authority
- CN
- China
- Prior art keywords
- shadow
- image frame
- sub
- image
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000005286 illumination Methods 0.000 claims abstract description 24
- 230000000007 visual effect Effects 0.000 claims abstract description 15
- 230000003044 adaptive effect Effects 0.000 claims abstract description 8
- 238000012545 processing Methods 0.000 claims abstract description 8
- 238000001514 detection method Methods 0.000 claims abstract description 7
- 238000006243 chemical reaction Methods 0.000 claims abstract 2
- 239000011159 matrix material Substances 0.000 claims description 12
- 230000033001 locomotion Effects 0.000 claims description 9
- 230000008859 change Effects 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 8
- 238000013519 translation Methods 0.000 claims description 4
- 238000003708 edge detection Methods 0.000 claims description 3
- 239000000203 mixture Substances 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims description 2
- 238000013507 mapping Methods 0.000 description 7
- 230000002159 abnormal effect Effects 0.000 description 3
- 230000004807 localization Effects 0.000 description 3
- 238000004883 computer application Methods 0.000 description 2
- 230000008030 elimination Effects 0.000 description 2
- 238000003379 elimination reaction Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 241000345998 Calamus manan Species 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 235000012950 rattan cane Nutrition 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a robust monocular vision SLAM method suitable for a shadow environment, which comprises the following steps: step one, shadow detection is carried out on an image frame acquired by a visual front end by using an image processing algorithm; dividing image frame subareas; step three, matching image frame sub-regions; finding other sub-regions corresponding to the non-shadow region for each sub-region in the shadow region by using a sub-region matching algorithm; step four, removing shadows in the image frame; for each matched pair of subregions, removing shadows in the subregions in the Lab color space using an adaptive illumination conversion method; after the shadow removal processing is finished, converting the result back to an RGB image; fifthly, extracting image frame characteristics; and step six, tracking the camera and constructing a map.
Description
Technical Field
The invention belongs to the field of computer vision, relates to visual positioning, image processing and three-dimensional reconstruction, and can realize real-time positioning and image construction in a shadow environment.
Background
SLAM is generally called Simultaneous Localization and Mapping, i.e., real-time synchronous positioning and Mapping, and generally refers to a robot or a moving rigid body, and is equipped with a specific sensor, and under the condition of no prior information, the SLAM estimates the motion of the SLAM and establishes a model of the surrounding environment. The monocular vision SLAM system can be directly used by various small-sized devices such as mobile phones, unmanned planes and the like due to the characteristic of simple structure, and can be applied to various scenes. Since such small devices are in a complex lighting environment at any time during operation [1], it is important to ensure that the visual front-end of the SLAM system can work stably in real time in various environments.
There are often a large number of natural shadows in some outdoor scenes during the day, such as: tree shade, building shelter, telegraph pole, etc. These typically strong visual features can result in a large change in the appearance of a place. Vision-based SLAM systems rely on camera poses that record scene appearance as an image. However, the image is formed under the interaction of the scene information and the current lighting conditions. Ideally, the same scene can always acquire the same image, but the observation result is often strongly influenced by illumination. In particular, shadows in natural environments are prone to cause false identifications by computers, and therefore shadow removal is an important research direction [2-5].
The traditional SLAM system mainly extracts and matches feature points from image frames acquired by a camera, and then carries out optimization solution to obtain the pose and the motion track of the camera. The PTAM [6] firstly plans the SLAM system into a structure of a visual front end and a mapping rear end, the visual front end is responsible for extracting characteristic information in a video frame to further calculate the pose and the motion trail of a camera, the rear end constructs a three-dimensional map through a geometric algorithm, and the front end and the rear end are parallel. PTAM lays the foundation for almost all late SLAM systems. However, PTAMs still have the disadvantage of poor robustness and are only suitable for small working spaces. ORB-SLAM [7] is one of the best-known successors of PTAM. The method continues to use the basic framework of the PTAM, improves part of components in the framework, and enables the system to have higher robustness and accuracy in the complex environment of a large scene. However, these current SLAM systems only consider their operation in well-lit scenes. In outdoor environments with complex shadows, these systems tend to suffer from insufficient robustness.
For the case of shadows present in the environment, finlayson et al [8] compute a map by analyzing images of observed material properties under different lighting conditions (e.g., the sun and the ground in the shadows). Corke et al [9] applies the invariant image of Finlayson to the single image localization problem to deal with the shadowing problem. This study shows that the transformed image of a location is more similar than the original color image, and thus the local mapping performance is improved. Wang et al [10] introduced an algorithm for multi-level image enhancement combined with a mutual information-based method, completed camera tracking by selecting a layer with optimal feature matching from different enhancement levels of an image frame, and improved the robustness of the SLAM visual front-end. However, the tracking process needs to rely on a local map built in advance, which is disadvantageous to real-time positioning in an unfamiliar environment. McManus et al [11] improve the robustness of their vision localization and mapping system to illumination problems by using a lidar-based sensor that is illumination invariant. Although the sensor achieves good effect, the sensor also has a series of problems of high cost, relative fragility, high power consumption requirement and the like. Kim et al [12] designed a set of SLAM systems with illumination robustness. By selecting the pre-built map with the matched illumination condition to carry out pose estimation in the changed illumination, the system has high accuracy and robustness because the used environment map is quite accurate. But since the system relies entirely on pre-established maps, this approach is only applicable to the problem of visual positioning in a fixed environment.
In summary, the existing SLAM method has low robustness to shadows existing in a natural environment, and is easily interfered by a shadow region during running, so that problems of accuracy reduction, tracking loss, running interruption and the like occur. However, a large amount of environment shadows often exist in the practical application scene of the monocular vision SLAM, so that the monocular vision SLAM method suitable for the shadow environment has a great significance.
Reference documents:
[1]Jiawei Huang,Shiguang Liu.Robust simultaneous localization and mapping in low-light environment.Computer Animation and Virtual Worlds,2019,30(9):e1895.
[2] zhu Biting, zheng Shibao Gaussian mixture model-based space domain background separation method and shadow elimination method [ J ]. Chinese graphics newspaper, 2008 (10): 1906-1909
[3] Zhang Gongying, li Hong, sun Yigang a mixed gaussian model based shadow removal algorithm [ J ]. Computer application, 2013,33 (01): 31-34.
[4] Steady, ni Kang. Background modeling and gesture shadow elimination based on YCbCr color space [ J ]. Chinese optics, 2015,8 (4): 589-595.
[5] Lanli, he Xiaohai, wu Xiaogong, rattan strange adaptive shadow removal based on superpixel and local color constancy [ J ] computer applications, 2016,36 (10): 2837-2841.
[6]Klein G,Murray D.Parallel tracking and mapping on a camera phone[C].In IEEE International Symposium on Mixed and Augmented Reality,2009:83–86.
[7]Mur-Artal R,Montiel J M M,Tardos J D.ORB-SLAM:a versatile and accurate monocular SLAM system[J].IEEE Transactions on Robotics,2017,31(5):1147–1163..
[8]Finlayson G,Drew M,Lu C.Intrinsic images by entropy minimization[C].In European Conference on Computer Vision,2004:582–595.
[9]Corke P,Paul R,Churchill W,et al.Dealing with shadows:Capturing intrinsic scene appearance for image-based outdoor localisation[C].In IEEE International Conference on Intelligent Robots and Systems,2013:2085–2092.
[10]Wang X,Christie M,Marchand E.Optimized contrast enhancements to improve robustness of visual tracking in a SLAM relocalisation context[C].In IEEE International Conference on Intelligent Robots and Systems,2018:103–108.
[11]Mcmanus C,Furgale P,Stenning B,et al.Visual teach and repeat using appearance-based lidar[C].In IEEE International Conference on Robotics and Automation,2012:389–396.
[12]Kim P,Coltin B,Alexandrov O,et al.Robust visual localization in changing lighting conditions[C].In IEEE International Conference on Robotics and Automation,2017:5447–5452.
Disclosure of Invention
The invention aims to overcome the defects in the prior art and provide a robust monocular vision SLAM method suitable for a shadow environment.
The purpose of the invention is realized by the following technical scheme:
a robust monocular vision SLAM method suitable for shadow environments, comprising the steps of:
step one, detecting shadow of an image frame, which specifically comprises the following steps:
and (4) carrying out shadow detection on the image frame acquired by the visual front end by using an image processing algorithm. The image frame is first converted from the RGB color space to the HSI color space. In the HSI color space, shadows are clearly distinguished from unshaded areas in both the chrominance and saturation channels. According to the characteristic of shadow, shadow and non-shadow areas can be effectively distinguished in the image frame by using HSI color space transformation twice in succession. By distinguishing shadow areas in the image frames, the influence of shadows on camera tracking can be avoided in the SLAM running process.
Step two, image frame subregion partition, include the following treatment specifically:
firstly, roughly dividing the region of an image frame by using an edge detection algorithm, and distinguishing a relatively obvious object outline. And then converting the pixel points and the adjacent pixel points in the image frame into a graph model, and performing region division on all the pixel points in the graph through a clustering algorithm. Finally, the image frame is divided into a plurality of sub-regions according to different texture attributes and pixel gradients of all the parts, and pixel points in each sub-region have similar pixel intensity.
Step three, image frame subregion matching, which specifically comprises the following processing:
a sub-region matching algorithm is used to find for each sub-region in the shadow region the other sub-regions to which it corresponds in the non-shadow region. The subregion matching algorithm consists of two steps. First, the image frame is divided into a shadow area and a non-shadow area, and then texture features are extracted for each of them. And then carrying out feature matching on each sub-region in the shadow region, and finding the sub-region which can be matched with the shadow region best from the illumination region to obtain a sub-region matching pair.
Step four, removing shadow of the image frame, which specifically comprises the following steps:
for each matching pair of subregions, shadows in the subregions are removed using an adaptive illumination translation method in the Lab color space. After the shadow removal process is completed, the result is converted back to an RGB image. By performing adaptive illumination transfer on the paired sub-regions, most of the shadows in the image frame can be better removed, and the regions after removal of the shadows can also blend into the surrounding scene.
Step five, image frame feature extraction, which specifically comprises the following steps:
the shadow removal algorithm is able to remove most of the shadows in the image frame. But the edges of the shadow cannot be completely removed. These residual edges may affect the whole image inter-frame feature matching process in the process of performing subsequent point feature extraction in the visual front end. Therefore, using the outlier rejection algorithm enables the SLAM visual front end to extract enough and accurate feature points from the image frame by identifying and rejecting feature points near shadow edges in the image frame.
Step six, tracking and constructing a map by a camera, and specifically comprises the following steps:
after feature extraction, when enough stable feature point pairs are obtained from the current image frame and the previous image frame, the motion change between the two image frames can be calculated through the corresponding relation between the image frames. This change is represented by two matrices, namely a rotation matrix R and a translation matrix t. The camera pose of each frame of image can be solved by calculating the essential matrix and the basic matrix, and the inter-frame tracking is completed. By triangulating pairs of feature points in an image frame, map point coordinates in three-dimensional space can be generated.
Compared with the prior art, the technical scheme of the invention has the following beneficial effects:
compared with the conventional SLAM method, the method firstly reserves the excellent performance of the conventional algorithm in the common illumination environment, and can generate an accurate tracking track in the good illumination environment. In addition, under the complex illumination environment full of shadows, the method can remove most of the shadow areas in the image frames through the processing of the visual front end on the image frames. For the shadow edge part, the invention can also reduce the influence of the shadow edge on the feature extraction by screening out abnormal feature points. Compared with the existing method, the method has higher precision when camera tracking is carried out in a shadow environment. Moreover, in some extreme cases, the existing method is interrupted in operation or the tracking is lost, and the invention can carry out continuous positioning and mapping.
Drawings
Fig. 1 is a frame diagram of a robust monocular vision SLAM method applicable to a shadow environment.
FIG. 2 is a graph comparing the trajectory error of the present invention with a prior art method.
Detailed Description
The invention is described in further detail below with reference to the figures and specific examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention discloses a robust SLAM method suitable for a shadow environment. As shown in fig. 1, the method specifically comprises the following steps:
1. implementing pre-processing of image frames
For an image frame acquired by a visual front end, edge detection is carried out on the image frame by using a Canny operator, continuous edges in the image are calculated, and a rough foreground area and a rough background area are divided for the image frame. And calculating the brightness intensity ratio and the difference value of the hue and the color saturation of the foreground area and the background area in the image frame, and finally establishing a corresponding HSI color model. For each pixel point in the image frame, whether the pixel point is a shadow point is judged by calculating the following discriminant.
(Q I (x，y)，B I (x，y))-γ I ≤1.5σ I (1)
||Q S (x，y)-B S (x，y)|-γ S |≤1.5σ S (2)
||Q H (x，y)-B H (x，y)|-γ H |≤1.5σ H (3)
Wherein Q (x, y), B (x, y) are the intensity of the pixel points in the foreground region and the background region in H, S, I three channels, gamma I ，γ S ，γ H Is the normalized average of the chrominance, saturation and luminance intensity, σ I ，σ S ，σ H Is three-channelStandard deviation.
The detection step is repeated two or three times depending on the brightness complexity of the image frame. The invention uses a shadow detection method from coarse to fine, firstly adopts a global continuous threshold method to detect a large-area shadow region, and then utilizes a local guide filter operator to improve the detected shadow so as to refine the shadow boundary.
2. Implementing image frame subregion partitioning
Firstly, an image frame is segmented by using an image segmentation method based on an image, each pixel point in the image frame is calculated to be connected with 8 adjacent pixel points to generate 8 edges, and the dissimilarity degree of the pixel and the surrounding pixels is calculated respectively. Sorting the edges according to dissimilarity from small to large (e) 1 ，e 2 ，e 3 …e n ) If the dissimilarity between the pixel point and the surrounding pixels is smaller than the minimum difference between the classes in the region where the pixel is located, the surrounding pixel points are combined into the class region where the pixel point is located. For a certain pixel point d in the sub-region C in the image frame, the intra-domain difference Int (C) of the sub-region where the pixel point d is located can be expressed as:
Int(C)＝max(e 1 ，e 2 ，e 3 …e n ) (4)
for two sub-areas C 1 ，C 2 Inter-domain difference Diff (C) thereof 1 ，C 2 ) The dissimilarity of the edge with the smallest dissimilarity among all edges connecting two regions can be defined as:
Diff(C 1 ，C 2 )＝min(α i ，β j )α i ∈C 1 ，β j ∈C 2 (5)
wherein alpha is i ，β j Respectively sub-region C 1 ，C 2 The edge in (1) is a region that, if the following formula is satisfied, merges two regions,
Diff(C 1 ，C 2 )≤min(Int(C 1 )，Int(C 2 )) (6)
3. implementing image frame subregion matching
For each sub-region in the shadow region, a sub-region matching algorithm is used to find the corresponding other sub-region for it. The subregion matching algorithm consists of two steps. First, an image frame is divided into a shadow area and a light area, and then texture features are extracted for each of them. The invention uses a feature representation method based on Gabor wavelet transform to extract texture features. And then carrying out sub-region matching on each sub-region in the shadow region, and finding the sub-region which is best matched with the shadow region from the illumination region to obtain a sub-region matching pair.
4. Adaptive illumination removal
And (4) aiming at each sub-region matching pair, adopting an adaptive illumination removal technology to complete shadow removal. In the imaging equation, the pixel intensity I (x) of a pixel x is the product of the luminance and the reflectivity, and if the pixel x is located in an illumination area, the luminance L (x) can be expressed as:
L(x)＝L d (x)+L a (x) (7)
wherein L is d (x) Is direct illumination, L a (x) Is indirect (ambient) lighting. Thus, the pixel intensity I at pixel x lit (x) Can be expressed as:
I lit (x)＝L d (x)R(x)+L a (x)R(x) (8)
wherein R (x) is the illumination reflectance. If some object blocks the primary light source, it will cast a shadow at pixel x, which will also block some ambient light, and therefore the intensity I of pixel x at the shadow shado w (x) can be expressed as:
I shadow (x)＝η(x)L a (x) R (x) (9) where η (x) is the attenuation factor of ambient lighting. From this, the illumination intensity at pixel point x can be found from the affine function between illumination intensity and shadow intensity:
5. feature point extraction
The shadow removal algorithm is able to remove a large portion of the shadows in the image frame. But the edges of the shadow cannot be completely removed. These residual edges may affect the whole image inter-frame feature matching process in the process of performing subsequent point feature extraction in the visual front end. The present embodiment uses an abnormal feature point screening method.
Firstly, converting an obtained image frame into an 8-layer image pyramid, and dividing each layer of image into uniform grids. And extracting FAST corners from each layer of image pyramid, determining that at least 5 corners are extracted from each grid, and reducing the threshold value if the extracted corners are not enough.
And aligning the binary image generated in the previous shadow detection step with each layer of pyramid image through size change. The edge part of the shadow area in each layer of pyramid image is detected, and if the corner point just extracted is in the 10 pixel range (with the radius of 10) near the edge point, the corner point is marked as a possible abnormal point.
For all possible outliers, the feature detection radius is expanded, the intensity difference is calculated over a larger range, and if the point still exhibits "features", it is retained, otherwise it is screened out as an outlier.
6. Camera tracking and mapping
After feature extraction, feature points in the image frame are described by means of binary character strings.
Where h (x), h (y) are pixel values of points x, y, and Z is the final descriptor. For two feature descriptions b1, b2 in two consecutive image frames acquired by the camera, the hamming distance D (b 1, b 2) between their feature descriptions is calculated to determine whether they are matching points.
The smaller the value of D is, the higher the similarity of the two features is, all map points in the previous image frame are traversed and projected to the current frame, and then a feature point with the closest descriptor distance is found in the current frame as a matching point.
When enough stable characteristic point pairs are obtained from the current image frame and the previous image frame, the motion change between the two image frames can be calculated through the corresponding relation between the image frames. This change in movement is represented by two matrices, namely a rotation matrix R and a translation matrix t. The real points in the three-dimensional space and the characteristic points in the image jointly form epipolar constraint, and the epipolar constraint is obtained by calculating an essential matrix E = tR and a basic matrix F = K -T EK -1 The camera pose of each frame of image can be solved, and inter-frame tracking is completed.
Wherein p is 1 Is a feature point in the previous frame,
is in the current frame with p 1 And matching feature points. And then triangularizing the image feature point pairs to generate map point coordinates in a three-dimensional space. Finally, by using the feature points in the image frame and the map points in the three-dimensional space, the generated camera movement track can be optimized through a beam adjustment algorithm.
The present invention is not limited to the embodiments described above. The foregoing description of the specific embodiments is intended to describe and illustrate the technical solutions of the present invention, and the above specific embodiments are merely illustrative and not restrictive. Those skilled in the art can make various changes in form and details without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (5)
1. A robust monocular vision SLAM method suitable for shadow environments, comprising the steps of:
step one, shadow detection is carried out on an image frame acquired by a visual front end by using an image processing algorithm;
dividing image frame subareas;
step three, matching image frame sub-regions; finding other sub-regions corresponding to the non-shadow region for each sub-region in the shadow region by using a sub-region matching algorithm;
step four, removing shadows in the image frame; for each matched pair of subregions, removing shadows in the subregions in the Lab color space using an adaptive illumination conversion method; after the shadow removal processing is finished, converting the result back to an RGB image;
fifthly, extracting image frame characteristics;
step six, tracking the camera and constructing a map, and specifically comprises the following steps:
after feature extraction, calculating the motion change between the current image frame and the previous image frame according to the corresponding relation between the image frames; representing the motion change between the two image frames by two matrixes, namely a rotation matrix R and a translation matrix t; calculating the camera pose of each frame of image by calculating the essential matrix and the basic matrix to complete inter-frame tracking; and triangularizing the characteristic point pairs in the image frame to generate map point coordinates in a three-dimensional space.
2. The robust monocular vision SLAM method of claim 1, wherein step one comprises the steps of: firstly, converting an image frame from an RGB color space to an HSI color space; in the HSI color space, the shadow has obvious difference from the unshaded area in the chroma and saturation channels; according to the characteristic of the shadow, shadow and non-shadow areas can be effectively distinguished in the image frame by using HSI color space transformation which is performed twice continuously; by distinguishing shadow areas in the image frames, the influence of the shadow on the tracking of the camera can be avoided in the SLAM running process.
3. The robust monocular vision SLAM method of claim 1, wherein step two specifically comprises the steps of: firstly, carrying out region division on an image frame by using an edge detection algorithm to distinguish a relatively obvious object outline; converting pixel points and adjacent pixel points in the image frame into a graph model, and performing region division on all the pixel points in the graph through a clustering algorithm; finally, dividing the image frame into a plurality of sub-regions according to the texture attributes and the pixel gradients of the regions, wherein the pixel points in each sub-region have similar pixel intensity.
4. The robust monocular vision SLAM method of claim 1, wherein the subregion matching algorithm in step three specifically comprises the following steps: firstly, dividing an image frame into a shadow area and a non-shadow area, then respectively extracting texture features from the shadow area and the non-shadow area, then performing feature matching on each sub-area in the shadow area, and finding the sub-area which can be optimally matched with the shadow area and the non-shadow area from an illumination area to obtain a sub-area matching pair.
5. The robust monocular vision SLAM method of claim 1 wherein, in step four, by performing adaptive illumination transfer on the paired sub-regions, most of the shadows in the image frame can be removed and the shadow-removed regions can blend into the surrounding scene.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911259076.5A CN111161219B (en) | 2019-12-10 | 2019-12-10 | Robust monocular vision SLAM method suitable for shadow environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911259076.5A CN111161219B (en) | 2019-12-10 | 2019-12-10 | Robust monocular vision SLAM method suitable for shadow environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111161219A CN111161219A (en) | 2020-05-15 |
| CN111161219B true CN111161219B (en) | 2023-04-11 |
Family
ID=70556674
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911259076.5A Active CN111161219B (en) | 2019-12-10 | 2019-12-10 | Robust monocular vision SLAM method suitable for shadow environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111161219B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114370882A (en) * | 2020-10-14 | 2022-04-19 | 蘑菇车联信息科技有限公司 | Method and related device for realizing SLAM positioning based on monocular automobile data recorder |
| CN113361315B (en) * | 2021-02-23 | 2021-12-07 | 仲恺农业工程学院 | Banana string identification method based on background saturation compression and difference threshold segmentation fusion |
| CN113343766B (en) * | 2021-05-11 | 2022-12-06 | 武汉工程大学 | Automatic detection method for double yellow lines of vehicle pressure |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109523589A (en) * | 2018-11-13 | 2019-03-26 | 浙江工业大学 | A kind of design method of more robust visual odometry |
| CN109658370A (en) * | 2018-11-29 | 2019-04-19 | 天津大学 | Image split-joint method based on mixing transformation |
| CN109934862A (en) * | 2019-02-22 | 2019-06-25 | 上海大学 | A kind of binocular vision SLAM method that dotted line feature combines |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106296666B (en) * | 2016-08-01 | 2019-03-01 | 北京大学深圳研究生院 | A kind of color image removes shadow method and application |
-
2019
- 2019-12-10 CN CN201911259076.5A patent/CN111161219B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109523589A (en) * | 2018-11-13 | 2019-03-26 | 浙江工业大学 | A kind of design method of more robust visual odometry |
| CN109658370A (en) * | 2018-11-29 | 2019-04-19 | 天津大学 | Image split-joint method based on mixing transformation |
| CN109934862A (en) * | 2019-02-22 | 2019-06-25 | 上海大学 | A kind of binocular vision SLAM method that dotted line feature combines |
Non-Patent Citations (2)
| Title |
|---|
| Jiawei Huang.Robust simultaneous localization and mapping in low-light environment.《Computer Animation and Virtual Worlds》.2019,全文. * |
| 罗韬 ; 姚素英 ; 史再峰 ; 陆尧 ; .自适应加权滤波的去隔行算法及其FPGA实现.吉林大学学报(工学版).2008,(第S2期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111161219A (en) | 2020-05-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10574905B2 (en) | System and methods for depth regularization and semiautomatic interactive matting using RGB-D images | |
| Alismail et al. | Direct visual odometry in low light using binary descriptors | |
| Bradley et al. | Adaptive thresholding using the integral image | |
| CN111161219B (en) | Robust monocular vision SLAM method suitable for shadow environment | |
| Hane et al. | Direction matters: Depth estimation with a surface normal classifier | |
| Chu et al. | Object tracking algorithm based on camshift algorithm combinating with difference in frame | |
| CN111160291B (en) | Human eye detection method based on depth information and CNN | |
| CN106407983A (en) | Image body identification, correction and registration method | |
| Li et al. | Fast visual odometry using intensity-assisted iterative closest point | |
| Wang et al. | Edge Enhanced Direct Visual Odometry. | |
| Concha et al. | Manhattan and Piecewise-Planar Constraints for Dense Monocular Mapping. | |
| CN109389617A (en) | A kind of motion estimate based on piece heterogeneous system and method for tracing and system | |
| Mishra et al. | Active segmentation for robotics | |
| Li et al. | Moving object detection in dynamic scenes based on optical flow and superpixels | |
| Mirante et al. | A fast image segmentation algorithm using color and depth map | |
| Yang et al. | Contrast limited adaptive histogram equalization for an advanced stereo visual slam system | |
| Kim et al. | Multi-view object extraction with fractional boundaries | |
| Yuan et al. | Rgb-d saliency detection: Dataset and algorithm for robot vision | |
| Wang et al. | Optical flow based plane detection for mobile robot navigation | |
| van de Wouw et al. | Hierarchical 2.5-d scene alignment for change detection with large viewpoint differences | |
| Bazin et al. | An original approach for automatic plane extraction by omnidirectional vision | |
| Yan et al. | Robust and efficient edge-based visual odometry | |
| Pan et al. | UFSM VO: Stereo odometry based on uniformly feature selection and strictly correspondence matching | |
| Chen et al. | High Precision ORB-SLAM Dense Reconstruction Based on Depth Visual Odometer in Dynamic Environments | |
| Das et al. | A survey on shadow detection and removal in images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |