CN117470230A - Visual laser sensor fusion positioning algorithm based on deep learning - Google Patents
Visual laser sensor fusion positioning algorithm based on deep learning Download PDFInfo
- Publication number
- CN117470230A CN117470230A CN202311372884.9A CN202311372884A CN117470230A CN 117470230 A CN117470230 A CN 117470230A CN 202311372884 A CN202311372884 A CN 202311372884A CN 117470230 A CN117470230 A CN 117470230A
- Authority
- CN
- China
- Prior art keywords
- deep learning
- data
- sensor
- feature extraction
- environment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 39
- 238000013135 deep learning Methods 0.000 title claims abstract description 21
- 230000000007 visual effect Effects 0.000 title claims description 27
- 238000000605 extraction Methods 0.000 claims abstract description 31
- 238000013136 deep learning model Methods 0.000 claims abstract description 16
- 230000007613 environmental effect Effects 0.000 claims description 10
- 238000005516 engineering process Methods 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 8
- 238000010276 construction Methods 0.000 claims description 5
- 238000013527 convolutional neural network Methods 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 description 10
- 238000000034 method Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000004807 localization Effects 0.000 description 3
- 238000007500 overflow downdraw method Methods 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 238000007499 fusion processing Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000002245 particle Substances 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000002226 simultaneous effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
- G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
- G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
- G01C21/1652—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments with ranging devices, e.g. LIDAR or RADAR
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
- G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
- G01C21/1656—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments with passive imaging devices, e.g. cameras
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Automation & Control Theory (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Molecular Biology (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- Electromagnetism (AREA)
- General Engineering & Computer Science (AREA)
- Biodiversity & Conservation Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention relates to the technical field of robot application, and discloses a vision laser sensor fusion positioning algorithm based on deep learning, which specifically comprises the following steps: acquiring real-time data through a laser radar, a camera and an IMU sensor on the robot; performing feature extraction and environment recognition on the preprocessed data by using a deep learning model; according to the feature extraction and the environment recognition result, weighting and fusing the different sensor data to obtain accurate pose estimation; and optimizing by using IMU data and GPS data through a back-end factor graph to obtain an accurate pose. The invention adopts a deep learning model to extract the dot line characteristics and identify the environment, thereby improving the characteristic quality and the positioning accuracy in a low-texture environment; through multi-sensor data fusion, the positioning accuracy and the robustness of the mobile robot SLAM system can be improved in a low-texture environment, and the real-time estimation of the position and the gesture of the robot can be realized.
Description
Technical Field
The invention belongs to the technical field of robot application, and particularly relates to a vision laser sensor fusion positioning algorithm based on deep learning.
Background
Currently, more common mobile robots include home service robots, AGV logistics robots, and the like. The home service robot mainly comprises an education robot, a cleaning robot, an old and disabled assisting robot and the like. In special occasions, such as exhibitions, restaurants, welcome, shopping guides, security and surveillance, special robots are playing an increasingly important role. Although mobile robots with complex functions bring subversive changes to the production and life of people, the functions of the robots are still limited. In the field with high requirements on complex scenes and performances, the functions and performances of the robot still have a large improvement space. For an autonomous mobile robot, robot positioning and navigation are key problems, and involve the aspects of accurate positioning, scene recognition, semantic understanding, obstacle analysis and decision-making and the like, which require deep analysis and research.
A SLAM (Simultaneouss Localization and Mapping, time-series localization and mapping) system is a system in which a robot generates localization of its own position and posture and scene map information by collecting and calculating various sensor data. SLAM systems typically include a variety of sensors and functional modules. The current mature and common SLAM systems can be classified into laser radar based SLAM (laser SLAM) and vision based SLAM (VSLAM) by core functional modules.
The visual sensor can collect abundant environmental information, has lower cost, but the perception frame is easy to lose frames under high-speed movement, and the environmental perception based on the visual information is influenced by illumination conditions. The laser sensor has the characteristics of high precision, large range and the like in the aspect of environment perception, and has the advantages in the aspect of detecting edge information (such as road edges). The laser sensor works relatively stably, performs well in severe weather, but cannot sense environment detail information, and is difficult to realize functions of repositioning and loop detection of a classical visual SLAM frame.
A single lidar sensor and vision sensor have not been able to achieve accurate map construction and navigation of mobile robots in complex scenarios. In order to realize accurate positioning and navigation of the mobile robot in a complex environment, avoid the influence of factors such as light change, scene rapid movement, dynamic objects and the like on the SLAM system, the SLAM technology mainstream direction of the mobile robot in the future is designed and researched into a multi-sensor fused SLAM system, and the stability and the environment adaptability of the SLAM system are improved.
The multi-sensor fusion SLAM can combine the advantages of each sensor in different environments. Under the condition of less texture or regular texture distribution of the surrounding environment, the positioning effect of the pure visual odometer is poor, and track drift is easy to generate. Also, fewer environmental obstructions or regular distribution of the environment can lead to positioning drift of the pure laser odometer. To address the problem of lost positioning of a single sensor in certain environments, many studies have combined visual and laser sensors with other sensors (e.g., IMUs, code wheels, etc.). Compared with a pure vision or pure laser sensor odometer, the fusion sensor has higher positioning precision, is not easy to lose precision in a complex environment, and has better robustness.
While SLAM system robustness is improved by fusing multiple sensors, many studies do not fully utilize individual sensor data. Such as image processing, point features are typically extracted or pixel tracking is performed using optical flow methods. In a low texture environment, the pixel variation is not obvious, so that the characteristic point extraction and optical flow tracking effects are poor. For this purpose, some studies have performed key frame matching by extracting line features. In a low-texture environment, the number of line features is large, and vision sensor data is fully utilized.
Under the low texture environment, the traditional feature extraction method is low in robustness and poor in quality of extracted feature effect. Meanwhile, the conventional multi-sensor fusion method generally adopts a low-coupling method, and cannot well cope with sudden changes of complex environments, so we propose a visual laser sensor fusion positioning algorithm based on deep learning, which aims at researching that under the low-texture environment, the point line characteristics are extracted by utilizing deep learning, the environment is identified by utilizing the deep learning, and then the multi-sensor data are fused, so that the positioning accuracy and the system robustness are improved.
Disclosure of Invention
In order to make up for the defects of the prior art, the invention provides a visual laser sensor fusion positioning algorithm based on deep learning, which aims to solve the problem that the quality of characteristics and the positioning accuracy are to be improved in a low-texture environment.
In order to achieve the above purpose, the present invention provides the following technical solutions: a visual laser sensor fusion positioning algorithm based on deep learning specifically comprises the following steps:
s1, sensor data acquisition: acquiring real-time data through a laser radar, a camera and an IMU sensor on the robot;
s2, feature extraction and environment recognition: performing feature extraction and environment recognition on the preprocessed data by using a deep learning model;
s3, multi-sensor data fusion: according to the feature extraction and the environment recognition result, different sensor data are subjected to weighted fusion to obtain more accurate pose estimation;
s4, back-end factor graph optimization: and optimizing by using IMU data and GPS data through a back-end factor graph to obtain more accurate pose.
Preferably, in the step S3, the multi-sensor data includes laser radar data, visual image data, and IMU data.
Preferably, in the step S3, the real-time estimation of the position and the posture of the robot is performed by using a multi-sensor data fusion technology through a SLAM system, and the SLAM system is based on a laser SLAM or a VSLAM.
Preferably, the SLAM system is a vision-based SLAM system, and the system captures environmental information by using a camera, performs feature extraction and environmental mapping by combining a computer vision technology, and improves the performance of the vision SLAM system in a low-texture environment by performing feature extraction by using a deep learning model, such as a Convolutional Neural Network (CNN).
Preferably, the SLAM system is a SLAM system based on a laser radar, the system uses the laser radar to perform environment sensing, and uses point cloud data to perform feature extraction and map construction, so that the laser radar has higher precision and robustness, and can realize a better positioning effect in a complex environment.
Preferably, the SLAM system is a SLAM system based on vision and laser radar fusion, the system combines the advantages of vision and laser radar to realize multi-sensor data fusion, the performance of the system in a low-texture environment can be improved through feature extraction of a deep learning model, and meanwhile, the robustness and the accuracy of the system can be improved through the addition of the laser radar.
Preferably, the SLAM system is a SLAM system based on a filtering algorithm, and the system realizes the estimation of the position and the posture of the robot by fusing and optimizing the data of multiple sensors through designing the filtering algorithm such as a Kalman filter, a particle filter and the like. The filtering algorithm can improve the robustness and the positioning accuracy of the system to a certain extent.
The invention has the technical effects and advantages that:
1. the invention adopts the deep learning model to extract the dot line characteristics and identify the environment, and improves the characteristic quality and the positioning accuracy in a low-texture environment. Compared with the prior art, the invention can realize higher positioning precision in a complex environment.
2. The invention improves the robustness of the system in complex environments through multi-sensor data fusion. Under the complex scenes such as occlusion, texture deletion and the like, the SLAM system has better stability and reliability.
3. The SLAM system provided by the invention can realize real-time estimation of the position and the gesture of the robot, can quickly respond to environmental changes, and improves the navigation precision. Compared with the prior art, the method has obvious advantages in real-time performance.
4. The SLAM system has higher positioning precision and robustness, and can expand the application range of the robot in complex environments, such as unmanned, robot navigation and the like.
5. Compared with the traditional feature extraction method, such as SIFT, SURF and the like, the method can reduce the calculation complexity and improve the calculation efficiency.
In conclusion, the method has the advantages of improving positioning accuracy, enhancing system robustness and instantaneity, expanding application range, reducing calculation complexity and the like, and has higher technical value and practical value compared with the prior art.
Drawings
FIG. 1 is a flowchart of an algorithm of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Although the prior art has made some progress in the mobile robot SLAM system, there are some drawbacks as follows:
1. limitations of a single sensor: accurate map construction and navigation are difficult to achieve in complex scenarios, both visual and lidar sensors. The visual sensor has poor characteristic extraction effect in a low-texture environment, and the laser radar sensor is difficult to capture environment detail information. This results in limited positioning and navigation performance of a single sensor in complex environments.
2. The deficiency of the multi-sensor fusion technique: although multisensor fusion SLAM systems have achieved some success, existing fusion methods still have some problems. For example, in the multi-sensor fusion process, the utilization rate of each sensor data is not high, and the feature extraction effect still needs to be improved. In addition, the conventional multi-sensor fusion method is not excellent in coping with abrupt changes in complex environments.
3. Insufficient adaptive capacity to the environment: the existing SLAM system is susceptible to positioning and navigation performance in the face of factors such as light change, scene rapid movement, dynamic objects and the like. This limits the wide application of mobile robots in complex environments.
In view of the above-mentioned drawbacks of the prior art, an object of the present invention is to provide a multi-sensor fusion mobile robot SLAM system in a low-texture environment, which has higher positioning accuracy and system robustness. Specifically, the invention performs dot line feature extraction and environment recognition through deep learning, and then fuses the multi-sensor data to improve positioning accuracy and system robustness. Therefore, the limitation of a single sensor can be overcome, the effect of the multi-sensor fusion technology is improved, and the adaptability of the SLAM system in a complex environment is enhanced.
The invention is described in further detail below with reference to fig. 1;
the embodiment of the application discloses a visual laser sensor fusion positioning algorithm based on deep learning, which specifically comprises the following steps:
s1, sensor data acquisition: acquiring real-time data through a laser radar, a camera and an IMU sensor on the robot;
s2, feature extraction and environment recognition: the method comprises the steps of performing feature extraction and environment recognition on preprocessed data by using a deep learning model, performing dot line feature extraction and environment recognition by using the deep learning model, extracting features with higher robustness and accuracy from sensor data by using a training model, and under a low-texture environment, the features extracted by the model have higher quality, so that the performance of an SLAM system is improved;
s3, multi-sensor data fusion: according to the characteristic extraction and the environment recognition result, weighting and fusing different sensor data to obtain more accurate pose estimation, in the fusion process, the deep learning model can be weighted according to the characteristics of different sensor data, so that more accurate pose estimation is obtained, and meanwhile, the robustness of the system under a complex environment can be improved through the fusion of multiple sensor data;
s4, back-end factor graph optimization: and optimizing by using IMU data and GPS data through a back-end factor graph to obtain more accurate pose.
Through the technical scheme, the positioning accuracy and the robustness of the SLAM system of the mobile robot can be improved in a low-texture environment, so that the application range of the robot in a complex environment is expanded; the deep learning model is adopted to extract the dot line characteristics and identify the environment, so that the characteristic quality and the positioning accuracy are improved in a low-texture environment; through multi-sensor data fusion, the robustness of the system in a complex environment is improved, and the real-time estimation of the position and the attitude of the robot is realized; the SLAM system has higher positioning precision and robustness, and can expand the application range of the robot in a complex environment.
Specifically, in the step S3, the multi-sensor data includes laser radar data, visual image data and IMU data, the real-time estimation of the position and the posture of the robot is performed by using a multi-sensor data fusion technology through a SLAM system, the SLAM system performs the posture estimation based on the laser SLAM or VSLAM and the characteristics extracted by a deep learning model, and the SLAM system has higher positioning precision and robustness in a complex environment.
Specifically, the SLAM system is a vision-based SLAM system, and the system captures environmental information by using a camera, performs feature extraction and environmental mapping by combining a computer vision technology, and can improve the performance of the vision SLAM system in a low-texture environment by performing feature extraction by using a deep learning model, such as a Convolutional Neural Network (CNN).
Specifically, the SLAM system is a laser radar-based SLAM system, the system uses the laser radar to perform environment sensing, and uses point cloud data to perform feature extraction and map construction, so that the laser radar has higher precision and robustness, and can achieve a better positioning effect in a complex environment.
Specifically, the SLAM system is an SLAM system based on vision and laser radar fusion, the system combines the advantages of vision and laser radar, multi-sensor data fusion is achieved, feature extraction is carried out through a deep learning model, the performance of the system in a low-texture environment can be improved, and meanwhile, the robustness and the accuracy of the system can be improved due to the addition of the laser radar.
Specifically, the SLAM system is a SLAM system based on a filtering algorithm, and the system realizes the estimation of the position and the posture of the robot by fusing and optimizing the data of multiple sensors through designing the filtering algorithm such as a Kalman filter, a particle filter and the like. The filtering algorithm can improve the robustness and the positioning accuracy of the system to a certain extent.
The visual laser sensor fusion positioning algorithm based on the deep learning is used for carrying out model training by utilizing the deep learning in part of research so as to improve the feature quality, thereby carrying out feature extraction; compared with the traditional method for extracting the dot line features, the dot line features extracted by the model are higher in quality and more in quantity, and the key frame matching effect is better than that of the features extracted by the traditional algorithm; and the environment around the robot is identified through deep learning, so that the current environment is more accurately obtained, which is favorable for a laser sensor or a visual sensor, thereby obtaining the weight of laser data and visual data, extracting the laser visual characteristics with higher quality and finally obtaining the positioning precision of the system.
Finally, it should be noted that: the foregoing description is only illustrative of the preferred embodiments of the present invention, and although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described, or equivalents may be substituted for elements thereof, and any modifications, equivalents, improvements or changes may be made without departing from the spirit and principles of the present invention.
Claims (7)
1. The visual laser sensor fusion positioning algorithm based on deep learning is characterized by comprising the following steps of:
s1, sensor data acquisition: acquiring real-time data through a laser radar, a camera and an IMU sensor on the robot;
s2, feature extraction and environment recognition: performing feature extraction and environment recognition on the preprocessed data by using a deep learning model;
s3, multi-sensor data fusion: according to the feature extraction and the environment recognition result, weighting and fusing the different sensor data to obtain accurate pose estimation;
s4, back-end factor graph optimization: and optimizing by using IMU data and GPS data through a back-end factor graph to obtain an accurate pose.
2. The visual laser sensor fusion positioning algorithm based on deep learning as claimed in claim 1, wherein: in the step S3, the multi-sensor data includes laser radar data, visual image data, and IMU data.
3. The visual laser sensor fusion positioning algorithm based on deep learning as claimed in claim 1, wherein: in the step S3, the real-time estimation of the position and the posture of the robot is performed by using a multi-sensor data fusion technology through a SLAM system, and the SLAM system is based on laser SLAM or VSLAM.
4. A visual laser sensor fusion positioning algorithm based on deep learning as claimed in claim 3, wherein: the SLAM system is a vision-based SLAM system which captures environmental information by using a camera, performs feature extraction and environmental mapping by combining computer vision technology, and performs feature extraction by using a deep learning model such as a convolutional neural network.
5. A visual laser sensor fusion positioning algorithm based on deep learning as claimed in claim 3, wherein: the SLAM system is based on a laser radar, and the system uses the laser radar to perform environment sensing, and performs feature extraction and map construction by utilizing point cloud data.
6. A visual laser sensor fusion positioning algorithm based on deep learning as claimed in claim 3, wherein: the SLAM system is based on vision and laser radar fusion, the system combines the advantages of vision and laser radar, multi-sensor data fusion is achieved, and feature extraction is carried out through a deep learning model.
7. A visual laser sensor fusion positioning algorithm based on deep learning as claimed in claim 3, wherein: the SLAM system is based on a filtering algorithm, and the system fuses and optimizes the multi-sensor data by designing the filtering algorithm so as to realize the estimation of the position and the posture of the robot.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311372884.9A CN117470230A (en) | 2023-10-23 | 2023-10-23 | Visual laser sensor fusion positioning algorithm based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311372884.9A CN117470230A (en) | 2023-10-23 | 2023-10-23 | Visual laser sensor fusion positioning algorithm based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117470230A true CN117470230A (en) | 2024-01-30 |
Family
ID=89624813
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311372884.9A Pending CN117470230A (en) | 2023-10-23 | 2023-10-23 | Visual laser sensor fusion positioning algorithm based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117470230A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114018236A (en) * | 2021-09-30 | 2022-02-08 | 哈尔滨工程大学 | Laser vision strong coupling SLAM method based on adaptive factor graph |
| CN115200588A (en) * | 2022-09-14 | 2022-10-18 | 煤炭科学研究总院有限公司 | SLAM autonomous navigation method and device for mobile robot |
| CN116222543A (en) * | 2023-04-26 | 2023-06-06 | 齐鲁工业大学(山东省科学院) | Multi-sensor fusion map construction method and system for robot environment perception |
| CN116399324A (en) * | 2023-03-24 | 2023-07-07 | 清水湾(深圳)自动驾驶智能研究中心(有限合伙) | Picture construction method and device, controller and unmanned vehicle |
-
2023
- 2023-10-23 CN CN202311372884.9A patent/CN117470230A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114018236A (en) * | 2021-09-30 | 2022-02-08 | 哈尔滨工程大学 | Laser vision strong coupling SLAM method based on adaptive factor graph |
| CN115200588A (en) * | 2022-09-14 | 2022-10-18 | 煤炭科学研究总院有限公司 | SLAM autonomous navigation method and device for mobile robot |
| CN116399324A (en) * | 2023-03-24 | 2023-07-07 | 清水湾(深圳)自动驾驶智能研究中心(有限合伙) | Picture construction method and device, controller and unmanned vehicle |
| CN116222543A (en) * | 2023-04-26 | 2023-06-06 | 齐鲁工业大学(山东省科学院) | Multi-sensor fusion map construction method and system for robot environment perception |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112014857B (en) | Three-dimensional laser radar positioning and navigation method for intelligent inspection and inspection robot | |
| CN109034018B (en) | Low-altitude small unmanned aerial vehicle obstacle sensing method based on binocular vision | |
| CN111626217B (en) | Target detection and tracking method based on two-dimensional picture and three-dimensional point cloud fusion | |
| CN109684921B (en) | Road boundary detection and tracking method based on three-dimensional laser radar | |
| Mühlfellner et al. | Summary maps for lifelong visual localization | |
| CN105825173B (en) | General road and lane detection system and method | |
| CN108230254B (en) | Automatic detection method for high-speed traffic full lane line capable of self-adapting scene switching | |
| CN104520732B (en) | The method and relevant device of alignment sensor | |
| Huang | Review on LiDAR-based SLAM techniques | |
| CN109949375A (en) | A kind of mobile robot method for tracking target based on depth map area-of-interest | |
| CN110487286B (en) | Robot pose judgment method based on point feature projection and laser point cloud fusion | |
| CN103278170A (en) | Mobile robot cascading map building method based on remarkable scenic spot detection | |
| CN113506318B (en) | Three-dimensional target perception method under vehicle-mounted edge scene | |
| CN105160649A (en) | Multi-target tracking method and system based on kernel function unsupervised clustering | |
| CN115032651A (en) | Target detection method based on fusion of laser radar and machine vision | |
| CN114120075A (en) | Three-dimensional target detection method integrating monocular camera and laser radar | |
| Zhou et al. | Asl-slam: A lidar slam with activity semantics-based loop closure | |
| Li et al. | An efficient point cloud place recognition approach based on transformer in dynamic environment | |
| CN114264297A (en) | Positioning and mapping method and system for UWB and visual SLAM fusion algorithm | |
| Fu et al. | Real-time infrared horizon detection in maritime and land environments based on hyper-laplace filter and convolutional neural network | |
| Hu et al. | A simple information fusion method provides the obstacle with saliency labeling as a landmark in robotic mapping | |
| CN113947636B (en) | Laser SLAM positioning system and method based on deep learning | |
| CN116664851A (en) | Automatic driving data extraction method based on artificial intelligence | |
| CN117470230A (en) | Visual laser sensor fusion positioning algorithm based on deep learning | |
| CN105809719A (en) | Object tracking method based on pixel multi-coding-table matching |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |