WO2022021661A1 - Procédé de positionnement visuel basé sur un processus gaussien, système et support de stockage - Google Patents

Procédé de positionnement visuel basé sur un processus gaussien, système et support de stockage Download PDF

Info

Publication number
WO2022021661A1
WO2022021661A1 PCT/CN2020/127187 CN2020127187W WO2022021661A1 WO 2022021661 A1 WO2022021661 A1 WO 2022021661A1 CN 2020127187 W CN2020127187 W CN 2020127187W WO 2022021661 A1 WO2022021661 A1 WO 2022021661A1
Authority
WO
WIPO (PCT)
Prior art keywords
gaussian process
position point
trajectory
features
visual positioning
Prior art date
Application number
PCT/CN2020/127187
Other languages
English (en)
Chinese (zh)
Inventor
李坚强
陈杰
李钦坚
胡书卿
梁中明
Original Assignee
深圳大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳大学 filed Critical 深圳大学
Publication of WO2022021661A1 publication Critical patent/WO2022021661A1/fr
Priority to US17/666,550 priority Critical patent/US20220168900A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G05D1/0253—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1628—Programme controls characterised by the control loop
    • B25J9/163—Programme controls characterised by the control loop learning, adaptive, model based, rule based expert control
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0088—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/207—Analysis of motion for motion estimation over a hierarchy of resolutions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/17—Terrestrial scenes taken from planes or by drones
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20024—Filtering details
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Definitions

  • the present invention relates to the technical field of artificial intelligence positioning, in particular to a visual positioning method, system and storage medium based on a Gaussian process.
  • Positioning is the most basic problem of robot navigation. In an unknown environment, there is a large error in the positioning of the robot itself, so it cannot provide navigation for the robot according to the positioning of the robot itself. Therefore, the positioning of the robot needs to be optimized. There are two main optimization frameworks:
  • Beam adjustment is a mature visual positioning theory. It has the ability to optimize the entire trajectory, but it is easy to lose the tracking of the robot's moving trajectory, and when the robot is outdoors, it changes. There is a large positioning error in a noisy environment.
  • the other is the optimization framework based on the filtering method, but this method can only estimate the latest state of the robot, but the current state of the robot and the historical state are interrelated.
  • the latest state is bound to reduce the accuracy of positioning.
  • the technical problem to be solved by the present invention is to provide a visual positioning method, system and storage medium based on a Gaussian process for the above-mentioned defects of the prior art, aiming to solve the problem of the optimization framework for optimizing robot positioning in the prior art.
  • the problem of poor positioning accuracy is to provide a visual positioning method, system and storage medium based on a Gaussian process for the above-mentioned defects of the prior art, aiming to solve the problem of the optimization framework for optimizing robot positioning in the prior art.
  • the problem of poor positioning accuracy is to provide a visual positioning method, system and storage medium based on a Gaussian process for the above-mentioned defects of the prior art, aiming to solve the problem of the optimization framework for optimizing robot positioning in the prior art.
  • the problem of poor positioning accuracy is to provide a visual positioning method, system and storage medium based on a Gaussian process for the above-mentioned defects of the prior art, aiming to solve the problem of the optimization framework for optimizing robot positioning in
  • a visual localization method based on a Gaussian process including:
  • the Bayesian filtering frame is reconstructed, and the current trajectory is given an initial position point, and the next position point of the current trajectory is generated through the reconstructed Bayesian filtering frame, and the next position point is used for navigation Provide positioning guidance.
  • Bayesian filter frame is reconstructed according to the Gaussian process expression, and an initial position point is given to the current track, and the next position point of the current track is generated by the reconstructed Bayesian filter frame, specifically including:
  • An initial position point is given to the current trajectory, and the next position point of the current trajectory is predicted according to the initial position point in the reconstructed Bayesian filtering framework;
  • the observation model is applied to the predicted next position point to correct the position, and the corrected predicted position point is obtained.
  • the methods for extracting the global features and semantic features in the collected image information are:
  • the highest probability value of different categories of things in each collected image is semantically extracted by the CenterNet algorithm.
  • the extracted global features and semantic features and the moving trajectory points are processed to obtain a Gaussian process expression, which specifically includes:
  • a dimension is established for the highest probability value of semantic extraction
  • the corresponding relationship between the eigenvalue matrix and the collected trajectory points is established, and the training is performed, and the Gaussian process expression representing the spatial relationship between the eigenvalues and the trajectory points is output.
  • the processing results can be applied to a Gaussian process, which is beneficial to Gaussian modeling.
  • a Gaussian process expression representing the spatial relationship between the eigenvalues and the trajectory points, specifically including:
  • the collection of image information of the surrounding environment on the way and the moving track points specifically include:
  • the track points that move on the way are located by GPS, and the track points located by GPS are randomly collected.
  • Using a camera for shooting can reduce costs while ensuring that the obtained image information is maximized, and the recognizable rate of the surrounding environment is improved.
  • the initial position point of the current track is the first position point of the current track positioned by GPS.
  • the present invention also discloses a system comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors to execute the one or more programs
  • the above program contains methods for performing the Gaussian process based visual localization method as described above.
  • the present invention also discloses a storage medium, wherein the storage medium stores a computer program, and the computer program can be executed to realize the above-mentioned Gaussian process-based visual positioning method.
  • the present invention provides a Gaussian process-based visual positioning method, system and storage medium, wherein the method includes: collecting image information of the surrounding environment and moving track points on the way;
  • Extracting the global features and semantic features in the collected image information processing the extracted global features and semantic features and the moving trajectory points according to preset processing rules to obtain a Gaussian process expression; reconstructing according to the Gaussian process expression Bayesian filtering framework, and assigning an initial position point to the current trajectory, and generating the next position point of the current trajectory through the reconstructed Bayesian filtering framework.
  • the association between the current state and the historical state can be established, thereby improving the accuracy of the predicted next position point and providing accurate navigation for the robot movement.
  • FIG. 1 is a flowchart of a preferred embodiment of a Gaussian process-based visual positioning method in the present invention.
  • FIG. 2 is a schematic diagram of the original trajectory of the prior art when using unmanned vehicles and drones to operate in an unknown environment.
  • FIG. 3 is a numerical schematic diagram of the operation result of the unmanned vehicle in the present invention.
  • FIG. 4 is a schematic diagram of the running track of the unmanned vehicle in the present invention.
  • FIG. 5 is a numerical schematic diagram of the operation result of the UAV in the present invention.
  • FIG. 6 is a schematic diagram of the running track of the UAV in the present invention.
  • FIG. 7 is a comparison diagram of effects when filtering different features in the present invention.
  • FIG. 8 is a schematic diagram of a preferred embodiment of numerical processing of observation values in the present invention.
  • FIG. 9 is a flowchart of a preferred embodiment of the manner in which the global feature, the semantic feature and the randomly adopted trajectory point are processed in step S300 of FIG. 1 .
  • FIG. 10 is a flowchart of a preferred embodiment of predicting a position point by using the reconstructed Bayesian filtering framework and correcting the predicted position point in step S400 of FIG. 1 .
  • Figure 11 is a traditional Bayesian filtering framework.
  • Fig. 12 is the Bayesian filtering framework after reconstruction in the present invention.
  • FIG. 13 is a functional principle block diagram of a preferred embodiment of the system in the present invention.
  • FIG. 14 is a functional principle block diagram of a specific embodiment of the system in the present invention.
  • SLAM Simultaneous localization and mapping
  • the robot starts to move from an unknown position in an unknown environment, and the robot moves from an unknown position according to the position estimation and the moving process.
  • the map performs its own positioning, and at the same time builds an incremental map on the basis of its own positioning to realize the autonomous positioning and navigation of the robot.
  • the SLAM algorithm only considers how the robot uses the information it obtains to construct a map of the environment, and locates the robot in the map. It does not consider the problem of how the robot can effectively detect the unknown environment, and cannot be used for the robot's path planning.
  • ORBSLAM is a real-time monocular SLAM system based on feature points, which can be run in large-scale, small-scale, indoor and outdoor environments.
  • the modules include: Tracking, Mapping, Loop closing, the principle is: by extracting ORB features from the image, performing pose estimation based on the previous frame, or initializing the pose by global relocation, and then tracking The local map has been reconstructed, and the pose is optimized.
  • the ORBSLAM system will still produce large errors due to various noises in the outdoor environment.
  • the other is the filtering method, which is computationally efficient but only estimates the latest robot pose.
  • the Bayes filter uses the motion model to estimate the trajectory and the observation model to correct the error. Given the current position of the robot, the observation model usually has Markov characteristics, and its current measurement is conditionally independent of the past measurement, which greatly reduces the localization performance of the robot due to the spatially correlated structure of the environment.
  • the above two commonly used robot positioning optimization frameworks still have drawbacks in specific applications. If the position prediction of the robot is not accurate enough, there will be certain errors, and the errors will accumulate, resulting in the subsequent predicted running trajectory and actual trajectory. There is a big deviation. Therefore, the present invention improves the problem of inaccurate positioning of the existing positioning optimization framework, so as to realize precise positioning of the robot in an unknown environment.
  • FIG. 1 is a flowchart of a Gaussian process-based visual positioning method in the present invention.
  • a Gaussian process-based visual positioning method according to an embodiment of the present invention includes the following steps:
  • S100 Collect image information of the surrounding environment and moving track points on the way.
  • Cameras can be used for shooting, or lidar can be used for detection.
  • the above two methods can be more accurate when used for outdoor environment detection.
  • lidar is a radar system that emits a laser beam to detect the position, speed and other characteristic quantities of the target.
  • the working principle is: send a detection signal (laser beam) to the target, and then compare the received signal (target echo) reflected from the target with the transmitted signal, and after proper processing, the relevant information of the target can be obtained, such as Target distance, azimuth, altitude, speed, attitude, and even shape and other parameters, so as to detect, track and identify targets such as aircraft and missiles.
  • Lidar ranging has the characteristics of long detection distance, high measurement accuracy and high angular resolution, and can more accurately perceive the surrounding environment, but due to its high cost, it will also increase the cost of robots, which is not conducive to commercial promotion.
  • a camera is used in this application to sense the surrounding environment. Compared with lidar, the price of the camera is cheaper, and it can be widely used, which is beneficial to the development of robot technology, and the visual information obtained by the camera contains rich visual information. Structural information, which is conducive to building maps and extracting semantic information.
  • collecting the image information of the surrounding environment on the way is: enabling the camera to capture the image of the surrounding environment on the way.
  • the structure of the robot can be improved, the camera can be loaded in the robot, or the structure of the robot can not be improved, the vehicle-mounted camera can be installed on the robot, or the aerial vehicle can be used for synchronous shooting, or the camera and the camera can be made in other ways.
  • the robot moves synchronously to take pictures of the robot's surroundings. It is understandable that there are various ways of using the camera to photograph the surrounding environment of the robot while traveling, and examples are not given here. Any method that can simultaneously photograph the surrounding environment of the robot can be used in this embodiment.
  • the method of collecting the track points moving on the way is: locating the track points moving during the traveling by GPS, and randomly collecting the track points located by the GPS.
  • a sensing device is installed on the robot, so that the real trajectory points of the robot's motion can be accurately located and randomly sampled from the real trajectory points.
  • the visual positioning systems of outdoor mobile robots mainly include ORBSLAM system and VINS system. Both use local features as observations to demonstrate the observations of the image at the moment, where local features refer to features extracted from local areas of the image, including edges, corners, lines, curves, and special features. property area, etc.
  • local features refer to features extracted from local areas of the image, including edges, corners, lines, curves, and special features. property area, etc.
  • Fig. 2 is a comparison diagram of the trajectory predicted in the prior art and the trajectory predicted in the present invention
  • Fig. 2 is the use of unmanned vehicles (a) (golf carts) in the prior art respectively. ) and UAV (b) (UAV) operating in an unknown environment, where the original trajectory of the unmanned vehicle (a) is (c) (ground truth of golf cart), the UAV (b) The original running trajectory is (d)(ground truth of UAV);
  • Fig. 3 is the numerical value of the experimental result of the unmanned vehicle
  • Fig. 4 is the experimental result of the unmanned vehicle
  • Figure 5 shows the experimental results of the UAV
  • Figure 6 shows the experimental results of the UAV, in which (a) (blind motion) is running in an unknown environment, (b) (semantic) is extracting semantics feature, (c) (global visual) is to extract visual features; (d) (global visual and semantic) is to combine semantic features and visual features.
  • the present application uses global features as observation values, where the global features refer to the overall attributes of the image, and common global features include color features, texture features, and shape features, such as intensity histograms. Global features have the characteristics of good invariance, simple calculation, and intuitive representation. It can better represent the correlation between adjacent positions.
  • the method of extracting the global feature as the observation value is: extracting the dimension of the global feature in the collected image information through the Steerable Pyramid algorithm.
  • the Steerable Pyramid algorithm is used to decompose the image into sub-band information of different scales and different directions. Through the decomposed sub-band images, we can observe the features we need at different scales and different directions.
  • this application also combines semantic features as observation values.
  • Semantic features are also a feature representation method that has been continuously developed with the development of deep learning in recent years. Semantics can often help us better understand images and allow robots The localization is more intelligent, and at the same time, the correlation of semantic information between adjacent locations is also great.
  • the method of extracting semantic features as observation values is: semantically extracting the highest probability values of different categories of things in each collected picture through the CenterNet algorithm. Specifically, classify the things on each picture, and find the one with the highest probability value that can be recognized among the same type of things.
  • the algorithm based on CenterNet is to predict the position of the upper left and lower right corners. Predict two heatmaps (two foot points), offset (map from heatmap to original image), and embedding (whether the two foot points are the same object) each time.
  • S300 Process the extracted global features, semantic features, and moving track points according to preset processing rules to obtain a Gaussian process expression.
  • the Gaussian process refers to: if the finite-dimensional distribution of the random process is normal distribution, the random process is called a Gaussian process or a normal process.
  • the relationship between the current state and the historical state of the robot can be established.
  • the adjacent positions represented by the global feature and the semantic feature have a large correlation, which is conducive to the construction of a Gaussian process, thereby forming a normal distribution between the observed value and the randomly sampled trajectory points.
  • the above-mentioned selection of the observed values and the processing of the observed values can be done in offline processing.
  • the specific method is to digitize the observed values, as shown in Figure 8.
  • the offline processing of the observed values does not occupy the system memory. Can improve system speed.
  • the method of processing the global feature, the semantic feature and the randomly adopted trajectory point is as follows:
  • the global features in the image are extracted by the open source Steerable Pyramid algorithm, and the global features are compressed.
  • the processing method is described here with experimental data: after the global features are extracted, 72-dimensional global features are extracted, However, 4*18 dimensions are repeated in 72 dimensions, so the original 72-dimensional global features are compressed into non-repeating 18-dimensional global features.
  • the highest probability value that can be identified by each category in each picture is extracted as the observation value. For example, there is only one probability value for a single category in each picture, so only one probability value of this category is extracted as the observation value; if There are multiple objects in the same category in each picture, and the probability value of the object with the highest recognition probability in the same category in different pictures is extracted as the observation value. After that, each extracted category is used as a dimension to complete the processing of semantic features.
  • S330 combine the dimensions corresponding to the extracted global features and semantic features into an eigenvalue matrix; wherein, the dimension of the row represents the feature category in the global feature or the category of things in the semantic feature, and the dimension of the column represents the estimated point or each frame. picture.
  • the formed eigenvalue matrix is an N*18 matrix
  • the current trajectory point forms an N*3 matrix, where N represents the number of pictures/number of positions.
  • the trajectory point is taken as the X value
  • the observed value is taken as the Y value
  • the observed value is the eigenvalue value after the semantic feature and the global feature are combined.
  • a series of covariance functions and mean square errors (including kernel functions) that represent the correlation between spatial structures, etc., as expressions of Gaussian processes, are called GP Models.
  • the relationship between each trajectory point and the observation value can be obtained, that is, the current observation value is known, and the position of the current trajectory point corresponding to the current observation value can be calculated through the trained model.
  • This estimation process is represented by a Gaussian kernel function.
  • the purpose of training the Gaussian process through the real sampled trajectory points and eigenvalues is to establish the relationship between the trajectory points and the observed values. Therefore, only one sampling is required, and the trajectory points are randomly selected to perform a Gaussian process. Process training, and then obtain a model, which can be used in similar scenarios, thereby reducing the need for frequent modeling and saving data processing procedures.
  • the principle of Bayesian filtering is to estimate the posture of the robot according to the probability state.
  • the Bayesian filter is a probability state estimation framework, as shown in Figure 11, which is a traditional Bayesian filtering framework.
  • the belief in the position/state of the robot is repeatedly updated. After the robot performs a series of control actions, the robot obtains some observations and corresponding control information, and then the current position of the robot can be obtained by The probability distribution is estimated, and the probability value (belief) that needs to be calculated is called belief.
  • the framework of Bayesian filtering is composed of a motion model and an observation model. The Bayesian filtering process is usually divided into two steps, that is, the position is estimated by the motion model, and the estimated position is corrected by the observation model.
  • the present invention introduces a Gaussian process with spatial correlation to reconstruct the Bayesian filtering framework, as shown in FIG. 12 , which is the reconstructed Bayesian filter in the present invention.
  • the filtering frame the part enclosed by the frame is added in the figure, is used to represent the modeling of the spatiotemporal correlation, which can represent the correlation between the current observation value of the robot and the past observation and control information, thereby improving the accuracy of the robot positioning.
  • the Gaussian process expression is introduced into GP-Localize, wherein, the GP-Localize algorithm can perform on-site measurements related to space during the movement of the robot.
  • GP-Localize can solve the Markov assumption, reduce the memory and time complexity of the cubic caused by the Gaussian process to calculate the inverse matrix of the covariance, and then realize the time and memory of the constant term.
  • the invention can maintain the time and memory complexity of GP-Localize constant item level by using Bayesian filtering and Gaussian process-based GP-Localize algorithm to extend to visual high-dimensional data, and is very suitable for real-time robot positioning and installation.
  • the sensor device of the camera is conducive to further promotion and commercialization.
  • the Bayesian filtering framework is reconstructed, and a real initial position point is given to the current trajectory, wherein the initial position point of the current trajectory is the first position of the current trajectory positioned by GPS. point, the first position point can be obtained by calculating the average value of repeated sampling for many times. Then, by entering other adjustment parameters, the Bayesian filtering process can be performed, and then all the position points on the current trajectory can be continuously iterated, so that the robot can provide accurate navigation for its own motion. It can be understood that it is the prior art to realize the prediction of the position point through Bayesian filtering, and details are not repeated here.
  • the steps of predicting the position point through the reconstructed Bayesian filtering framework and revising the predicted position point include:
  • the distribution is updated by Bayesian formula to obtain the posterior distribution:
  • the present invention obtains the process of GP-Localize by introducing a Gaussian process into the Bayesian framework:
  • X be a set of positions, and zx or Zx are associated, each position stands for observation if x is observed or not observed, although not observed also denote a GP, each finite A subset of has a multivariate Gaussian distribution.
  • the distribution of unobserved observations using Gaussian prediction is expressed as:
  • GP-Localize can achieve constant time and memory, ideal for persistent robot localization.
  • the correction expression is : Among them, p(z t
  • Bayesian filtering not only visual information is used, but also the positioning sensor of the robot itself is used for positioning.
  • the Gaussian process is introduced into GP-Localize to establish an observation model, and then the robot can be accurately navigated by drawing a map.
  • the global feature and the semantic feature the present invention not only facilitates the processing of the Gaussian process, but also the semantic feature can be used for later semantic segmentation and recognition, which can be better used for robot positioning and has strong practicability.
  • step S430 it includes:
  • the present invention also discloses a system, as shown in FIG. 13 , which includes a memory 20 and one or more programs, wherein one or more programs are stored in the memory 20 and configured to be composed of one or more programs.
  • the execution of the one or more programs by the processor 10 includes performing the Gaussian process-based visual positioning method as described above; specifically as described above.
  • the system of the present invention executes the following steps during operation:
  • the first step select the observation value (observation choise): select the semantic feature (steerable pyramid) and the global feature (centernet) of the surrounding environment, and then use the trajectory point as x (loc: X), and the observation value as y (obs: Y) to get the feature matrix.
  • Step 2 Perform Gaussian process transformation (Date Processing), input the feature matrix into the Gaussian model (GP Model), and combine with the initial position point (Initial GPS) of the actual positioning, by inputting a series of parameters (parameters, slice, block- way, etc.) to get the Gaussian process expression.
  • GP Model Gaussian model
  • Initial position point Initial GPS
  • the third step is to substitute the obtained Gaussian process expression into the Bayesian framework for trajectory prediction, in which the motion model is used for trajectory prediction, and the observation model is used for trajectory correction, wherein , the observation model is created online via a Gaussian process expression (online global-semantic gaussian process).
  • the present invention also discloses a storage medium, wherein the storage medium stores a computer program, and the computer program can be executed to implement the above-mentioned Gaussian process-based visual positioning method; the details are as described above.
  • the present invention discloses a Gaussian process-based visual positioning method, system and storage medium, wherein the method includes: collecting image information of the surrounding environment and moving track points on the way; extracting the collected image information The global features and semantic features in the algorithm; the extracted global features and semantic features, and the moving trajectory points are processed according to the preset processing rules to obtain a Gaussian process expression; the Bayesian filtering framework is reconstructed according to the Gaussian process expression. , and give the current trajectory an initial position point, and generate the next position point of the current trajectory through the reconstructed Bayesian filtering framework. The association between the current state and the historical state can be established, thereby improving the accuracy of the predicted next position point and providing accurate navigation for the robot movement.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Remote Sensing (AREA)
  • Mathematical Analysis (AREA)
  • Algebra (AREA)
  • Mathematical Optimization (AREA)
  • Computational Mathematics (AREA)
  • Medical Informatics (AREA)
  • Health & Medical Sciences (AREA)
  • Pure & Applied Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Probability & Statistics with Applications (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Electromagnetism (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Business, Economics & Management (AREA)
  • Game Theory and Decision Science (AREA)
  • Image Analysis (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

La présente invention concerne un procédé de positionnement visuel basé sur un processus gaussien, un système et un support de stockage. Le procédé comprend les étapes consistant à acquérir des informations d'image d'un environnement ambiant et à déplacer des points de piste dans un processus de déplacement (S100); à extraire des caractéristiques globales et des caractéristiques sémantiques dans les informations d'image acquises (S200); à traiter les caractéristiques globales et les caractéristiques sémantiques extraites et les points de piste en mouvement selon une règle de traitement prédéfinie pour obtenir une expression de processus gaussien (S300); et à reconstruire une structure de filtrage bayésien en fonction de l'expression de processus gaussien, conférant à la piste en cours un point de position initial, et à générer le point de position suivant de la piste en cours au moyen de la structure de filtrage bayésien reconstruite, le point de position suivant étant utilisé pour fournir un guidage de positionnement pour la navigation (S400). L'association entre l'état actuel et un état historique peut être établie, ainsi la précision du point de position suivant prédit est améliorée, et une navigation précise peut être fournie pour le mouvement d'un robot.
PCT/CN2020/127187 2020-07-27 2020-11-06 Procédé de positionnement visuel basé sur un processus gaussien, système et support de stockage WO2022021661A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/666,550 US20220168900A1 (en) 2020-07-27 2022-02-07 Visual positioning method and system based on gaussian process, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010731969.1A CN111739066B (zh) 2020-07-27 2020-07-27 一种基于高斯过程的视觉定位方法、系统及存储介质
CN202010731969.1 2020-07-27

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/666,550 Continuation US20220168900A1 (en) 2020-07-27 2022-02-07 Visual positioning method and system based on gaussian process, and storage medium

Publications (1)

Publication Number Publication Date
WO2022021661A1 true WO2022021661A1 (fr) 2022-02-03

Family

ID=72657932

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/127187 WO2022021661A1 (fr) 2020-07-27 2020-11-06 Procédé de positionnement visuel basé sur un processus gaussien, système et support de stockage

Country Status (3)

Country Link
US (1) US20220168900A1 (fr)
CN (1) CN111739066B (fr)
WO (1) WO2022021661A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111739066B (zh) * 2020-07-27 2020-12-22 深圳大学 一种基于高斯过程的视觉定位方法、系统及存储介质
CN112767476B (zh) * 2020-12-08 2024-04-26 中国科学院深圳先进技术研究院 一种快速定位系统、方法及应用
CN114897929A (zh) * 2022-05-31 2022-08-12 工业云制造(四川)创新中心有限公司 基于视觉降噪的机器人运动方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150316383A1 (en) * 2012-12-03 2015-11-05 Navisens, Inc. Systems and methods for estimating the motion of an object
CN105509755A (zh) * 2015-11-27 2016-04-20 重庆邮电大学 一种基于高斯分布的移动机器人同步定位与地图构建方法
CN108256572A (zh) * 2018-01-16 2018-07-06 哈尔滨职业技术学院 一种基于改进朴素贝叶斯的室内视觉特征分类方法
CN110304075A (zh) * 2019-07-04 2019-10-08 清华大学 基于混合动态贝叶斯网络和高斯过程的车辆轨迹预测方法
CN110751671A (zh) * 2018-07-23 2020-02-04 中国科学院长春光学精密机械与物理研究所 一种基于核相关滤波与运动估计的目标跟踪方法
CN111340939A (zh) * 2020-02-21 2020-06-26 广东工业大学 一种室内三维语义地图构建方法
CN111739066A (zh) * 2020-07-27 2020-10-02 深圳大学 一种基于高斯过程的视觉定位方法、系统及存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050246307A1 (en) * 2004-03-26 2005-11-03 Datamat Systems Research, Inc. Computerized modeling method and a computer program product employing a hybrid Bayesian decision tree for classification
CN107833241A (zh) * 2017-10-20 2018-03-23 东华大学 对环境光照变化具有鲁棒性的实时视觉目标检测方法
US11040729B2 (en) * 2018-05-31 2021-06-22 Nissan North America, Inc. Probabilistic object tracking and prediction framework
CN109903338A (zh) * 2019-03-14 2019-06-18 中国计量大学 一种基于改进orb算法的移动机器人定位方法
CN110558977A (zh) * 2019-09-09 2019-12-13 西北大学 基于机器学习模糊特征选择的癫痫发作脑电信号分类方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150316383A1 (en) * 2012-12-03 2015-11-05 Navisens, Inc. Systems and methods for estimating the motion of an object
CN105509755A (zh) * 2015-11-27 2016-04-20 重庆邮电大学 一种基于高斯分布的移动机器人同步定位与地图构建方法
CN108256572A (zh) * 2018-01-16 2018-07-06 哈尔滨职业技术学院 一种基于改进朴素贝叶斯的室内视觉特征分类方法
CN110751671A (zh) * 2018-07-23 2020-02-04 中国科学院长春光学精密机械与物理研究所 一种基于核相关滤波与运动估计的目标跟踪方法
CN110304075A (zh) * 2019-07-04 2019-10-08 清华大学 基于混合动态贝叶斯网络和高斯过程的车辆轨迹预测方法
CN111340939A (zh) * 2020-02-21 2020-06-26 广东工业大学 一种室内三维语义地图构建方法
CN111739066A (zh) * 2020-07-27 2020-10-02 深圳大学 一种基于高斯过程的视觉定位方法、系统及存储介质

Also Published As

Publication number Publication date
US20220168900A1 (en) 2022-06-02
CN111739066B (zh) 2020-12-22
CN111739066A (zh) 2020-10-02

Similar Documents

Publication Publication Date Title
Li et al. Building and optimization of 3D semantic map based on Lidar and camera fusion
Shin et al. Roarnet: A robust 3d object detection based on region approximation refinement
Taheri et al. SLAM; definition and evolution
CN110335337B (zh) 一种基于端到端半监督生成对抗网络的视觉里程计的方法
WO2022021661A1 (fr) Procédé de positionnement visuel basé sur un processus gaussien, système et support de stockage
Li et al. Deep sensor fusion between 2D laser scanner and IMU for mobile robot localization
Wang et al. Pointloc: Deep pose regressor for lidar point cloud localization
Kwon et al. Visual graph memory with unsupervised representation for visual navigation
Chen et al. Milestones in autonomous driving and intelligent vehicles—part ii: Perception and planning
Cheng et al. Improving visual localization accuracy in dynamic environments based on dynamic region removal
CN113568435B (zh) 一种基于无人机自主飞行态势感知趋势的分析方法与系统
CN113674416A (zh) 三维地图的构建方法、装置、电子设备及存储介质
CN114998276B (zh) 一种基于三维点云的机器人动态障碍物实时检测方法
Huang et al. Fish tracking and segmentation from stereo videos on the wild sea surface for electronic monitoring of rail fishing
CN115731268A (zh) 基于视觉/毫米波雷达信息融合的无人机多目标跟踪方法
Yin et al. Radar-on-lidar: metric radar localization on prior lidar maps
CN111812978B (zh) 一种多无人机协作slam方法与系统
Zhao et al. Dynamic object tracking for self-driving cars using monocular camera and lidar
Shamsfakhr et al. GSR: Geometrical scan registration algorithm for robust and fast robot pose estimation
Chen et al. I2D-Loc: Camera localization via image to lidar depth flow
Liu et al. Hybrid metric-feature mapping based on camera and Lidar sensor fusion
Xiong et al. Multi-uncertainty captured multi-robot lidar odometry and mapping framework for large-scale environments
Jaenal et al. Improving visual slam in car-navigated urban environments with appearance maps
Deng et al. Improved closed-loop detection and Octomap algorithm based on RGB-D SLAM
Spampinato et al. Deep learning localization with 2D range scanner

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20946800

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 16.05.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20946800

Country of ref document: EP

Kind code of ref document: A1