WO2022012229A1 - 机器人的重定位方法及装置、存储介质、电子设备 - Google Patents

机器人的重定位方法及装置、存储介质、电子设备 Download PDF

Info

Publication number
WO2022012229A1
WO2022012229A1 PCT/CN2021/098981 CN2021098981W WO2022012229A1 WO 2022012229 A1 WO2022012229 A1 WO 2022012229A1 CN 2021098981 W CN2021098981 W CN 2021098981W WO 2022012229 A1 WO2022012229 A1 WO 2022012229A1
Authority
WO
WIPO (PCT)
Prior art keywords
robot
matching result
pose
data
initial value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/098981
Other languages
English (en)
French (fr)
Inventor
姚秀军
马福强
王超
桂晨光
崔丽华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jingdong Technology Information Technology Co Ltd
Original Assignee
Jingdong Technology Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jingdong Technology Information Technology Co Ltd filed Critical Jingdong Technology Information Technology Co Ltd
Priority to EP21842606.2A priority Critical patent/EP4160474A4/en
Priority to US18/005,362 priority patent/US20230266470A1/en
Publication of WO2022012229A1 publication Critical patent/WO2022012229A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/005—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 with correlation of navigation data from several sources, e.g. map or contour matching
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G01C21/1652—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments with ranging devices, e.g. LIDAR or RADAR
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G01C21/1656—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments with passive imaging devices, e.g. cameras
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/42—Simultaneous measurement of distance and other co-ordinates
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/50—Systems of measurement based on relative movement of target
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/50—Systems of measurement based on relative movement of target
    • G01S17/58—Velocity or trajectory determination systems; Sense-of-movement determination systems
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
    • G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/4808—Evaluating distance, position or velocity data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches

Definitions

  • the application date is July 15, 2020
  • the application number is 202010680255.2
  • the invention title is "Robot Relocation Method and Device, Storage Medium, Electronic Device” as priority.
  • Embodiments of the present disclosure relate to the field of robotics, and in particular, to a method for relocating a robot, a device for relocating a robot, a computer-readable storage medium, and an electronic device.
  • localization methods based on vision and laser SLAM can use environmental information for active localization without the need to modify the environment.
  • a method for relocating a robot comprising:
  • the pose of the robot is solved according to each of the retrieval results to obtain the initial value of the first pose, and the laser point cloud data is analyzed in a preset two-dimensional grid map according to the speed data and displacement data. perform matching to obtain the first matching result;
  • a second matching result is generated according to the initial value of the first pose and the first matching result
  • a second matching result is generated according to the initial value of the second pose , the first matching result and the second matching result to reposition the robot.
  • performing feature encoding on the local visual features of the image data to obtain image vector features includes:
  • the image vector feature is obtained by connecting the residual and the vector according to the index order of the visual word in the preset visual dictionary.
  • the image vector features after dimension reduction processing are retrieved in a preset key frame database, and multiple retrieval results obtained include:
  • an image vector corresponding to the cluster center is obtained according to the index of the cluster center, and a plurality of the retrieval results are obtained according to a plurality of the image vectors.
  • the pose of the robot is solved according to each of the retrieval results, and obtaining the initial value of the first pose includes:
  • the pose of the robot is solved according to each of the matching features and the matching relationship between the matching features, so as to obtain a first initial value of the pose.
  • matching the laser point cloud data in a preset two-dimensional grid map according to the velocity data and the displacement data, and obtaining a first matching result includes:
  • the velocity data and the displacement data are fused to obtain the second initial value of the pose
  • the laser point cloud data is matched in a preset two-dimensional grid map based on the initial value of the second pose to obtain a first matching result.
  • repositioning the robot according to the initial value of the second pose, the first matching result, and the second matching result includes:
  • the robot is relocated according to the weighted summation.
  • the relocation method of the robot further includes:
  • the preset condition includes at least one of the following:
  • the difference between the speed data and the speed data at the previous moment corresponding to the speed data is greater than a preset speed threshold
  • the difference between the displacement data and the displacement data at the previous moment corresponding to the displacement data is greater than a preset displacement threshold
  • the overlapping portion between the image data and the image data of the previous frame corresponding to the image data is smaller than a preset overlapping threshold.
  • a repositioning device for a robot comprising:
  • the data acquisition module is used to acquire the image data of the robot collected by the camera device, the speed data and displacement data of the robot collected by the inertial measurement device and the displacement measurement device, and the laser point cloud of the robot collected by the lidar data;
  • a feature encoding module configured to perform feature encoding on the local visual features of the image data to obtain image vector features, and retrieve the dimension-reduced image vector features in a preset key frame database to obtain multiple retrieval results;
  • the data matching module is used to solve the pose of the robot according to each of the retrieval results, obtain the initial value of the first pose, and match it in the preset two-dimensional grid map according to the speed data and the displacement data.
  • the laser point cloud data is matched to obtain a first matching result;
  • a relocation module configured to generate a second matching result according to the initial value of the first pose and the first matching result when judging that the initial value of the first pose and the first matching result are inconsistent, and generate a second matching result according to the initial value of the first pose and the first matching result, The second pose initial value, the first matching result, and the second matching result reposition the robot.
  • a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements any one of the above-mentioned methods for relocating a robot.
  • an electronic device comprising:
  • a memory for storing executable instructions for the processor
  • the processor is configured to execute the relocation method of the robot described in any one of the above by executing the executable instructions.
  • FIG. 1 schematically shows a flowchart of a method for relocating a robot according to an exemplary embodiment of the present disclosure.
  • FIG. 2 schematically shows a structural example diagram of a robot according to an exemplary embodiment of the present disclosure.
  • FIG. 3 schematically shows a flowchart of a method for constructing a two-dimensional grid map according to an exemplary embodiment of the present disclosure.
  • FIG. 4 schematically shows a flowchart of a method for obtaining image vector features by feature encoding local visual features of image data according to an exemplary embodiment of the present disclosure.
  • FIG. 5 schematically shows a flowchart of a method for retrieving image vector features after dimension reduction processing in a preset key frame database to obtain multiple retrieval results according to an exemplary embodiment of the present disclosure.
  • FIG. 6 schematically shows a flowchart of a method for relocating the robot according to the initial value of the second pose, the first matching result and the second matching result according to an exemplary embodiment of the present disclosure.
  • FIG. 7 schematically shows a flowchart of another method for relocating a robot according to an exemplary embodiment of the present disclosure.
  • FIG. 8 schematically shows a block diagram of a repositioning apparatus of a robot according to an exemplary embodiment of the present disclosure.
  • FIG. 9 schematically shows an electronic device for implementing the above-mentioned relocation method of a robot according to an exemplary embodiment of the present disclosure.
  • Example embodiments will now be described more fully with reference to the accompanying drawings.
  • Example embodiments can be embodied in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
  • the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
  • Navigation and positioning technology is one of the key technologies of indoor robot system.
  • the commonly used positioning technologies mainly include Outside-In (outside-in positioning) and Inside-Out (inside-out positioning).
  • the positioning methods of Outside-In include the visual positioning method based on April tag (visual reference library) code and the positioning method based on visible light communication;
  • the methods of Inside-Out include vision and laser SLAM (simultaneous localization and mapping, simultaneous positioning and construction Fig.) positioning method, etc.
  • the positioning method of Outside-In requires pre-setting prior information in the use environment, and transforms the work environment to a certain extent, which limits the expansion of use scenarios; on the other hand, Inside The positioning method of -Out can use environmental information for active positioning, and does not need to transform the environment.
  • the positioning robustness of traditional positioning algorithms in complex and unstructured scenarios cannot meet the application requirements, such as : Relocation problem in repeated scenes; localization accuracy problem in unstructured scene; localization robustness problem in open scene, etc.
  • This exemplary embodiment first provides a method for relocating a robot, which can be run on a robot; of course, those skilled in the art can also run the method of the present disclosure on other platforms according to requirements, and in this exemplary embodiment, this No special restrictions are made.
  • the relocation method of the robot may include the following steps:
  • Step S110 Obtain the image data of the robot collected by the camera device, the speed data and displacement data of the robot collected by the inertial measurement device and the displacement measurement device, and the laser point cloud data of the robot collected by the lidar;
  • Step S120 Perform feature encoding on the local visual features of the image data to obtain image vector features, and retrieve the dimension-reduced image vector features in a preset key frame database to obtain multiple retrieval results;
  • Step S130 Solve the pose of the robot according to each of the retrieval results, obtain the initial value of the first pose, and analyze the laser in a preset two-dimensional grid map according to the speed data and displacement data. The point cloud data is matched to obtain the first matching result;
  • Step S140 When judging that the initial value of the first pose and the first matching result are inconsistent, generate a second matching result according to the initial value of the first pose and the first matching result, and generate a second matching result according to the second matching result.
  • the initial value of the pose, the first matching result, and the second matching result reposition the robot.
  • the image vector features are obtained by feature encoding the local visual features of the image data, and the dimension-reduced image vector features are retrieved in a preset key frame database to obtain multiple features. And according to each retrieval result, the pose of the robot is solved to obtain the initial value of the first pose, and the laser point cloud data is matched in the preset two-dimensional grid map according to the speed data and displacement data.
  • the first matching result is obtained; finally, the robot is repositioned according to the initial value of the first pose, the first matching result and the second matching result, which solves the problem that the repositioning in the repeated scene cannot be realized in the prior art; another In the aspect, by solving the pose of the robot according to each retrieval result, the initial value of the first pose is obtained, and the laser point cloud data is matched in the preset two-dimensional grid map according to the speed data and displacement data, and the first position is obtained.
  • the robot is relocated according to the initial value of the first pose, the first matching result and the second matching result, which solves the problem of low positioning accuracy in the prior art in an unstructured scene;
  • image data, velocity data, displacement data and laser point cloud data are collected by different sensors, multi-sensor fusion is realized, which further improves the accuracy of the relocation results of the robot.
  • the robot may include a lidar 201 , a binocular camera (camera device) 202 , an IMU (Inertial measurement unit, inertial measurement unit) 203 , a wheel encoder (Odom) 204 and a controller 205 .
  • a lidar 201 a lidar 201 , a binocular camera (camera device) 202 , an IMU (Inertial measurement unit, inertial measurement unit) 203 , a wheel encoder (Odom) 204 and a controller 205 .
  • Lidar can be used to collect the laser point cloud data of the robot
  • binocular camera can be used to collect the image data of the robot
  • the inertial measurement unit can be used to collect the acceleration data and angular velocity data of the robot
  • mapping process involved in the exemplary embodiments of the present disclosure, that is, the process of constructing a two-dimensional grid map, is explained and described.
  • a camera is used to capture key frame images, and feature encoding is performed on the key frame images.
  • the feature encoding method can be BoW (Bog of words, bag of words), VLAD (Vector of Aggragate Locally Descriptor, local aggregation descriptor vector), NetVLAD, etc.
  • the specific mapping process can be referred to as shown in FIG. 3 .
  • Step S310 collecting the image data of the robot through the binocular camera, collecting the speed data and displacement data of the robot through the inertial measurement unit and the wheel encoder, and collecting the laser point cloud data of the robot through the lidar;
  • Step S320 judging whether the image data satisfies a preset condition; wherein, the preset condition includes at least one of the following:
  • the difference between the speed data and the speed data at the previous moment corresponding to the speed data is greater than a preset speed threshold
  • the difference between the displacement data and the displacement data at the previous moment corresponding to the displacement data is greater than a preset displacement threshold
  • the overlapped portion between the image data and the image data of the previous frame corresponding to the image data is less than a preset overlap threshold
  • Step S330 if yes, perform feature encoding on the image data, and store the encoded image data as a key frame in the key frame database; if not, jump to step S310;
  • Step S340 constructing a two-dimensional (2D) grid map based on the key frame database.
  • the VLAD algorithm is taken as an example to explain and describe the specific process of feature encoding for image data.
  • the same function can be achieved by using the tree BoW algorithm, NetVLAD, etc.
  • the residual vectors of a visual word are accumulated and summed; finally, the residual vectors formed by all visual words are linked end to end to form a super-long vector.
  • the dimension of the super-long vector is k*d, where k is the cluster center.
  • step S110 acquire the image data of the robot collected by the camera device, the speed data and displacement data of the robot collected by the inertial measurement device and the displacement measurement device, and the laser point cloud data of the robot collected by the lidar .
  • the camera device may be, for example, a binocular camera
  • the inertial measurement device may be, for example, an IMU (Inertial measurement unit, inertial measurement unit)
  • the only measurement device may be, for example, a wheel encoder (Odom).
  • the controller can obtain the image data of the robot from the binocular camera, the acceleration data and angular velocity data of the robot from the IMU, the displacement data of the robot from the wheel encoder, and finally obtain the robot from the lidar. laser point cloud data.
  • step S120 feature encoding is performed on the local visual features of the image data to obtain image vector features, and the dimension-reduced image vector features are retrieved in a preset key frame database to obtain multiple retrieval results.
  • the local visual features of the image data are feature-coded to obtain the image vector features.
  • performing feature encoding on local visual features of image data to obtain image vector features may include steps S410 to S430. in:
  • step S410 the local visual features of the image data are extracted, and the visual words adjacent to the local visual features are searched in a preset visual dictionary;
  • step S420 the residual vector between the local visual feature and the visual word is calculated, and the residual vector belonging to the same visual word is accumulated and summed to obtain the residual sum vector;
  • step S430 the residual and the vector are connected according to the index order of the visual word in the preset visual dictionary to obtain the image vector feature.
  • a visual dictionary which can be trained using a general image database and K-means algorithm, or a third-party public visual dictionary, which contains k visual words;
  • it can be based on SIFT (Scale-invariant feature transform, Scale-invariant feature transformation), SURF (Speeded Up Robust Features, accelerated robust features), ORB (Oriented FAST and Rotated BRIEF, fast feature point extraction and description) and other algorithms to extract local visual features of image data.
  • SIFT Scale-invariant feature transform, Scale-invariant feature transformation
  • SURF Speeded Up Robust Features, accelerated robust features
  • ORB Oriented FAST and Rotated BRIEF, fast feature point extraction and description
  • the SIFT algorithm is taken as an example to explain and illustrate how to extract the local features of the image data.
  • the scale space of the image data is generated, and the extreme points of the scale space are detected; then, the extreme points are precisely located, and direction parameters are specified for each key point; finally, the key point descriptor is generated based on the direction parameters, to obtain local visual features. Further, the residual vectors of local visual features and the nearest neighbor visual words are calculated, and the residual vectors belonging to the same visual word are accumulated and summed; after the above calculation of all visual features is completed, all residuals and vectors The index order of the words is connected end to end to form a super long vector as the expression of an image.
  • the image vector features after dimension reduction processing need to be retrieved in a preset key frame database to obtain multiple retrieval results.
  • searching the image vector features after dimension reduction processing in a preset key frame database, and obtaining multiple search results may include steps S510 to S530. in:
  • step S510 dimensionality reduction processing is performed on the image vector features, and space division is performed on the image vector features after the dimensionality reduction processing to obtain a plurality of sub-vectors;
  • step S520 the cluster center corresponding to each of the sub-vectors and the index of each of the cluster centers are obtained in a preset key frame database, and the relationship between each of the sub-vectors and each of the cluster centers is calculated. the distance between;
  • step S530 when it is judged that the distance is smaller than a preset threshold, an image vector corresponding to the cluster center is obtained according to the index of the cluster center, and a plurality of the retrievals are obtained according to a plurality of the image vectors result.
  • PCA Principal Component Analysis, principal component analysis
  • PCA Principal Component Analysis, principal component analysis
  • it may include: performing mean normalization on the image vector features, and then calculating the coordinator based on the normalized image vector features.
  • Variance matrix Perform singular value decomposition on the covariance matrix, select the eigenvectors corresponding to the first K eigenvalues, and output the dimension-reduced projection feature matrix, and finally output the dimension-reduced image vector features.
  • the image vector features after dimension reduction processing are retrieved, and multiple retrieval results are obtained.
  • the KNN algorithm is used to find the nearest neighbor cluster center for the N vectors of each subspace, and the N subvectors of all M subspaces and the index of the corresponding nearest neighbor cluster center are made into a lookup table of the database;
  • retrieving the image vector features after dimensionality reduction processing based on the PQ algorithm, and obtaining multiple retrieval results may include: first, spatially dividing the image vector features after dimensionality reduction processing to obtain multiple sub-vectors, and calculating each sub-vector.
  • the vector corresponds to the indices and distances of all cluster centers, and an online lookup table is made; then, when calculating the similarity expressed by the current image vector and any image vector in the database, look up the database image vector in the lookup table of the database.
  • the index numbers of the M subspaces finally, look up the corresponding distances in the online lookup table according to the index numbers, accumulate the distances of the M subspaces, and use the accumulated sum of the distances as the similarity between the current image vector and the image vector in the database.
  • step S130 the pose of the robot is solved according to each of the retrieval results to obtain the initial value of the first pose, and according to the speed data and displacement data, the pose of the robot is determined in a preset two-dimensional grid map.
  • the laser point cloud data is matched to obtain the first matching result.
  • first, feature matching is performed on each of the retrieval results to obtain matching features among the retrieval results and a matching relationship between the matching features; secondly, according to the matching features and the matching features
  • the matching relationship between the matching features is used to solve the pose of the robot to obtain the initial value of the first pose; then, the velocity data and the displacement data are fused to obtain the initial value of the second pose; further,
  • the laser point cloud data is matched in a preset two-dimensional grid map based on the initial value of the second pose to obtain a first matching result.
  • step S140 when it is determined that the initial value of the first posture and the first matching result are inconsistent, a second matching result is generated according to the initial value of the first posture and the first matching result, and a second matching result is generated according to the first matching result.
  • the initial value of the two poses, the first matching result and the second matching result reposition the robot.
  • the initial value of the first pose and the first matching result are consistent. If they are consistent, it proves that the position of the robot is accurate and relocation is not required;
  • the initial pose and the first matching result generate a new visual matching result (the second matching result), and then the robot is repositioned according to the initial value of the second pose, the first matching result and the second matching result.
  • repositioning the robot according to the initial value of the second pose, the first matching result and the second matching result may include steps S610-S630, wherein:
  • step S610 configure a first weight value, a second weight value and a third weight value for the initial value of the second pose, the first matching result and the second matching result, respectively;
  • step S620 a weighted sum is performed on the initial value of the second pose, the first matching result, the second matching result, and the first weight value, the second weight value, and the third weight value, respectively;
  • step S630 the robot is relocated according to the weighted summation result.
  • ⁇ argmin ⁇ R ekf + ⁇ R csm + ⁇ R vision ⁇ ;
  • ⁇ is the relocation result of the robot
  • R ekf is the initial value of the second pose
  • R csm is the first matching result
  • R vision is the second matching result
  • ⁇ , ⁇ and ⁇ are the first weight value
  • the relocation method of the robot may include the following steps:
  • Step S710 acquiring a frame of image, and extracting local visual features from the image
  • Step S720 image coding is performed on the frame image, and a dimensionality reduction operation is performed using the PCA model;
  • Step S730 use the PQ model to perform image retrieval in the database to obtain the most similar N images
  • Step S740 performing feature-level matching on the acquired N images, images satisfying a certain threshold are retained, and the matching relationship of the features is saved;
  • Step S750 use the PnP algorithm to solve the pose of the matching feature, and use the least squares to optimize the pose to obtain the initial value of the first pose;
  • Step S760 fuse IMU and Odom data based on EKF (Extended Kalman Filter, Extended Kalman Filter) to obtain the initial value of the second pose; and based on the CSM algorithm and the initial value of the second pose, in the preset 2D grid map Match the laser point cloud data to obtain the first matching result;
  • EKF Extended Kalman Filter
  • CSM Extended Kalman Filter
  • Step S770 when judging that the initial value of the first pose and the first matching result are inconsistent, generate a second matching result according to the initial value of the first pose and the first matching result;
  • Step S780 repositioning the robot according to the initial value of the second pose, the first matching result and the second matching result.
  • the robot positioning method provided by the exemplary embodiment of the present disclosure can deal with the problems of positioning errors and long positioning time caused by using laser point cloud matching in large scenes; at the same time, using the second position initial value to provide a laser point cloud matching
  • the initial value of positioning can greatly improve the speed and accuracy of relocation.
  • Example embodiments of the present disclosure also provide a repositioning device for a robot.
  • the robot relocation device may include a data acquisition module 810 , a feature encoding module 820 , a data matching module 830 and a relocation module 840 . in:
  • the data acquisition module 810 can be used to acquire the image data of the robot collected by the camera device, the speed data and displacement data of the robot collected by the inertial measurement device and the displacement measurement device, and the laser point of the robot collected by the lidar cloud data;
  • the feature encoding module 820 can be used to perform feature encoding on the local visual features of the image data to obtain image vector features, and retrieve the dimension-reduced image vector features in a preset key frame database to obtain multiple retrieval results. ;
  • the data matching module 830 can be used to solve the pose of the robot according to each of the retrieval results, obtain the initial value of the first pose, and use the speed data and displacement data in the preset two-dimensional grid map. Matching the laser point cloud data to obtain a first matching result;
  • the relocation module 840 may be configured to generate a second matching result according to the initial value of the first pose and the first matching result when it is judged that the initial value of the first pose is inconsistent with the first matching result, and generate a second matching result according to the first matching result.
  • the robot is repositioned according to the initial value of the second pose, the first matching result and the second matching result.
  • performing feature encoding on the local visual features of the image data to obtain image vector features includes:
  • the image vector feature is obtained by connecting the residual and the vector according to the index order of the visual word in the preset visual dictionary.
  • the image vector features after dimension reduction processing are retrieved in a preset key frame database, and multiple retrieval results obtained include:
  • an image vector corresponding to the cluster center is obtained according to the index of the cluster center, and a plurality of the retrieval results are obtained according to a plurality of the image vectors.
  • the pose of the robot is solved according to each of the retrieval results, and obtaining the initial value of the first pose includes:
  • the pose of the robot is solved according to each of the matching features and the matching relationship between the matching features, so as to obtain a first initial value of the pose.
  • matching the laser point cloud data in a preset two-dimensional grid map according to the velocity data and the displacement data, and obtaining a first matching result includes:
  • the velocity data and the displacement data are fused to obtain the second initial value of the pose
  • the laser point cloud data is matched in a preset two-dimensional grid map based on the initial value of the second pose to obtain a first matching result.
  • repositioning the robot according to the initial value of the second pose, the first matching result, and the second matching result includes:
  • the robot is relocated according to the weighted summation.
  • the relocation device of the robot further includes:
  • a judgment module which can be used to judge whether the image data meets a preset condition; if so, store the image data as a key frame in the key frame database;
  • the preset condition includes at least one of the following:
  • the difference between the speed data and the speed data at the previous moment corresponding to the speed data is greater than a preset speed threshold
  • the difference between the displacement data and the displacement data at the previous moment corresponding to the displacement data is greater than a preset displacement threshold
  • the overlapping portion between the image data and the image data of the previous frame corresponding to the image data is smaller than a preset overlapping threshold.
  • modules or units of the apparatus for action performance are mentioned in the above detailed description, this division is not mandatory. Indeed, according to embodiments of the present disclosure, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above may be further divided into multiple modules or units to be embodied.
  • an electronic device capable of implementing the above method is also provided.
  • aspects of the present disclosure may be implemented as a system, method or program product. Therefore, various aspects of the present disclosure can be embodied in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, which may be collectively referred to herein as implementations "circuit", “module” or "system”.
  • FIG. 9 An electronic device 900 according to this embodiment of the present disclosure is described below with reference to FIG. 9 .
  • the electronic device 900 shown in FIG. 9 is only an example, and should not impose any limitation on the function and scope of use of the embodiments of the present disclosure.
  • electronic device 900 takes the form of a general-purpose computing device.
  • Components of the electronic device 900 may include, but are not limited to, the above-mentioned at least one processing unit 910 , the above-mentioned at least one storage unit 920 , a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910 ), and a display unit 940 .
  • the storage unit stores program codes, which can be executed by the processing unit 910, so that the processing unit 910 executes various exemplary methods according to the present disclosure described in the above-mentioned “Exemplary Methods” section of this specification.
  • the processing unit 910 may perform step S110 as shown in FIG. 1 : acquiring image data of the robot collected by a camera device, speed data and displacement data of the robot collected by an inertial measurement device and a displacement measurement device, and the laser point cloud data of the robot collected by the laser radar;
  • Step S120 perform feature encoding on the local visual features of the image data to obtain image vector features, and perform dimension reduction processing in a preset key frame database.
  • the image vector feature is retrieved to obtain a plurality of retrieval results;
  • Step S130 Solve the pose of the robot according to each of the retrieval results, obtain the initial value of the first pose, and calculate the initial value of the first pose according to the speed data and displacement data.
  • Step S140 when it is judged that the initial value of the first pose and the first matching result are inconsistent, according to the first matching result
  • the initial pose value and the first matching result generate a second matching result, and the robot is repositioned according to the second initial pose value, the first matching result, and the second matching result.
  • the storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 9201 and/or a cache storage unit 9202 , and may further include a read only storage unit (ROM) 9203 .
  • RAM random access storage unit
  • ROM read only storage unit
  • the storage unit 920 may also include a program/utility 9204 having a set (at least one) of program modules 9205 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, An implementation of a network environment may be included in each or some combination of these examples.
  • the bus 930 may be representative of one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local area using any of a variety of bus structures bus.
  • the electronic device 900 may also communicate with one or more external devices 1000 (eg, keyboards, pointing devices, Bluetooth devices, etc.), with one or more devices that enable a user to interact with the electronic device 900, and/or with Any device (eg, router, modem, etc.) that enables the electronic device 900 to communicate with one or more other computing devices. Such communication may take place through input/output (I/O) interface 950 . Also, the electronic device 900 may communicate with one or more networks (eg, a local area network (LAN), a wide area network (WAN), and/or a public network such as the Internet) through a network adapter 960 . As shown, network adapter 960 communicates with other modules of electronic device 900 via bus 930 . It should be understood that, although not shown, other hardware and/or software modules may be used in conjunction with electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives and data backup storage systems.
  • the exemplary embodiments described herein may be implemented by software, or may be implemented by software combined with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure may be embodied in the form of software products, and the software products may be stored in a non-volatile storage medium (which may be CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to cause a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to an embodiment of the present disclosure.
  • a computing device which may be a personal computer, a server, a terminal device, or a network device, etc.
  • a computer-readable storage medium on which a program product capable of implementing the above-described method of the present specification is stored.
  • various aspects of the present disclosure may also be implemented in the form of a program product comprising program code for causing the program product to run on a terminal device when the program product is run on a terminal device.
  • the terminal device performs the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned "Example Method" section of this specification.
  • a program product for implementing the above method according to an embodiment of the present disclosure may adopt a portable compact disc read only memory (CD-ROM) and include program codes, and may run on a terminal device, such as a personal computer.
  • CD-ROM compact disc read only memory
  • the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • the program product may employ any combination of one or more readable media.
  • the readable medium may be a readable signal medium or a readable storage medium.
  • the readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
  • a computer readable signal medium may include a propagated data signal in baseband or as part of a carrier wave with readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a readable signal medium can also be any readable medium, other than a readable storage medium, that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
  • Program code embodied on a readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
  • Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages—such as Java, C++, etc., as well as conventional procedural Programming Language - such as the "C" language or similar programming language.
  • the program code may execute entirely on the user computing device, partly on the user device, as a stand-alone software package, partly on the user computing device and partly on a remote computing device, or entirely on the remote computing device or server execute on.
  • the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (eg, using an Internet service provider business via an Internet connection).
  • LAN local area network
  • WAN wide area network

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • General Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Automation & Control Theory (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Probability & Statistics with Applications (AREA)
  • Manipulator (AREA)
  • Image Analysis (AREA)

Abstract

一种机器人的重定位方法及装置、存储介质、电子设备，该方法包括：对图像数据的局部视觉特征进行特征编码得到图像矢量特征，并在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果（S120）；根据各检索结果对机器人的位姿进行求解，得到第一位姿初始值，并根据速度数据以及位移数据在预设的二维栅格地图中对激光点云数据进行匹配，得到第一匹配结果（S130）；根据第一位姿初始值以及第一匹配结果生成第二匹配结果，并根据第二位姿初始值、第一匹配结果以及第二匹配结果对机器人进行重定位。

Description

机器人的重定位方法及装置、存储介质、电子设备
本申请以申请日为2020年7月15日、申请号为202010680255.2、发明名称为“机器人的重定位方法及装置、存储介质、电子设备”作为优先权。
技术领域
本公开实施例涉及机器人技术领域，具体而言，涉及一种机器人的重定位方法、机器人的重定位装置、计算机可读存储介质以及电子设备。
背景技术
随着人工智能的不断发展，机器人的应用场景得到极大拓展，给人们的生产生活带来了极大的便利，同时，对于机器人的效率以及智能化的要求也越来越高，例如，对于自主行走机器人来说，要求机器人对于行走路线要有较高效的行走路径规划。
目前，基于视觉和激光SLAM(simultaneous localization and mapping，同步定位与建图)的定位方法，可以利用环境信息进行主动定位，不需要对环境进行改造。
发明内容
根据本公开的一个方面，提供一种机器人的重定位方法，包括：
获取通过摄像装置采集的机器人的图像数据、通过惯性测量装置以及位移测量装置采集的所述机器人的速度数据以及位移数据，以及通过激光雷达采集的所述机器人的激光点云数据；
对所述图像数据的局部视觉特征进行特征编码得到图像矢量特征，并在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果；
根据各所述检索结果对所述机器人的位姿进行求解，得到第一位姿初始值，并根据所述速度数据以及位移数据在预设的二维栅格地图中对所述激光点云数据进行匹配，得到第一匹配结果；
在判断所述第一位姿初始值以及第一匹配结果不一致时，根据所述第一位姿初始值以及所述第一匹配结果生成第二匹配结果，并根据所述第二位姿初始值、第一匹配结果以及第二匹配结果对所述机器人进行重定位。
在本公开的一种示例性实施例中，对所述图像数据的局部视觉特征进行特征编码得到图像矢量特征包括：
提取所述图像数据的局部视觉特征，并在预设的视觉词典中查找与所述局部视觉特征临近的视觉单词；
计算所述局部视觉特征与所述视觉单词之间的残差向量，并将属于同一所述视觉单 词的残差向量进行累加求和操作得到残差和向量；
根据所述视觉单词在所述预设的视觉词典中的索引顺序对所述残差和向量进行连接，得到所述图像矢量特征。
在本公开的一种示例性实施例中，在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果包括：
对所述图像矢量特征进行降维处理，并对降维处理后的图像矢量特征进行空间划分，得到多个子向量；
在预设的关键帧数据库中获取与各所述子向量对应的聚类中心以及各所述聚类中心的索引，并计算各所述子向量与各所述聚类中心之间的距离；
在判断所述距离小于预设阈值时，根据所述聚类中心的索引获取与所述聚类中心对应的图像矢量，并根据多个所述图像矢量得到多个所述检索结果。
在本公开的一种示例性实施例中，根据各所述检索结果对所述机器人的位姿进行求解，得到第一位姿初始值包括：
对各所述检索结果进行特征匹配，得到各所述检索结果之间的匹配特征以及匹配特征之间的匹配关系；
根据各所述匹配特征以及各所述匹配特征之间的匹配关系对所述机器人的位姿进行求解，得到第一位姿初始值。
在本公开的一种示例性实施例中，根据所述速度数据以及位移数据在预设的二维栅格地图中对所述激光点云数据进行匹配，得到第一匹配结果包括：
对所述速度数据以及位移数据进行融合，得到第二位姿初始值；
基于所述第二位姿初始值在预设的二维栅格地图中对所述激光点云数据进行匹配，得到第一匹配结果。
在本公开的一种示例性实施例中，根据所述第二位姿初始值、第一匹配结果以及第二匹配结果对所述机器人进行重定位包括：
分别为所述第二位姿初始值、第一匹配结果以及第二匹配结果配置第一权重值、第二权重值以及第三权重值；
对所述第二位姿初始值、第一匹配结果、第二匹配结果，以及第一权重值、第二权重值、第三权重值分别进行加权求和；
根据加权求和结果对所述机器人进行重定位。
在本公开的一种示例性实施例中，所述机器人的重定位方法还包括：
判断所述图像数据是否满足预设条件；若是，则将所述图像数据作为关键帧存储至所述关键帧数据库中；
其中，所述预设条件至少包括以下一项：
所述速度数据以及与所述速度数据对应的上一时刻的速度数据之间的差值大于预设速度阈值；
所述位移数据以及与所述位移数据对应的上一时刻的位移数据之间的差值大于预设位移阈值；
所述图像数据以及与所述图像数据对应的上一帧图像数据之间的重叠部分小于预设重叠阈值。
根据本公开的一个方面，提供一种机器人的重定位装置，包括：
数据获取模块，用于获取通过摄像装置采集的机器人的图像数据、通过惯性测量装置以及位移测量装置采集的所述机器人的速度数据以及位移数据，以及通过激光雷达采集的所述机器人的激光点云数据；
特征编码模块，用于对所述图像数据的局部视觉特征进行特征编码得到图像矢量特征，并在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果；
数据匹配模块，用于根据各所述检索结果对所述机器人的位姿进行求解，得到第一位姿初始值，并根据所述速度数据以及位移数据在预设的二维栅格地图中对所述激光点云数据进行匹配，得到第一匹配结果；
重定位模块，用于在判断所述第一位姿初始值以及第一匹配结果不一致时，根据所述第一位姿初始值以及所述第一匹配结果生成第二匹配结果，并根据所述第二位姿初始值、第一匹配结果以及第二匹配结果对所述机器人进行重定位。
根据本公开的一个方面，提供一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述任意一项所述的机器人的重定位方法。
根据本公开的一个方面，提供一种电子设备，包括：
处理器；以及
存储器，用于存储所述处理器的可执行指令；
其中，所述处理器配置为经由执行所述可执行指令来执行上述任意一项所述的机器人的重定位方法。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。在附图中：
图1示意性示出根据本公开示例实施例的一种机器人的重定位方法的流程图。
图2示意性示出根据本公开示例实施例的一种机器人的结构示例图。
图3示意性示出根据本公开示例实施例的一种二维栅格地图的构建方法的流程图。
图4示意性示出根据本公开示例实施例的一种对图像数据的局部视觉特征进行特征编码得到图像矢量特征的方法流程图。
图5示意性示出根据本公开示例实施例的一种在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果的方法流程图。
图6示意性示出根据本公开示例实施例的一种根据所述第二位姿初始值、第一匹配结果以及第二匹配结果对所述机器人进行重定位的方法流程图。
图7示意性示出根据本公开示例实施例的另一种机器人的重定位方法的流程图。
图8示意性示出根据本公开示例实施例的一种机器人的重定位装置的框图。
图9示意性示出根据本公开示例实施例的一种用于实现上述机器人的重定位方法的电子设备。
具体实施方式
现在将参照附图更全面地描述示例实施方式。然而，示例实施方式能够以多种形式实施，且不应被理解为限于在此阐述的范例；相反，提供这些实施方式使得本公开将更加全面和完整，并将示例实施方式的构思全面地传达给本领域的技术人员。所描述的特征、结构或特性可以以任何合适的方式结合在一个或更多实施方式中。
此外，附图仅为本公开的示意性图解，并非一定是按比例绘制。图中相同的附图标记表示相同或类似的部分，因而将省略对它们的重复描述。附图中所示的一些方框图是功能实体，不一定必须与物理或逻辑上独立的实体相对应。可以采用软件形式来实现这些功能实体，或在一个或多个硬件模块或集成电路中实现这些功能实体，或在不同网络和/或处理器装置和/或微控制器装置中实现这些功能实体。
导航和定位技术是室内机器人系统的关键技术之一，目前常用的定位技术主要包括Outside-In(外向内定位)和Inside-Out(内向外定位)两类。其中，Outside-In的定位方式包括基于April tag(视觉基准库)码的视觉定位方法以及基于可见光通信的定位方法；Inside-Out的方法包括视觉和激光SLAM(simultaneous localization and mapping，同步定位与建图)的定位方法等。
但是，上述方法存在如下缺陷：一方面，Outside-In的定位方式需要在使用环境中预先设定先验信息，对工作环境进行一定程度的改造，限制了使用场景的扩展；另一方面，Inside-Out的定位方式可以利用环境信息进行主动定位，不需要对环境进行改造，但随着场景的扩展，传统的定位算法在复杂和非结构化场景下的定位鲁棒性无法满足应用要求，如：在重复场景下的重定位问题；在非结构化场景下的定位准确性问题；在空旷场景下的定位鲁棒性问题等。
本示例实施方式中首先提供了一种机器人的重定位方法，该方法可以运行于机器人； 当然，本领域技术人员也可以根据需求在其他平台运行本公开的方法，本示例性实施例中对此不做特殊限定。参考图1所示，该机器人的重定位方法可以包括以下步骤：
步骤S110.获取通过摄像装置采集的机器人的图像数据、通过惯性测量装置以及位移测量装置采集的所述机器人的速度数据以及位移数据，以及通过激光雷达采集的所述机器人的激光点云数据；
步骤S120.对所述图像数据的局部视觉特征进行特征编码得到图像矢量特征，并在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果；
步骤S130.根据各所述检索结果对所述机器人的位姿进行求解，得到第一位姿初始值，并根据所述速度数据以及位移数据在预设的二维栅格地图中对所述激光点云数据进行匹配，得到第一匹配结果；
步骤S140.在判断所述第一位姿初始值以及第一匹配结果不一致时，根据所述第一位姿初始值以及所述第一匹配结果生成第二匹配结果，并根据所述第二位姿初始值、第一匹配结果以及第二匹配结果对所述机器人进行重定位。
上述机器人的重定位方法中，一方面，通过对图像数据的局部视觉特征进行特征编码得到图像矢量特征，并在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果；并根据各检索结果对机器人的位姿进行求解，得到第一位姿初始值，并根据速度数据以及位移数据在预设的二维栅格地图中对激光点云数据进行匹配，得到第一匹配结果；最后根据第一位姿初始值、第一匹配结果以及第二匹配结果对机器人进行重定位，解决了现有技术中无法实现在重复场景下的重定位的问题；另一方面，通过根据各检索结果对机器人的位姿进行求解，得到第一位姿初始值，并根据速度数据以及位移数据在预设的二维栅格地图中对激光点云数据进行匹配，得到第一匹配结果；最后根据第一位姿初始值、第一匹配结果以及第二匹配结果对机器人进行重定位，解决了现有技术中在非结构化场景下，定位的准确性较低的问题；再一方面，由于图像数据、速度数据、位移数据以及激光点云数据是通过不同的传感器采集得到的，因此实现了多传感器的融合，进一步的提升了机器人的重定位结果的准确率。
以下，将结合附图对本公开示例实施例机器人的重定位方法中涉及的步骤进行详细的解释以及说明。
首先，对本公开示例实施例中所涉及的机器人进行解释以及说明。参考图2所示，该机器人中可以包括激光雷达201、双目摄像头(摄像装置)202、IMU(Inertial measurement unit，惯性测量单元)203、轮式编码器(Odom)204以及控制器205。其中：
激光雷达可以用于采集机器人的激光点云数据，双目摄像头可以用于采集机器人的图像数据，惯性测量单元可以用于采集机器人的加速度数据以及角速度数据，轮式编码器可以用于采集机器人的移动距离；控制器可以用于执行本申请所记载的机器人的重定位方法。
其次，对本公开示例实施例中所涉及的建图过程，也就是构建二维栅格地图的过程进行解释以及说明。具体的，使用激光SLAM建立2D栅格地图过程中，使用摄像头捕捉关 键帧图像，并对关键帧图像进行特征编码，其中特征编码的方法可以为BoW(Bog of words，词袋)、VLAD(Vector of Aggragate Locally Descriptor，局部聚集描述子向量)、NetVLAD等。
其中，具体的建图过程可以参考图3所示。
步骤S310，通过双目摄像头采集机器人的图像数据、通过惯性测量单元以及轮式编码器采集所述机器人的速度数据以及位移数据，以及通过激光雷达采集所述机器人的激光点云数据；
步骤S320，判断所述图像数据是否满足预设条件；其中，所述预设条件至少包括以下一项：
所述速度数据以及与所述速度数据对应的上一时刻的速度数据之间的差值大于预设速度阈值；
所述位移数据以及与所述位移数据对应的上一时刻的位移数据之间的差值大于预设位移阈值；
所述图像数据以及与所述图像数据对应的上一帧图像数据之间的重叠部分小于预设重叠阈值；
步骤S330，若是，则对所述图像数据进行特征编码，并将特征编码后的图像数据作为关键帧存储至所述关键帧数据库中；若否，则跳转至步骤S310；
步骤S340，基于所述关键帧数据库构建二维(2D)栅格地图。
其中，以VLAD算法为例，对图像数据进行特征编码的具体过程进行解释以及说明，当然，使用树形BoW算法、NetVLAD等可以实现同样的功能。首先，获取一帧图像，并对图像提取局部视觉特征；其次，使用K-means算法获取视觉词典；然后，对当前图像的局部视觉特征查找最近邻视觉单词，然后计算残差向量，并将同一个视觉单词的残差向量进行累加求和；最后，将所有视觉单词形成的残差向量进行首尾链接，形成超长矢量，超长矢量的维度为k*d，其中，k为聚类中心的数量，d为局部视觉特征的维度；进一步的，当建图完成后对所有数据库图像的图像编码进行PCA降维，如将维度降到D＝128；再使用PCA降维后的数据库数据训练PQ(Product Quantization，乘积量化)模型。
以下，对步骤S110-步骤S140进行解释以及说明。
在步骤S110中，获取通过摄像装置采集的机器人的图像数据、通过惯性测量装置以及位移测量装置采集的所述机器人的速度数据以及位移数据，以及通过激光雷达采集的所述机器人的激光点云数据。
在本示例实施例中，摄像装置例如可以是双目摄像头、惯性测量装置例如可以是IMU(Inertial measurement unit，惯性测量单元)、唯一测量装置例如可以是轮式编码器(Odom)。具体的，控制器可以分别从双目摄像机中获取机器人的图像数据、从IMU中获取机器人的加速度数据以及角速度数据、从轮式编码器中获取机器人的位移数据，最后再从激光雷达中获取机器人的激光点云数据。
在步骤S120中，对所述图像数据的局部视觉特征进行特征编码得到图像矢量特征，并在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果。
在本示例实施例中，首先，对图像数据的局部视觉特征进行特征编码得到图像矢量特征。具体的，参考图4所示，对图像数据的局部视觉特征进行特征编码得到图像矢量特征可以包括步骤S410-步骤S430。其中：
在步骤S410中，提取所述图像数据的局部视觉特征，并在预设的视觉词典中查找与所述局部视觉特征临近的视觉单词；
在步骤S420中，计算所述局部视觉特征与所述视觉单词之间的残差向量，并将属于同一所述视觉单词的残差向量进行累加求和操作得到残差和向量；
在步骤S430中，根据所述视觉单词在所述预设的视觉词典中的索引顺序对所述残差和向量进行连接，得到所述图像矢量特征。
以下，将对步骤S410-步骤S430进行解释以及说明。首先，获取视觉词典，该词典可以使用通用图像数据库和K-means算法进行训练，也可以使用第三方公开视觉词典，视觉词典包含k个视觉单词；其次，可以基于SIFT(Scale-invariant feature transform，尺度不变特征变换)、SURF(Speeded Up Robust Features，加速稳健特征)、ORB(Oriented FAST and Rotated BRIEF，快速特征点提取和描述)等算法，提取图像数据的局部视觉特征。具体的，以SIFT算法为例，对如何提取图像数据的局部特征进行解释以及说明。首先，生成图像数据的尺度空间，并检测尺度空间的极值点；然后，对极值点进行精确定位，并为每个关键点指定方向参数；最后，基于该方向参数生成关键点描述子，以得到局部视觉特征。进一步的，计算局部视觉特征和最近邻视觉单词的残差向量，并将属于同一视觉单词的残差向量进行累加求和操作；完成所有视觉特征的上述计算之后，将所有残差和向量按照视觉单词的索引顺序进行首尾连接，形成一个超长矢量作为一张图像的表达。
进一步的，在本示例实施例中，当得到图像矢量特征以后，还需要在在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果。具体的，参考图5所示，在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果可以包括步骤S510-步骤S530。其中：
在步骤S510中，对所述图像矢量特征进行降维处理，并对降维处理后的图像矢量特征进行空间划分，得到多个子向量；
在步骤S520中，在预设的关键帧数据库中获取与各所述子向量对应的聚类中心以及各所述聚类中心的索引，并计算各所述子向量与各所述聚类中心之间的距离；
在步骤S530中，在判断所述距离小于预设阈值时，根据所述聚类中心的索引获取与所述聚类中心对应的图像矢量，并根据多个所述图像矢量得到多个所述检索结果。
以下，将对步骤S510-步骤S530进行解释以及说明。首先，使用PCA(Principal Component Analysis，主成分分析)模型对图像矢量特征进行降维操作；具体可以包括：对图像矢量特征进行均值归一化，然后根据归一化处理后的图像矢量特征计算协方差矩阵； 对协方差矩阵进行奇异值分解，选取前K个特征值对应的特征向量，并输出降维的投影特征矩阵，最后输出降维后的图像矢量特征。然后，基于PQ算法对降维处理后的图像矢量特征进行检索，得到多个检索结果。其中：
PQ(Product Quantization，乘积量化)模型的原理为：
首先，获取使用PCA压缩后的图像矢量表达数据库，包含N张图像的D维表达，即N*D维矩阵；
其次，将上述矩阵划分成M个子矩阵，称为子空间，每个子空间的维度为D/M，如当D＝128，M＝8时，d＝D/M＝128/8＝16；
然后，对每个子空间内的N个向量使用k-means算法进行聚类分析，获取k个聚类中心；
最后，对每个子空间的N个向量使用KNN算法查找最近邻聚类中心，将所有M个子空间的N个子向量和对应最近邻聚类中心的索引制作成数据库的查找表；
进一步的，基于PQ算法对降维处理后的图像矢量特征进行检索，得到多个检索结果可以包括：首先，对降维处理后的图像矢量特征进行空间划分，得到多个子向量，并计算每个子向量对应所有聚类中心的索引和距离，并制成在线查找表；然后，当计算当前图像矢量和数据库中任一图像矢量表达的相似性时，在数据库的查找表中查找数据库图像矢量的在M个子空间的索引号；最后，根据索引号在在线查找表中查找对应的距离，将M个子空间的距离累积，距离的累加和作为当前图像矢量和数据库中该图像矢量的相似性。
在步骤S130中，根据各所述检索结果对所述机器人的位姿进行求解，得到第一位姿初始值，并根据所述速度数据以及位移数据在预设的二维栅格地图中对所述激光点云数据进行匹配，得到第一匹配结果。
在本示例实施例中，首先，对各所述检索结果进行特征匹配，得到各所述检索结果之间的匹配特征以及匹配特征之间的匹配关系；其次，根据各所述匹配特征以及各所述匹配特征之间的匹配关系对所述机器人的位姿进行求解，得到第一位姿初始值；然后，对所述速度数据以及位移数据进行融合，得到第二位姿初始值；进一步的，基于所述第二位姿初始值在预设的二维栅格地图中对所述激光点云数据进行匹配，得到第一匹配结果。详细而言：
首先，对获取的N张图像(检索结果)进行特征级匹配，得到各检索之间的匹配特征以及匹配特征之间的匹配关系，并将满足一定匹配阈值的检索结果保留；其次，使用PnP(Perspective-n-Point，三维到二维的点对运动)算法对匹配特征以及匹配关系进行位姿求解，并使用最小二乘进行位姿优化得到第一位姿初始值；进一步的，基于EKF(Extended Kalman Filter，扩展卡尔曼滤波)融合IMU和Odom数据，获取第二位姿初值(平移和旋转)；最后，基于CSM(Canonical Scan Matcher，标准扫描匹配)算法以及第二位姿初始值，在预设的2D栅格地图中对激光点云数据进行匹配，得到第一匹配结果。
在步骤S140中，在判断所述第一位姿初始值以及第一匹配结果不一致时，根据所述 第一位姿初始值以及所述第一匹配结果生成第二匹配结果，并根据所述第二位姿初始值、第一匹配结果以及第二匹配结果对所述机器人进行重定位。
在本示例实施例中，首先，判断第一位姿初始值以及第一匹配结果是否一致，如果一致，则证明机器人的位置是准确的，不需要进行重定位；如果不一致，则根据第一位姿初始之以及第一匹配结果生成新的视觉匹配结果(第二匹配结果)，然后再根据所述第二位姿初始值、第一匹配结果以及第二匹配结果对所述机器人进行重定位。
具体的，参考图6所示，根据所述第二位姿初始值、第一匹配结果以及第二匹配结果对所述机器人进行重定位可以包括步骤S610-步骤S630，其中：
在步骤S610中，分别为所述第二位姿初始值、第一匹配结果以及第二匹配结果配置第一权重值、第二权重值以及第三权重值；
在步骤S620中，对所述第二位姿初始值、第一匹配结果、第二匹配结果，以及第一权重值、第二权重值、第三权重值分别进行加权求和；
在步骤S630中，根据加权求和结果对所述机器人进行重定位。
以下，将对步骤S610-步骤S630进行解释以及说明。具体的：
ε＝argmin{αR ekf+βR csm+γR vision}；
其中，ε为机器人的重定位结果，R ekf为第二位姿初始值、R csm为第一匹配结果、R vision为第二匹配结果，α、β以及γ分别为第一权重值、第二权重值以及第三权重值，并且有α+β+γ＝1。
以下，将结合图7对本公开示例实施例的机器人的重定位方法进行进一步的解释以及说明。参考图7所示，该机器人的重定位方法可以包括以下步骤：
步骤S710，获取一帧图像，并对图像提取局部视觉特征；
步骤S720，对该帧图像进行图像编码，并使用PCA模型进行降维操作；
步骤S730，使用PQ模型在数据库中进行图像检索，获取最相似的N张图像；
步骤S740，对获取的N张图像进行特征级匹配，满足一定阈值的图像被保留，并保存特征的匹配关系；
步骤S750，使用PnP算法对匹配特征进行位姿求解，并使用最小二乘进行位姿优化得到第一位姿初始值；
步骤S760，基于EKF(Extended Kalman Filter，扩展卡尔曼滤波)融合IMU和Odom数据，获取第二位姿初值；并基于CSM算法以及第二位姿初始值，在预设的2D栅格地图中对激光点云数据进行匹配，得到第一匹配结果；
步骤S770，在判断所述第一位姿初始值以及第一匹配结果不一致时，根据所述第一位姿初始值以及所述第一匹配结果生成第二匹配结果；
步骤S780，根据所述第二位姿初始值、第一匹配结果以及第二匹配结果对所述机器人进行重定位。
本公开示例实施例提供的机器人定位方法，可以应对在大场景下使用激光点云匹配带来的定位错误，定位时间较长等问题；同时，使用第二位置初始值为激光点云匹配提供一个定位初值，可以极大提升重定位的速度和准确性。
本公开示例实施例还提供了一种机器人的重定位装置。参考图8所示，该机器人重定位装置可以包括数据获取模块810、特征编码模块820、数据匹配模块830以及重定位模块840。其中：
数据获取模块810可以用于获取通过摄像装置采集的机器人的图像数据、通过惯性测量装置以及位移测量装置采集的所述机器人的速度数据以及位移数据，以及通过激光雷达采集的所述机器人的激光点云数据；
特征编码模块820可以用于对所述图像数据的局部视觉特征进行特征编码得到图像矢量特征，并在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果；
数据匹配模块830可以用于根据各所述检索结果对所述机器人的位姿进行求解，得到第一位姿初始值，并根据所述速度数据以及位移数据在预设的二维栅格地图中对所述激光点云数据进行匹配，得到第一匹配结果；
重定位模块840可以用于在判断所述第一位姿初始值以及第一匹配结果不一致时，根据所述第一位姿初始值以及所述第一匹配结果生成第二匹配结果，并根据所述第二位姿初始值、第一匹配结果以及第二匹配结果对所述机器人进行重定位。
在本公开的一种示例性实施例中，对所述图像数据的局部视觉特征进行特征编码得到图像矢量特征包括：
提取所述图像数据的局部视觉特征，并在预设的视觉词典中查找与所述局部视觉特征临近的视觉单词；
计算所述局部视觉特征与所述视觉单词之间的残差向量，并将属于同一所述视觉单词的残差向量进行累加求和操作得到残差和向量；
根据所述视觉单词在所述预设的视觉词典中的索引顺序对所述残差和向量进行连接，得到所述图像矢量特征。
在本公开的一种示例性实施例中，在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果包括：
对所述图像矢量特征进行降维处理，并对降维处理后的图像矢量特征进行空间划分，得到多个子向量；
在预设的关键帧数据库中获取与各所述子向量对应的聚类中心以及各所述聚类中心的索引，并计算各所述子向量与各所述聚类中心之间的距离；
在判断所述距离小于预设阈值时，根据所述聚类中心的索引获取与所述聚类中心对应的图像矢量，并根据多个所述图像矢量得到多个所述检索结果。
在本公开的一种示例性实施例中，根据各所述检索结果对所述机器人的位姿进行求解， 得到第一位姿初始值包括：
对各所述检索结果进行特征匹配，得到各所述检索结果之间的匹配特征以及匹配特征之间的匹配关系；
根据各所述匹配特征以及各所述匹配特征之间的匹配关系对所述机器人的位姿进行求解，得到第一位姿初始值。
在本公开的一种示例性实施例中，根据所述速度数据以及位移数据在预设的二维栅格地图中对所述激光点云数据进行匹配，得到第一匹配结果包括：
对所述速度数据以及位移数据进行融合，得到第二位姿初始值；
基于所述第二位姿初始值在预设的二维栅格地图中对所述激光点云数据进行匹配，得到第一匹配结果。
在本公开的一种示例性实施例中，根据所述第二位姿初始值、第一匹配结果以及第二匹配结果对所述机器人进行重定位包括：
分别为所述第二位姿初始值、第一匹配结果以及第二匹配结果配置第一权重值、第二权重值以及第三权重值；
对所述第二位姿初始值、第一匹配结果、第二匹配结果，以及第一权重值、第二权重值、第三权重值分别进行加权求和；
根据加权求和结果对所述机器人进行重定位。
在本公开的一种示例性实施例中，所述机器人的重定位装置还包括：
判断模块，可以用于判断所述图像数据是否满足预设条件；若是，则将所述图像数据作为关键帧存储至所述关键帧数据库中；
其中，所述预设条件至少包括以下一项：
所述速度数据以及与所述速度数据对应的上一时刻的速度数据之间的差值大于预设速度阈值；
所述位移数据以及与所述位移数据对应的上一时刻的位移数据之间的差值大于预设位移阈值；
所述图像数据以及与所述图像数据对应的上一帧图像数据之间的重叠部分小于预设重叠阈值。
上述机器人的重定位装置中各模块的具体细节已经在对应的机器人的的重定位方法中进行了详细的描述，因此此处不再赘述。
应当注意，尽管在上文详细描述中提及了用于动作执行的设备的若干模块或者单元，但是这种划分并非强制性的。实际上，根据本公开的实施方式，上文描述的两个或更多模块或者单元的特征和功能可以在一个模块或者单元中具体化。反之，上文描述的一个模块或者单元的特征和功能可以进一步划分为由多个模块或者单元来具体化。
此外，尽管在附图中以特定顺序描述了本公开中方法的各个步骤，但是，这并非要求或者暗示必须按照该特定顺序来执行这些步骤，或是必须执行全部所示的步骤才能实现期 望的结果。附加的或备选的，可以省略某些步骤，将多个步骤合并为一个步骤执行，以及/或者将一个步骤分解为多个步骤执行等。
在本公开的示例性实施例中，还提供了一种能够实现上述方法的电子设备。
所属技术领域的技术人员能够理解，本公开的各个方面可以实现为系统、方法或程序产品。因此，本公开的各个方面可以具体实现为以下形式，即：完全的硬件实施方式、完全的软件实施方式(包括固件、微代码等)，或硬件和软件方面结合的实施方式，这里可以统称为“电路”、“模块”或“系统”。
下面参照图9来描述根据本公开的这种实施方式的电子设备900。图9显示的电子设备900仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图9所示，电子设备900以通用计算设备的形式表现。电子设备900的组件可以包括但不限于：上述至少一个处理单元910、上述至少一个存储单元920、连接不同系统组件(包括存储单元920和处理单元910)的总线930以及显示单元940。
其中，所述存储单元存储有程序代码，所述程序代码可以被所述处理单元910执行，使得所述处理单元910执行本说明书上述“示例性方法”部分中描述的根据本公开各种示例性实施方式的步骤。例如，所述处理单元910可以执行如图1中所示的步骤S110：获取通过摄像装置采集的机器人的图像数据、通过惯性测量装置以及位移测量装置采集的所述机器人的速度数据以及位移数据，以及通过激光雷达采集的所述机器人的激光点云数据；步骤S120：对所述图像数据的局部视觉特征进行特征编码得到图像矢量特征，并在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果；步骤S130：根据各所述检索结果对所述机器人的位姿进行求解，得到第一位姿初始值，并根据所述速度数据以及位移数据在预设的二维栅格地图中对所述激光点云数据进行匹配，得到第一匹配结果；步骤S140：在判断所述第一位姿初始值以及第一匹配结果不一致时，根据所述第一位姿初始值以及所述第一匹配结果生成第二匹配结果，并根据所述第二位姿初始值、第一匹配结果以及第二匹配结果对所述机器人进行重定位。
存储单元920可以包括易失性存储单元形式的可读介质，例如随机存取存储单元(RAM)9201和/或高速缓存存储单元9202，还可以进一步包括只读存储单元(ROM)9203。
存储单元920还可以包括具有一组(至少一个)程序模块9205的程序/实用工具9204，这样的程序模块9205包括但不限于：操作系统、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或某种组合中可能包括网络环境的实现。
总线930可以为表示几类总线结构中的一种或多种，包括存储单元总线或者存储单元控制器、外围总线、图形加速端口、处理单元或者使用多种总线结构中的任意总线结构的局域总线。
电子设备900也可以与一个或多个外部设备1000(例如键盘、指向设备、蓝牙设备等)通信，还可与一个或者多个使得用户能与该电子设备900交互的设备通信，和/或与 使得该电子设备900能与一个或多个其它计算设备进行通信的任何设备(例如路由器、调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口950进行。并且，电子设备900还可以通过网络适配器960与一个或者多个网络(例如局域网(LAN)，广域网(WAN)和/或公共网络，例如因特网)通信。如图所示，网络适配器960通过总线930与电子设备900的其它模块通信。应当明白，尽管图中未示出，可以结合电子设备900使用其它硬件和/或软件模块，包括但不限于：微代码、设备驱动器、冗余处理单元、外部磁盘驱动阵列、RAID系统、磁带驱动器以及数据备份存储系统等。
通过以上的实施方式的描述，本领域的技术人员易于理解，这里描述的示例实施方式可以通过软件实现，也可以通过软件结合必要的硬件的方式来实现。因此，根据本公开实施方式的技术方案可以以软件产品的形式体现出来，该软件产品可以存储在一个非易失性存储介质(可以是CD-ROM，U盘，移动硬盘等)中或网络上，包括若干指令以使得一台计算设备(可以是个人计算机、服务器、终端装置、或者网络设备等)执行根据本公开实施方式的方法。
在本公开的示例性实施例中，还提供了一种计算机可读存储介质，其上存储有能够实现本说明书上述方法的程序产品。在一些可能的实施方式中，本公开的各个方面还可以实现为一种程序产品的形式，其包括程序代码，当所述程序产品在终端设备上运行时，所述程序代码用于使所述终端设备执行本说明书上述“示例性方法”部分中描述的根据本公开各种示例性实施方式的步骤。
根据本公开的实施方式的用于实现上述方法的程序产品，其可以采用便携式紧凑盘只读存储器(CD-ROM)并包括程序代码，并可以在终端设备，例如个人电脑上运行。然而，本公开的程序产品不限于此，在本文件中，可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。
所述程序产品可以采用一个或多个可读介质的任意组合。可读介质可以是可读信号介质或者可读存储介质。可读存储介质例如可以为但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。
计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了可读程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。可读信号介质还可以是可读存储介质以外的任何可读介质，该可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。
可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于无线、有线、光缆、RF等等，或者上述的任意合适的组合。
可以以一种或多种程序设计语言的任意组合来编写用于执行本公开操作的程序代码，所述程序设计语言包括面向对象的程序设计语言—诸如Java、C++等，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户计算设备上部分在远程计算设备上执行、或者完全在远程计算设备或服务器上执行。在涉及远程计算设备的情形中，远程计算设备可以通过任意种类的网络，包括局域网(LAN)或广域网(WAN)，连接到用户计算设备，或者，可以连接到外部计算设备(例如利用因特网服务提供商来通过因特网连接)。
此外，上述附图仅是根据本公开示例性实施例的方法所包括的处理的示意性说明，而不是限制目的。易于理解，上述附图所示的处理并不表明或限制这些处理的时间顺序。另外，也易于理解，这些处理可以是例如在多个模块中同步或异步执行的。
本领域技术人员在考虑说明书及实践这里发明的发明后，将容易想到本公开的其他实施例。本申请旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未发明的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本公开的真正范围和精神由权利要求指出。

Claims (10)

  1. 一种机器人的重定位方法，包括：
    获取通过摄像装置采集的机器人的图像数据、通过惯性测量装置以及位移测量装置采集的所述机器人的速度数据以及位移数据，以及通过激光雷达采集的所述机器人的激光点云数据；
    对所述图像数据的局部视觉特征进行特征编码得到图像矢量特征，并在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果；
    根据各所述检索结果对所述机器人的位姿进行求解，得到第一位姿初始值，并根据所述速度数据以及位移数据在预设的二维栅格地图中对所述激光点云数据进行匹配，得到第一匹配结果；
    在判断所述第一位姿初始值以及第一匹配结果不一致时，根据所述第一位姿初始值以及所述第一匹配结果生成第二匹配结果，并根据所述第二位姿初始值、第一匹配结果以及第二匹配结果对所述机器人进行重定位。
  2. 根据权利要求1所述的机器人的重定位方法，其中，对所述图像数据的局部视觉特征进行特征编码得到图像矢量特征包括：
    提取所述图像数据的局部视觉特征，并在预设的视觉词典中查找与所述局部视觉特征临近的视觉单词；
    计算所述局部视觉特征与所述视觉单词之间的残差向量，并将属于同一所述视觉单词的残差向量进行累加求和操作得到残差和向量；
    根据所述视觉单词在所述预设的视觉词典中的索引顺序对所述残差和向量进行连接，得到所述图像矢量特征。
  3. 根据权利要求1所述的机器人的重定位方法，其中，在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果包括：
    对所述图像矢量特征进行降维处理，并对降维处理后的图像矢量特征进行空间划分，得到多个子向量；
    在预设的关键帧数据库中获取与各所述子向量对应的聚类中心以及各所述聚类中心的索引，并计算各所述子向量与各所述聚类中心之间的距离；
    在判断所述距离小于预设阈值时，根据所述聚类中心的索引获取与所述聚类中心对应的图像矢量，并根据多个所述图像矢量得到多个所述检索结果。
  4. 根据权利要求1所述的机器人的重定位方法，其中，根据各所述检索结果对所述机器人的位姿进行求解，得到第一位姿初始值包括：
    对各所述检索结果进行特征匹配，得到各所述检索结果之间的匹配特征以及匹配特征之间的匹配关系；
    根据各所述匹配特征以及各所述匹配特征之间的匹配关系对所述机器人的位姿进行求解，得到第一位姿初始值。
  5. 根据权利要求1所述的机器人的重定位方法，其中，根据所述速度数据以及位移数据在预设的二维栅格地图中对所述激光点云数据进行匹配，得到第一匹配结果包括：
    对所述速度数据以及位移数据进行融合，得到第二位姿初始值；
    基于所述第二位姿初始值在预设的二维栅格地图中对所述激光点云数据进行匹配，得到第一匹配结果。
  6. 根据权利要求1所述的机器人的重定位方法，其中，根据所述第二位姿初始值、第一匹配结果以及第二匹配结果对所述机器人进行重定位包括：
    分别为所述第二位姿初始值、第一匹配结果以及第二匹配结果配置第一权重值、第二权重值以及第三权重值；
    对所述第二位姿初始值、第一匹配结果、第二匹配结果，以及第一权重值、第二权重值、第三权重值分别进行加权求和；
    根据加权求和结果对所述机器人进行重定位。
  7. 根据权利要求1所述的机器人的重定位方法，其中，所述机器人的重定位方法还包括：
    判断所述图像数据是否满足预设条件；若是，则将所述图像数据作为关键帧存储至所述关键帧数据库中；
    其中，所述预设条件至少包括以下一项：
    所述速度数据以及与所述速度数据对应的上一时刻的速度数据之间的差值大于预设速度阈值；
    所述位移数据以及与所述位移数据对应的上一时刻的位移数据之间的差值大于预设位移阈值；
    所述图像数据以及与所述图像数据对应的上一帧图像数据之间的重叠部分小于预设重叠阈值。
  8. 一种机器人的重定位装置，包括：
    数据获取模块，用于获取通过摄像装置采集的机器人的图像数据、通过惯性测量装置以及位移测量装置采集的所述机器人的速度数据以及位移数据，以及通过激光雷达采集的所述机器人的激光点云数据；
    特征编码模块，用于对所述图像数据的局部视觉特征进行特征编码得到图像矢量特征，并在预设的关键帧数据库中对降维处理后的图像矢量特征进行检索，得到多个检索结果；
    数据匹配模块，用于根据各所述检索结果对所述机器人的位姿进行求解，得到第一位姿初始值，并根据所述速度数据以及位移数据在预设的二维栅格地图中对所述激光点云数据进行匹配，得到第一匹配结果；
    重定位模块，用于在判断所述第一位姿初始值以及第一匹配结果不一致时，根据所述第一位姿初始值以及所述第一匹配结果生成第二匹配结果，并根据所述第二位姿初始值、第一匹配结果以及第二匹配结果对所述机器人进行重定位。
  9. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1-7任一项所述的机器人的重定位方法。
  10. 一种电子设备，包括：
    处理器；以及
    存储器，用于存储所述处理器的可执行指令；
    其中，所述处理器配置为经由执行所述可执行指令来执行权利要求1-7任一项所述的机器人的重定位方法。
PCT/CN2021/098981 2020-07-15 2021-06-08 机器人的重定位方法及装置、存储介质、电子设备 Ceased WO2022012229A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21842606.2A EP4160474A4 (en) 2020-07-15 2021-06-08 ROBOT RELOCATION METHOD AND APPARATUS, AND STORAGE MEDIUM AND ELECTRONIC DEVICE
US18/005,362 US20230266470A1 (en) 2020-07-15 2021-06-08 Robot relocalization method and apparatus, and storage medium and electronic device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010680255.2A CN111931589B (zh) 2020-07-15 2020-07-15 机器人的重定位方法及装置、存储介质、电子设备
CN202010680255.2 2020-07-15

Publications (1)

Publication Number Publication Date
WO2022012229A1 true WO2022012229A1 (zh) 2022-01-20

Family

ID=73313051

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/098981 Ceased WO2022012229A1 (zh) 2020-07-15 2021-06-08 机器人的重定位方法及装置、存储介质、电子设备

Country Status (4)

Country Link
US (1) US20230266470A1 (zh)
EP (1) EP4160474A4 (zh)
CN (1) CN111931589B (zh)
WO (1) WO2022012229A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114595238A (zh) * 2022-03-04 2022-06-07 高德软件有限公司 基于矢量的地图处理方法及装置
CN114913226A (zh) * 2022-03-31 2022-08-16 阿里巴巴（中国）有限公司 定位方法、装置及计算机存储介质
CN114967540A (zh) * 2022-05-10 2022-08-30 禾多科技（北京）有限公司 车载传感器的数据采集设备
CN115683129A (zh) * 2023-01-04 2023-02-03 苏州尚同墨方智能科技有限公司 一种基于高清地图的长期重定位方法和装置
CN116358552A (zh) * 2023-03-14 2023-06-30 麦岩智能科技(北京)有限公司 用于机器人的重定位方法、存储介质及设备

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111931589B (zh) * 2020-07-15 2024-06-21 京东科技信息技术有限公司 机器人的重定位方法及装置、存储介质、电子设备
CN112710299B (zh) * 2020-12-04 2024-05-17 深圳市优必选科技股份有限公司 一种重定位方法、装置、终端设备及存储介质
CN112621753B (zh) * 2020-12-09 2022-12-06 浙江大华技术股份有限公司 机器人定位方法、电子设备及存储介质
CN113984064B (zh) * 2021-10-27 2024-10-08 山东亚历山大智能科技有限公司 一种用于室内移动机器人的定位方法及系统
CN114519817A (zh) * 2021-12-30 2022-05-20 北京盈迪曼德科技有限公司 机器人重定位方法、装置及机器人
CN114638943A (zh) * 2022-03-24 2022-06-17 广州高新兴机器人有限公司 巡逻机器人重定位方法及巡逻机器人
CN115128621B (zh) * 2022-06-29 2025-05-13 广州高新兴机器人有限公司 基于二维栅格地图的机器人重定位方法、装置及机器人
CN116007609A (zh) * 2022-12-19 2023-04-25 北京理工大学 一种多光谱图像和惯导融合的定位方法和计算系统
CN115980775A (zh) * 2022-12-21 2023-04-18 广东电网有限责任公司 一种配网树障的无人机检测方法及相关装置
CN120451273B (zh) * 2025-07-09 2025-09-19 重庆长安汽车股份有限公司 视觉重定位方法、装置、计算机设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106092104A (zh) * 2016-08-26 2016-11-09 深圳微服机器人科技有限公司 一种室内机器人的重定位方法及装置
CN108759844A (zh) * 2018-06-07 2018-11-06 科沃斯商用机器人有限公司 机器人重定位与环境地图构建方法、机器人及存储介质
CN109084732A (zh) * 2018-06-29 2018-12-25 北京旷视科技有限公司 定位与导航方法、装置及处理设备
WO2019062651A1 (zh) * 2017-09-26 2019-04-04 上海思岚科技有限公司 一种定位建图的方法及系统
CN110533722A (zh) * 2019-08-30 2019-12-03 的卢技术有限公司 一种基于视觉词典的机器人快速重定位方法及系统
CN111931589A (zh) * 2020-07-15 2020-11-13 北京海益同展信息科技有限公司 机器人的重定位方法及装置、存储介质、电子设备

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2776216B1 (en) * 2011-11-11 2022-08-31 iRobot Corporation Robot apparautus and control method for resuming operation following a pause.
CN109389641A (zh) * 2017-08-02 2019-02-26 北京贝虎机器人技术有限公司 室内地图综合数据生成方法及室内重定位方法
CN110657803B (zh) * 2018-06-28 2021-10-29 深圳市优必选科技有限公司 机器人定位方法、装置以及存储装置
CN109460267B (zh) * 2018-11-05 2021-06-25 贵州大学 移动机器人离线地图保存与实时重定位方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106092104A (zh) * 2016-08-26 2016-11-09 深圳微服机器人科技有限公司 一种室内机器人的重定位方法及装置
WO2019062651A1 (zh) * 2017-09-26 2019-04-04 上海思岚科技有限公司 一种定位建图的方法及系统
CN108759844A (zh) * 2018-06-07 2018-11-06 科沃斯商用机器人有限公司 机器人重定位与环境地图构建方法、机器人及存储介质
CN109084732A (zh) * 2018-06-29 2018-12-25 北京旷视科技有限公司 定位与导航方法、装置及处理设备
CN110533722A (zh) * 2019-08-30 2019-12-03 的卢技术有限公司 一种基于视觉词典的机器人快速重定位方法及系统
CN111931589A (zh) * 2020-07-15 2020-11-13 北京海益同展信息科技有限公司 机器人的重定位方法及装置、存储介质、电子设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4160474A4

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114595238A (zh) * 2022-03-04 2022-06-07 高德软件有限公司 基于矢量的地图处理方法及装置
CN114913226A (zh) * 2022-03-31 2022-08-16 阿里巴巴（中国）有限公司 定位方法、装置及计算机存储介质
CN114967540A (zh) * 2022-05-10 2022-08-30 禾多科技（北京）有限公司 车载传感器的数据采集设备
CN115683129A (zh) * 2023-01-04 2023-02-03 苏州尚同墨方智能科技有限公司 一种基于高清地图的长期重定位方法和装置
CN116358552A (zh) * 2023-03-14 2023-06-30 麦岩智能科技(北京)有限公司 用于机器人的重定位方法、存储介质及设备

Also Published As

Publication number Publication date
CN111931589B (zh) 2024-06-21
US20230266470A1 (en) 2023-08-24
EP4160474A4 (en) 2024-08-07
CN111931589A (zh) 2020-11-13
EP4160474A1 (en) 2023-04-05

Similar Documents

Publication Publication Date Title
WO2022012229A1 (zh) 机器人的重定位方法及装置、存储介质、电子设备
Tsintotas et al. The revisiting problem in simultaneous localization and mapping: A survey on visual loop closure detection
US11182612B2 (en) Systems and methods for place recognition based on 3D point cloud
Masone et al. A survey on deep visual place recognition
Zhou et al. To learn or not to learn: Visual localization from essential matrices
EP4131030B1 (en) Method and apparatus for searching for target
KR102433743B1 (ko) 로봇의 지도 구축 및 위치 추적
CN111652934B (zh) 定位方法及地图构建方法、装置、设备、存储介质
Garg et al. Seqmatchnet: Contrastive learning with sequence matching for place recognition & relocalization
CN110146080B (zh) 一种基于移动机器人的slam回环检测方法及装置
Du et al. 3-D LiDAR-based place recognition techniques: A review of the past ten years
Song et al. Semantic mapping techniques for indoor mobile robots: Review and prospect
Pradeep et al. Visual loop closing using multi-resolution sift grids in metric-topological slam
Tsintotas et al. Visual place recognition for simultaneous localization and mapping
Nowakowski et al. Topological localization using Wi-Fi and vision merged into FABMAP framework
WO2020194079A1 (en) Method and system for performing localization of an object in a 3d
Xu et al. Point2graph: An end-to-end point cloud-based 3d open-vocabulary scene graph for robot navigation
CN114943766A (zh) 重定位方法、装置、电子设备及计算机可读存储介质
Jung et al. GOTPR: General Outdoor Text-based Place Recognition Using Scene Graph Retrieval with OpenStreetMap
Yoshiki et al. Scalable change detection from 3d point cloud maps: Invariant map coordinate for joint viewpoint-change localization
Wen et al. Efficient indoor positioning with visual experiences via lifelong learning
Garcia-Fidalgo et al. Methods for Appearance-based Loop Closure Detection
Chen et al. Astra: Toward General-Purpose Mobile Robots via Hierarchical Multimodal Learning
Gong et al. Adaptive soft encoding: A general unsupervised feature aggregation method for place recognition
Yin et al. A PCLR-GIST algorithm for fast image retrieval in visual indoor localization system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21842606

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021842606

Country of ref document: EP

Effective date: 20221230

NENP Non-entry into the national phase

Ref country code: DE