WO2023198088A1 - 机器人的位姿确定方法、装置、机器人以及存储介质 - Google Patents

机器人的位姿确定方法、装置、机器人以及存储介质 Download PDF

Info

Publication number
WO2023198088A1
WO2023198088A1 PCT/CN2023/087710 CN2023087710W WO2023198088A1 WO 2023198088 A1 WO2023198088 A1 WO 2023198088A1 CN 2023087710 W CN2023087710 W CN 2023087710W WO 2023198088 A1 WO2023198088 A1 WO 2023198088A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
pose
nodes
robot
measurement
Prior art date
Application number
PCT/CN2023/087710
Other languages
English (en)
French (fr)
Inventor
龚喜
Original Assignee
深圳市普渡科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202210380299.2A external-priority patent/CN114463429B/zh
Priority claimed from CN202211580709.4A external-priority patent/CN116168080A/zh
Application filed by 深圳市普渡科技有限公司 filed Critical 深圳市普渡科技有限公司
Publication of WO2023198088A1 publication Critical patent/WO2023198088A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods

Definitions

  • the present application relates to the field of robotic technology, and in particular, to a method, device, robot, and storage medium for determining the pose of a robot.
  • forward-looking lasers are usually used to obtain the natural features in front of the robot for positioning.
  • the natural features in front of the robot are generally tables, chairs, sofas and other items on the ground. These items may be moved, causing Positioning fails or drifts, affecting the positioning effect.
  • a robot pose determination method is provided.
  • this application provides a method for determining the pose of a robot, including:
  • the visual map information database includes multiple top view images collected by the robot in different postures in the driving area;
  • the current robot pose when the robot collects the current positioning image is determined.
  • this application also provides a method for establishing a visual map information database of a robot.
  • the visual map information database is used in any of the above mentioned pose determination methods.
  • the establishment method includes:
  • top-view images of the driving area are obtained through the RGB camera, and the top-view images include multiple reference feature points;
  • a visual map information database is constructed based on each top perspective image, the feature points in each top perspective image, and the fourth measurement pose corresponding to each top perspective image.
  • this application also provides a robot, including a memory and a processor.
  • the memory stores a computer program.
  • the processor executes the computer program, the steps of any of the above methods are implemented.
  • the present application also provides a computer-readable storage medium on which a computer program is stored.
  • a computer program is stored on which a computer program is stored.
  • the present application also provides a computer program product, including a computer program that implements the steps of any of the above methods when executed by a processor.
  • Figure 1 is a schematic flowchart of a method for determining the pose of a robot in one embodiment
  • Figure 2 is a schematic flowchart of a method for determining the pose of a robot in another embodiment
  • Figure 3 is a schematic flowchart of determining matching feature point pairs in one embodiment
  • Figure 4 is a schematic flowchart of a method for determining the pose of a robot in another embodiment
  • Figure 5 is a schematic diagram of the correspondence between the positioning image and the measured pose in one embodiment
  • Figure 6 is a schematic flowchart of determining the robot pose corresponding to each first measurement pose in one embodiment
  • Figure 7 is a schematic flowchart of a method for determining the pose of a robot in another embodiment
  • Figure 8 is a schematic flowchart of building a visual map information database in one embodiment
  • Figure 9 is a schematic flowchart of building a visual map information database in another embodiment
  • Figure 10 is a schematic flow chart of constructing a visual map information database in another embodiment
  • Figure 11 is a schematic flowchart of a map creation and positioning method in one embodiment
  • Figure 12 is a structural block diagram of a robot pose determination device in one embodiment
  • Figure 13 is a structural block diagram of a device for establishing a robot's visual map information database in one embodiment
  • Figure 14 is an internal structural diagram of the robot in one embodiment.
  • This application is applied to various robots such as sweeping robots and shopping mall robots.
  • the robots can map a certain area through image sensors (cameras).
  • image sensors cameras
  • the images collected by the camera have richer features than laser images, it is beneficial to the accuracy of positioning.
  • a method for determining the pose of a robot which can also be called a positioning method.
  • This embodiment illustrates the application of this method to a robot, which includes the following steps:
  • the current positioning image is the top perspective image collected by the robot in the driving area.
  • the current positioning image is used to position the current robot pose (Pose) when the robot collects this image.
  • the robot's driving area refers to the ground area where the robot travels
  • the top perspective image is the image of the top area corresponding to the ground area.
  • the RGB camera is specifically disposed on the top of the robot, and its main axis of view is upward, which can be vertically upward or tilted upward, thereby obtaining a top view image as the current positioning image.
  • the top perspective image when the driving area is an indoor scene, may specifically be an indoor ceiling image.
  • the top perspective image includes objects above the robot, such as the sky, leaves, eaves, etc., which are not limited here.
  • the RGB camera can also be installed in other places on the robot, as long as it can obtain a top-view image, and there is no specific limitation here.
  • the current positioning image may be a top-view image collected when the robot determines the robot's posture for the first time in the driving area, or may be a top-view image collected when the robot's posture is not determined for the first time.
  • the current positioning image may be an RGB image.
  • an RGB camera is provided on the top of the robot.
  • the top perspective image at the current moment can be collected through the RGB camera as the above-mentioned current positioning image.
  • the RGB camera can acquire top-view images when triggered by the operator, or can automatically acquire top-view images periodically.
  • the robot can also be equipped with other image acquisition devices other than RGB cameras to obtain the current positioning image. In this embodiment, there is no specific limit on the type of image acquisition device.
  • the visual map information database includes multiple top-view images collected by the robot in different postures in the driving area.
  • the robot uses an image similarity algorithm to calculate the similarity between the current positioning image and the top perspective image of each frame in the visual map information database, and obtains the top perspective image with the greatest similarity as the target top perspective image.
  • the above image similarity algorithm may be a Structural Similarity (SSIM) algorithm, which determines the similarity between images through the comparison of brightness, contrast and structure between the current positioning image and the top view image.
  • the above image similarity algorithm can also be a vector similarity algorithm. By using vector representations of the current positioning image and the top perspective image, the cosine value between the two vectors is calculated as the similarity between the images.
  • the above image similarity algorithm can also be a histogram similarity algorithm, which calculates the histograms of the current positioning image and the top perspective image, and then calculates the normalized correlation coefficient between the two histograms (such as Bhattacharyya distance, histogram Intersection distance, etc.) to obtain the similarity between images.
  • the above image similarity algorithm may also be a hash similarity algorithm, which determines the similarity between images by calculating respective hash values of the current positioning image and the top perspective image.
  • the candidate feature points are the feature points in the current positioning image
  • the reference feature points are the feature points in the target top perspective image.
  • the above candidate feature points and reference feature points can be ORB (Oriented FAST and Rotated BRIEF) points, which can be detected based on the FAST (features from accelerated segment test) algorithm, or other feature points, such as sift points or other The points of deep learning are obtained by using the corresponding detection algorithm.
  • the visual map information database also includes the three-dimensional coordinates of the reference feature points in each top view image in the real scene.
  • the robot determines the target top perspective image in the visual map information database, it can determine the current position when the robot collects the current positioning image based on the two-dimensional coordinates of the candidate feature points in the current positioning image and the three-dimensional coordinates of the reference feature points in the target top perspective image. Robot pose.
  • the reference feature points in the top view image are processed by global optimization (Global BA) to minimize the reprojection error of each feature point in the top view image, and then remove the feature points whose reprojection error is greater than the error threshold.
  • the obtained feature points that is, the top view image includes multiple feature points, and based on the reprojection error of each feature point, some feature points are selected as reference feature points in the top view image.
  • the robot can extract candidate feature points in the current positioning image through the feature point recognition model, and then obtain the two-dimensional coordinates of each candidate feature point in the current positioning image.
  • the robot further determines matching feature point pairs between candidate feature points and reference feature points, and uses PNP (Perspective-n-Point, Multi-point perspective imaging) algorithm determines the current robot pose when the robot collects the current positioning image.
  • PNP Perspective-n-Point, Multi-point perspective imaging
  • the robot collects top-view images in the driving area through the RGB camera installed on the top of the robot as the current positioning image, and obtains the target top-view image with the greatest similarity to the current positioning image in the visual map information database, and then based on The two-dimensional coordinates of the candidate feature points in the current positioning image and the three-dimensional coordinates of the reference feature points in the target top view image determine the current robot pose when the robot collects the current positioning image.
  • the visual map information database includes multiple top-view images collected by the robot in different postures in the driving area.
  • both candidate feature points and reference feature points include multiple, as shown in Figure 2, the above S130, according to the two-dimensional coordinates of the candidate feature points in the current positioning image and the three-dimensional coordinates of the reference feature points in the target top perspective image, Determine the current robot pose when the robot collects the current positioning image, including:
  • the feature vector is used to characterize the direction and position of the feature point in the image.
  • any position in the current positioning image is used as a reference point, and the vector formed from the reference point to the candidate feature point is the feature vector of the candidate feature point in the current positioning image.
  • any position in the target top perspective image is used as a reference point, and the vector formed from the reference point to the reference feature point is the feature vector of the reference feature point in the target top perspective image.
  • the position of the reference point in the current positioning image and the reference point in the target top perspective image are the same.
  • the reference point in the current positioning image is the center point of the current positioning image
  • the reference point in the target top perspective image is the center point of the target top perspective image.
  • the robot vectorizes the candidate feature points in the current positioning image and the reference feature points in the target top view image, determines the feature vector of each candidate feature point through the reference point in the current positioning image, and determines the feature vector of each candidate feature point through the target top
  • the reference points in the perspective image determine the feature vectors of each reference feature point.
  • S220 Determine matching feature point pairs between the current positioning image and the target top perspective image based on the feature vectors of each candidate feature point and the feature vector of each reference feature point.
  • the robot may determine matching feature point pairs based on the similarity between each candidate feature point and each reference feature point.
  • the robot vectorizes the candidate feature points and the reference feature points to determine the similarity between each candidate feature point and each reference feature point based on the feature vector of each candidate feature point and the feature vector of each reference feature point. degree, and then determine the matching feature point pair between the current positioning image and the target top perspective image among multiple candidate feature points and multiple reference feature points based on the obtained similarity.
  • the robot can use the PNP algorithm to determine the position of the robot when the robot collects the current positioning image based on the two-dimensional coordinates of the candidate feature points in the matching feature point pair and the three-dimensional coordinates of the reference feature point.
  • the matching feature point pairs include at least three pairs.
  • the cosine value between vectors can be used to represent the similarity between each candidate feature point and each reference feature point.
  • the above S220 according to the feature vector of each candidate feature point and each reference feature
  • the feature vector of the point determines the matching feature point pair between the current positioning image and the target top view image, including:
  • the robot uses the inter-vector cosine calculation formula to calculate the cosine value between the feature vector of each candidate feature point and the feature vector of each reference feature point, so that the feature vector of each candidate feature point and any reference feature point can be obtained The cosine value between the eigenvectors.
  • the cosine value constraint condition includes that the cosine value is greater than the cosine value threshold.
  • the robot compares the obtained cosine value with a preset cosine value threshold, and determines the candidate feature points and reference feature points corresponding to the cosine value greater than the cosine value threshold as matching feature point pairs.
  • the current positioning image includes candidate feature points A, B, C, and D. Taking the center point O of the current positioning image as the reference point, the corresponding feature vector is obtained.
  • the target top perspective image includes reference feature points a, b, and c.
  • the robot can determine that the candidate feature point A and the reference feature point a are a matching feature point pair, and the candidate feature point B and the reference feature point b are a matching feature point pair.
  • the candidate feature point C and the reference feature point c are matching feature point pairs.
  • the robot can determine the candidate feature point A and the reference feature point a as a matching feature point pair.
  • the robot obtains the feature vector of each candidate feature point in the current positioning image and the feature vector of each reference feature point in the target top view image, and then based on each The feature vectors of the candidate feature points and the feature vectors of each reference feature point are used to determine the matching feature point pairs between the current positioning image and the target top view image, so as to match the candidate feature points in the current positioning image based on the matching feature points.
  • the three-dimensional coordinates of the reference feature point in the matching feature point pair determine the current robot pose when the robot collects the current positioning image.
  • the matching feature point pair is based on vectorization processing to determine the matching feature point pair based on the similarity between the cosine values between the feature vectors of each candidate feature point and the feature vector of each reference feature point.
  • the matching accuracy of the matched feature point pairs can be improved.
  • the PNP algorithm is used to solve the current robot pose, which accordingly improves the accuracy of the determined current robot pose.
  • the above method also includes:
  • the preset sensor can directly acquire the pose of the robot, and the pose of the robot acquired by the preset sensor is called the measurement pose.
  • the preset sensor may be a positioning sensor, for example, an odometer provided on the robot, such as a wheel odometer, or an IMU (Inertial measurement unit).
  • the robot can continuously obtain the measurement postures of the robot at different times through the preset sensor.
  • the first measurement posture multiple first measurement postures can be obtained.
  • S420 Obtain the second measurement pose corresponding to the time of the current positioning image based on the two first measurement poses before and after the time of the current positioning image.
  • the time at which the current positioning image is located is the current time at which the current positioning image is sampled.
  • the sampling frequency of the preset sensor and the RGB camera is different. Normally, the sampling frequency of the preset sensor for pose sampling is higher than the sampling frequency of the RGB camera.
  • the robot obtains two first measurement poses whose measurement moments are adjacent to and before the moment of the current positioning image (i.e., the current moment).
  • the two first measurement poses can be The pose is interpolated to obtain the second measured pose corresponding to the moment of the current positioning image.
  • the robot obtains two first measurement poses whose measurement moments are adjacent to and before the moment of the current positioning image (i.e., the current moment).
  • the two first measurement poses can be The pose is interpolated to obtain the second measured pose corresponding to the moment of the current positioning image.
  • the first measurement poses Pose1 and Pose2 that are adjacent to and before the current positioning image A at the measurement time to obtain the second measurement corresponding to the time at which the current positioning image A is located.
  • PoseA optionally, a commonly used interpolation algorithm can be used to obtain the second measurement pose PoseA. There are no restrictions here.
  • the current positioning image B interpolate the first measurement poses Pose4 and Pose5 that are adjacent to and before the current positioning image B at the measurement time to obtain the second measurement pose PoseB corresponding to the time at which the current positioning image B is located; optional Yes, commonly used interpolation algorithms can be used to obtain the second measurement pose PoseB. There are no restrictions here.
  • the robot can determine the corresponding moment of each first measured pose based on the relative relationship between the current robot pose and the second measured pose. robot pose.
  • the above-mentioned S430 determining the robot posture at the corresponding moment of each first measurement posture based on the current robot posture and the second measurement posture, includes:
  • T_o0 represents the second measurement pose
  • T0_inv represents the inverse matrix of the current robot pose
  • the current positioning image is updated periodically, and accordingly, the relative pose dT is also updated with the update of the current positioning image.
  • the time of the first measurement posture is after the time of the current positioning image and before the time of the next positioning image, that is, for the robot
  • positioning image B is the next positioning image of current positioning image A.
  • the poses are Pose2, Pose3 and Pose4.
  • the robot can take Pose2 as T_o0, and bring Pose2 as T_o0 and the relative pose dT calculated above into formula (1) to obtain the inverse matrix of the robot pose at the corresponding moment of Pose2. , and then obtain the robot pose Pose2' at the corresponding time.
  • the robot poses Pose3', Pose4' and Pose5' at the corresponding time of Pose3, Pose4 and Pose5 are obtained respectively.
  • the robot obtains the measurement postures of the robot at different times through the preset sensor, and obtains multiple first measurement postures, and obtains the current positioning image based on the two first measurement postures before and after the current positioning image.
  • the second measurement posture corresponding to the current time is determined based on the current robot posture and the second measurement posture to determine the robot posture corresponding to each first measurement posture at the time.
  • the relative pose between the current robot pose and the second measured pose can be obtained, so that when the time of the first measured pose is located after the time of the current positioning image and before the time of the next positioning image, by The first measured pose and the relative pose obtain the robot pose at the corresponding moment of the first measured pose.
  • the robot pose is determined at the corresponding moment between the current positioning image and the next positioning image. It not only ensures higher positioning frequency but also improves positioning accuracy, making it suitable for real-time positioning.
  • a map is first created based on the image obtained by the camera and the pose of the robot obtained from the positioning sensor, and then the created map is stored in the robot's visual map information library as a pre-stored map.
  • the robot can then use the pre-stored map and the robot's position to The images taken during the movement are positioned.
  • the images taken by the camera during the movement are matched with the feature points in the pre-stored map, thereby reducing the impact of light.
  • the images obtained by the camera and the pre-stored map contain rich features. Therefore, the accuracy of positioning is improved.
  • this embodiment of the present application also provides a positioning method, which can also be called a robot pose determination method.
  • the method includes:
  • the positioning method in this embodiment can be applied to any positioning equipment, such as cleaning robots, navigation robots, etc.
  • the positioning equipment is equipped with cameras and positioning sensors. Taking a robot as an example to illustrate, during the positioning process, the image of the robot's posture is first obtained through the camera on the robot, that is, the current positioning image, which is defined as the first image.
  • the camera can be an RGB camera.
  • the visual map information library includes pre-stored maps.
  • the pre-stored map is a visual map that is created by the robot in advance based on the image obtained by the camera and the robot's posture obtained based on the positioning sensor, and is stored in the visual map information database.
  • the robot compares the similarity between the first image and the images in the pre-stored map, and determines the image in the pre-stored map that is most similar to the first image, which is defined as the target image, where the pre-stored map is generated by the robot based on at least two nodes.
  • the pre-stored map includes at least two second images, at least two The coordinate information in the three-dimensional space of the feature points of each of the second images and the corresponding pose of each of the at least two second images.
  • DBOW word band model
  • DBOW word band model
  • Extract feature points from the first image and the target image match the feature points in the first image and the target image, and determine the matching feature points in the first image and the target image.
  • use image feature point matching technology Match the feature points in the first image and the target image to determine the matching feature points in the first image and the target image.
  • the robot can calculate the pose of the first image according to the feature points in the target image that match the first image.
  • the current pose of the robot when the robot shoots the first image is the positioning information of the robot.
  • the three-dimensional coordinates of the feature points in the first image that match the target image are compared with the three-dimensional coordinates of the feature points in the first image that match the first image, and then the obtained three-dimensional coordinates of the feature points in the first image are used as the PnP algorithm
  • the pose of the shooting position of the first image in the visual map is calculated, that is, the positioning information of the robot is obtained.
  • the visual map information database stores top-view images of the robot's driving area, so the above target image is the target top-view image.
  • the first image corresponding to the current position of the robot is acquired through the camera, the similarity between the first image and the image in the pre-stored map is compared, and the target image in the pre-stored map with the highest similarity to the first image is determined, where,
  • the pre-stored map includes at least two second images, coordinate information in three-dimensional space of feature points of each of the at least two second images, and poses corresponding to each of the at least two second images; Extracting feature points in the first image to obtain a plurality of first feature points, matching the plurality of first feature points with feature points of the target image, and determining which of the plurality of first feature points are consistent with the feature points of the target image.
  • the pre-stored map in this application extracts feature points based on the poses of at least two nodes and the second image corresponding to each node in the poses of at least two nodes, and creates the map after optimizing the pose based on the feature points.
  • the pre-stored map is mainly created using feature points in the image, which is less affected by lighting, and the image is taken by a camera, which has richer features than the laser map, so the map accuracy is high.
  • the first image obtained by the camera and the pre-stored map are used. Compare to determine the target image in the pre-stored map that is most similar to the first image, and then use the target image and the matching feature points in the first image to calculate the pose to determine the location of the robot, that is to say, the camera is used to capture the image during the positioning process.
  • the image and the feature points in the pre-stored map are used to calculate the pose, taking advantage of the rich features of the image captured by the camera and reducing the influence of light, thus improving the positioning accuracy.
  • feature points in the first image are extracted to obtain a plurality of first feature points, including:
  • Feature points are extracted from the first histogram to obtain a plurality of candidate feature points in the first image as first feature points.
  • histogram equalization is first performed on the first image to obtain the corresponding first histogram, and then the first histogram in the first histogram is extracted. Feature points.
  • the above method further includes:
  • the measurement pose corresponding to the current position of the robot is obtained through the positioning sensor.
  • the measurement pose corresponding to the current position includes the coordinate information of the robot at the current position;
  • Obtain the top perspective image of the target that is most similar to the current positioning image in the visual map information database including:
  • the robot when the robot obtains the first image, it also obtains the measurement pose corresponding to the current position of the robot through the positioning sensor.
  • the measurement pose includes the coordinate information of the robot at the current position. Among them, the measured pose corresponding to the current position of the robot is the current pose corresponding to the current position of the robot.
  • to obtain the measurement posture corresponding to the current position of the robot through the positioning sensor it is only necessary to compare the similarity between the first image and the image in the pre-stored map, and determine the top perspective of the target in the pre-stored map that is the most similar to the first image. Just execute it before the image.
  • the process of determining the top-view image of the target with the highest similarity to the first image in the pre-stored map may include: first determining in the pre-stored map the current position of the robot. The position corresponding to the coordinate information in the corresponding measurement pose is defined as the target position, and then the target position is taken as the center and the radius is the preset radius to obtain the target range, and the images of the target range are screened in the pre-stored map as candidate images, where, The coordinates in the pose corresponding to the candidate image are within the target range. Finally, the target top perspective image with the highest similarity to the first image is found in the candidate image.
  • the positioning sensor is used to record the coordinates of the robot's movement in the corresponding area in the pre-stored map (for example, by recording the distance and direction of the robot's movement in the area, the coordinates of the robot in the area can be obtained correspondingly), and based on the coordinates, it can be determined
  • the approximate position of the robot in the pre-stored map (such as within 5m of the position corresponding to the coordinates in the pre-stored map), so directly comparing the obtained first image with the determined approximate position, you can quickly determine that the pre-stored map is similar to the first image
  • the top view image of the target with the highest degree of accuracy is used to record the coordinates of the robot's movement in the corresponding area in the pre-stored map (for example, by recording the distance and direction of the robot's movement in the area, the coordinates of the robot in the area can be obtained correspondingly), and based on the coordinates, it can be determined
  • the approximate position of the robot in the pre-stored map (such as within 5m
  • the above method further includes:
  • Obtain the robot's positioning information based on the first posture Specifically include:
  • the current robot pose is used as the first pose of the robot, the measured pose corresponding to the current position and the first pose are fused to obtain the fused pose, and the fused pose is used as the robot's positioning information.
  • the robot is equipped with a positioning sensor.
  • the measurement pose corresponding to the current position of the robot is also acquired through the positioning sensor.
  • the measured pose and the first pose corresponding to the current position are fused to obtain the fused pose, and the fused pose is Output as the robot's positioning information.
  • the measured pose and the first pose corresponding to the current position of the robot can be fused using extended Kalman filtering technology, or other technologies can be used for fusion, which is not limited here.
  • the frequency of the positioning sensor's input posture is Q
  • the frequency of the robot's first posture calculated through the above method is P.
  • Calculating P requires multiple steps, and Q will be much larger than P. Therefore, in this application, the positioning sensor is used
  • the sum of the measured poses is calculated to calculate the first pose of the robot in the predicted map, and the measured poses and the first pose are fused. Subsequently, positioning information can be output based on the measured poses obtained by the positioning sensor, which can greatly Increase the frequency of positioning information output.
  • This application also provides a method for constructing a robot's visual map information database, which can also be called a map creation method.
  • the construction method applies In the pose determination/positioning method of any of the above embodiments. As shown in Figure 8, the method includes:
  • S810 Acquire multiple top perspective images of the driving area through the RGB camera, and the top perspective images include multiple reference feature points.
  • the robot constructs the visual map information database corresponding to the driving area
  • the driving area when the driving area is an annular area, it can travel around the driving area in one circle clockwise and one circle counterclockwise.
  • the driving area is a non-circular area, you can drive from the starting point of the driving area to the key point, and then return to the starting point from the end point.
  • While the robot is driving, it acquires multiple top-view images of the driving area including multiple reference feature points through the RGB camera.
  • S820 Acquire the measurement postures of the robot at different times through the preset sensors, obtain a plurality of third measurement postures, and determine a fourth measurement posture corresponding to the top perspective image based on the third measurement postures.
  • the robot can obtain the measurement postures of the robot at different times through preset sensors, and as the third measurement postures, multiple third measurement postures can be obtained.
  • the sampling frequency of the preset sensor for pose sampling is higher than the sampling frequency of the RGB camera.
  • the two third measurement poses before and after are used to obtain the fourth measurement pose corresponding to the moment of the top view image.
  • S830 Construct a visual map information database based on each top perspective image, the feature points in each top perspective image, and the fourth measurement pose corresponding to each top perspective image.
  • the above-mentioned S830 constructs a visual map information database based on each top perspective image, the feature points in each top perspective image, and the fourth measurement pose corresponding to each top perspective image, include:
  • the robot after determining the fourth measurement pose corresponding to the time of each top view image, the robot further determines the reference special diagnosis point in the top view image based on the top view images corresponding to the two adjacent fourth measurement poses at the time.
  • the three-dimensional coordinates in the real scene can be obtained specifically using the PNP algorithm, which will not be described again here.
  • S920 Perform loopback processing on the plurality of fourth measurement poses to correct the plurality of fourth measurement poses, and obtain multiple corrected image poses.
  • the robot when the robot is circling the driving area to obtain top-view images, the robot can use the "word band" model to perform loop detection.
  • a loop When a loop is detected, it means that the robot has circled the driving area and returned to the starting position.
  • the robot can use the pose correction algorithm to correct the pose drift and obtain multiple corrected image poses.
  • S930 Use a preset optimization algorithm to optimize multiple image poses and three-dimensional coordinates of reference feature points to obtain multiple optimized image poses and multiple optimized three-dimensional coordinates.
  • the robot uses the corrected image pose to recalculate the three-dimensional coordinates of all feature points in the top-view image, and perform global optimization to minimize reprojection errors and remove re-projection errors. Feature points whose projection error is greater than the error threshold. After the above series of correction and optimization, multiple optimized image poses and multiple optimized three-dimensional coordinates are obtained.
  • the robot associates each top-view image, the reference feature points corresponding to the top-view image, the optimized three-dimensional coordinates of the reference feature points, and multiple optimized image poses one by one to form a visual map information database, that is, a visual map.
  • the information database includes each frame of the top perspective image acquired by the robot, as well as the reference feature points in the corresponding top perspective image, the three-dimensional coordinates of the reference feature points, and the image pose when the top perspective image of the frame was acquired.
  • the robot when determining the current robot pose of the current positioning image, the robot can obtain the current positioning in the visual map information library.
  • the image pose corresponding to the top view image of the target with the greatest image similarity is used as the current robot pose.
  • the parameters of the RGB camera and preset sensors (such as wheeled odometer) integrated/installed on the robot need to be calibrated.
  • the visual map information database is pre-established so that the robot can subsequently perform robot positioning directly based on the data in the visual map information database, thereby further improving the robot's pose determination efficiency.
  • the above construction/creation method also includes the following steps:
  • S1010 Use the acquired multiple top view images as second images, extract feature points of all second images, obtain the feature points of each second image in all second images, and determine any phase in at least two nodes. Correspondence of feature points in the second image of adjacent nodes, where the adjacent nodes are two nodes that are adjacent in creation time among at least two nodes.
  • the robot also uses the camera to capture multiple top-view images of the area at each node, which are defined as second images.
  • the images corresponding to the nodes adjacent to the creation time/creation position contain similar image content, that is, at the nodes adjacent to the creation time/creation position, the directions of the images taken by the robot are the same, or
  • the rotation angle is small (for example, the shooting angle of two nodes is less than 5°), so that the images corresponding to adjacent nodes have more identical feature points, which is beneficial to the subsequent creation process.
  • each time the robot creates a node, or after creating all nodes, or every time a preset number of nodes is created the robot determines the correspondence between the feature points in the image corresponding to the adjacent nodes. Specifically, during the movement of the robot, each shooting angle direction is close to or in the same direction, so there are more identical pixels in the second image taken corresponding to adjacent nodes. After creating a new node, perform feature point extraction on the second image corresponding to each node until feature points are extracted on the second image corresponding to all nodes, and then based on the features extracted from the second image of the adjacent node Matching is performed among the points to determine the relationship between the feature points in the second graph corresponding to adjacent nodes.
  • the map creation method in this embodiment can be applied to a map creation system consisting of an acquisition terminal and a map creation terminal.
  • the acquisition terminal is equipped with a camera and a positioning sensor.
  • the positioning sensor can be a wheeled odometer or other sensors.
  • the camera is used to obtain images of the environment where the terminal is located, and the positioning sensor can be used to record and obtain the pose of the terminal, which can include position and posture;
  • the map creation terminal is used to receive the images and poses transmitted by the acquisition terminal, and generate Maps are created using images and poses, and the map creation terminal can be a server, a personal computer, etc.
  • the map creation method of this application can also be applied to a robot equipped with a camera and a positioning sensor.
  • the robot can create a map based on the image obtained by its own camera and the pose obtained by the positioning sensor.
  • the map creation method is applied to a robot as an example.
  • the robot moves in the area where the map is to be created.
  • the robot follows a straight line or s A route moves through the area where the map is to be created.
  • the second pose at least includes the position, In other embodiments pose may include position and attitude.
  • Each node created by the robot during its movement can be created according to preset rules.
  • a new node is created every preset movement distance; or it can be created according to the time interval of the robot's movement. New nodes are then created, and the pose of the machine is obtained through the positioning sensor.
  • the number of nodes created by the robot is as large as possible, so that the images corresponding to all nodes can cover the entire map area to be created.
  • the nodes created by the robot can be evenly distributed in the area of the map to be created; or The nodes created by the robot can be connected end to end to form a complete graph, that is, the first node created and the last node created can be created at the same location, but at different creation times, forming a moving path connected end to end; or the robot creates
  • the nodes can be evenly distributed in the area of the map to be created, and each node can be connected to form a complete graph.
  • S1030 Use the preset first rule to optimize the third measurement pose of each node in the at least two nodes, and obtain the fourth measurement pose of each node in the at least two nodes.
  • the map is mapped according to the preset first rule.
  • the obtained third measurement pose of each node is optimized, and the updated third pose of each created node is obtained, which is defined as the fourth measurement pose.
  • the preset first rule may be a pose or pose graph optimization rule in traditional technology.
  • the map is created based on the corresponding relationship of the feature points in the second image of any adjacent node and the fourth measurement pose obtained by the update.
  • the map can be created based on the feature points of any adjacent node.
  • Correspondence of feature points in the second image and update obtained The obtained fourth measurement pose can use the triangulation algorithm to calculate the position of each feature point in the three-dimensional space, and then use the LM algorithm to optimize and adjust the position of each feature point in the three-dimensional space and the pose of each second image, that is A map with scales can be obtained to complete the creation of the map.
  • the created map is saved and stored in the visual map information library as a pre-stored map, or sent to the server for storage, so that other devices can obtain the map of the area from the server for use.
  • the saved map may include each feature point and the relationship between the feature points.
  • the third measurement pose of each node in at least two nodes created by the robot during its movement in the map area to be created is acquired through the positioning sensor, and the acquired multiple top perspective images are used as the second image, Perform feature point extraction on each second image in all second images to obtain feature points of all second images, and determine the correspondence between the feature points in the second image of any adjacent node in at least two nodes, where adjacent The nodes are two nodes that are adjacent in creation time among at least two nodes; use the preset first rule to optimize the third measured pose of each node in the at least two pose nodes, and obtain each of the at least two nodes.
  • the fourth measurement pose of the node create a map based on the corresponding relationship of the feature points in the second image of any adjacent node and the fourth measurement pose of each node in at least two nodes, and store it in the visual map information database.
  • the robot obtains the pose of the nodes created by itself during the movement of the map area to be created and obtains the image of each node through the camera, thereby using the camera to obtain an image with richer map features than the laser, and then extracts the features in the image point, which reduces the impact of lighting on the image and can obtain a higher-precision map, thereby laying the foundation for other devices to position in this area to improve the accuracy and accuracy of positioning.
  • feature point extraction is performed on all second images to obtain feature points of all second images, including:
  • Feature points are extracted for all rotated images, and third feature points of each rotated image in all rotated images are obtained.
  • the feature points of the second image include second feature points and third feature points.
  • determining the correspondence between the feature points in the second image of any adjacent nodes among at least two nodes includes:
  • the corresponding relationship between the feature points in the first image of any adjacent node among at least two nodes includes a first corresponding relationship and a second corresponding relationship.
  • the process of obtaining the pixel points in the second image may include: obtaining the feature points of the second image obtained twice: first time, for each third Feature points are extracted from the original images of the two images, and this time the extracted feature points are defined as second feature points; after rotating all the second images for the second time, the rotated images of each second image are obtained, which is defined as Rotate the image, and then extract feature points from each rotated image. This time, the extracted feature points are defined as third feature points.
  • the second image can be rotated 180 degrees to obtain a rotated image.
  • the second feature point and the third feature point of each second image are obtained in this way, and the relationship between the second feature points in the two connected second images is determined based on the original image of the second image (defined as the first correspondence relationship), And determine the relationship (second correspondence) between the third feature points in the adjacent rotated images based on the rotated images corresponding to the two adjacent second images.
  • the adjacent second images are A1 and A2, and the image A1 includes feature points a11, a12, and a13, and image A2 includes feature points a21, a22, and a23. Then determine the corresponding relationships between a11, a12, a13, and a21, a22, and a23 respectively.
  • the adjacent second images are A1 and A2, and the corresponding rotated images of the second images A1 and A2 are B1 and B2 respectively.
  • Image B1 includes feature points b11, b12, and b13
  • image B2 includes feature points b21, b22, and b23. , then determine the corresponding relationships between b11, b12, and b13 and b21, b22, and b23 respectively.
  • the obtained second image is rotated to obtain feature points in the original image and the rotated image of each second image, which can increase the number of feature points in the image, thereby providing more feature points for calculation in subsequent steps. , thereby improving the accuracy of the calculation results and laying the foundation for improving the accuracy of the map.
  • obtaining the pixel points in the second image may include: obtaining scale-invariant feature transform SIFT feature points in the second image.
  • SIFT feature points are implemented through the SIFT algorithm. The essence of the SIFT algorithm is to find key points (feature points) on different scale spaces and calculate the direction of the key points.
  • the key points extracted by SIFT are some very prominent points that will not change due to factors such as lighting, affine transformation, and noise, such as corner points, edge points, bright spots in dark areas, and dark points in bright areas. This can also improve the accuracy of the map.
  • the third measurement pose is optimized using the preset first rule to obtain the fourth measurement pose of each node in at least two nodes, including:
  • the third measurement pose is optimized according to the corresponding relationship of the third measurement pose of the adjacent nodes and the third measurement pose, and a fourth measurement pose of each node in the adjacent nodes is obtained.
  • the first correspondence relationship and the second correspondence relationship of adjacent nodes can be used to calculate the correspondence relationship between the shooting points (imaging points) of the adjacent second images.
  • the corresponding relationship The first correspondence relationship and the second correspondence relationship of the feature points in adjacent nodes can determine the three-dimensional image, the feature points in the three-dimensional image, and the correspondence relationship between the feature points in the three-dimensional space and the shooting points (imaging points) in the adjacent second image, That is, the corresponding relationship between the third measurement poses of adjacent nodes can be calculated based on the first correspondence relationship and the second correspondence relationship, thereby optimizing the third measurement pose of each node and obtaining the optimized third measurement position of each node. pose, and define the optimized pose as the fourth measurement pose.
  • feature points in the second original image and the rotated image are used.
  • the number of feature points is larger than that in a single original image. Therefore, the accuracy of the shooting points (imaging points) of each node can be improved.
  • the map Accuracy lays the foundation.
  • using a preset first rule to optimize the third measurement pose of each node in at least two nodes to obtain the fourth measurement pose of each node in at least two nodes includes:
  • the total residual of all nodes is calculated, including:
  • the measurement value between any adjacent nodes among at least two nodes is calculated by calculating the inverse matrix of the third measurement pose of any adjacent node among at least two nodes;
  • the total residual of all nodes is calculated based on the residual between any adjacent nodes in at least two nodes.
  • the fourth measurement pose of all nodes is also optimized. Specifically, the inverse matrix of the third measurement pose of adjacent nodes is first calculated, and then based on The inverse matrix of the third measurement pose of the adjacent node calculates a measurement value between the second image of the adjacent node, the measurement value is based on the fourth measurement pose of the second image of the adjacent node and the fourth measurement pose.
  • T w1 is the pose of image A1
  • T 1w is the inverse matrix of T w1
  • T w2 is the pose of image A2
  • T 2w is the inverse matrix of T w2 .
  • the total residuals of all nodes are calculated based on the residuals of adjacent nodes.
  • the total residuals of all adjacent nodes are summed to obtain the total residuals of all nodes.
  • the corresponding pose of each node is used as the updated pose of each node, that is, the fourth measured pose.
  • the starting condition of the robot can be: determine whether the data of all nodes planned in the map area to be created (the poses and second images of all nodes) have been obtained. In this embodiment, judging whether there are images taken at the same location is used as a basis for obtaining data of all nodes planned in the map area to be created. Specifically, after obtaining the image of each node, it is determined whether there is an image at the same position, that is, whether there is a substantially identical image (two images with a similarity close to 100% indicate that the two images are substantially the same).
  • creating a map based on the correspondence between the feature points in the second image of any adjacent node and the fourth measured pose of each node in at least two nodes includes:
  • any phase among the at least two nodes is calculated through a triangulation algorithm.
  • the coordinate information of all feature points in the second image and the fourth measurement pose of each node in at least two nodes are optimized through a preset iterative algorithm to obtain the optimized coordinate information and optimization of all feature points in the second image.
  • a corresponding pre-stored map is obtained from all the second images, the optimized coordinate information of the feature points in all the second images, and the optimized fourth measured pose of each node in at least two nodes.
  • the process of creating the map may include: based on the corresponding relationship between the fourth measured pose of any adjacent node and the feature point in the first image of any adjacent node, calculating and obtaining the three-dimensional position of the feature point in the image of the adjacent node. The position in space, and then a map with scale can be constructed based on the position of the feature point in the three-dimensional space and the fourth measurement pose.
  • the position of the feature point in the first image in the three-dimensional space can be calculated through a triangulation algorithm according to the fourth measurement pose and the first image corresponding to the fourth measurement pose; further, according to the position of the feature point in the three-dimensional space
  • the position and the fourth measured pose are optimized through an iterative algorithm.
  • the iterative algorithm can be an LM algorithm to obtain the optimized coordinate information of all feature points in the second image and the optimized fourth coordinate information of each node in at least two nodes. Measure the pose, and then obtain the corresponding pre-stored map based on all the second images, the optimized coordinate information of the feature points in all the second images, and the optimized fourth measured pose of each node in at least two nodes.
  • the optimization of the fourth measurement posture can also be obtained by optimizing based on the optimized third measurement posture and the posture provided by the positioning sensor in the above embodiment.
  • the above method can be run in different devices, or can be run in the same device.
  • the process of running in the same device can include: controlling the robot to create a map Move along the preset path within the area.
  • the camera in the robot takes photos of the environment where the robot is located at preset time intervals to obtain multiple top-view images of the corresponding driving area, that is, the second image.
  • the odometer in the robot takes photos at preset time intervals.
  • a plurality of third measurement poses of the robot are acquired at intervals, and each third measurement pose corresponds to a second image.
  • the preset path can be a graphic connected end to end. After acquiring multiple second images, extract feature points from each second image, match the feature points in any two adjacent second images, and determine the correspondence of the feature points in any two adjacent second images. relationship, as shown in Figure 11.
  • the robot uses the third measurement posture to determine whether the robot has completed the preset path. Specifically, determine whether there are basically the same images (two images with a similarity close to 100% indicate that the two images are basically the same) to determine whether the robot has completed the preset path. .
  • the robot uses the corresponding relationship between the feature points in the second image and the feature points in the adjacent second images to optimize the pose of all the second images, and then based on the corresponding relationship between the feature points in the adjacent second images.
  • the corresponding relationship determines the position of each feature point in the three-dimensional space, and finally a map with scale is created based on the position of each feature point in the three-dimensional space and the pose corresponding to each optimized second image.
  • the created map is saved to the robot, and the robot can use the saved map for positioning.
  • the robot obtains the image of the current positioning image, which is recorded as the first image. Compare the first image and the saved map, find the image in the map that is most similar to the first image, define the image in the map that is most similar to the first image as the target image (i.e., the target top perspective image), and then Feature points are extracted from the image and the first image, and feature points in the first image and the target image are matched to determine matching feature points in the first image and the target image. Finally, the pose of the first image taken can be calculated based on the feature points in the target image that match the first image. As the pose of the robot taking the first image, this pose is the positioning information of the robot, thereby achieving positioning.
  • the specific processes implemented in this embodiment are the same as the above-mentioned positioning method and map creation method, and will not be described in detail here.
  • this application provides a robot posture determination device, including: a first acquisition module 1201, a second acquisition module 1202, and a posture determination module 1203, wherein:
  • the first acquisition module 1201 is used to acquire the current positioning image through the RGB camera; wherein the current positioning image is a top perspective image collected by the robot in the driving area;
  • the second acquisition module 1202 is used to obtain the top perspective image of the target with the greatest similarity to the current positioning image in the visual map information database; wherein the visual map information database includes multiple top views collected by the robot in different poses in the driving area. perspective image;
  • the pose determination module 1203 is used to determine the current robot pose when the robot collects the current positioning image based on the two-dimensional coordinates of the candidate feature points in the current positioning image and the three-dimensional coordinates of the reference feature points in the target top view image.
  • both the candidate feature points and the reference feature points include multiple; the pose determination module 1203 is specifically used to:
  • the pose determination module 1203 is specifically used to:
  • the above device further includes a real-time positioning module for:
  • the measurement postures of the robot at different times are obtained through preset sensors to obtain multiple first measurement postures; based on the two first measurement postures before and after the time of the current positioning image, a second measurement corresponding to the time of the current positioning image is obtained. pose; determine the robot pose at the corresponding moment of each first measured pose based on the current robot pose and the second measured pose.
  • the real-time positioning module is specifically used to:
  • the pose and relative pose obtain the robot pose at the corresponding moment of the first measured pose.
  • the pose determination module 1203 is also used to:
  • the visual map information library includes pre-stored maps, and the real-time positioning module is also used to:
  • the measurement pose corresponding to the current position of the robot is obtained through the positioning sensor.
  • the measurement pose corresponding to the current position includes the coordinate information of the robot at the current position;
  • the second acquisition module 1202 is also used to:
  • the pose determination module 1203 is also used to:
  • the current robot pose is used as the first pose of the robot, the measured pose corresponding to the current position and the first pose are fused to obtain the fused pose, and the fused pose is used as the robot's positioning information.
  • the present application provides a device for establishing a visual map information database of a robot.
  • the visual map information database is applied to any of the above mentioned pose determination methods.
  • the establishment method includes: acquisition module 1301, measurement module 1302 and construction module 1303, wherein:
  • the image acquisition module 1301 is used to acquire multiple top perspective images of the driving area through an RGB camera, where the top perspective images include multiple reference feature points;
  • the pose measurement module 1302 is used to obtain the measurement poses of the robot at different times through preset sensors, obtain a plurality of third measurement poses, and determine a fourth measurement pose corresponding to the top perspective image based on the third measurement poses;
  • the map construction module 1303 is used to map each top perspective image, the feature points in each top perspective image, and the corresponding top perspective image.
  • the fourth measurement pose constructs a visual map information database.
  • map building module 1303 is specifically used to:
  • the above device further includes:
  • the second feature extraction module is used to use the acquired multiple top view images as second images, extract feature points of all second images, obtain the feature points of each second image in all second images, and determine at least The correspondence relationship between the feature points in the second image of any adjacent nodes among the two nodes, where the adjacent nodes are at least two nodes whose creation time is adjacent among the two nodes;
  • the third measurement pose acquisition module is used to obtain the third measurement pose of each node of at least two nodes created by the robot during its movement in the driving area through the positioning sensor; wherein each top perspective image corresponds to each node;
  • a pose optimization module configured to optimize the third measurement pose of each of the at least two nodes using a preset first rule to obtain a fourth measurement pose of each of the at least two nodes;
  • a map creation module configured to create a pre-stored map based on the corresponding relationship between the feature points in the second image of any adjacent node among the at least two nodes and the fourth measurement pose of each node among the at least two nodes, and store it in the visual map database.
  • the second feature extraction module is also used to:
  • the feature points of the second image include the second feature point and the third feature point.
  • the second feature extraction module is also used to:
  • the corresponding relationship between the feature points in the first image of any adjacent node among at least two nodes includes a first corresponding relationship and a second corresponding relationship.
  • the pose optimization module is also used to:
  • the pose optimization module is also used to:
  • the pose optimization module is also used to:
  • the pose optimization module is also used to:
  • the map creation module is also used to:
  • any phase among the at least two nodes is calculated through a triangulation algorithm.
  • the coordinate information of the feature points in the second image of the adjacent node in the three-dimensional space; the coordinate information of all the feature points in the second image and the fourth of each node in at least two nodes are calculated through a preset iterative algorithm.
  • Optimize the measured poses to obtain the coordinate information of the optimized feature points in all second images and the optimized fourth measured poses of each node in at least two nodes; combine all second images, all optimized second images
  • the coordinate information of the feature points in the second image and the optimized fourth measured pose of each node in at least two nodes are used to obtain a corresponding pre-stored map.
  • a component may be, but is not limited to, a process, a processor, an object, an executable code, a thread of execution, a program, and/or a computer running on a processor.
  • applications and servers running on the server can be components.
  • One or more components can reside within a process and/or thread of execution, and a component can be localized on one computer and/or distributed between two or more computers.
  • a robot is provided, the internal structure diagram of which can be shown in Figure 14 .
  • the computer device includes a processor, memory, communication interface, display screen and input device connected through a system bus. Wherein, the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes non-volatile storage media and internal memory.
  • the non-volatile storage medium stores operating systems and computer programs. This internal memory provides an environment for the execution of operating systems and computer programs in non-volatile storage media.
  • the communication interface of the computer device is used for wired or wireless communication with external terminals.
  • the wireless mode can be implemented through WIFI, mobile cellular network, NFC (Near Field Communication) or other technologies.
  • the robot's display screen can be a liquid crystal display or an electronic ink display.
  • the robot's input device can be a touch layer covered on the display screen, or it can be a button, trackball or touch pad provided on the robot shell. It can also be It is an external keyboard, trackpad or mouse, etc.
  • Figure 14 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied.
  • Specific computer equipment can May include more or fewer parts than shown, or combine certain parts, or have a different arrangement of parts.
  • a robot including a memory and a processor.
  • a computer program is stored in the memory.
  • the processor executes the computer program, it implements the steps of any of the above methods.
  • a computer-readable storage medium is provided, a computer program is stored thereon, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.
  • a computer program product including a computer program that implements the steps of any one of the above methods when executed by a processor.
  • the computer program can be stored in a non-volatile computer-readable storage.
  • the computer program when executed, may include the processes of the above method embodiments.
  • Any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory.
  • Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetic variable memory (Magnetoresistive Random Access Memory (MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc.
  • Volatile memory may include random access memory (Random Access Memory, RAM) or external cache memory, etc.
  • RAM Random Access Memory
  • RAM random access memory
  • RAM Random Access Memory
  • the databases involved in the various embodiments provided in this application may include at least one of a relational database and a non-relational database.
  • Non-relational databases may include blockchain-based distributed databases, etc., but are not limited thereto.
  • the processors involved in the various embodiments provided in this application may be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to this.

Abstract

本申请涉及一种机器人的位姿确定方法、装置、机器人和存储介质。所述方法包括：通过设置在机器人顶部的RGB相机在行驶区域采集顶部视角图像，作为当前定位图像，并在视觉地图信息库中获取与当前定位图像相似度最大的目标顶部视角图像，进而根据当前定位图像中候选特征点的二维坐标和目标顶部视角图像中参考特征点的三维坐标，确定机器人采集当前定位图像时的当前机器人位姿。其中，视觉地图信息库中包括机器人在行驶区域中不同位姿采集到的多幅顶部视角图像。

Description

机器人的位姿确定方法、装置、机器人以及存储介质
相关申请的交叉引用
本申请要求于2022年04月12日提交中国专利局、申请号为2022103802992、申请名称为“机器人、地图创建方法、定位方法及介质”，以及于2022年12月09日提交中国专利局、申请号为2022115807094、申请名称为“机器人的位姿确定方法、装置、机器人以及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及机器人技术领域，特别是涉及一种机器人的位姿确定方法、装置、机器人以及存储介质。
背景技术
随着科技的进步，越来越多的机器人被应用到人们的日常生活中。其中，在已知地图中对机器人进行准确定位，是机器人做复杂任务的基础。
传统的室内定位技术中，通常利用前视激光获取机器人的前方的自然特征来进行定位，其前方的自然特征一般为地面上的桌子、椅子、沙发等用品，这些用品可能会被移动，从而使得定位失效或者出现漂移，影响定位效果。
发明内容
根据本申请的各种实施例，提供一种机器人的位姿确定方法、装置、机器人以及存储介质。
第一方面，本申请提供了一种机器人的位姿确定方法，包括：
通过RGB相机获取当前定位图像；其中，当前定位图像为机器人在行驶区域采集的顶部视角图像；
在视觉地图信息库中获取与当前定位图像相似度最大的目标顶部视角图像；其中，视觉地图信息库中包括机器人在行驶区域中以不同位姿采集到的多幅顶部视角图像；
根据当前定位图像中候选特征点的二维坐标和目标顶部视角图像中参考特征点的三维坐标，确定机器人采集当前定位图像时的当前机器人位姿。
第二方面，本申请还提供了一种机器人的视觉地图信息库的建立方法，该视觉地图信息库应用于上述任一项所述的位姿确定方法中，该建立方法包括：
通过RGB相机获取行驶区域的多幅顶部视角图像，顶部视角图像包括多个参考特征点；
通过预设传感器获取机器人在不同时刻的测量位姿，得到多个第三测量位姿，并根据第三测量位姿确定与顶部视角图像对应的第四测量位姿；
根据各顶部视角图像、各顶部视角图像中的特征点以及各顶部视角图像对应的第四测量位姿构建视觉地图信息库。
第三方面，本申请还提供了一种机器人，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现上述任一项所述方法的步骤。
第四方面，本申请还提供了一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述任一项所述方法的步骤。
第五方面，本申请还提供了一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现上述任一项所述方法的步骤。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他实施例的附图。
图1为一个实施例中机器人的位姿确定方法的流程示意图；
图2为另一个实施例中机器人的位姿确定方法的流程示意图；
图3为一个实施例中确定匹配的特征点对的流程示意图；
图4为另一个实施例中机器人的位姿确定方法的流程示意图；
图5为一个实施例中定位图像与测量位姿之间的对应关系示意图；
图6为一个实施例中确定各第一测量位姿对应时刻的机器人位姿的流程示意图；
图7为另一个实施例中机器人的位姿确定方法的流程示意图；
图8为一个实施例中构建视觉地图信息库的流程示意图；
图9为另一个实施例中构建视觉地图信息库的流程示意图；
图10为另一个实施例中构建视觉地图信息库的流程示意图；
图11为一个实施例中地图创建和定位方法的流程示意图；
图12为一个实施例中机器人的位姿确定装置的结构框图；
图13为一个实施例中机器人的视觉地图信息库的建立装置的结构框图；
图14为一个实施例中机器人的内部结构图。
具体实施方式
为了便于理解本申请，下面将参照相关附图对本申请进行更全面的描述。附图中给出了本申请的较佳实施例。但是，本申请可以以许多不同的形式来实现，并不限于本文所描述的实施例。相反地，提供这些实施例的目的是使对本申请的公开内容的理解更加透彻全面。
除非另有定义，本文所使用的所有的技术和科学术语与属于发明的技术领域的技术人员通常理解的含义相同。本文中在发明的说明书中所使用的术语只是为了描述具体的实施例的目的，不是旨在限制本申请。本文所使用的术语“和/或”包括一个或多个相关的所列项目的任意的和所有的组合。
本申请应用于扫地机器人、商场机器人等各种机器人，机器人能够通过图像传感器(相机)对某个区域进行建图(地图绘制)。在实际应用中，因相机采集的图像相对于激光图像，相机采集的图像中的特征更丰富，有利于定位的准确性。
在一个实施例中，如图1所示，提供了一种机器人的位姿确定方法，也可称为定位方法，本实施例以该方法应用于机器人进行举例说明，包括以下步骤：
S110、通过RGB相机获取当前定位图像。
其中，当前定位图像为机器人在行驶区域采集的顶部视角图像。当前定位图像用于定位机器人采集这幅图像时的当前机器人位姿(Pose)。机器人的行驶区域指的是机器人行驶的地面区域，顶部视角图像即为该地面区域所对应的顶部区域的图像。
在可选实施例中，RGB相机具体设置于机器人的顶部，其视角主轴线向上，其具体可以是垂直向上，也可以倾斜向上，从而获得顶部视角图像作为当前定位图像。
在可选场景中，行驶区域为室内场景时，其顶部视角图像具体可以是室内的天花板图像。行驶区域为室外场景时，其顶部视角图像则包括机器人上方物体，如天空、树叶、屋檐等等，这里不做限定。
在其他实施例中，RGB相机也可以设置于机器人的其他地方，只要可以实现获取顶部视角图像即可，这里不做具体限定。
可选地，当前定位图像可以是机器人在行驶区域中首次确定机器人位姿时所采集的顶部视角图像，也可以是非首次确定机器人位姿时所采集的顶部视角图像。
在可选实施例中，当前定位图像具体可以是RGB图像。
可选地，机器人顶部设置有RGB相机，机器人在行驶区域行驶的过程中，可通过该RGB相机采集当前时刻下的顶部视角图像，作为上述当前定位图像。其中，RGB相机可在操作者触发的情况下获取顶部视角图像，也可周期性自动获取顶部视角图像。可选地，机器人还可以搭载RGB相机以外的其他图像采集设备，以获取当前定位图像，本实施例中对于图像采集设备的类型并不做具体限制。
S120、在视觉地图信息库中获取与当前定位图像相似度最大的目标顶部视角图像。
其中，视觉地图信息库中包括机器人在行驶区域中不同位姿采集到的多幅顶部视角图像。
可选地，机器人采用图像相似性算法计算当前定位图像与视觉地图信息库中每一帧顶部视角图像之间的相似度，并从中获取相似度最大的顶部视角图像作为目标顶部视角图像。
可选地，上述图像相似性算法可以是结构相似度(Structural Similarity，SSIM)算法，通过当前定位图像与顶部视角图像之间亮度、对比度以及结构三方面的对比来确定图像之间的相似度。上述图像相似性算法还可以是向量相似度算法，通过将当前定位图像和顶部视角图像采用向量表征，计算两向量之间的余弦值作为图像之间的相似度。上述图像相似性算法还可以是直方图相似度算法，通过计算当前定位图像与顶部视角图像各自的直方图，再计算两个直方图之间的归一化相关系数(如巴氏距离、直方图相交距离等)得到图像之间的相似度。上述图像相似性算法还可以是哈希相似度算法，通过计算当前定位图像与顶部视角图像各自的哈希值来确定图像之间的相似度。
S130、根据当前定位图像中候选特征点的二维坐标和目标顶部视角图像中参考特征点的三维坐标，确定机器人采集当前定位图像时的当前机器人位姿。
其中，候选特征点为当前定位图像中的特征点，参考特征点为目标顶部视角图像中的特征点。可选地，上述候选特征点和参考特征点可以是ORB(Oriented FAST and Rotated BRIEF)点，可基于FAST(features from accelerated segment test)算法检测得到，还可以是其他特征点，如sift点或者其他深度学习的点，并采用相应地的检测算法得到。
可选地，视觉地图信息库还包括各顶部视角图像中参考特征点在现实场景中的三维坐标。机器人在视觉地图信息库中确定了目标顶部视角图像后，即可基于当前定位图像中候选特征点的二维坐标和目标顶部视角图像中参考特征点的三维坐标确定机器人采集当前定位图像时的当前机器人位姿。
需要说明的是，顶部视角图像中的参考特征点为经过全局优化处理(Global BA)，最小化顶部视角图像中每一特征点的重投影误差，再去除重投影误差大于误差阈值的特征点后所得到的特征点，即顶部视角图像中包括多个特征点，基于每一特征点的重投影误差筛选出部分特征点作为顶部视角图像中的参考特征点。
可选地，机器人可通过特征点识别模型提取得到当前定位图像中候选特征点，进而获取每一候选特征点在当前定位图像中二维坐标。机器人进一步在候选特征点和参考特征点之间确定匹配的特征点对，并根据匹配的特征点对中候选特征点的二维坐标和参考特征点的三维坐标采用PNP(Perspective-n-Point，多点透视成像)算法确定机器人采集当前定位图像时的当前机器人位姿。
本实施例中，机器人通过设置在机器人顶部的RGB相机在行驶区域采集顶部视角图像，作为当前定位图像，并在视觉地图信息库中获取与当前定位图像相似度最大的目标顶部视角图像，进而根据当前定位图像中候选特征点的二维坐标和目标顶部视角图像中参考特征点的三维坐标，确定机器人采集当前定位图像时的当前机器人位姿。其中，视觉地图信息库中包括机器人在行驶区域中不同位姿采集到的多幅顶部视角图像。通过上述方法即可实现基于顶部视角图像确定当前机器人位姿，顶部视角图像中多为静态特征，不会被移动，从而避免了定位失效或漂移，大大提高了机器人定位的准确性。同时，对于图像中的特征点而言，其三维坐标可准确反映该特征点的空间位置，并不会随着机器人位姿的变化而变化，进而根据特征点在当前定位图像中的二维坐标和实际的三维坐标确定机器人采集当前定位图像时的当前机器人位姿，可进一步提高所确定的当前机器人位姿的准确性，以提高定位效果。
实际应用中，候选特征点和参考特征点均包括多个，如图2所示，上述S130、根据当前定位图像中候选特征点的二维坐标和目标顶部视角图像中参考特征点的三维坐标，确定机器人采集当前定位图像时的当前机器人位姿，包括：
S210、获取各候选特征点在当前定位图像中的特征向量和各参考特征点在目标顶部视角图像中的特征向量。
其中，特征向量用于表征特征点在图像中的方向和位置。可选地，以当前定位图像中的任一位置作为参考点，取参考点到候选特征点所形成的向量即该候选特征点在当前定位图像中的特征向量。同理，以目标顶部视角图像中的任一位置作为参考点，取参考点到参考特征点所形成的向量即该参考特征点在目标顶部视角图像中的特征向量。
需要说明的是，当前定位图像中参考点和目标顶部视角图像中参考点的位置相同。例如，当前定位图像中的参考点为当前定位图像的中心点，目标顶部视角图像中的参考点则为目标顶部视角图像的中心点。
可选地，机器人将当前定位图像中的候选特征点和目标顶部视角图像中的参考特征点进行向量化处理，通过当前定位图像中的参考点确定各候选特征点的特征向量，以及通过目标顶部视角图像中的参考点确定各参考特征点的特征向量。
S220、根据各候选特征点的特征向量与各参考特征点的特征向量，确定当前定位图像和目标顶部视角图像之间匹配的特征点对。
可选地，机器人可根据各候选特征点与各参考特征点之间的相似度确定匹配的特征点对。本实施例中，机器人将候选特征点和参考特征点进行向量化处理，以根据各候选特征点的特征向量与各参考特征点的特征向量确定各候选特征点与各参考特征点之间的相似度，进而基于得到的相似度在多个候选特征点和多个参考特征点中确定当前定位图像和目标顶部视角图像之间匹配的特征点对。
S230、根据匹配的特征点对中候选特征点在当前定位图像中的二维坐标，以及匹配的特征点对中参考特征点的三维坐标确定机器人采集当前定位图像时的当前机器人位姿。
可选地，机器人在确定出匹配的特征点对后，即可根据匹配的特征点对中候选特征点的二维坐标和参考特征点的三维坐标采用PNP算法确定机器人采集当前定位图像时的机器人位姿。其中，匹配的特征点对至少包括三对。
在上述实施例中，可采用向量间的余弦值表征各候选特征点与各参考特征点之间的相似度，如图3所示，上述S220、根据各候选特征点的特征向量与各参考特征点的特征向量，确定当前定位图像和目标顶部视角图像之间匹配的特征点对，包括：
S310、获取各候选特征点的特征向量与各参考特征点的特征向量之间的余弦值。
可选地，机器人采用向量间余弦计算公式计算各候选特征点的特征向量与各参考特征点的特征向量之间的余弦值，即可得到每一候选特征点的特征向量与任一参考特征点的特征向量之间的余弦值。
S320、将满足余弦值约束条件的余弦值所对应的候选特征点和参考特征点确定为匹配的特征点对。
其中，余弦值约束条件包括余弦值大于余弦值阈值。
可选地，机器人将得到的余弦值与预设的余弦值阈值进行比较，将大于余弦值阈值的余弦值所对应的候选特征点和参考特征点确定为匹配的特征点对。例如，当前定位图像中包括候选特征点A、B、C以及D，以当前定位图像的中心点O为参考点，对应得到特征向量以及目标顶部视角图像中包括参考特征点a、b、c，以目标顶部视角图像的中心点o为参考点，对应得到特征向量进行向量间的余弦值计算后，得到与之间的余弦值Cosθ1大于余弦值阈值Cosθ0，与之间的余弦值Cosθ2大于余弦值阈值Cosθ0，与之间的余弦值Cosθ3大于余弦值阈值Cosθ0，机器人即可确定候选特征点A和参考特征点a为匹配的特征点对，候选特征点B和参考特征点b为匹配的特征点对，候选特征点C和参考特征点c为匹配的特征点对。
可选地，当存在同一参考特征点的特征向量与至少两个候选特征点的特征向量之间的余弦值均大于余弦值阈值，则确定得到最大余弦值对应的参考特征点和候选特征点为匹配的特征点对。例如，继续上述举例，当得到与之间的余弦值Cosθ1大于余弦值阈值Cosθ0，且与之间的余弦值Cosθ4也大于余弦值阈值Cosθ0，其中，Cosθ1＞Cosθ4，机器人即可确定候选特征点A和参考特征点a为匹配的特征点对。
本实施例中，候选特征点和参考特征点均包括多个，机器人通过获取各候选特征点在当前定位图像中的特征向量和各参考特征点在目标顶部视角图像中的特征向量，进而根据各候选特征点的特征向量与各参考特征点的特征向量，确定当前定位图像和目标顶部视角图像之间匹配的特征点对，以根据匹配的特征点对中候选特征点在当前定位图像中的二维坐标，以及匹配的特征点对中参考特征点的三维坐标确定机器人采集当前定位图像时的当前机器人位姿。具体基于向量化处理，以根据各候选特征点的特征向量与各参考特征点的特征向量之间余弦值表征向量间的相似度，进而确定匹配的特征点对。通过上述方式可提高匹配的特征点对的匹配准确性，基于准确匹配的特征点对利用PNP算法求解当前机器人位姿，相应提高了所确定的当前机器人位姿的准确性。
在机器人周期性自动获取天花图像进行定位的应用场景下，为适用于实时定位，如图4所示，上述方法还包括：
S410、通过预设传感器获取机器人在不同时刻的测量位姿，得到多个第一测量位姿。
其中，预设传感器可直接获取机器人的位姿，由预设传感器所获取的机器人的位姿称为测量位姿。可 选地，预设传感器可以是定位传感器，例如，设置在机器人上的里程计，如轮式里程计，或者IMU(Inertial measurement unit，惯性测量单元)。
可选地，机器人在行驶区域行驶的过程中，机器人可通过预设传感器连续的获取机器人在不同时刻的测量位姿，作为第一测量位姿，即可得到多个第一测量位姿。
S420、基于当前定位图像所在时刻的前后两个第一测量位姿得到与当前定位图像所在时刻对应的第二测量位姿。
其中，当前定位图像所在时刻即为采样当前定位图像的当前时刻。需要说明的是，预设传感器和RGB相机的采样频率不同，通常情况下，进行位姿采样的预设传感器的采样频率要高于RGB相机的采样频率，为确定同一时刻下的当前定位图像和测量位姿，需要基于当前定位图像所在时刻的前后两个第一测量位姿得到与当前定位图像所在时刻对应的第二测量位姿。
可选地，在多个第一测量位姿中，机器人从中获取测量时刻位于当前定位图像所在时刻(即当前时刻)相邻前后的两个第一测量位姿，可对这两个第一测量位姿进行插值，得到与当前定位图像所在时刻对应的第二测量位姿。如图5所示，针对当前定位图像A，对测量时刻位于当前定位图像A相邻前后的第一测量位姿Pose1和Pose2进行插值，即可得到与当前定位图像A所在时刻对应的第二测量位姿PoseA；可选的，可以利用常用的插值算法获取第二测量位姿PoseA，这里不多做限定。
针对当前定位图像B，对测量时刻位于当前定位图像B相邻前后的第一测量位姿Pose4和Pose5进行插值，即可得到与当前定位图像B所在时刻对应的第二测量位姿PoseB；可选的，可以利用常用的插值算法获取第二测量位姿PoseB，这里不多做限定。
S430、根据当前机器人位姿和第二测量位姿确定各第一测量位姿对应时刻的机器人位姿。
可选地，机器人在得到当前时刻下机器人的当前机器人位姿和第二测量位姿后，可基于当前机器人位姿与第二测量位姿之间的相对关系确定各第一测量位姿对应时刻的机器人位姿。
在一可选地实施例中，如图6所示，上述S430、根据当前机器人位姿和第二测量位姿确定各第一测量位姿对应时刻的机器人位姿，包括：
S610、获取当前机器人位姿与第二测量位姿之间的相对位姿。
可选地，机器人可根据如下公式(1)计算当前机器人位姿T0与第二测量位姿T_o0之间相对位姿dT：
dT＝T_o0*T0_inv         (公式1)
其中，T_o0表示第二测量位姿，T0_inv表示当前机器人位姿的逆矩阵。
需要说明的是，当前定位图像是周期性不断更新地，相应地，相对位姿dT也在随着当前定位图像的更新而更新。
S620、当第一测量位姿的所在时刻位于当前定位图像所在时刻之后且位于下一个定位图像的所在时刻之前时，通过第一测量位姿与相对位姿得到第一测量位姿对应时刻的机器人位姿。
可选地，对于机器人所获取到的多个第一测量位姿，在第一测量位姿的所在时刻位于当前定位图像所在时刻之后且位于下一个定位图像的所在时刻之前时，即针对在机器人已获取当前定位图像，还未获取下一个定位图像的时段内所采集到的第一测量位姿，通过各第一测量位姿与相对位姿得到各第一测量位姿对应时刻的机器人位姿。
继续上述图5中的举例，在定位图像A为当前定位图像时，定位图像B则为当前定位图像A的下一个定位图像，针对当前定位图像A与定位图像B之间的每一个第一测量位姿，即Pose2、Pose3和Pose4，机器人可将Pose2作为T_o0，并将作为T_o0的Pose2以及前述计算得到的相对位姿dT带入公式(1)，以得到Pose2对应时刻的机器人位姿逆矩阵，进而得到对应时刻的机器人位姿Pose2’，同理，分别得到Pose3、Pose4和Pose5对应时刻的机器人位姿Pose3’，Pose4’以及Pose5’。
本实施例中，机器人通过预设传感器获取机器人在不同时刻的测量位姿，得到多个第一测量位姿，并基于当前定位图像所在时刻的前后两个第一测量位姿得到与当前定位图像所在时刻对应的第二测量位姿，进而根据当前机器人位姿和第二测量位姿确定各第一测量位姿对应时刻的机器人位姿。具体可获取当前机器人位姿与第二测量位姿之间的相对位姿，以在第一测量位姿的所在时刻位于当前定位图像所在时刻之后且位于下一个定位图像的所在时刻之前时，通过第一测量位姿与相对位姿得到第一测量位姿对应时刻的机器人位姿。通过上述方法实现了在当前定位图像与下一个定位图像之间相应时刻机器人位姿的确定，在保 证较高定位频率的同时也提高了定位精度，以适用于实时定位。
由于相机在采集图像的过程中容易受到光线的影响，从而导致清晰度受到影响。在一实施例中，先根据相机获得的图像和根据定位传感器获得机器人的位姿创建地图，然后将创建的地图作为预存地图存储于机器人的视觉地图信息库中，机器人即可根据预存地图以及在移动过程中拍摄的图像进行定位，定位过程中根据移动过程中相机拍摄的图像和预存地图中的特征点进行匹配，从而减少光线的影响，且利用相机获得的图像以及预存地图均包含丰富特征，因此提高定位的准确性。
在一个实施例中，如图7所示，本申请实施例还提供了一种定位方法，也可称为机器人的位姿确定方法，该方法包括：
S710、通过相机获取机器人在当前位置对应的第一图像；
本实施例中的定位方法可以应用于任意定位设备，比如清洁机器人、导航机器人等，该定位设备中搭载有相机和定位传感器。以机器人为例进行说明，在定位过程中，先通过机器人上的相机获得机器人所在位姿的图像，即当前定位图像，定义为第一图像。其中，相机可以是RGB相机。
S720、将第一图像与预存地图中的图像进行相似度对比，确定预存地图中与第一图像相似度最高的目标图像，其中，预存地图包括至少两个第二图像、至少两个第二图像中每个第二图像的特征点在三维空间中的坐标信息和至少两个第二图像中每个第二图像对应的位姿。
在一实施例中，视觉地图信息库包括预存地图。该预存地图即为机器人预先根据相机获得的图像和根据定位传感器获得机器人的位姿所创建，并存入视觉地图信息库中的视觉地图。
可选地，机器人将第一图像和预存地图中的图像进行相似度对比，确定预存地图中与第一图像相似度最高的图像，定义为目标图像，其中预存的地图由机器人根据至少两个节点的位姿以及至少两个节点的位姿中每个节点对应的第二图像进行特征点提取，并根据特征点进行位姿优化后所创建，该预存地图包括至少两个第二图像、至少两个第二图像中每个第二图像的特征点在三维空间中的坐标信息和至少两个第二图像中每个第二图像对应的位姿。示例性的，可以通过词带模型(DBOW)将第一图像和预存的地图中图像进行相似度对比，从而在预存地图中确定与第一图像最相似的目标图像。
S730、提取第一图像中的特征点得到多个第一特征点，并将多个第一特征点和目标图像的特征点进行特征点匹配，确定多个第一特征点中与目标图像的特征点相匹配的目标特征点。
将第一图像和目标图像进行特征点的提取，将第一图像和目标图像中的特征点进行匹配，确定第一图像和目标图像中匹配的特征点，具体地，利用图像特征点匹配技术，对第一图像和目标图像中特征点进行匹配，从而确定第一图像和目标图像中匹配的特征点。
S740、根据目标特征点在第一图像中的坐标，和与目标特征点相匹配的特征点在三维空间中的坐标信息，计算得到机器人的当前机器人位姿，作为第一位姿，并根据第一位姿获得机器人的定位信息。
可选地，机器人根据目标图像中与第一图像匹配的特征点即可计算出拍摄第一图像的位姿，作为机器人拍摄第一图像时的当前机器人位姿，即为机器人的定位信息。具体地，通过目标图像中与第一图像匹配的特征点的三维坐标对第一图像中与目标图像匹配的特征点的三维坐标，然后将获得的第一图像中特征点的三维坐标作为PnP算法的输入，计算出第一图像的拍摄位置在视觉地图中的位姿，即获得机器人的定位信息。其中，视觉地图信息库存储的均为机器人行驶区域的顶部视角图像，因此上述目标图像即为目标顶部视角图像。
本实施例中，通过相机获取机器人在当前位置对应的第一图像，将第一图像与预存地图中的图像进行相似度对比，确定预存地图中与第一图像相似度最高的目标图像，其中，预存地图包括至少两个第二图像、至少两个第二图像中每个第二图像的特征点在三维空间中的坐标信息和至少两个第二图像中每个第二图像对应的位姿；提取第一图像中的特征点得到多个第一特征点，并将多个第一特征点和目标图像的特征点进行特征点匹配，确定多个第一特征点中与目标图像的特征点相匹配的目标特征点；根据目标特征点在第一图像中的坐标，和与目标特征点相匹配的特征点在三维空间中的坐标信息，计算得到机器人的当前机器人位姿，作为第一位姿，并根据第一位姿获得机器人的定位信息。通过上述方式，本申请中预存地图根据至少两个节点的位姿以及至少两个节点的位姿中每个节点对应的第二图像进行特征点提取，并根据特征点进行位姿优化后所创建，预存地图主要利用图像中特征点创建，受到光照影响较小，且图像是通过相机拍摄，相对激光地图特征更丰富，因此地图精度高。在定位过程中根据相机获取到的第一图像与预存地图进 行对比，确定预存地图中与第一图像相似度最高的目标图像，然后利用目标图像和第一图像中匹配特征点进行位姿计算，确定机器人所在位置，也就是说在定位过程中采用相机拍摄图像与预存地图中特征点进行位姿计算，利用了相机拍摄图像特征丰富特性，且减少了光线的影响，因此提高了定位精度。
在一实施例中，提取第一图像中的特征点得到多个第一特征点，包括：
将当前定位图像作为第一图像，对第一图像进行直方图均衡化，获得对应的第一直方图；
对第一直方图进行特征点提取得到第一图像中的多个候选特征点，作为第一特征点。
为了减少光照对图像的影响，进而影响定位的精度，本实施例中，先对第一图像进行直方图均衡化，获得对应的第一直方图，然后提取第一直方图中的第一特征点。
基于上述实施例，在视觉地图信息库中获取与当前定位图像相似度最大的目标顶部视角图像之前，上述方法还包括：
通过定位传感器获取机器人在当前位置对应的测量位姿，当前位置对应的测量位姿包括机器人在当前位置的坐标信息；
在视觉地图信息库中获取与当前定位图像相似度最大的目标顶部视角图像，包括：
在预存地图中确定与机器人在当前位置对应的测量位姿中坐标信息对应的目标位置；
将第一图像与预存地图中以目标位置为中心的预设范围内图像进行相似度对比，确定预存地图中与第一图像相似度最高的目标顶部视角图像。
本实施例中，机器人在获取第一图像时，还通过定位传感器获取机器人的当前位置对应的测量位姿，该测量位姿包括机器人在当前位置的坐标信息。其中，该机器人的当前位置对应的测量位姿即为机器人在当前位置对应的当前位姿。具体实施中，通过定位传感器获取机器人的当前位置对应的测量位姿只需要在将第一图像与预存地图中的图像进行相似度对比，确定预存地图中与第一图像相似度最高的目标顶部视角图像之前执行即可。
此时在将第一图像与预存地图中的图像进行相似度对比，确定预存地图中与第一图像相似度最高的目标顶部视角图像的过程可包括：先在预存地图中确定与机器人在当前位置对应的测量位姿中坐标信息对应的位置，定义为目标位置，然后以目标位置为中心，半径为预设半径的范围得到目标范围，在预存地图中筛选目标范围的图像作为候选图像，其中，候选图像对应的位姿中的坐标位于目标范围内，最后，在候选图像中查找到与第一图像相似度最高的目标顶部视角图像，这样在对比过程中，则不需要将第一图像与预存地图中所有图像进行对比，仅需要利用第一图像与候选图像进行相似度比对，则可从候选图像中查找到与第一图像相似度最高的目标顶部视角图像，通过该方式则能够快速在预存地图中确定与第一图像相似度最高的目标顶部视角图像。示例性的，通过定位传感器记录机器人在预存地图中对应区域中运动的坐标(如记录机器人在区域中运动的距离和方向，则可以对应获得机器人在该区域的坐标)，根据该坐标则可以确定机器人在预存地图中的大概位置(如在预存地图中坐标对应位置的5m范围内)，因此直接将获得的第一图像与确定大概位置进行对比，即可快速确定预存地图中与第一图像相似度最高的目标顶部视角图像。
在一实施例中，上述方法还包括：
根据第一位姿获得机器人的定位信息。具体包括：
将当前机器人位姿作为机器人的第一位姿，对当前位置对应的测量位姿和第一位姿进行融合处理得到融合位姿，将融合位姿作为机器人的定位信息。
本实施例中，机器人上搭载有定位传感器，在获取第一图像时，还通过定位传感器获取机器人的当前位置对应的测量位姿。在计算出机器人的第一位姿后，根据当前位置对应的测量位姿和第一位姿，将当前位置对应的测量位姿和第一位姿进行融合处理得到融合位姿，将融合位姿作为机器人的定位信息进行输出。机器人的当前位置对应的测量位姿和第一位姿可以采用扩展卡尔曼滤波技术进行融合，也就采用其他技术进行融合，此处不做限定。
因定位传感器输入位姿的频率为Q，通过上述方式计算机器人的第一位姿的频率为P，计算P需要经过多个步骤进行，则Q会远远大于P，因此本申请中根据定位传感器的测量位姿的和计算出机器人在预测地图中第一位姿，将测量位姿和第一位姿进行融合，后续则可以根据定位传感器获得的测量位姿进行定位信息的输出，从而可以大大提高定位信息输出的频率。
本申请还提供了一种机器人的视觉地图信息库的构建方法，也可称为地图创建方法，该构建方法应用 于上述任一项实施例的位姿确定/定位方法中。如图8所示，该方法包括：
S810、通过RGB相机获取行驶区域的多幅顶部视角图像，顶部视角图像包括多个参考特征点。
可选地，机器人在构建行驶区域所对应的视觉地图信息库时，在可选实施例中，当行驶区域为环状区域时，可围绕行驶区域顺时针行驶一圈，逆时针行驶一圈，当行驶区域为非环状区域时，可以从其行驶区域的起点行驶到重点，随后从终点返回起点。
其中，在存在边路的情况下，也是先推入边路后返回。
机器人在行驶的过程中，通过RGB相机获取行驶区域的多幅包括多个参考特征点的顶部视角图像。
S820、通过预设传感器获取机器人在不同时刻的测量位姿，得到多个第三测量位姿，并根据第三测量位姿确定与顶部视角图像对应的第四测量位姿。
可选地，在机器人围绕行驶区域行驶的过程中，机器人可通过预设传感器获取机器人在不同时刻的测量位姿，作为第三测量位姿，即可得到多个第三测量位姿。
承前所述，通常情况下，进行位姿采样的预设传感器的采样频率要高于RGB相机的采样频率，为确定同一时刻下的顶部视角图像和测量位姿，则需要基于顶部视角图像所在时刻的前后两个第三测量位姿(如进行插值)得到与顶部视角图像所在时刻对应的第四测量位姿。
S830、根据各顶部视角图像、各顶部视角图像中的特征点以及各顶部视角图像对应的第四测量位姿构建视觉地图信息库。
在一可选地实施例中，如图9所示，上述S830、根据各顶部视角图像、各顶部视角图像中的特征点以及各顶部视角图像对应的第四测量位姿构建视觉地图信息库，包括：
S910、基于相邻的两个第四测量位姿对应的顶部视角图像确定参考特征点的三维坐标。
可选地，确定出各顶部视角图像所在时刻对应的第四测量位姿后，机器人则进一步根据时刻相邻的两个第四测量位姿对应顶部视角图像确定顶部视角图像中参考特诊点在现实场景中的三维坐标，具体可采用PNP算法得到，在此不再赘述。
S920、对多个第四测量位姿进行回环处理以修正多个第四测量位姿，得到修正后的多个图像位姿。
可选地，机器人环绕行驶区域获取顶部视角图像的过程中，机器人可采用“词带”模型进行回环检测，当检测到回环，即表征机器人已环绕行驶区域一圈，回到了起始位置。此时，机器人即可使用位姿修正算法，矫正位姿漂移，得到修正后的多个图像位姿。
S930、采用预设优化算法对多个图像位姿以及参考特征点的三维坐标进行优化，得到多个优化后的图像位姿以及多个优化后的三维坐标。
可选地，机器人在得到修正后的图像位姿后，再采用修正后的图像位姿，重新计算顶部视角图像中所有特征点的三维坐标，并进行全局优化，最小化重投影误差，去除重投影误差大于误差阈值的特征点。经过上述一系列的校正优化后，得到多个优化后的图像位姿以及多个优化后的三维坐标。
S940、将各顶部视角图像、对应顶部视角图像的参考特征点、参考特征点优化后的三维坐标以及多个优化后的图像位姿关联，形成视觉地图信息库。
可选地，机器人将各顶部视角图像、对应顶部视角图像的参考特征点、参考特征点优化后的三维坐标以及多个优化后的图像位姿一一关联，形成视觉地图信息库，即视觉地图信息库中包括机器人所获取的每一帧顶部视角图像，以及对应顶部视角图像中的参考特征点、参考特征点的三维坐标以及获取该帧顶部视角图像时的图像位姿。
可选地，在视觉地图信息库中包括每一帧顶部视角图像所对应的图像位姿的情况下，确定当前定位图像的当前机器人位姿时，机器人可在视觉地图信息库中获取与当前定位图像相似度最大的目标顶部视角图像所对应的图像位姿，作为当前机器人位姿。
在上述实施例中，在构建视觉地图信息库之前，还需要对集成/安装在机器人上的RGB相机和预设传感器(如轮式里程计)进行参数标定。例如，所要标定的参数包括：摄像设备的相机内参(K＝[fx,fy,cx,cy]，畸变系数d＝[d1,d2,d3,d4])、轮式里程计外参(左轮半径r1，右轮半径r2，两轮之间的直径D)以及RGB相机与轮式里程计之间的外参。
本实施例中，通过预先建立视觉地图信息库，以便机器人后续直接基于视觉地图信息库中的数据进行机器人定位，进一步提高机器人的位姿确定效率。
在一个实施例中，如图10所示，上述构建/创建方法还包括以下步骤：
S1010、将所获取的多幅顶部视角图像作为第二图像，对所有第二图像进行特征点提取，获得所有第二图像中每个第二图像的特征点，并确定至少两个节点中任意相邻节点的第二图像中特征点的对应关系，其中相邻节点为至少两个节点中创建时间相邻的两个节点。
在每个节点机器人还通过相机在各节点拍摄所在区域得到多幅顶部视角图像，定义为第二图像。
在一实施例中，为了地图顺利创建，创建时间/创建位置相邻的节点对应的图像包含的图像内容较为近似，即在创建时间/创建位置相邻的节点，机器人拍摄图像的方向相同，或者旋转角度较小(如两个节点拍摄角度小于5°)，从而使得相邻的节点对应的图像中相同特征点较多，这样有利于后续创建流程。
本实施例中，机器人在每次创建一节点时，或者在创建所有节点后，或者每隔创建预设数量节点时，机器人则会确定相邻节点对应的图像中特征点的对应关系。具体地，机器人在移动过程中，每次拍摄角度方向接近或者为同一方向，因此相邻节点对应的拍摄的第二图像中相同的像素点较多。在创建一个新的节点后，对每个节点对应的第二图像进行特征点提取，直到将所有节点对应的第二图像进行特征点提取，然后根据相邻节点的第二图像中提取出的特征点中进行匹配，从而确定相邻节点对应的第二图形中特征点的关系。
S1020、通过定位传感器获取机器人在行驶区域移动过程中创建的至少两个节点中每个节点的第三测量位姿；其中，各顶部视角图像与各节点对应。
本实施例中地图创建方法可以应用于包括获取终端和地图创建终端组成的地图创建系统中，其中获取终端上搭载有相机和定位传感器，定位传感器可以是轮式里程计，也可以是其他传感器，相机用于得到获取终端所处环境的图像，定位传感器可以用于记录获取终端的位姿，位姿可以包括位置和姿态；地图创建终端用于接收获取终端传输的图像、位姿，并根据接收的图像和位姿进行地图的创建，地图创建终端可以为服务器、个人电脑等。
作为另一种实施例，本申请的地图创建方法也可以应用于机器人，该机器人上搭载有相机和定位传感器，该机器人能够根据自身相机获取的图像和定位传感器获得的位姿进行地图创建。为方便描述本申请中地图创建方法以应用于机器人为例进行说明。
将机器人放入待创建地图的区域，通过遥控器控制机器人在待创建地图的区域中进行移动，或者机器人按照预存运动规则在待创建地图的区域中进行移动，示例性的，机器人按照直线或者s型路线在待创建地图的区域中进行移动。通过机器人中的定位传感器获取机器人自身在待创建地图区域移动过程中创建的多个节点，获取每个节点的第二位姿，定义为第三测量位姿，该第二位姿至少包括位置，在其他实施例中位姿可以包括位置和姿态。机器人在移动过程中创建各个节点可以依照预设规则进行创建，示例性的：根据定位传感器记录的移动距离，每隔预设移动距离则创建一个新的节点；或者可以根据机器人移动的时间间隔创建新的节点，然后在创建节点时，通过定位传感器获取机器的位姿。
为了保证地图完整创建，机器人创建的节点数量尽可能多，从而使得所有节点对应的图像能够覆盖整个待创建地图区域，示例性的，机器人创建的节点可以均匀分布在待创建地图的区域中；或者机器人创建的节点可以首尾相接连成一个完整的图形，即第一个创建的节点和最后一个创建的节点可以为同一位置所创建，只是创建时间不同，形成首尾相接的移动路径；或者机器人创建的节点可以均匀分布在待创建地图的区域中，且各节点可以连成一个完整的图形。
S1030、利用预设第一规则对至少两个节点中每个节点的第三测量位姿进行优化，获得至少两个节点中每个节点的第四测量位姿。
在实际应用中，由于可能存在定位传感器或者相机中其他单元的精度不高的情形，容易导致的地图精度不高，为提高创建地图的准确度，本实施例中根据预设的第一规则对获得的每个节点的第三测量位姿进行优化，获得创建的各个节点更新后的第三位姿，定义为第四测量位姿。其中预设的第一规则可以为传统技术中位姿或位姿图优化规则。
S1040、根据至少两个节点中任意相邻节点的第二图像中特征点的对应关系和至少两个节点中每个节点的第四测量位姿创建地图，并存入视觉地图信息库。
在各个节点的位姿更新后，根据任意相邻节点的第二图像中特征点的对应关系和更新所获得的第四测量位姿进行地图的创建，示例性的，可以根据任意相邻节点的第二图像中特征点的对应关系和更新所获 得的第四测量位姿利用三角化算法即可计算出各个特征点在三维空间中的位置，然后使用LM算法优化调整三维空间中各个特征点的位置以及每张第二图像的位姿，即可获得带有尺度的地图，完成地图的创建。地图创建完成后将创建的地图进行保存，作为预存地图存入视觉地图信息库，或者发送到服务器保存，从而方便其他设备可以从服务器获取该区域的地图进行使用。进一步地，为方便后续使用，保存的地图中可以包括各个特征点和特征点之间存在的关系。
本实施例中，通过定位传感器获取机器人在待创建地图区域移动过程中创建的至少两个节点中每个节点的第三测量位姿，和将所获取的多幅顶部视角图像作为第二图像，对所有第二图像中每个第二图像进行特征点提取，获得所有第二图像的特征点，并确定至少两个节点中任意相邻节点的第二图像中特征点的对应关系，其中相邻节点为至少两个节点中创建时间相邻的两个节点；利用预设第一规则对至少两个位姿节点中每个节点的第三测量位姿进行优化，获得至少两个节点中每个节点的第四测量位姿；根据任意相邻节点的第二图像中特征点的对应关系和至少两个节点中每个节点的第四测量位姿创建地图，并存入视觉地图信息库。这样，本申请中机器人获取自身在待创建地图区域移动过程中创建的节点的位姿以及通过相机获得各节点的图像，从而利用相机获得地图特征相对激光更丰富的图像，然后提取图像中的特征点，减少了光照对图像的影响，能够获得精度更高的地图，进而为前他设备在该区域进行定位提高定位的准确性和精度奠定基础。
在一个实施例中，对所有第二图像进行特征点提取，获得所有第二图像的特征点，包括：
对所有第二图像进行特征点提取，获得所有第二图像中每个第二图像的第二特征点；
按照预设角度旋转所有第二图像，获得所有第二图像中每个第二图像对应的旋转图像；
对所有旋转图像进行特征点提取，获得所有旋转图像中每个旋转图像的第三特征点，第二图像的特征点包括第二特征点和第三特征点。
对应的，确定至少两个节点中任意相邻节点的第二图像中特征点的对应关系，包括：
确定至少两个节点中任意相邻节点的第二图像的第二特征点的第一对应关系，以及至少两个节点中任意相邻节点的第二图像对应的旋转图像的第三特征点的第二对应关系，至少两个节点中任意相邻节点的第一图像中特征点的对应关系包括第一对应关系和第二对应关系。
具体地，为了进一步提高定位的精度，在本实施例中，获得第二图像中像素点的过程可以包括：对获得的第二图像进行两次特征点的获取：第一次，对每个第二图像的原图进行特征点的提取，此次将提取的特征点定义为第二特征点；第二次将所有的第二图像进行旋转后，获得各第二图像旋转后的图像，定义为旋转图像，然后对各旋转图像进行特征点的提取，此次将提取的特征点定义为第三特征点，示例性的，可以将第二图像旋转180度，获得旋转图像。
如此获得每张第二图像的第二特征点和第三特征点，根据第二图像原图确定相连的两张第二图像中第二特征点之间的关系(定义为第一对应关系)，并且根据相邻两张第二图像对应的旋转图像确定相邻旋转图像中第三特征点之间的关系(第二对应关系)，示例性的：相邻的第二图像为A1和A2，图像A1中包括特征点a11、a12、a13，图像A2中包括特征点a21、a22、a23，则确定a11、a12、a13分别与a21、a22、a23的对应关系。相邻的第二图像为A1和A2，第二图像A1和A2分别对应的旋转图像为B1和B2，图像B1中包括特征点b11、b12、b13，图像B2中包括特征点b21、b22、b23，则确定b11、b12、b13分别与b21、b22、b23的对应关系。
本实施例中根据获取的第二图像进行旋转，获取每个第二图像原始图像和旋转图像中的特征点，能够增加图像中特征点的数量，从而为后续步骤提供更多的特征点进行计算，从而提高计算结果的准确性，进而为提高地图的精度奠定基础。
进一步，为了提高定位的精度，获取第二图像中像素点可以包括：获取第二图像中的尺度不变特征变换SIFT特征点。SIFT特征点通过SIFT算法实现，该SIFT算法的实质是在不同的尺度空间上查找关键点(特征点)，并计算出关键点的方向。SIFT所提取到的关键点是一些十分突出，不会因光照，仿射变换和噪音等因素而变化的点，如角点、边缘点、暗区的亮点及亮区的暗点等。如此也能够提高了地图的精度。
作为一种实施例，基于上述实施例，利用预设第一规则对第三测量位姿进行优化，获得至少两个节点中每个节点的第四测量位姿，包括：
根据相邻节点的第一对应关系、第二对应关系以及相邻节点的第三测量位姿确定相邻节点的第三测量 位姿的对应关系；
根据相邻节点的第三测量位姿的对应关系和第三测量位姿对第三测量位姿进行优化，获得相邻节点中每个节点的第四测量位姿。
本实施例中利用相邻节点的第一对应关系、第二对应关系可计算出对相邻第二图像的拍摄点(成像点)的对应关系，具体地，根据相邻的两张二维图像、相邻节点中特征点的第一对应关系、第二对应关系可以确定三维图像，该三维图像中特征点，以及三维空间中特征点在相邻第二图像中拍摄点(成像点)的对应关系，即可以根据第一对应关系、第二对应关系可计算出对相邻节点的第三测量位姿的对应关系，从而优化各节点的第三测量位姿，获得各节点优化后的第三测量位姿，定义优化后的位姿为第四测量位姿。本实施例中使用了第二图像原图和旋转图像中的特征点，特征点数量相对单一原图中特征点数量多，因此能够提高各节点的拍摄点(成像点)的精度，为提高地图精度奠定基础。
作为一种实施例，利用预设第一规则对至少两个节点中每个节点的第三测量位姿进行优化，获得至少两个节点中每个节点的第四测量位姿，包括：
根据至少两个节点中每个节点的第三测量位姿，计算得到所有节点的总残差，并判断总残差是否处于预设残差范围内；
当总残差不处于预设残差范围内，利用预设第一规则对至少两个节点中每个节点的第三测量位姿进行优化，获得至少两个节点中每个节点的第四测量位姿，以使得优化后根据至少两个节点中每个节点的第四测量位姿计算得到的所有节点的总残差处于预设残差范围内。
其中，根据至少两个节点中每个节点的第三测量位姿，计算得到所有节点的总残差，包括：
利用至少两个节点中任意相邻节点的第三测量位姿计算得到至少两个节点中任意相邻节点第三测量位姿的逆矩阵；
通过至少两个节点中任意相邻节点第三测量位姿的逆矩阵计算得到至少两个节点中任意相邻节点之间的测量值；
根据至少两个节点中任意相邻节点之间的测量值计算得到至少两个节点中任意相邻节点之间的残差；
根据至少两个节点中任意相邻节点之间的残差，计算得到所有节点的总残差。
本实施例中为了提高地图的精度，还对所有节点的第四测量位姿(各节点的成像点)进行优化，具体地，先计算相邻节点的第三测量位姿的逆矩阵，然后根据相邻节点的第三测量位姿的逆矩阵计算相邻节点的第二图像之间的测量值，该测量值根据相邻节点的第二图像的第四测量位姿和第四测量位姿的逆矩阵计算获得，示例性的，相邻节点的第二图像分别为A1和A2，图像A1和A2之间的测量值dTmeasure＝T2w*T1w，其中Tw1为图像A1的位姿，T1w为Tw1的逆矩阵；Tw2为图像A2的位姿，T2w为Tw2的逆矩阵。再根据相邻节点之间的测量值计算相邻节点之间的残差，示例性的，相邻节点之间的残差为Err，Err＝Log(T1w*Tw2*dTmeasure)，其中Log：表示矩阵到向量的映射。最后根据相邻节点的残差计算所有节点的总残差，即将所有相邻节点的残差求和获得所有节点的总残差，根据所有节点的总残差判断总残差是否在预设范围内，当不在预设范围内，通过预设算法(示例性的，预设算法可以为LM算法)调整一个或者多个节点的Tw1和Tw2的值，然后重复上述步骤，即重新计算相邻节点的第二图像之间的测量值，并根据测量值计算所有节点的总残差。直到所有节点的总残差处于预设范围后，将对应的各个节点的位姿作为各节点更新后的位姿，即第四测量位姿。
作为一种实施例，计算得到所有节点的总残差之前，还包括：
根据节点的第二图像确定是否存在机器人处于同一位置拍摄的图像；
在存在的情况下，则执行步骤：计算所有节点的总残差。
计算总残差需要获得所有的节点，在计算总残差之前机器人的启动条件可以为：确定是否获得了待创建地图区域中规划的所有节点的数据(所有节点的位姿和第二图像)。本实施例中，通过判断是否存在同一位置拍摄的图像作为是否获得了待创建地图区域中规划的所有节点的数据的依据。具体地，在获得各节点的图像后，判断是否存在同一位置的图像，即判断是否存在基本相同的图像(相似度接近100％的两张图像说明两张图像基本相同)。
示例性的，判断是否存在基本相同的图像可以通过词带模型进行搜索判断，即可以通过词带模型在所有获得的图像搜索是否存在基本相同的两张图像，在存在的情况下，说明两个图像的拍摄点(成像点)相 同，然后再计算所有节点的总残差，再计算时，同一位置拍摄的多张图像只使用其中一张图像进行计算，同一位置拍摄的其他图像则不参与计算。作为另一种实施例，也可能将同一位置的节点的图像都采用上述方式进行计算。
作为一种实施例，根据任意相邻节点的第二图像中特征点的对应关系和至少两个节点中每个节点的第四测量位姿创建地图，包括：
根据至少两个节点中任意相邻节点的第四测量位姿和至少两个节点中任意相邻节点的第二图像中特征点的对应关系，通过三角化算法计算得到至少两个节点中任意相邻节点的第二图像中特征点在三维空间中的坐标信息；
通过预设迭代算法对所有第二图像中特征点的坐标信息、至少两个节点中每个节点的第四测量位姿进行优化，得到优化后的所有第二图像中特征点的坐标信息和优化后的至少两个节点中每个节点的第四测量位姿；
将所有第二图像、优化后的所有第二图像中特征点的坐标信息和优化后的至少两个节点中每个节点的第四测量位姿得到对应的预存地图。
具体地，创建地图的过程可以包括：根据任意相邻节点的第四测量位姿和任意相邻节点的第一图像中特征点的对应关系，计算获得该相邻节点的图像中特征点在三维空间中的位置，然后根据特征点的在三维空间的位置和第四测量位姿则可以构成带有尺度的地图。
示例性的，可以根据第四测量位姿和第四测量位姿对应的第一图像通过三角化算法计算第一图像中特征点在三维空间中的位置；进一步地，根据特征点的在三维空间的位置和第四测量位姿通过迭代算法进行优化，迭代算法可以为LM算法，获得优化后的所有第二图像中特征点的坐标信息和优化后的至少两个节点中每个节点的第四测量位姿，再基于将所有第二图像、优化后的所有第二图像中特征点的坐标信息和优化后的至少两个节点中每个节点的第四测量位姿得到对应的预存地图。作为一种实施例，第四测量位姿的优化还可以根据上述实施例中优化后的第三测量位姿和定位传感器提供的位姿进行优化所获得。
作为一种实施例，上述方法可以在不同的设备中运行，也可以在同一设备中运行，在同一设备(以设备为机器人为例进行说明)中运行的过程可以包括：控制机器人在待创建地图的区域内沿预设路径进行移动，机器人中的相机按预设时间间隔对机器人所在环境进行拍照获得相应行驶区域的多幅顶部视角图像，即第二图像，机器人中的里程计按预设时间间隔获取机器人的多个第三测量位姿，每个第三测量位姿和一张第二图像对应。为保证创建地图的完整，预设路径可以为一个首尾相连的图形。获取多张第二图像后，对每张第二图像进行特征点提取，将任意相邻的两张第二图像中特征点进行匹配，确定任意相邻的两张第二图像中特征点的对应关系，如图11所示。
利用第三测量位姿判断机器人是否走完了预设路径，具体通过判断是否存在基本相同的图像(相似度接近100％的两张图像说明两张图像基本相同)来判断机器人是否走完了预设路径。在机器人走完了预设路径后，则利用第二图像中特征点以及相邻第二图像中特征点的对应关系对所有第二图像进行位姿优化，然后根据相邻第二图像中特征点的对应关系确定各特征点在三维空间中的位置，最后根据各特征点在三维空间中的位置以及优化后的各第二图像对应的位姿创建带有尺度的地图。
地图创建完成后，则将创建的地图保存至机器人中，机器人则可以使用保存的地图进行定位，具体地，机器人获取所在位置的图像即当前定位图像，记为第一图像。将第一图像和保存的地图进行对比，找到地图中与第一图像相似度最高的图像，定义地图中与第一图像相似度最高的图像为目标图像(即目标顶部视角图像)，然后对目标图像和第一图像进行特征点提取，并将第一图像和目标图像中的特征点进行匹配，确定第一图像和目标图像中匹配的特征点。最后根据目标图像中与第一图像匹配的特征点即可计算出拍摄第一图像的位姿，作为机器人拍摄第一图像的位姿，该位姿即为机器人的定位信息，从而实现定位。本实施例中各实现的具体过程与上述定位方法、地图创建方法相同，此处不多做赘述。
应该理解的是，虽然如上所述的各实施例所涉及的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，如上所述的各实施例所涉及的流程图中的至少一部分步骤可以包括多个步骤或者多个阶段，这些步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其 它步骤中的步骤或者阶段的至少一部分轮流或者交替地执行。
在一个实施例中，如图12所示，本申请提供了一种机器人的位姿确定装置，包括：第一获取模块1201、第二获取模块1202和位姿确定模块1203，其中：
第一获取模块1201用于通过RGB相机获取当前定位图像；其中，当前定位图像为机器人在行驶区域采集的顶部视角图像；
第二获取模块1202用于在视觉地图信息库中获取与当前定位图像相似度最大的目标顶部视角图像；其中，视觉地图信息库中包括机器人在行驶区域中以不同位姿采集到的多幅顶部视角图像；
位姿确定模块1203用于根据当前定位图像中候选特征点的二维坐标和目标顶部视角图像中参考特征点的三维坐标，确定机器人采集当前定位图像时的当前机器人位姿。
在其中一个实施例中，候选特征点和参考特征点均包括多个；位姿确定模块1203具体用于：
获取各候选特征点在当前定位图像中的特征向量和各参考特征点在目标顶部视角图像中的特征向量；根据各候选特征点的特征向量与各参考特征点的特征向量，确定当前定位图像和目标顶部视角图像之间匹配的特征点对；根据匹配的特征点对中候选特征点在当前定位图像中的二维坐标，以及匹配的特征点对中参考特征点的三维坐标确定机器人采集当前定位图像时的当前机器人位姿。
在其中一个实施例中，位姿确定模块1203具体用于：
获取各候选特征点的特征向量与各参考特征点的特征向量之间的余弦值；将满足余弦值约束条件的余弦值所对应的候选特征点和参考特征点确定为匹配的特征点对。
在其中一个实施例中，上述装置还包括实时定位模块，用于：
通过预设传感器获取机器人在不同时刻的测量位姿，得到多个第一测量位姿；基于当前定位图像所在时刻的前后两个第一测量位姿得到与当前定位图像所在时刻对应的第二测量位姿；根据当前机器人位姿和第二测量位姿确定各第一测量位姿对应时刻的机器人位姿。
在其中一个实施例中，实时定位模块具体用于：
获取当前机器人位姿与第二测量位姿之间的相对位姿；当第一测量位姿的所在时刻位于当前定位图像所在时刻之后且位于下一个定位图像的所在时刻之前时，通过第一测量位姿与相对位姿得到第一测量位姿对应时刻的机器人位姿。
在其中一个实施例中，位姿确定模块1203还用于：
将当前定位图像作为第一图像，对第一图像进行直方图均衡化，获得对应的第一直方图；对第一直方图进行特征点提取得到第一图像中的多个候选特征点，作为第一特征点。
在其中一个实施例中，视觉地图信息库中包括预存地图，实时定位模块还用于：
通过定位传感器获取机器人在当前位置对应的测量位姿，当前位置对应的测量位姿包括机器人在当前位置的坐标信息；
第二获取模块1202还用于：
在预存地图中确定与机器人在当前位置对应的测量位姿中坐标信息对应的目标位置；将第一图像与预存地图中以目标位置为中心的预设范围内图像进行相似度对比，确定预存地图中与第一图像相似度最高的目标顶部视角图像。
在其中一个实施例中，位姿确定模块1203还用于：
将当前机器人位姿作为机器人的第一位姿，对当前位置对应的测量位姿和第一位姿进行融合处理得到融合位姿，将融合位姿作为机器人的定位信息。
在一个实施例中，如图13所示，本申请提供了一种机器人的视觉地图信息库的建立装置，该视觉地图信息库应用于上述任一项所述的位姿确定方法中。该建立方法包括：获取模块1301、测量模块1302和构建模块1303，其中：
图像获取模块1301用于通过RGB相机获取行驶区域的多幅顶部视角图像，顶部视角图像包括多个参考特征点；
位姿测量模块1302用于通过预设传感器获取机器人在不同时刻的测量位姿，得到多个第三测量位姿，并根据第三测量位姿确定与顶部视角图像对应的第四测量位姿；
地图构建模块1303用于根据各顶部视角图像、各顶部视角图像中的特征点以及各顶部视角图像对应 的第四测量位姿构建视觉地图信息库。
在其中一个实施例中，地图构建模块1303具体用于：
基于相邻的两个第四测量位姿对应的顶部视角图像确定参考特征点的三维坐标；对多个第四测量位姿进行回环处理以修正多个第四测量位姿，得到修正后的多个图像位姿；采用预设优化算法对多个图像位姿以及参考特征点的三维坐标进行优化，得到多个优化后的图像位姿以及多个优化后的三维坐标；将各顶部视角图像、对应顶部视角图像的参考特征点、参考特征点优化后的三维坐标以及多个优化后的图像位姿关联，形成视觉地图信息库。
在其中一个实施例中，上述装置还包括：
第二特征提取模块，用于将所获取的多幅顶部视角图像作为第二图像，对所有第二图像进行特征点提取，获得所有第二图像中每个第二图像的特征点，并确定至少两个节点中任意相邻节点的第二图像中特征点的对应关系，其中相邻节点为至少两个节点中创建时间相邻的两个节点；
第三测量位姿获取模块，用于通过定位传感器获取机器人在行驶区域移动过程中创建的至少两个节点中每个节点的第三测量位姿；其中，各顶部视角图像与各节点对应；
位姿优化模块，用于利用预设第一规则对至少两个节点中每个节点的第三测量位姿进行优化，获得至少两个节点中每个节点的第四测量位姿；
地图创建模块，用于根据至少两个节点中任意相邻节点的第二图像中特征点的对应关系和至少两个节点中每个节点的第四测量位姿创建预存地图，并存入视觉地图信息库。
在其中一个实施例中，第二特征提取模块还用于：
对所有第二图像进行特征点提取，获得所有第二图像中每个第二图像的第二特征点；按照预设角度旋转所有第二图像，获得所有第二图像中每个第二图像对应的旋转图像；对所有旋转图像进行特征点提取，获得所有旋转图像中每个旋转图像的第三特征点，第二图像的特征点包括第二特征点和第三特征点。
在其中一个实施例中，第二特征提取模块还用于：
确定至少两个节点中任意相邻节点的第二图像的第二特征点的第一对应关系，以及至少两个节点中任意相邻节点的第二图像对应的旋转图像的第三特征点的第二对应关系，至少两个节点中任意相邻节点的第一图像中特征点的对应关系包括第一对应关系和第二对应关系。
在其中一个实施例中，位姿优化模块还用于：
根据相邻节点的第一对应关系、第二对应关系以及相邻节点的第三测量位姿确定相邻节点的第三测量位姿的对应关系；根据相邻节点的第三测量位姿的对应关系和第三测量位姿对第三测量位姿进行优化，获得相邻节点中每个节点的第四测量位姿。
在其中一个实施例中，位姿优化模块还用于：
根据至少两个节点中每个节点的第三测量位姿，计算得到所有节点的总残差，并判断总残差是否处于预设残差范围内；当总残差不处于预设残差范围内，利用预设第一规则对至少两个节点中每个节点的第三测量位姿进行优化，获得至少两个节点中每个节点的第四测量位姿，以使得优化后根据至少两个节点中每个节点的第四测量位姿计算得到的所有节点的总残差处于预设残差范围内。
在其中一个实施例中，位姿优化模块还用于：
利用至少两个节点中任意相邻节点的第三测量位姿计算得到至少两个节点中任意相邻节点第三测量位姿的逆矩阵；通过至少两个节点中任意相邻节点第三测量位姿的逆矩阵计算得到至少两个节点中任意相邻节点之间的测量值；根据至少两个节点中任意相邻节点之间的测量值计算得到至少两个节点中任意相邻节点之间的残差；根据至少两个节点中任意相邻节点之间的残差，计算得到所有节点的总残差。
在其中一个实施例中，位姿优化模块还用于：
根据节点的第二图像确定是否存在机器人处于同一位置拍摄的图像；在存在的情况下，则执行步骤：计算所有节点的总残差。
在其中一个实施例中，地图创建模块还用于：
根据至少两个节点中任意相邻节点的第四测量位姿和至少两个节点中任意相邻节点的第二图像中特征点的对应关系，通过三角化算法计算得到至少两个节点中任意相邻节点的第二图像中特征点在三维空间中的坐标信息；通过预设迭代算法对所有第二图像中特征点的坐标信息、至少两个节点中每个节点的第四 测量位姿进行优化，得到优化后的所有第二图像中特征点的坐标信息和优化后的至少两个节点中每个节点的第四测量位姿；将所有第二图像、优化后的所有第二图像中特征点的坐标信息和优化后的至少两个节点中每个节点的第四测量位姿得到对应的预存地图。
如在本申请中所使用的，术语“组件”、“模块”和“系统”等旨在表示计算机相关的实体，它可以是硬件、硬件和软件的组合、软件、或者执行中的软件。例如，组件可以是但不限于是，在处理器上运行的进程、处理器、对象、可执行码、执行的线程、程序和/或计算机。作为说明，运行在服务器上的应用程序和服务器都可以是组件。一个或多个组件可以驻留在进程和/或执行的线程中，并且组件可以位于一个计算机内和/或分布在两个或更多的计算机之间。
在一个实施例中，提供了一种机器人，其内部结构图可以如图14所示。该计算机设备包括通过系统总线连接的处理器、存储器、通信接口、显示屏和输入装置。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统和计算机程序。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的通信接口用于与外部的终端进行有线或无线方式的通信，无线方式可通过WIFI、移动蜂窝网络、NFC(近场通信)或其他技术实现。该计算机程序被处理器执行时以实现一种机器人的位姿确定方法，还可以实现一种机器人的视觉地图信息库的建立方法。该机器人的显示屏可以是液晶显示屏或者电子墨水显示屏，该机器人的输入装置可以是显示屏上覆盖的触摸层，也可以是机器人外壳上设置的按键、轨迹球或触控板，还可以是外接的键盘、触控板或鼠标等。
本领域技术人员可以理解，图14中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
在一个实施例中，提供了一种机器人，包括存储器和处理器，存储器中存储有计算机程序，该处理器执行计算机程序时实现上述任一项所述方法的步骤。
在一个实施例中，提供了一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现上述任一项所述方法的步骤。
在一个实施例中，提供了一种计算机程序产品，包括计算机程序，该计算机程序被处理器执行时实现上述任一项所述方法的步骤。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、数据库或其它介质的任何引用，均可包括非易失性和易失性存储器中的至少一种。非易失性存储器可包括只读存储器(Read-Only Memory，ROM)、磁带、软盘、闪存、光存储器、高密度嵌入式非易失性存储器、阻变存储器(ReRAM)、磁变存储器(Magnetoresistive Random Access Memory，MRAM)、铁电存储器(Ferroelectric Random Access Memory，FRAM)、相变存储器(Phase Change Memory，PCM)、石墨烯存储器等。易失性存储器可包括随机存取存储器(Random Access Memory，RAM)或外部高速缓冲存储器等。作为说明而非局限，RAM可以是多种形式，比如静态随机存取存储器(Static Random Access Memory，SRAM)或动态随机存取存储器(Dynamic Random Access Memory，DRAM)等。本申请所提供的各实施例中所涉及的数据库可包括关系型数据库和非关系型数据库中至少一种。非关系型数据库可包括基于区块链的分布式数据库等，不限于此。本申请所提供的各实施例中所涉及的处理器可为通用处理器、中央处理器、图形处理器、数字信号处理器、可编程逻辑器、基于量子计算的数据处理逻辑器等，不限于此。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对本申请专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请的保护范围应以所附权利要求为准。

Claims (21)

  1. 一种机器人的位姿确定方法，所述方法包括：
    通过RGB相机获取当前定位图像；其中，所述当前定位图像为所述机器人在行驶区域采集的顶部视角图像；
    在视觉地图信息库中获取与所述当前定位图像相似度最大的目标顶部视角图像；其中，所述视觉地图信息库中包括机器人在所述行驶区域中以不同位姿采集到的多幅顶部视角图像；
    根据所述当前定位图像中候选特征点的二维坐标和所述目标顶部视角图像中参考特征点的三维坐标，确定所述机器人采集所述当前定位图像时的当前机器人位姿。
  2. 根据权利要求1所述的方法，所述候选特征点和所述参考特征点均包括多个；所述根据所述当前定位图像中候选特征点的二维坐标和所述目标顶部视角图像中参考特征点的三维坐标，确定所述机器人采集所述当前定位图像时的当前机器人位姿，包括：
    获取各所述候选特征点在所述当前定位图像中的特征向量和各所述参考特征点在所述目标顶部视角图像中的特征向量；
    根据各所述候选特征点的特征向量与各所述参考特征点的特征向量，确定所述当前定位图像和所述目标顶部视角图像之间匹配的特征点对；
    根据所述匹配的特征点对中候选特征点在所述当前定位图像中的二维坐标，以及所述匹配的特征点对中参考特征点的三维坐标确定所述机器人采集所述当前定位图像时的当前机器人位姿。
  3. 根据权利要求2所述的方法，所述根据各所述候选特征点的特征向量与各所述参考特征点的特征向量，确定所述当前定位图像和所述目标顶部视角图像之间匹配的特征点对，包括：
    获取各所述候选特征点的特征向量与各所述参考特征点的特征向量之间的余弦值；
    将满足余弦值约束条件的余弦值所对应的候选特征点和参考特征点确定为匹配的特征点对。
  4. 根据权利要求1-3中任一项所述的方法，所述方法还包括：
    通过预设传感器获取机器人在不同时刻的测量位姿，得到多个第一测量位姿；
    基于所述当前定位图像所在时刻的前后两个第一测量位姿得到与所述当前定位图像所在时刻对应的第二测量位姿；
    根据所述当前机器人位姿和所述第二测量位姿确定各所述第一测量位姿对应时刻的机器人位姿。
  5. 根据权利要求4所述的方法，所述根据所述当前机器人位姿和所述第二测量位姿确定各所述第一测量位姿对应时刻的机器人位姿，包括：
    获取所述当前机器人位姿与所述第二测量位姿之间的相对位姿；
    当所述第一测量位姿的所在时刻位于所述当前定位图像所在时刻之后且位于下一个定位图像的所在时刻之前时，通过所述第一测量位姿与所述相对位姿得到所述第一测量位姿对应时刻的机器人位姿。
  6. 根据权利要求1-5中任一项所述的方法，所述方法还包括：
    将所述当前定位图像作为第一图像，对所述第一图像进行直方图均衡化，获得对应的第一直方图；
    对所述第一直方图进行特征点提取得到所述第一图像中的多个候选特征点，作为第一特征点。
  7. 根据权利要求1-5中任一项所述的方法，所述视觉地图信息库中包括预存地图，在所述在视觉地图信息库中获取与所述当前定位图像相似度最大的目标顶部视角图像之前，所述方法还包括：
    通过定位传感器获取所述机器人在当前位置对应的测量位姿，所述当前位置对应的测量位姿包括所述机器人在当前位置的坐标信息；
    所述在视觉地图信息库中获取与所述当前定位图像相似度最大的目标顶部视角图像，包括：
    在所述预存地图中确定与所述机器人在当前位置对应的测量位姿中坐标信息对应的目标位置；
    将所述第一图像与所述预存地图中以所述目标位置为中心的预设范围内图像进行相似度对比，确定所述预存地图中与所述第一图像相似度最高的目标顶部视角图像。
  8. 根据权利要求7所述的方法，所述方法还包括：
    将所述当前机器人位姿作为所述机器人的第一位姿，对所述当前位置对应的测量位姿和所述第一位姿进行融合处理得到融合位姿，将所述融合位姿作为所述机器人的定位信息。
  9. 一种机器人的视觉地图信息库的建立方法，所述视觉地图信息库应用于权利要求1-8中任一项所述的位姿确定方法中，所述方法包括：
    通过所述RGB相机获取所述行驶区域的多幅顶部视角图像，所述顶部视角图像包括多个参考特征点；
    通过预设传感器获取所述机器人在不同时刻的测量位姿，得到多个第三测量位姿，并根据所述第三测量位姿确定与所述顶部视角图像对应的第四测量位姿；
    根据各所述顶部视角图像、各所述顶部视角图像中的特征点以及各所述顶部视角图像对应的第四测量位姿构建所述视觉地图信息库。
  10. 根据权利要求9所述的方法，所述根据各所述顶部视角图像、各所述顶部视角图像中的特征点以及各所述顶部视角图像对应的第四测量位姿构建所述视觉地图信息库，包括：
    基于相邻的两个第四测量位姿对应的顶部视角图像确定所述参考特征点的三维坐标；
    对多个所述第四测量位姿进行回环处理以修正多个所述第四测量位姿，得到修正后的多个图像位姿；
    采用预设优化算法对所述多个图像位姿以及所述参考特征点的三维坐标进行优化，得到多个优化后的图像位姿以及多个优化后的三维坐标；
    将各所述顶部视角图像、对应所述顶部视角图像的参考特征点、所述参考特征点优化后的三维坐标以及所述多个优化后的图像位姿关联，形成所述视觉地图信息库。
  11. 根据权利要求9所述的方法，所述方法还包括：
    将所获取的多幅顶部视角图像作为第二图像，对所有第二图像进行特征点提取，获得所述所有第二图像中每个第二图像的特征点，并确定所述至少两个节点中任意相邻节点的第二图像中特征点的对应关系，其中所述相邻节点为所述至少两个节点中创建时间相邻的两个节点；
    通过定位传感器获取所述机器人在所述行驶区域移动过程中创建的至少两个节点中每个节点的第三测量位姿；其中，各所述顶部视角图像与各所述节点对应；
    利用预设第一规则对所述至少两个节点中每个节点的第三测量位姿进行优化，获得所述至少两个节点中每个节点的第四测量位姿；
    根据所述至少两个节点中任意相邻节点的第二图像中特征点的对应关系和所述至少两个节点中每个节点的第四测量位姿创建预存地图，并存入所述视觉地图信息库。
  12. 根据权利要求11所述的方法，所述对所有第二图像进行特征点提取，获得所述所有第二图像中每个第二图像的特征点，包括：
    对所述所有第二图像进行特征点提取，获得所述所有第二图像中每个第二图像的第二特征点；
    按照预设角度旋转所述所有第二图像，获得所述所有第二图像中每个第二图像对应的旋转图像；
    对所有旋转图像进行特征点提取，获得所述所有旋转图像中每个旋转图像的第三特征点，所述第二图像的特征点包括所述第二特征点和所述第三特征点。
  13. 根据权利要求12所述的方法，所述确定所述至少两个节点中任意相邻节点的第二图像中特征点的对应关系，包括：
    确定所述至少两个节点中任意相邻节点的第二图像的第二特征点的第一对应关系，以及所述至少两个节点中任意相邻节点的第二图像对应的旋转图像的第三特征点的第二对应关系，所述至少两个节点中任意相邻节点的第一图像中特征点的对应关系包括第一对应关系和第二对应关系。
  14. 根据权利要求13所述的方法，所述利用预设第一规则对所述至少两个节点中每个节点的第三测量位姿进行优化，获得所述至少两个节点中每个节点的第四测量位姿，包括：
    根据相邻节点的第一对应关系、第二对应关系以及相邻节点的第三测量位姿确定相邻节点的第三测量位姿的对应关系；
    根据相邻节点的第三测量位姿的对应关系和第三测量位姿对所述第三测量位姿进行优化，获得相邻节点中每个节点的第四测量位姿。
  15. 根据权利要求11所述的方法，所述利用预设第一规则对所述至少两个节点中每个节点的第三测量位姿进行优化，获得所述至少两个节点中每个节点的第四测量位姿，包括：
    根据所述至少两个节点中每个节点的第三测量位姿，计算得到所有节点的总残差，并判断所述总残差是否处于预设残差范围内；
    当所述总残差不处于预设残差范围内，利用预设第一规则对所述至少两个节点中每个节点的第三测量位姿进行优化，获得所述至少两个节点中每个节点的第四测量位姿，以使得优化后根据至少两个节点中每 个节点的第四测量位姿计算得到的所述所有节点的总残差处于预设残差范围内。
  16. 根据权利要求15所述的方法，所述根据所述至少两个节点中每个节点的第三测量位姿，计算得到所有节点的总残差，包括：
    利用所述至少两个节点中任意相邻节点的第三测量位姿计算得到所述至少两个节点中任意相邻节点第三测量位姿的逆矩阵；
    通过所述至少两个节点中任意相邻节点第三测量位姿的逆矩阵计算得到所述至少两个节点中任意相邻节点之间的测量值；
    根据所述至少两个节点中任意相邻节点之间的测量值计算得到所述至少两个节点中任意相邻节点之间的残差；
    根据所述至少两个节点中任意相邻节点之间的残差，计算得到所有节点的总残差。
  17. 根据权利要求15所述的方法，所述计算得到所有节点的总残差之前，所述处理器还用于执行所述计算机程序时实现如下步骤：
    根据所述节点的第二图像确定是否存在所述机器人处于同一位置拍摄的图像；
    在存在的情况下，则执行步骤：计算所有节点的总残差。
  18. 根据权利要求11所述的方法，所述根据至少两个节点中任意相邻节点的第二图像中特征点的对应关系和所述至少两个节点中每个节点的第四测量位姿创建预存地图，包括：
    根据所述至少两个节点中任意相邻节点的第四测量位姿和所述至少两个节点中任意相邻节点的第二图像中特征点的对应关系，通过三角化算法计算得到所述至少两个节点中任意相邻节点的第二图像中特征点在三维空间中的坐标信息；
    通过预设迭代算法对所有第二图像中特征点的坐标信息、所述至少两个节点中每个节点的第四测量位姿进行优化，得到优化后的所述所有第二图像中特征点的坐标信息和优化后的所述至少两个节点中每个节点的第四测量位姿；
    将所述所有第二图像、优化后的所述所有第二图像中特征点的坐标信息和优化后的所述至少两个节点中每个节点的第四测量位姿得到对应的预存地图。
  19. 一种机器人，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现权利要求1至18中任一项所述的方法的步骤。
  20. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1至18中任一项所述的方法的步骤。
  21. 一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现权利要求1至18中任一项所述的方法的步骤。
PCT/CN2023/087710 2022-04-12 2023-04-12 机器人的位姿确定方法、装置、机器人以及存储介质 WO2023198088A1 (zh)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202210380299.2A CN114463429B (zh) 2022-04-12 2022-04-12 机器人、地图创建方法、定位方法及介质
CN202210380299.2 2022-04-12
CN202211580709.4 2022-12-09
CN202211580709.4A CN116168080A (zh) 2022-12-09 2022-12-09 机器人的位姿确定方法、装置、机器人以及存储介质

Publications (1)

Publication Number Publication Date
WO2023198088A1 true WO2023198088A1 (zh) 2023-10-19

Family

ID=88328967

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/087710 WO2023198088A1 (zh) 2022-04-12 2023-04-12 机器人的位姿确定方法、装置、机器人以及存储介质

Country Status (1)

Country Link
WO (1) WO2023198088A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140235267A1 (en) * 2011-08-27 2014-08-21 Korea University Research And Business Foundation Method for recognizing position of mobile robot by using features of arbitrary shapes on ceiling
CN110853095A (zh) * 2019-10-11 2020-02-28 北京达佳互联信息技术有限公司 相机定位方法、装置、电子设备及存储介质
CN112101378A (zh) * 2020-08-20 2020-12-18 上海姜歌机器人有限公司 机器人重定位方法、装置及设备
CN113761255A (zh) * 2021-08-19 2021-12-07 劢微机器人科技(深圳)有限公司 机器人室内定位方法、装置、设备及存储介质
CN114463429A (zh) * 2022-04-12 2022-05-10 深圳市普渡科技有限公司 机器人、地图创建方法、定位方法及介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140235267A1 (en) * 2011-08-27 2014-08-21 Korea University Research And Business Foundation Method for recognizing position of mobile robot by using features of arbitrary shapes on ceiling
CN110853095A (zh) * 2019-10-11 2020-02-28 北京达佳互联信息技术有限公司 相机定位方法、装置、电子设备及存储介质
CN112101378A (zh) * 2020-08-20 2020-12-18 上海姜歌机器人有限公司 机器人重定位方法、装置及设备
CN113761255A (zh) * 2021-08-19 2021-12-07 劢微机器人科技(深圳)有限公司 机器人室内定位方法、装置、设备及存储介质
CN114463429A (zh) * 2022-04-12 2022-05-10 深圳市普渡科技有限公司 机器人、地图创建方法、定位方法及介质

Similar Documents

Publication Publication Date Title
Walch et al. Image-based localization using lstms for structured feature correlation
Park et al. Elastic lidar fusion: Dense map-centric continuous-time slam
CN111780764B (zh) 一种基于视觉地图的视觉定位方法、装置
Gao et al. Jigsaw: Indoor floor plan reconstruction via mobile crowdsensing
WO2020233724A1 (zh) 一种基于视觉slam的电网作业环境地图的构建方法及系统
Whelan et al. Deformation-based loop closure for large scale dense RGB-D SLAM
WO2022121640A1 (zh) 机器人重定位方法、装置、机器人和可读存储介质
WO2019169540A1 (zh) 紧耦合视觉slam的方法、终端及计算机可读存储介质
US9761002B2 (en) Stereo-motion method of three-dimensional (3-D) structure information extraction from a video for fusion with 3-D point cloud data
WO2019196403A1 (zh) 定位方法、定位服务器及定位系统
Gao et al. Robust RGB-D simultaneous localization and mapping using planar point features
CN110660098B (zh) 基于单目视觉的定位方法和装置
CN110648363A (zh) 相机姿态确定方法、装置、存储介质及电子设备
Xie et al. RDC-SLAM: A real-time distributed cooperative SLAM system based on 3D LiDAR
US11367263B1 (en) Image-guided three dimensional modeling
Ding et al. Persistent stereo visual localization on cross-modal invariant map
Teng et al. CloudNavi: Toward ubiquitous indoor navigation service with 3D point clouds
CN113223078A (zh) 标志点的匹配方法、装置、计算机设备和存储介质
Zheng et al. A multi-frame graph matching algorithm for low-bandwidth RGB-D SLAM
Restrepo et al. Evaluation of feature-based 3-d registration of probabilistic volumetric scenes
CN114463429B (zh) 机器人、地图创建方法、定位方法及介质
WO2023198088A1 (zh) 机器人的位姿确定方法、装置、机器人以及存储介质
Lee et al. Robust uncertainty-aware multiview triangulation
CN114742967B (zh) 一种基于建筑数字孪生语义图的视觉定位方法及装置
US9852542B1 (en) Methods and apparatus related to georeferenced pose of 3D models

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23787731

Country of ref document: EP

Kind code of ref document: A1