WO2018028649A1 - 一种移动装置及其定位方法、计算机存储介质 - Google Patents

一种移动装置及其定位方法、计算机存储介质 Download PDF

Info

Publication number
WO2018028649A1
WO2018028649A1 PCT/CN2017/096945 CN2017096945W WO2018028649A1 WO 2018028649 A1 WO2018028649 A1 WO 2018028649A1 CN 2017096945 W CN2017096945 W CN 2017096945W WO 2018028649 A1 WO2018028649 A1 WO 2018028649A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature descriptors
mobile device
feature
descriptors
visual
Prior art date
Application number
PCT/CN2017/096945
Other languages
English (en)
French (fr)
Inventor
庞富民
陈子冲
Original Assignee
纳恩博（北京）科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 纳恩博（北京）科技有限公司 filed Critical 纳恩博（北京）科技有限公司
Publication of WO2018028649A1 publication Critical patent/WO2018028649A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20—Instruments for performing navigational calculations
    • G01C21/206—Instruments for performing navigational calculations specially adapted for indoor navigation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20—Instruments for performing navigational calculations

Definitions

  • the present invention relates to the field of positioning technologies, and in particular, to a mobile device, a positioning method thereof, and a computer storage medium.
  • Existing indoor positioning methods based on visual and inertial devices are mainly divided into two categories: 1) Establishing environmental map positioning methods, such as: SLAM (Simultaneous Localization And Mapping) technology, 2) No Need to establish a positioning method for the environment map, such as: visual / inertial odometer technology.
  • SLAM Simultaneous Localization And Mapping
  • the method for locating the environment map includes: the robot usually establishes a map for the environment while estimating the position and posture of the robot, and obtains the position information of the robot by optimizing the trajectory of the robot itself and the relative positional relationship between each position gesture in the trajectory and the road sign in the map.
  • the method for establishing the environment map is highly accurate. However, if the environment information map is to be incorporated into the positioning optimization algorithm, it needs to consume a large amount of computing resources of the robot. Therefore, the calculation amount of the optimization algorithm often becomes an influence on the establishment environment. The bottleneck of the real-time nature of the map location method.
  • the application of the positioning method that does not need to establish an environment map in the mobile device ensures the real-time performance, but as the motion trajectory grows, the mobile device accumulates an error in estimating the position and posture of the position during the movement, resulting in the movement The estimation error of the position and posture of the equipment will continue to increase, which will seriously affect the Bit precision.
  • the embodiment of the present invention is to provide a mobile device, a positioning method thereof, and a computer storage medium, so as to solve the technical problem that the estimation error of the position of the mobile device is continuously increased based on the positioning of the mobile device in the prior art, and the positioning accuracy is seriously affected. .
  • an embodiment of the present invention provides a method for locating a mobile device, including:
  • the performing by using the first set of feature descriptors, each of the plurality of sets of feature descriptors that are extracted in advance, performs closed-loop detection, including:
  • comparing the first set of feature descriptors with each of the plurality of sets of feature descriptors that are extracted first, and determining whether the preset similar conditions are met include:
  • the spatial coordinates of the visual feature points described by the second set of feature descriptors determine the pose of the mobile device at the current moment, including:
  • T is the transition matrix
  • X i is the descriptor of the plurality of spatial coordinates of feature points of the described visual features
  • K is an internal parameter matrix of the image acquisition unit built in the mobile device
  • R is a posture of the mobile device
  • t is the location of the mobile device.
  • the method further includes: during the moving of the mobile device to collect the visual feature points, the method further includes:
  • Estimating the mobile device during the moving process based on the inertial data and the visual information The trajectory in motion.
  • the method further includes:
  • the motion trajectory is corrected based on the determined pose of the mobile device at the current time in place of the pose of the corresponding time estimated based on the inertial data and the visual information.
  • each of the plurality of sets of feature descriptors that are previously extracted includes: the key frame image from the key frame image data acquired at a previous time compared to the current time.
  • a set of feature descriptors is extracted from the data, wherein the key frame image data is sequentially determined from the collected frame image data according to a preset spatial interval.
  • the embodiment of the present invention further provides a mobile device, including:
  • An extracting unit configured to extract a first set of feature descriptors of the collected visual feature points at the current time during the movement of the collected visual feature points
  • the detecting unit is configured to perform closed-loop detection on each of the first set of feature descriptors collected by the extracting unit and each of the plurality of sets of feature descriptors that are extracted first;
  • a determining unit configured to, when the detecting unit detects a closed loop based on the first set of feature descriptors and the second set of feature descriptors, the spatial coordinates of the visual feature points described by the second set of feature descriptors, Determining a pose of the mobile device at the current time, wherein the second set of feature descriptors is one of the plurality of previously extracted sets of feature descriptors.
  • the detecting unit includes:
  • the comparison subunit is configured to compare the first set of feature descriptors with each of the plurality of sets of feature descriptors that are extracted first, and respectively determine the plurality of sets of feature descriptors that are extracted first. a number of descriptors of each set of feature descriptors and the first set of feature descriptors satisfying a preset similar condition;
  • a determining subunit configured to respectively determine the plurality of previously extracted groups determined by the comparing subunit Whether the number of descriptors of each of the feature descriptors and the first group of feature descriptors satisfying the preset similarity condition is greater than a preset number threshold; and the number of descriptors satisfying the preset similarity condition is greater than the preset
  • the threshold is set, the first set of feature descriptors and the corresponding group feature descriptors are characterized to detect a closed loop.
  • the comparison subunit is configured to: each of the first set of feature descriptors and each of the plurality of sets of feature descriptors of the first extracted feature descriptors Each feature descriptor is compared one by one; determining whether a vector angle between the feature descriptors for comparison is less than a preset angle threshold, wherein the comparison is performed when the vector angle is smaller than the preset angle threshold The feature descriptor satisfies the preset similar condition.
  • the determining unit includes:
  • a first determining subunit configured to determine a plurality of feature descriptors in the second set of feature descriptors
  • a second determining subunit configured to determine that the plurality of feature descriptors determined by the first determining subunit correspond to two-dimensional image coordinates in the frame image data collected at the current time;
  • a matrix establishing subunit configured to establish the representation of the movement based on spatial coordinates of the visual feature points described by the plurality of feature descriptors, the two-dimensional image coordinates, and an internal parameter matrix of the built-in image acquisition unit of the mobile device a transfer matrix of the pose of the device;
  • Solving a subunit configured to obtain a pose of the mobile device at the current moment based on the transition matrix
  • T is the transition matrix
  • X i is the descriptor of the plurality of spatial coordinates of feature points of the described visual features
  • K is an internal parameter matrix of the built-in image acquisition unit of the mobile device
  • R is a posture of the mobile device
  • t is the location of the mobile device.
  • the mobile device further includes:
  • An acquisition unit configured to collect inertial data and visual information during the movement
  • a trajectory estimating unit configured to estimate a motion trajectory during the movement based on the inertial data collected by the collecting unit and the visual information.
  • the mobile device further includes:
  • a correction unit configured to replace the pose of the corresponding moment estimated based on the inertial data and the visual information by the pose of the mobile device determined by the determining unit at the current moment to correct the motion trajectory .
  • each of the plurality of sets of feature descriptors that are previously extracted includes: the key frame image from the key frame image data acquired at a previous time compared to the current time.
  • a set of feature descriptors is extracted from the data, wherein the key frame image data is sequentially determined from the collected frame image data according to a preset spatial interval.
  • the embodiment of the present invention further provides a computer storage medium, where the computer storage medium stores computer executable instructions, and the computer executable instructions are used to perform positioning of the mobile device according to the embodiment of the present invention. method.
  • the mobile device and the positioning method thereof and the computer storage medium provided by the embodiment of the present invention collect the visual feature points during the moving process by the mobile device, and extract the first set of feature descriptors of the visual feature points collected at the current time;
  • the group feature descriptors perform closed-loop detection with each of the plurality of sets of feature descriptors that are extracted first, thereby determining whether the mobile device has reached the same region that has arrived; in the first group based on the feature description and prior
  • the extracted second set of feature descriptors detects the closed loop, the spatial coordinates of the visual feature points described by the second set of feature descriptors determine the pose of the mobile device at the current moment, so that the mobile device can pass through the same region again.
  • FIG. 1 is a schematic flowchart of a positioning method of a mobile device according to an embodiment of the present invention
  • step S103 in FIG. 1 is a detailed flowchart of step S103 in FIG. 1;
  • FIG. 3 is a functional unit diagram of a mobile device according to an embodiment of the present invention.
  • the mobile device and the positioning method thereof and the computer storage medium provided by the embodiments of the present invention solve the technical problem that the mobile device accumulates errors in estimating the posture of the mobile device during the moving process, and seriously affects the positioning accuracy.
  • the technical solution of the embodiment of the present invention is to solve the above technical problem, and the technical solution proposed includes:
  • the first set of feature descriptors of the visual feature points collected at the current time is extracted, and the first set of feature descriptors and each of the plurality of sets of feature descriptors extracted online are respectively selected.
  • the descriptor performs closed loop detection.
  • the mobile device is a robot equipped with an image capturing unit, and the image capturing unit mounted thereon may be a fisheye camera or other camera or scanning device that functions better than a fisheye camera.
  • the spatial coordinates of the visual feature points described by the group of feature descriptors extracted are determined. Taking out the pose of the mobile device at the current time, it can be seen that the spatial position of the visual feature point described by the set of feature descriptors extracted first determines the pose of the mobile device at the current moment, and can detect when the closed loop is detected. Correcting the bit posture of the mobile device, thereby eliminating the error of the position estimation of the mobile device accumulated during the movement process, thereby improving the positioning accuracy based on the positioning of the mobile device, thereby realizing the situation without establishing an environment map. Accurate positioning to ensure real-time and positioning accuracy based on mobile device positioning.
  • FIG. 1 is a schematic flowchart diagram of a positioning method of a mobile device according to an embodiment of the present invention, where the positioning method includes:
  • a Feature Descriptor may be represented by a vector in one example, specifically a vector for describing visual feature points in the acquired frame image data.
  • the first set of feature descriptors is a set of vectors for describing each visual feature point in the frame image data acquired at the current time.
  • the visual feature point is a point of the surrounding environment feature in the collected image, such as a table corner, a stool leg, a door corner, etc., as visual feature points.
  • the image acquisition unit (the image acquisition unit can be implemented by the camera in one example) on the mobile device performs image acquisition, each time image acquisition After acquiring one frame of image data, the unit identifies visual feature points in the image data, and extracts feature descriptors of the visual features to obtain a set of feature descriptors corresponding to one frame of image data. That is, before extracting the first set of feature descriptors corresponding to the current time, multiple sets of feature descriptors have been extracted beforehand.
  • extracting the first set of feature descriptors of the collected visual feature points at the current time comprises: extracting feature descriptors of the visual feature points in the frame image data collected at the current time as the first set of feature descriptors, and recording the extracted The first set of descriptors.
  • extracting the first set of feature descriptors of the collected visual feature points at the current time comprises: when determining that the frame image data acquired at the current time is key frame image data, Extracting the feature descriptor of the visual feature point in the frame image data acquired at the current time is a first group of feature descriptors.
  • S102 Perform closed-loop detection on each of the first set of feature descriptors and each of the plurality of sets of feature descriptors that are extracted first.
  • each of the plurality of sets of feature descriptors is extracted first is the same as or similar to the embodiment in which the first set of feature descriptors are extracted in S101.
  • each set of feature descriptors of the plurality of sets of feature descriptors that are extracted first includes: extracting the collected frames when the frame image data is collected by the image acquisition unit at a previous time compared with the current time. a feature descriptor of the visual feature point in the image data, each frame image data is correspondingly extracted to a set of feature descriptors, and a set of feature descriptors corresponding to each frame image data is recorded, thereby obtaining the prior extraction described in S102. Multiple sets of feature descriptors.
  • the plurality of sets of feature descriptors extracted in advance correspond to the key image data collected from a plurality of prior moments compared to the current time At least one set of feature descriptors extracted from the frame image data, and when the collected frame image data is not key frame image data, the feature descriptor is not extracted.
  • the key frame image data is all the frame image data sequentially collected from the mobile device according to the preset spatial interval. Determined.
  • each time the image acquisition unit collects frame image data it is determined whether the captured frame image data is key frame image data based on a preset spatial interval, and if the frame image data is determined to be key frame image data, The feature descriptor of the visual feature point in the acquired frame image data, if the frame image data is determined not to be the key frame image data, the feature descriptor is not extracted.
  • the preset space interval is set according to the computing resources and the positioning accuracy requirements of the mobile device. For example, if the preset spatial interval is 0.5 m, the frame image data collected by the initial position of the mobile device is determined as the key frame image data, and then the frame image data acquired by the mobile device from the initial position after each movement of 0.5 is determined as the key. Frame image data, and the frame image data collected by the mobile device at other locations is not key frame image data, such as: (0m, 0.5m), (0.5m, 1m), (1m, 1.5m)... The frame image data is determined to be non-key frame image data.
  • performing closed-loop detection on each of the first set of feature descriptors and each of the plurality of sets of feature descriptors that are extracted first including: respectively, the first set of feature descriptors are respectively Each set of feature descriptors in the first set of feature descriptors is similarly compared, and each of the plurality of sets of feature descriptors extracted firstly determines that each set of feature descriptors and the first set of feature descriptors satisfy a preset The number of descriptors of the similar condition; respectively, determining whether the number of descriptors of each of the plurality of sets of feature descriptors and the first set of feature descriptors satisfying the preset similar condition is greater than a preset number threshold; When the number of descriptors of the similar condition is greater than the preset number threshold, the first set of feature descriptors and the corresponding group feature descriptor are characterized to detect a closed loop. Therefore, when a closed loop is detected based on a certain group of feature de
  • Three sets of feature descriptors are correspondingly extracted at the T1 time, the T2 time, and the T3 time before the current time (for example, the time T4).
  • they are named as: group A feature descriptors extracted at time T1, group B feature descriptors extracted at time T2, group C feature descriptors extracted at time T3, and group D feature descriptors extracted at time T4 (ie, For a set of feature descriptors, the following three steps are performed independently: the D group feature descriptors are compared with the A group feature descriptors to determine that the D group feature descriptors and the A group feature descriptors satisfy the preset similarity.
  • the number of descriptors of the condition is a; the similarity between the D group feature descriptor and the B group feature descriptor is determined, and the number of descriptors satisfying the preset similarity condition between the D group feature descriptor and the B group feature descriptor is determined as b.
  • the D group feature descriptors are similarly compared with the C group feature descriptors, and the number of descriptors satisfying the preset similar conditions between the D group feature descriptors and the C group feature descriptors is determined to be c.
  • the number of descriptors a is greater than a preset number threshold, whether the number of descriptors b is greater than a preset threshold, and whether the number of descriptors c is greater than a preset threshold. If it is determined that the number of descriptors a is greater than the preset number threshold, it is considered that the current time reaches the same area that has been reached at time T1; if the number of description sub-b is greater than the preset number threshold, it is considered that the current time arrives at the same time that T2 has arrived. If the number of descriptors c is greater than the preset number threshold, the current time is considered to have reached the same area that was reached at time T3.
  • the preset number threshold may be set according to actual requirements.
  • the preset number threshold may be set to 3, and when the number of descriptors satisfying the preset similar condition is greater than 3, the closed loop is detected.
  • the number of descriptors satisfying the preset similarity condition is 4, 5, or 6, etc., and the closed loop is detected.
  • the first set of feature descriptors of the collected visual feature points (this is a group B feature descriptor) is extracted, and the A group feature descriptors extracted at the time of T1 are closed-loop detected.
  • the first set of feature descriptors (currently group C feature descriptors) of the collected visual feature points are extracted, and the group A feature descriptors extracted at time T1 are closed-loop detected.
  • the B group feature descriptor extracted at time T2 Ring detection is closing with the B group feature descriptor extracted at time T2 Ring detection.
  • the first set of feature descriptors (this is the D set of feature descriptors) for extracting the collected visual feature points are respectively closed-loop detection with the A set of feature descriptors extracted at time T1.
  • the B group feature descriptors extracted at time T2 are closed-loop detection
  • the C group feature descriptors extracted at T3 time are closed-loop detection. Looping sequentially, so that when the first set of feature descriptors are extracted at each current time of T2, T3, T4, T4, T6, respectively, each set of features in the plurality of sets of feature descriptors extracted earlier than the current time is respectively The descriptor performs closed loop detection.
  • the preset angle threshold can be set according to actual needs. For example, if the preset angle threshold is set to 30 degrees, the vector angle between the two compared feature descriptors is [0, 30] degrees, and it is determined that the preset similar condition is satisfied, otherwise it is determined that the condition is not satisfied. Presuppose similar conditions. For example, if the preset angle threshold is set to 15 degrees, the vector angle between the two compared feature descriptors is [0, 15] degrees, and it is determined that the preset similar condition is satisfied, otherwise it is determined that the preset similarity is not satisfied. condition.
  • the second set of feature descriptors is one of the plurality of sets of feature descriptors that are extracted first.
  • the posture of the mobile device at the current moment includes at least a mobile device. Position and posture at the current moment.
  • the spatial coordinates of the visual feature points described by the second set of feature descriptors determine the pose of the mobile device at the current moment, including:
  • S1011 Determine a plurality of feature descriptors in the second set of feature descriptors.
  • the determined plurality of feature descriptors are feature descriptors that satisfy a preset similar condition with the feature descriptors in the first group of feature descriptors.
  • the number of feature descriptors determined from the second set of feature descriptors is set according to a preset number threshold. For example, if the preset number threshold is 3, four feature descriptors that satisfy the preset similarity condition with the feature descriptors in the first group of feature descriptors are determined from the second group of feature descriptors.
  • the technical description provided in step S102 can determine, from the second set of feature descriptors, feature descriptors that satisfy the preset similarity condition with the first set of feature descriptors, for example, five features. a descriptor, or six feature descriptors, or seven feature descriptors, etc., in which four feature descriptors can be determined from the determined five or six or seven feature descriptors in S1031 of the present embodiment.
  • the four feature descriptors in the second set of feature descriptors and the first set of feature descriptors satisfy the preset similarity condition are determined by the four feature descriptors.
  • step S102 has determined, from the second group of feature descriptors, a feature descriptor that satisfies a preset similar condition with the first group of feature descriptors, for example, there are five feature descriptors, or 6 feature descriptors, or 7 feature descriptors, or 8 feature descriptors, etc., can be determined from the 5 or 6 or 7 or 8 feature descriptors in S1031 of the present embodiment. There are only five feature descriptors in the second set of feature descriptors, and the first set of feature descriptors satisfy the preset similar conditions.
  • S1012 Determine corresponding two-dimensional image coordinates in the frame image data collected by the plurality of feature descriptors at the current time.
  • the determined plurality of feature descriptors are different from each other.
  • the determined feature descriptors may include: a feature descriptor of “table angle 1”, a feature descriptor of “table angle 2”, and “stool leg 1” ”
  • the feature descriptor, the feature descriptor of the "stool leg 2" determines the two-dimensional image coordinates of the feature descriptor of the "table angle 1" in the frame image data acquired at the current time, and determines the "table angle 2”
  • the feature descriptor collects the two-dimensional image coordinates in the frame image data at the current time, determines the two-dimensional image coordinates of the feature descriptor of the "stool leg 1" in the frame image data acquired at the current time, and determines the "stool leg”
  • the feature of 2" describes the two-dimensional image coordinates in the frame image data acquired at the current time.
  • T is a transition matrix
  • X i is a plurality of spatial vision characteristic feature point coordinates of the sub-described description
  • K is an internal parameter matrix of the image acquisition unit built in the mobile device
  • R is the posture of the mobile device
  • t is the position of the mobile device.
  • the spatial coordinates of the visual feature points described by the plurality of feature descriptors are obtained by the following embodiments: the image acquisition unit collects frame image data, and records each visual feature point in the acquired frame image data. Spatial coordinates.
  • the spatial coordinates of the visual feature points described by the plurality of feature descriptors are obtained by the following embodiments: the image acquisition unit collects key frame image data, and records each of the captured key frame image data. The spatial coordinates of the feature points.
  • the spatial coordinates of the visual feature points described by the plurality of feature descriptors may be determined from the spatial coordinates of each of the recorded second feature descriptors based on any of the above embodiments.
  • the transfer matrix T may be determined according to the number of feature descriptors determined by S1031.
  • S1031 may include: determining, from the second set of feature descriptors, four feature descriptors that satisfy a preset similarity condition with the first set of feature descriptors, and then describing the visual features based on the four feature descriptors.
  • the spatial coordinates of the points (for example, the spatial coordinates of the visual feature points described by the four feature descriptors are X 1 , X 2 , X 3 , X 4 , respectively ), and the visual feature points described by the four feature descriptors are in the current Collecting corresponding two-dimensional image coordinates in the frame image data at any time (for example, it can be recorded as
  • the internal parameter matrix K of the image acquisition unit built in the mobile device establishes a 4 ⁇ 4 transfer matrix T indicating the pose of the mobile device.
  • S1034 Obtain a pose of the mobile device at the current moment based on the transfer matrix. Specifically, the solution obtains the pose at the current moment including the posture R and the position t of the mobile device at the current time.
  • the pose of the mobile device determined by the embodiment of the present invention is used to correct the motion trajectory estimated based on the inertial data and the visual information.
  • the inertial data and the visual information of the mobile device during the moving process are collected, and the motion trajectory of the mobile device during the moving process is estimated based on the inertial data and the visual information.
  • the inertial data of the mobile device during the movement is collected by an IMU (Inertial Measurement Unit) mounted on the mobile device.
  • the IMU includes an accelerometer and a gyroscope.
  • the accelerometer and the gyroscope correspondingly measure the acceleration and angular velocity during the movement of the mobile device itself, and then calculate the position and posture of the mobile device at each moment, and the image acquisition unit mounted on the mobile device performs
  • the visual information of the mobile device during the movement is collected, and the position and posture of the estimated mobile device are further estimated using the visual information to obtain a motion trajectory of the mobile device during the movement.
  • the pose of the corresponding moment estimated based on the inertial data and the visual information is replaced to correct the motion trajectory estimated based on the inertial data and the visual information.
  • the posture of the mobile device obtained at the current time is obtained by solving the transfer matrix established by S1033.
  • the position t replaces the posture R and the position t of the corresponding time estimated based on the inertial data and the visual information to achieve an effect of correcting the motion trajectory estimated based on the inertial data and the visual information.
  • the embodiment of the invention further provides a mobile device, as shown in FIG. 3, comprising:
  • the extracting unit 201 is configured to extract, during the moving process of collecting the visual feature points, the first set of feature descriptors of the collected visual feature points at the current time;
  • the detecting unit 202 is configured to perform closed-loop detection on each of the first set of feature descriptors collected by the extracting unit 201 and each of the plurality of sets of feature descriptors that are extracted first;
  • a determining unit 203 configured to determine, by the detecting unit 202, when the closed loop is detected by the first set of feature descriptors and the second set of feature descriptors, by using spatial coordinates of the visual feature points described by the second set of feature descriptors The pose of the mobile device at the current moment, wherein the second set of feature descriptors is one of a plurality of sets of feature descriptors that are previously extracted.
  • the detecting unit 202 includes:
  • the comparison subunit is configured to compare the first set of feature descriptors with each of the plurality of sets of feature descriptors that are extracted first, and respectively determine each of the plurality of previously selected sets of feature descriptors a feature descriptor and a number of descriptors satisfying a preset similar condition with the first group of feature descriptors;
  • a determining subunit configured to determine, respectively, whether the number of descriptors of each of the plurality of sets of feature descriptors determined by the comparison subunit and the first set of feature descriptors satisfy a preset similarity condition is greater than a preset number threshold; when the number of descriptors satisfying the preset similar condition is greater than the preset number threshold, characterizing the first set of feature descriptors and the corresponding group feature descriptors detects a closed loop.
  • the comparison subunit is configured to perform each feature descriptor in each of the first group of feature descriptors and each of the group of feature descriptors in the first group of feature descriptors. Contrast; determining whether the vector angle between the feature descriptors for comparison is less than a preset angle threshold, wherein the feature that is compared when the vector angle is less than the preset angle threshold The descriptor satisfies the preset similar conditions.
  • the determining unit 203 includes:
  • a first determining subunit configured to determine a plurality of feature descriptors in the second set of feature descriptors
  • a second determining subunit configured to determine a corresponding two-dimensional image coordinate in the frame image data acquired by the plurality of feature descriptors determined by the first determining subunit at the current time;
  • a matrix establishing subunit configured to establish a transfer matrix representing a pose of the mobile device based on spatial coordinates of the visual feature points described by the plurality of feature descriptors, two-dimensional image coordinates, and an internal parameter matrix of the built-in image acquisition unit of the mobile device;
  • Solving the subunit configured to solve the transition matrix to obtain the pose of the mobile device at the current moment
  • T is a transition matrix
  • X i is a plurality of spatial vision characteristic feature point coordinates of the sub-described description
  • K is an internal parameter matrix of the built-in image acquisition unit of the mobile device
  • R is a posture of the mobile device
  • t is a mobile device position.
  • the mobile device further includes:
  • An acquisition unit configured to collect inertial data and visual information during the movement
  • a trajectory estimating unit configured to estimate a motion trajectory during the movement based on the inertial data and the visual information collected by the collecting unit.
  • the mobile device further includes:
  • a correction unit configured to replace the pose of the corresponding moment estimated based on the inertial data and the visual information based on the pose of the mobile device determined by the determining unit at the current moment to correct the motion trajectory.
  • the descriptor includes: extracting a set of feature descriptors from the key frame image data when the key frame image data acquired at a previous time compared with the current time, wherein the key frame image data is according to a preset space The intervals are sequentially determined from the acquired image data of all the frames.
  • the extracting unit 201, the detecting unit 202, the determining unit 203, the collecting unit, the trajectory estimating unit and the correcting unit of the mobile device, and the comparing subunit and the determining subunit included in the detecting unit 202 are included in the determining unit 203.
  • the first determining subunit, the second determining subunit, the matrix establishing subunit, and the solving subunit are, in practical applications, a CPU (Central Processing Unit), a DSP (Digital Signal Processor) Or FPGA (Field-Programmable Gate Array) implementation.
  • an embodiment of the present invention further provides a mobile device, including a memory, a processor, and a computer program stored on the memory and operable on the processor, where the processor implements the implementation of the present invention when the program is executed.
  • a mobile device including a memory, a processor, and a computer program stored on the memory and operable on the processor, where the processor implements the implementation of the present invention when the program is executed.
  • the step of the positioning method of the mobile device according to the example, specifically, when the processor executes the program, when the mobile device collects the visual feature points, the first time the visual feature points collected at the current time are extracted a set of feature descriptors; performing closed-loop detection on each of the first set of feature descriptors and each of the plurality of sets of feature descriptors previously extracted; and describing the second and second features based on the first set of features
  • the group feature describes the closed loop, the pose of the mobile device at the current moment is determined by the spatial coordinates of the visual feature points described by the second set of
  • the first set of feature descriptors are compared with each of the plurality of sets of feature descriptors that are extracted first, and are respectively determined. Determining the number of descriptors of each set of feature descriptors and the first set of feature descriptors satisfying preset similar conditions in the plurality of sets of feature descriptors extracted first; respectively determining each of the plurality of sets of feature descriptors extracted first Whether the number of descriptors of the descriptor and the first set of feature descriptors satisfying the preset similarity condition is greater than a preset number threshold; when the preset similar condition is satisfied When the number of the descriptors is greater than the preset number threshold, the first set of feature descriptors and the corresponding group feature descriptors are characterized to detect a closed loop.
  • each of the feature descriptors in the first set of feature descriptors and each of the plurality of sets of feature descriptors in the first set of feature descriptors are respectively The feature descriptors are compared one by one; determining whether the vector angle between the feature descriptors for comparison is less than a preset angle threshold, wherein the feature that is compared when the vector angle is smaller than the preset angle threshold The descriptor satisfies the preset similar condition.
  • the processor executes the program, determining: determining a plurality of feature descriptors in the second set of feature descriptors; determining that the plurality of feature descriptors are collected at the current time Corresponding two-dimensional image coordinates in the frame image data; based on spatial coordinates of the visual feature points described by the plurality of feature descriptors, the two-dimensional image coordinates, and an internal parameter matrix of the image acquisition unit built in the mobile device a transfer matrix representing a pose of the mobile device; obtaining a pose of the mobile device at the current time based on the transfer matrix;
  • T is the transition matrix
  • X i is the descriptor of the plurality of spatial coordinates of feature points of the described visual features
  • K is an internal parameter matrix of the image acquisition unit built in the mobile device
  • R is a posture of the mobile device
  • t is the location of the mobile device.
  • the inertial data and the visual information of the mobile device during the moving process are collected during the movement of the mobile device to collect the visual feature points;
  • the inertial data and the visual information estimate a motion trajectory of the mobile device during the moving.
  • the mobile device when the processor executes the program, the mobile device is determined by the spatial coordinates of the visual feature points described by the second set of feature descriptors. After the pose of the current time is set, the posture of the corresponding moment estimated based on the inertial data and the visual information is replaced based on the determined pose of the mobile device at the current moment to correct the posture Movement track.
  • the first feature set of the visual feature points collected at the current time is extracted by the mobile device collecting the visual feature points during the moving process; and the first set of feature descriptors are respectively extracted from the previous ones.
  • Each set of feature descriptors in the group feature descriptor performs closed-loop detection to determine whether the mobile device has reached the same region that has arrived; detecting a closed loop based on the first set of feature descriptors and the second set of feature descriptors extracted earlier And determining, by the spatial coordinates of the visual feature points described by the second set of feature descriptors, the pose of the mobile device at the current time, so that the space of the visual feature points recorded according to the previous time when the mobile device passes through the same region again The coordinates recalculate the current pose of the mobile device to correct the pose of the mobile device in the closed loop position, thereby eliminating the error accumulation of the pose estimation, and solving the error of the mobile device accumulating the pose estimation of the pose during the movement.
  • the embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are used to perform the positioning method of the mobile device according to the embodiment of the invention.
  • the computer executable instructions are executed by the processor: when the mobile device collects the visual feature points, extracting a first set of feature descriptors of the collected visual feature points at the current time; A set of feature descriptors respectively performs closed-loop detection with each set of feature descriptors of the plurality of sets of feature descriptors extracted first; and when a closed loop is detected based on the first set of feature descriptors and the second set of feature descriptors, Determining a spatial coordinate of the visual feature point described by the second set of feature descriptors, determining a pose of the mobile device at the current time, wherein the second set of feature descriptors is the prior extracted Multiple sets Evaluate one of the descriptors.
  • the first set of feature descriptors are compared with each of the previously extracted sets of feature descriptors, Determining, respectively, the number of descriptors of each of the plurality of sets of feature descriptors that are extracted first and the first set of feature descriptors satisfying a preset similar condition; respectively determining each of the plurality of sets of feature descriptors extracted first Whether the number of descriptors of the group feature descriptor and the first group of feature descriptors satisfying the preset similarity condition is greater than a preset number threshold; when the number of descriptors satisfying the preset similar condition is greater than the preset number threshold, Characterizing the first set of feature descriptors and the corresponding set of feature descriptors detects a closed loop.
  • each feature descriptor in the first set of feature descriptors is respectively associated with each of the plurality of sets of feature descriptors extracted earlier;
  • Each feature descriptor is compared one by one; determining whether a vector angle between the feature descriptors for comparison is less than a preset angle threshold, wherein the representation is compared when the vector angle is less than the preset angle threshold The feature descriptor satisfies the preset similar condition.
  • the computer executable instructions are executed by the processor to: determine a plurality of feature descriptors in the second set of feature descriptors; and determine the plurality of feature descriptors at the current moment Corresponding two-dimensional image coordinates in the acquired frame image data; spatial coordinates of the visual feature points described based on the plurality of feature descriptors, the two-dimensional image coordinates, and internal parameters of the image acquisition unit built in the mobile device
  • the matrix establishes a transition matrix representing the pose of the mobile device; obtaining a pose of the mobile device at the current time based on the transition matrix.
  • the inertial data and the visual information of the mobile device during the moving process are collected during the movement of the mobile device to collect the visual feature points;
  • the inertial data and the visual information estimate a motion trajectory of the mobile device during the moving.
  • the computer executable instructions are executed by the processor: determining, by the spatial coordinates of the visual feature points described by the second set of feature descriptors, that the mobile device is in the After the pose of the current time, the pose of the corresponding moment estimated based on the inertial data and the visual information is replaced based on the determined pose of the mobile device at the current moment to correct the motion trajectory.
  • modules in the devices of the embodiments can be adaptively changed and placed in one or more devices different from the embodiment.
  • the modules or units or components of the embodiments may be combined into one module or unit or component, and further they may be divided into a plurality of sub-modules or sub-units or sub-components.
  • any combination of the features disclosed in the specification, including the accompanying claims, the abstract and the drawings, and any methods so disclosed, or All processes or units of the device are combined.
  • Each feature disclosed in this specification (including the accompanying claims, the abstract and the drawings) may be replaced by alternative features that provide the same, equivalent or similar purpose.
  • Various component embodiments of the present invention may be implemented in hardware, or in a client module running on one or more processors, or in a combination thereof.
  • a microprocessor or digital signal processor may be used in practice to implement some or all of the functionality of some or all of the components of the security protection device of the software installation package in accordance with embodiments of the present invention.
  • the invention can also be implemented as a device or device program (e.g., a computer program and a computer program product) for performing some or all of the methods described herein.
  • a program implementing the invention may be stored on a computer readable medium or may be in the form of one or more signals. Such signals may be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
  • the technical solution of the embodiment of the present invention collects a visual feature point during the moving process by the mobile device, and extracts a first set of feature descriptors of the collected visual feature points at the current time; and respectively extracts the first set of feature descriptors with the previously extracted
  • Each set of feature descriptors of the plurality of sets of feature descriptors performs closed loop detection to determine whether the mobile device has reached the same area that has arrived; and is detected based on the first set of feature descriptors and the second set of feature descriptors extracted earlier
  • the spatial coordinates of the visual feature points described by the second set of feature descriptors are used to determine the pose of the mobile device at the current time, so that the visual feature points recorded according to the previous time can be used when the mobile device passes through the same region again.
  • the spatial coordinates recalculate the current pose of the mobile device to correct the pose of the mobile device in the closed loop position, thereby eliminating the error accumulation of the pose estimation, and solving the estimation of the pose of the mobile device accumulated during the movement.
  • Errors technical problems that seriously affect positioning accuracy, effective in the absence of an environmental map
  • Based on high precision positioning of mobile devices enabling accurate positioning without the establishment of environmental conditions map to simultaneously ensure that real-time mobile-based positioning device and positioning accuracy.

Abstract

一种移动装置及其定位方法，其中，该方法包括：在移动装置采集视觉特征点的移动过程中，提取当前时刻所采集视觉特征点的第一组特征描述子（S101）；将第一组特征描述子分别与之前提取的每组特征描述子进行闭环检测（S102）；在基于第一组特征描述子与第二组特征描述子检测到闭环时，通过第二组特征描述子所描述的视觉特征点的空间坐标，确定出移动装置在当前时刻的位姿，其中，第二组特征描述子为之前提取的各组特征描述子中的其中一组（S103）。该方法解决了移动装置在移动过程中积累下对自身位姿估计的误差，而严重影响定位精度的技术问题，进而提高了基于移动装置定位的精度。

Description

一种移动装置及其定位方法、计算机存储介质
相关申请的交叉引用
本申请基于申请号为201610652818.0、申请日为2016年08月10日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此以引入方式并入本申请。
技术领域
本发明涉及定位技术领域，尤其涉及一种移动装置及其定位方法、计算机存储介质。
背景技术
现有基于视觉和惯性器件的机器人进行室内定位方法主要分为两大类：1)建立环境地图的定位方法，如：视觉SLAM(Simultaneous Localization And Mapping，即时定位与地图构建)技术，2)不需要建立环境地图的定位方法，如：视觉/惯性里程计技术。
建立环境地图的定位方法包括：机器人通常在估计自身位置姿态的同时，要对环境建立地图，通过优化机器人自身轨迹以及轨迹中各个位置姿态与地图中路标的相对位置关系来获得机器人的位置信息。该建立环境地图的定位方法精度较高，但是室内环境地图的建立若要将环境信息纳入到定位的优化算法中，就需要消耗机器人大量的运算资源，因此优化算法的运算量往往成为影响建立环境地图的定位方法的实时性的瓶颈。而现有不需要建立环境地图的定位方法在移动设备中的应用使实时性得到保证，但是随着运动轨迹的增长，移动装置在移动过程中积累下对自身位置姿态估计的误差，导致对移动设备位置姿态的估计误差会持续增加，严重影响定 位精度。
发明内容
本发明实施例期望提供一种移动装置及其定位方法、计算机存储介质，以解决现有技术中基于移动装置的定位对移动设备位姿的估计误差会持续增加，而严重影响定位精度的技术问题。
第一方面，本发明实施例提供了一种移动装置的定位方法，包括：
在所述移动装置采集视觉特征点的移动过程中，提取当前时刻所采集视觉特征点的第一组特征描述子；
将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行闭环检测；
在基于所述第一组特征描述子与第二组特征描述子检测到闭环时，通过所述第二组特征描述子所描述的视觉特征点的空间坐标，确定出所述移动装置在所述当前时刻的位姿，其中，所述第二组特征描述子为所述在先提取的多组特征描述子中的其中一组。
在一实施例中，所述将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行闭环检测，包括：
将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行相似对比，分别确定出在先提取的多组特征描述子中每组特征描述子与所述第一组特征描述子满足预设相似条件的描述子数量；
分别判断在先提取的多组特征描述子中每组特征描述子与所述第一组特征描述子满足预设相似条件的描述子数量是否大于预设数量阈值；当满足所述预设相似条件的描述子数量大于所述预设数量阈值时，表征所述第一组特征描述子与对应的组特征描述子检测到闭环。
在一实施例中，将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行相似对比，判断是否满足预设相似条件， 包括：
将所述第一组特征描述子中的每个特征描述子分别与在先提取的多组特征描述子中的每组特征描述子中的每个特征描述子进行一一对比；
判断进行对比的特征描述子之间的向量夹角是否小于预设角度阈值，其中，在所述向量夹角小于所述预设角度阈值时表征进行对比的特征描述子满足所述预设相似条件。
在一实施例中，所述通过所述第二组特征描述子所描述的视觉特征点的空间坐标，确定出所述移动装置在所述当前时刻的位姿，包括：
确定出所述第二组特征描述子中的多个特征描述子；
确定所述多个特征描述子在所述当前时刻采集的帧图像数据中对应的二维图像坐标；
基于所述多个特征描述子所描述视觉特征点的空间坐标、所述二维图像坐标、以及所述移动装置内置的图像采集单元的内参数矩阵建立表示所述移动装置的位姿的转移矩阵；
基于所述转移矩阵得到所述移动装置在所述当前时刻的位姿；
其中，所述转移矩阵满足：
Figure PCTCN2017096945-appb-000001
其中，T为所述转移矩阵，Xi为所述多个特征描述子所描述视觉特征点的空间坐标，
Figure PCTCN2017096945-appb-000002
为所述多个特征描述子在所述当前时刻采集的帧图像数据中对应的二维图像坐标，K为所述移动装置内置的图像采集单元的内参数矩阵，R为所述移动装置的姿态，t为所述移动装置的位置。
在一实施例中，所述在移动装置采集视觉特征点的移动过程中，所述方法还包括：
采集所述移动装置在所述移动过程中的惯性数据和视觉信息；
基于所述惯性数据和所述视觉信息估计所述移动装置在所述移动过程 中的运动轨迹。
在一实施例中，在所述通过所述第二组特征描述子所描述的视觉特征点的空间坐标，确定出所述移动装置在所述当前时刻的位姿之后，所述方法还包括：
基于确定出的所述移动装置在所述当前时刻的位姿代替基于所述惯性数据和所述视觉信息估计的对应时刻的位姿，以修正所述运动轨迹。
在一实施例中，所述在先提取的多组特征描述子中的每组特征描述子包括：与所述当前时刻相比的在先时刻采集的关键帧图像数据时从所述关键帧图像数据中提取一组特征描述子，其中，所述关键帧图像数据为按照预设空间间隔依次从采集的全部帧图像数据中确定出。
第二方面，本发明实施例还提供了一种移动装置，包括：
提取单元，配置为在采集视觉特征点的移动过程中，提取当前时刻所采集视觉特征点的第一组特征描述子；
检测单元，配置为将所述提取单元采集的所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行闭环检测；
确定单元，配置为在所述检测单元基于所述第一组特征描述子与第二组特征描述子检测到闭环时，通过所述第二组特征描述子所描述的视觉特征点的空间坐标，确定出所述移动装置在所述当前时刻的位姿，其中，所述第二组特征描述子为所述在先提取的多组特征描述子中的其中一组。
在一实施例中，所述检测单元，包括：
对比子单元，配置为将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行相似对比，分别确定出在先提取的多组特征描述子中每组特征描述子与所述第一组特征描述子满足预设相似条件的描述子数量；
判断子单元，配置为分别判断所述对比子单元确定的在先提取的多组 特征描述子中每组特征描述子与所述第一组特征描述子满足预设相似条件的描述子数量是否大于预设数量阈值；当满足所述预设相似条件的描述子数量大于所述预设数量阈值时，表征所述第一组特征描述子与对应的组特征描述子检测到闭环。
在一实施例中，所述对比子单元，配置为将所述第一组特征描述子中的每个特征描述子分别与在先提取的多组特征描述子中的每组特征描述子中的每个特征描述子进行一一对比；判断进行对比的特征描述子之间的向量夹角是否小于预设角度阈值，其中，在所述向量夹角小于所述预设角度阈值时表征进行对比的特征描述子满足所述预设相似条件。
在一实施例中，所述确定单元，包括：
第一确定子单元，配置为确定出所述第二组特征描述子中的多个特征描述子；
第二确定子单元，配置为确定所述第一确定子单元确定的所述多个特征描述子对应在当前时刻采集帧图像数据中的二维图像坐标；
矩阵建立子单元，配置为基于所述多个特征描述子所描述视觉特征点的空间坐标、所述二维图像坐标、以及所述移动装置的内置图像采集单元的内参数矩阵建立表示所述移动装置的位姿的转移矩阵；
求解子单元，配置为基于所述转移矩阵得到所述移动装置在所述当前时刻的位姿；
其中，所述转移矩阵满足：
Figure PCTCN2017096945-appb-000003
其中，T为所述转移矩阵，Xi为所述多个特征描述子所描述视觉特征点的空间坐标，
Figure PCTCN2017096945-appb-000004
为所述多个特征描述子在所述当前时刻采集的帧图像数据中的二维图像坐标，K为所述移动装置的内置图像采集单元的内参数矩阵，R为所述移动装置的姿态，t为所述移动装置的位置。
在一实施例中，所述移动装置还包括：
采集单元，配置为采集移动过程中的惯性数据和视觉信息；
轨迹估计单元，配置为基于所述采集单元采集的惯性数据和所述视觉信息估计在移动过程中的运动轨迹。
在一实施例中，所述移动装置还包括：
修正单元，配置为基于所述确定单元确定出的所述移动装置在所述当前时刻的位姿代替基于所述惯性数据和所述视觉信息估计的对应时刻的位姿，以修正所述运动轨迹。
在一实施例中，所述在先提取的多组特征描述子中的每组特征描述子包括：与所述当前时刻相比的在先时刻采集的关键帧图像数据时从所述关键帧图像数据中提取一组特征描述子，其中，所述关键帧图像数据为按照预设空间间隔依次从采集的全部帧图像数据中确定出。
第三方面，本发明实施例还提供了一种计算机存储介质，所述计算机存储介质中存储有计算机可执行指令，所述计算机可执行指令用于执行本发明实施例所述的移动装置的定位方法。
本发明实施例提供的移动装置及其定位方法、计算机存储介质，通过移动装置在移动过程中采集视觉特征点，提取当前时刻所采集视觉特征点的第一组特征描述子；将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行闭环检测，从而确定移动装置是否到达已经到达过的同一区域；在基于第一组特征描述子与在先提取的第二组特征描述子检测到闭环时，通过第二组特征描述子描述的视觉特征点的空间坐标，确定出移动装置在当前时刻的位姿，从而能够在移动装置再一次经过同一区域时根据先前时刻记录的视觉特征点的空间坐标重新计算移动装置当前的位姿，以修正移动装置在闭环位置的位姿，从而消除了对位姿估计的误差累计，以解决了移动装置在移动过程中积累下对自身位姿估计的 误差，而严重影响定位精度的技术问题，以在不建立环境地图情况下有效提高了基于移动装置定位的精度，从而实现了在不建立环境地图情况下的准确定位，以同时确保基于移动装置定位的实时性和定位精度。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。
图1为本发明实施例的移动装置的定位方法的流程示意图；
图2为图1中步骤S103的细化流程图；
图3为本发明实施例中移动装置的功能单元图。
具体实施方式
本发明实施例提供的移动装置及其定位方法、计算机存储介质，以解决移动装置在移动过程中积累下对自身位姿估计的误差，而严重影响定位精度的技术问题。本发明实施例的技术方案为解决上述技术问题，提出的技术方案包括：
在移动装置采集视觉特征点的移动过程中，提取当前时刻所采集视觉特征点的第一组特征描述子，将第一组特征描述子分别与在线提取的多组特征描述子中的每组特征描述子进行闭环检测。比如，移动装置为搭载有图像采集单元的机器人，搭载的图像采集单元可以为鱼眼相机、或功能优于鱼眼相机的其他相机、扫描设备。通过这两个步骤可以看出，通过移动装置在移动过程中采集用于闭环检测的特征描述子，从而在每次采集到一组特征描述子时，均与在线提取的每组特征描述子进行闭环检测，从而通过循环的进行闭环检测，确定每个当前时刻到达的区域是否为之前曾经到 达过的同一区域。
进而在基于第一组特征描述子与在先提取的各组特征描述子中的其中一组检测到闭环时，通过在先提取的该组特征描述子所描述的视觉特征点的空间坐标，确定出移动装置在当前时刻的位姿，可以看出，通过在先提取的该组特征描述子所描述的视觉特征点的空间坐标确定出移动装置在当前时刻的位姿，能够在检测到闭环时进行修正移动装置的位姿态，从而消除了移动装置在移动过程中积累下的对自身位姿估计的误差，从而提高了基于移动装置定位的定位精度，从而实现了在不建立环境地图情况下的准确定位，以同时确保了基于移动装置定位的实时性和定位精度。
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
参考图1所示，图1为本发明实施例的移动装置的定位方法的流程示意图，该定位方法包括：
S101、在移动装置采集视觉特征点的移动过程中，提取当前时刻所采集视觉特征点的第一组特征描述子。
本实施例中，特征描述子(Feature Descriptor)在一种示例中可通过向量表示，具体为用于描述所采集帧图像数据中的视觉特征点的向量。第一组特征描述子为用于描述当前时刻所采集的帧图像数据中每个视觉特征点的一组向量。具体的，视觉特征点为所采集到的图像中的周围环境特征的点，比如：桌角、凳腿、门角等均可作为视觉特征点。
在移动装置进行移动过程中，移动装置上搭载的图像采集单元(图像采集单元在一种示例中可通过摄像头实现)进行图像采集，每次图像采集 单元采集到一帧图像数据后识别图像数据中的视觉特征点，提取这些视觉特征的特征描述子，以得到一帧图像数据所对应的一组特征描述子。即在提取当前时刻对应的第一组特征描述子之前，在先已提取到多组特征描述子。
在一实施例中，提取当前时刻所采集视觉特征点的第一组特征描述子包括：提取当前时刻所采集帧图像数据中视觉特征点的特征描述子为第一组特征描述子，记录提取的第一组描述子。为了减少对移动装置计算资源的消耗，在另一实施例中，提取当前时刻所采集视觉特征点的第一组特征描述子包括：在确定当前时刻采集的帧图像数据为关键帧图像数据时，提取当前时刻采集的帧图像数据中的视觉特征点的特征描述子为第一组特征描述子。
S102、将第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行闭环检测。
在S102中，在先提取多组特征描述子中的每组特征描述子的实施方式与S101中提取第一组特征描述子的实施方式相同或相似。
在一实施例中，在先提取的多组特征描述子中的每组特征描述子包括：与当前时刻相比的在先时刻，通过图像采集单元采集到帧图像数据时，提取所采集的帧图像数据中视觉特征点的特征描述子，每个帧图像数据对应提取到一组特征描述子，记录每个帧图像数据对应的一组特征描述子，从而得到S102中所述的在先提取的多组特征描述子。
为了减少对移动装置计算资源的消耗，在另一实施例中：在先提取的多组特征描述子对应于与当前时刻相比的多个在先时刻采集到关键帧图像数据时从所述关键帧图像数据中提取的至少一组特征描述子，采集到的帧图像数据不是关键帧图像数据时则不进行特征描述子的提取。其中，关键帧图像数据为按照预设空间间隔依次从移动装置采集的全部帧图像数据中 确定出的。作为一种实施方式，图像采集单元每次采集帧图像数据时，基于预设空间间隔进行判断所采集的帧图像数据是否为关键帧图像数据，若判定帧图像数据为关键帧图像数据则提取所采集的帧图像数据中视觉特征点的特征描述子，若判定帧图像数据不是关键帧图像数据则不进行特征描述子的提取。
其中，预设空间间隔根据移动装置的运算资源和定位精度需求进行设置。例如，预设空间间隔为0.5m，则移动装置启动的初始位置采集的帧图像数据确定为关键帧图像数据后，则以移动装置从初始位置之后每移动0.5时采集的帧图像数据判定为关键帧图像数据，而移动装置在其他位置采集到的帧图像数据均不是关键帧图像数据，比如：(0m，0.5m)、(0.5m，1m)、(1m，1.5m)…距离内采集到的帧图像数据均判定为非关键帧图像数据。
作为一种实施方式，所述将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行闭环检测，包括：将第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行相似对比，分别确定出在先提取的多组特征描述子中每组特征描述子与所述第一组特征描述子满足预设相似条件的描述子数量；分别判断在先提取的多组特征描述子中每组特征描述子与第一组特征描述子满足预设相似条件的描述子数量是否大于预设数量阈值；当满足预设相似条件的描述子数量大于预设数量阈值时，表征所述第一组特征描述子与对应的组特征描述子检测到闭环。从而在基于第一组特征描述子与在先提取的多组特征描述子中的某一组特征描述子检测到闭环时，则认为移动装置到达之前经过的对应同一区域。
作为一种示例，下面以在当前时刻之前已经提取了三组特征描述子为例，对进行一次闭环检测的实施方式举例描述，从而本领域技术人员可以根据如下举例描述知晓其他时刻进行闭环检测的实施方式。
在当前时刻(例如T4时刻)之前的T1时刻、T2时刻、T3时刻对应提取了三组特征描述子。为了描述方便，分别命名为：T1时刻提取的A组特征描述子、T2时刻提取的B组特征描述子、T3时刻提取的C组特征描述子，T4时刻提取的D组特征描述子(即第一组特征描述子)，则独立执行如下三个步骤：将D组特征描述子与A组特征描述子进行相似对比，确定出D组特征描述子与A组特征描述子之间满足预设相似条件的描述子数量为a；将D组特征描述子与B组特征描述子进行相似对比，确定出D组特征描述子与B组特征描述子之间满足预设相似条件的描述子数量为b，将D组特征描述子与C组特征描述子进行相似对比，确定出D组特征描述子与C组特征描述子之间满足预设相似条件的描述子数量为c。进一步地，判断描述子数量a是否大于预设数量阈值、判断描述子数量b是否大于预设数量阈值，以及判断描述子数量c是否大于预设数量阈值。若判断出描述子数量a大于预设数量阈值，则认为当前时刻到达T1时刻所达到过的同一区域；若描述子数量b大于预设数量阈值，则认为当前时刻到达T2时刻所到达过的同一区域；若描述子数量c大于预设数量阈值，则认为当前时刻到达T3时刻所达到过的同一区域。
在具体实施过程中，预设数量阈值可根据实际需求设置，例如，在本实施例中可设置预设数量阈值为3，在满足预设相似条件的描述子数量大于3时表征检测到闭环。例如，满足预设相似条件的描述子数量为4个、5个或6个等均表征检测到闭环。
本实施例中，在当前时刻为T2时刻，提取所采集视觉特征点的第一组特征描述子(本次为B组特征描述子)，与T1时刻提取的A组特征描述子进行闭环检测。随着时间的推移，在当前时刻为T3时刻，提取所采集视觉特征点的第一组特征描述子(本次为C组特征描述子)与T1时刻提取的A组特征描述子进行闭环检测、以及与T2时刻提取的B组特征描述子进行闭 环检测。随着时间的推移，在当前时刻为T4时刻，提取所采集视觉特征点的第一组特征描述子(本次为D组特征描述子)分别与T1时刻提取的A组特征描述子进行闭环检测、T2时刻提取的B组特征描述子进行闭环检测，以及T3时刻提取的C组特征描述子进行闭环检测。依次循环，从而在T2、T3、T4、T4、T6…每个当前时刻提取到第一组特征描述子时，分别与在当前时刻相比在先提取的多组特征描述子中的每组特征描述子进行闭环检测。
本实施例中，预设相似条件为描述子之间的向量夹角小于预设角度阈值。则判断是否满足预设相似条件包括：将第一组特征描述子中的每个特征描述子分别与在先提取的多组特征描述子中的每组特征描述子中的每个特征描述子进行一一对比；判断进行对比的特征描述子之间的向量夹角是否小于预设角度阈值，其中，在进行对比的特征描述子之间的向量夹角小于预设角度阈值时表征进行对比的两个特征描述子满足预设相似条件，从而判定描述子之间的匹配程度。
在具体实施过程中，预设角度阈值可根据实际需求设置。例如，预设角度阈值设置为30度，则两个进行对比的特征描述子之间的向量夹角为[0,30]度则判定为满足所述预设相似条件，否则判定为不满足所述预设相似条件。例如，预设角度阈值设置为15度，则两个进行对比的特征描述子之间的向量夹角为[0,15]度则判定为满足预设相似条件，否则判定为不满足预设相似条件。
S103、在基于第一组特征描述子与第二组特征描述子检测到闭环时，通过第二组特征描述子所描述的视觉特征点的空间坐标，确定出移动装置在当前时刻的位姿，其中，第二组特征描述子为所述在先提取的多组特征描述子中的其中一组。
作为一种实施方式，移动装置在所述当前时刻的位姿至少包括移动装 置在当前时刻的位置和姿态。参考图2所示，在一实施例中，所述通过第二组特征描述子所描述的视觉特征点的空间坐标，确定出移动装置在所述当前时刻的位姿，包括：
S1031、确定出第二组特征描述子中的多个特征描述子。
具体的，确定出的多个特征描述子为与第一组特征描述子中的特征描述子满足预设相似条件的特征描述子。从第二组特征描述子中确定出的特征描述子的数量根据预设数量阈值设置。比如预设数量阈值为3，则从第二组特征描述子中确定出与第一组特征描述子中的特征描述子满足预设相似条件的4个特征描述子。
以预设数量阈值为3为例，采用步骤S102提供的技术方案可以从第二组特征描述子中确定出与第一组特征描述子满足预设相似条件的特征描述子，比如有5个特征描述子、或6个特征描述子、或7个特征描述子等等，则在本实施方式的S1031中可以从确定的5个或6个或7个特征描述子中确定出4个特征描述子，第二组特征描述子中与第一组特征描述子满足预设相似条件的只有4个特征描述子，则这4个特征描述子均确定出。以预设数量阈值为4为例：步骤S102已经从第二组特征描述子中确定出与第一组特征描述子满足预设相似条件的特征描述子，比如：有5个特征描述子、或6个特征描述子、或7个特征描述子、或8个特征描述子等等，则在本实施方式的S1031中可以从这5个或6个或7个或8个特征描述子中确定出5个特征描述子，第二组特征描述子中与第一组特征描述子满足预设相似条件的只有5个特征描述子。
S1032、确定所述多个特征描述子在所述当前时刻采集的帧图像数据中对应的二维图像坐标。
具体的，确定出的多个特征描述子互不相同，比如，确定出的特征描述子可包括：“桌角1”的特征描述子、“桌角2”的特征描述子、“凳腿1” 的特征描述子、“凳腿2”的特征描述子，则确定“桌角1”的特征描述子在所述当前时刻采集的帧图像数据中的二维图像坐标，确定“桌角2”的特征描述子在所述当前时刻采集帧图像数据中的二维图像坐标，确定“凳腿1”的特征描述子在所述当前时刻采集的帧图像数据中的二维图像坐标，确定“凳腿2”的特征描述子在所述当前时刻采集的帧图像数据中的二维图像坐标。在具体实施过程中，可通过视觉特征匹配，匹配出S1031中已经确定出的多个特征描述子所对应描述的多个视觉特征点在当前时刻采集的帧图像数据中的二维图像坐标。
S1033、基于多个特征描述子所描述视觉特征点的空间坐标、所述二维图像坐标、以及所述移动装置内置的图像采集单元的内参数矩阵建立表示移动装置的位姿的转移矩阵。
其中，所述转移矩阵满足以下表达式：
Figure PCTCN2017096945-appb-000005
其中，T为转移矩阵，Xi为多个特征描述子所描述视觉特征点的空间坐标，
Figure PCTCN2017096945-appb-000006
为多个特征描述子在当前时刻采集的帧图像数据中对应的二维图像坐标，K为移动装置内置的图像采集单元的内参数矩阵，R为移动装置的姿态，t为移动装置的位置。
在一实施例中，所述多个特征描述子所描述视觉特征点的空间坐标通过如下实施方式获得：图像采集单元采集到帧图像数据，记录所采集的帧图像数据中每个视觉特征点的空间坐标。在另一实施例中，所述多个特征描述子所描述视觉特征点的空间坐标通过如下实施方式获得：图像采集单元采集到关键帧图像数据，记录所采集的关键帧图像数据中每个视觉特征点的空间坐标。可基于上述任一实施方式，从记录的第二组特征描述子中每个视觉特征点的空间坐标确定出该多个特征描述子所描述的视觉特征点的空间坐标。
具体的，转移矩阵T可根据S1031确定出的特征描述子的数量确定。在一示例中，S1031可包括：从第二组特征描述子中确定出与第一组特征描述子满足预设相似条件的4个特征描述子，则基于这4个特征描述子所描述视觉特征点的空间坐标(例如这4个特征描述子所描述视觉特征点的空间坐标分别为X1、X2、X3、X4)，这4个特征描述子所描述视觉特征点在所述当前时刻采集帧图像数据中对应的二维图像坐标(例如可记为
Figure PCTCN2017096945-appb-000007
Figure PCTCN2017096945-appb-000008
以及移动装置内置的图像采集单元的内参数矩阵K建立表示移动装置的位姿的4×4的转移矩阵T。
S1034：基于所述转移矩阵得到移动装置在所述当前时刻的位姿。具体的，求解得到在当前时刻的位姿包括移动装置在当前时刻的姿态R和位置t。
作为一种实施方式，本发明实施例确定出的移动装置在当前时刻的位姿用于修正基于惯性数据和视觉信息估计的运动轨迹。
作为一种实施方式：在移动装置采集视觉特征点的移动过程中，采集移动装置在移动过程中的惯性数据和视觉信息，基于惯性数据和视觉信息估计移动装置在移动过程中的运动轨迹。具体的，通过移动装置上搭载的IMU(Inertial Measurement Unit，惯性测量单元)采集移动装置在移动过程中的惯性数据。而IMU包括加速计和陀螺仪，加速计与陀螺仪对应测量移动装置自身移动过程中的加速度和角速度后，推算出移动装置在每个时刻的位置和姿态，移动装置上搭载的图像采集单元进行采集移动装置在移动过程中的视觉信息，使用视觉信息对推算出的移动装置的位置和姿态进行进一步估计，以得到移动装置在移动过程中的运动轨迹。进一步基于S103确定出的移动装置在当前时刻的位姿代替基于惯性数据和视觉信息估计的对应时刻的位姿，以修正基于惯性数据和视觉信息估计的运动轨迹。具体的，将求解S1033建立的转移矩阵得到的移动装置在当前时刻的姿态R和 位置t代替基于惯性数据和视觉信息估计的对应时刻的姿态R和位置t，以达到修正基于惯性数据和视觉信息估计的运动轨迹的效果。
本发明实施例还提供了一种移动装置，参考图3所示，包括：
提取单元201，配置为在采集视觉特征点的移动过程中，提取当前时刻所采集视觉特征点的第一组特征描述子；
检测单元202，配置为将所述提取单元201采集的所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行闭环检测；
确定单元203，配置为在所述检测单元202基于第一组特征描述子与第二组特征描述子检测到闭环时，通过第二组特征描述子所描述的视觉特征点的空间坐标，确定出移动装置在当前时刻的位姿，其中，第二组特征描述子为在先提取的多组特征描述子中的其中一组。
作为一种实施方式，所述检测单元202，包括：
对比子单元，配置为将第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行相似对比，分别确定出在先提取的多组特征描述子中每组特征描述子与与所述第一组特征描述子满足预设相似条件的描述子数量；
判断子单元，配置为分别判断所述对比子单元确定的在先提取的多组特征描述子中每组特征描述子与所述第一组特征描述子满足预设相似条件的描述子数量是否大于预设数量阈值；当满足预设相似条件的描述子数量大于预设数量阈值时，表征所述第一组特征描述子与对应的组特征描述子检测到闭环。
其中，对比子单元，配置为将第一组特征描述子中的每个特征描述子分别与在先提取的多组特征描述子中的每组特征描述子中的每个特征描述子进行一一对比；判断进行对比的特征描述子之间的向量夹角是否小于预设角度阈值，其中，在向量夹角小于预设角度阈值时表征进行对比的特征 描述子满足预设相似条件。
作为一种实施方式，所述确定单元203，包括：
第一确定子单元，配置为确定出第二组特征描述子中的多个特征描述子；
第二确定子单元，配置为确定所述第一确定子单元确定的所述多个特征描述子在所述当前时刻采集的帧图像数据中对应的二维图像坐标；
矩阵建立子单元，配置为基于多个特征描述子所描述视觉特征点的空间坐标、二维图像坐标、以及移动装置的内置图像采集单元的内参数矩阵建立表示移动装置的位姿的转移矩阵；
求解子单元，配置为求解转移矩阵得到移动装置在当前时刻的位姿；
其中，所述转移矩阵满足：
Figure PCTCN2017096945-appb-000009
其中，T为转移矩阵，Xi为多个特征描述子所描述视觉特征点的空间坐标，
Figure PCTCN2017096945-appb-000010
为多个特征描述子在所述当前时刻采集的帧图像数据中对应的二维图像坐标，K为移动装置的内置图像采集单元的内参数矩阵，R为移动装置的姿态，t为移动装置的位置。
作为一种实施方式，该移动装置还包括：
采集单元，配置为采集移动过程中的惯性数据和视觉信息；
轨迹估计单元，配置为基于所述采集单元采集的惯性数据和视觉信息估计在移动过程中的运动轨迹。
作为一种实施方式，该移动装置还包括：
修正单元，配置为基于所述确定单元确定出的移动装置在当前时刻的位姿代替基于惯性数据和视觉信息估计的对应时刻的位姿，以修正运动轨迹。
作为一种实施方式，所述在先提取的多组特征描述子中的每组特征描 述子包括：与所述当前时刻相比的在先时刻采集的关键帧图像数据时从所述关键帧图像数据中提取一组特征描述子，其中，所述关键帧图像数据为按照预设空间间隔依次从采集的全部帧图像数据中确定出。
本实施例中，移动装置的提取单元201、检测单元202、确定单元203、采集单元、轨迹估计单元和修正单元，以及检测单元202所包括的对比子单元和判断子单元，确定单元203所包括的第一确定子单元、第二确定子单元、矩阵建立子单元和求解子单元，在实际应用中，可由CPU(Central Processing Unit，中央处理器)、DSP(Digital Signal Processor，数字信号处理器)或FPGA(Field－Programmable Gate Array，可编程门阵列)实现。
另一方面，本发明实施例还提供了一种移动装置，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现本发明实施例所述的移动装置的定位方法的步骤，具体的，所述处理器执行所述程序时实现：在所述移动装置采集视觉特征点的移动过程中，提取当前时刻所采集视觉特征点的第一组特征描述子；将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行闭环检测；在基于所述第一组特征描述子与第二组特征描述子检测到闭环时，通过所述第二组特征描述子所描述的视觉特征点的空间坐标，确定出所述移动装置在所述当前时刻的位姿，其中，所述第二组特征描述子为所述在先提取的多组特征描述子中的其中一组。
作为一种实施方式，所述处理器执行所述程序时实现：将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行相似对比，分别确定出在先提取的多组特征描述子中每组特征描述子与所述第一组特征描述子满足预设相似条件的描述子数量；分别判断在先提取的多组特征描述子中每组特征描述子与所述第一组特征描述子满足预设相似条件的描述子数量是否大于预设数量阈值；当满足所述预设相似条件的描 述子数量大于所述预设数量阈值时，表征所述第一组特征描述子与对应的组特征描述子检测到闭环。
其中，所述处理器执行所述程序时实现：将所述第一组特征描述子中的每个特征描述子分别与在先提取的多组特征描述子中的每组特征描述子中的每个特征描述子进行一一对比；判断进行对比的特征描述子之间的向量夹角是否小于预设角度阈值，其中，在所述向量夹角小于所述预设角度阈值时表征进行对比的特征描述子满足所述预设相似条件。
作为一种实施方式，所述处理器执行所述程序时实现：确定出所述第二组特征描述子中的多个特征描述子；确定所述多个特征描述子在所述当前时刻采集的帧图像数据中对应的二维图像坐标；基于所述多个特征描述子所描述视觉特征点的空间坐标、所述二维图像坐标、以及所述移动装置内置的图像采集单元的内参数矩阵建立表示所述移动装置的位姿的转移矩阵；基于所述转移矩阵得到所述移动装置在所述当前时刻的位姿；
其中，所述转移矩阵满足：
Figure PCTCN2017096945-appb-000011
其中，T为所述转移矩阵，Xi为所述多个特征描述子所描述视觉特征点的空间坐标，
Figure PCTCN2017096945-appb-000012
为所述多个特征描述子在所述当前时刻采集的帧图像数据中对应的二维图像坐标，K为所述移动装置内置的图像采集单元的内参数矩阵，R为所述移动装置的姿态，t为所述移动装置的位置。
作为一种实施方式，所述处理器执行所述程序时实现：在移动装置采集视觉特征点的移动过程中，采集所述移动装置在所述移动过程中的惯性数据和视觉信息；基于所述惯性数据和所述视觉信息估计所述移动装置在所述移动过程中的运动轨迹。
作为一种实施方式，所述处理器执行所述程序时实现：在所述通过所述第二组特征描述子所描述的视觉特征点的空间坐标，确定出所述移动装 置在所述当前时刻的位姿之后，基于确定出的所述移动装置在所述当前时刻的位姿代替基于所述惯性数据和所述视觉信息估计的对应时刻的位姿，以修正所述运动轨迹。
本发明实施例中，通过移动装置在移动过程中采集视觉特征点，提取当前时刻所采集视觉特征点的第一组特征描述子；将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行闭环检测，从而确定移动装置是否到达已经到达过的同一区域；在基于第一组特征描述子与在先提取的第二组特征描述子检测到闭环时，通过第二组特征描述子描述的视觉特征点的空间坐标，确定出移动装置在当前时刻的位姿，从而能够在移动装置再一次经过同一区域时根据先前时刻记录的视觉特征点的空间坐标重新计算移动装置当前的位姿，以修正移动装置在闭环位置的位姿，从而消除了对位姿估计的误差累计，以解决了移动装置在移动过程中积累下对自身位姿估计的误差，而严重影响定位精度的技术问题，以在不建立环境地图情况下有效提高了基于移动装置定位的精度，从而实现了在不建立环境地图情况下的准确定位，以同时确保基于移动装置定位的实时性和定位精度。
本发明实施例还提供了一种计算机存储介质，所述计算机存储介质中存储有计算机可执行指令，所述计算机可执行指令用于执行本发明实施例所述的移动装置的定位方法。具体的，所述计算机可执行指令被处理器执行时实现：在所述移动装置采集视觉特征点的移动过程中，提取当前时刻所采集视觉特征点的第一组特征描述子；将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行闭环检测；在基于所述第一组特征描述子与第二组特征描述子检测到闭环时，通过所述第二组特征描述子所描述的视觉特征点的空间坐标，确定出所述移动装置在所述当前时刻的位姿，其中，所述第二组特征描述子为所述在先提取的多组特 征描述子中的其中一组。
作为一种实施方式，所述计算机可执行指令被处理器执行时实现：将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行相似对比，分别确定出在先提取的多组特征描述子中每组特征描述子与所述第一组特征描述子满足预设相似条件的描述子数量；分别判断在先提取的多组特征描述子中每组特征描述子与所述第一组特征描述子满足预设相似条件的描述子数量是否大于预设数量阈值；当满足所述预设相似条件的描述子数量大于所述预设数量阈值时，表征所述第一组特征描述子与对应的组特征描述子检测到闭环。
其中，所述计算机可执行指令被处理器执行时实现：将所述第一组特征描述子中的每个特征描述子分别与在先提取的多组特征描述子中的每组特征描述子中的每个特征描述子进行一一对比；判断进行对比的特征描述子之间的向量夹角是否小于预设角度阈值，其中，在所述向量夹角小于所述预设角度阈值时表征进行对比的特征描述子满足所述预设相似条件。
作为一种实施方式，所述计算机可执行指令被处理器执行时实现：确定出所述第二组特征描述子中的多个特征描述子；确定所述多个特征描述子在所述当前时刻采集的帧图像数据中对应的二维图像坐标；基于所述多个特征描述子所描述视觉特征点的空间坐标、所述二维图像坐标、以及所述移动装置内置的图像采集单元的内参数矩阵建立表示所述移动装置的位姿的转移矩阵；基于所述转移矩阵得到所述移动装置在所述当前时刻的位姿。
作为一种实施方式，所述计算机可执行指令被处理器执行时实现：在移动装置采集视觉特征点的移动过程中，采集所述移动装置在所述移动过程中的惯性数据和视觉信息；基于所述惯性数据和所述视觉信息估计所述移动装置在所述移动过程中的运动轨迹。
作为一种实施方式，所述计算机可执行指令被处理器执行时实现：在所述通过所述第二组特征描述子所描述的视觉特征点的空间坐标，确定出所述移动装置在所述当前时刻的位姿之后，基于确定出的所述移动装置在所述当前时刻的位姿代替基于所述惯性数据和所述视觉信息估计的对应时刻的位姿，以修正所述运动轨迹。
在此处所提供的说明书中，说明了大量具体细节。然而，能够理解，本发明的实施例可以在没有这些具体细节的情况下实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。
类似地，应当理解，为了精简本公开并帮助理解各个发明方面中的一个或多个，在上面对本发明的示例性实施例的描述中，本发明的各个特征有时被一起分组到单个实施例、图、或者对其的描述中。然而，并不应将该公开的方法解释成反映如下意图：即所要求保护的本发明要求比在每个权利要求中所明确记载的特征更多的特征。更确切地说，如下面的权利要求书所反映的那样，发明方面在于少于前面公开的单个实施例的所有特征。因此，遵循具体实施方式的权利要求书由此明确地并入该具体实施方式，其中每个权利要求本身都作为本发明的单独实施例。
本领域那些技术人员可以理解，可以对实施例中的设备中的模块进行自适应性地改变并且把它们设置在与该实施例不同的一个或多个设备中。可以把实施例中的模块或单元或组件组合成一个模块或单元或组件，以及此外可以把它们分成多个子模块或子单元或子组件。除了这样的特征和/或过程或者单元中的至少一些是相互排斥之外，可以采用任何组合对本说明书(包括伴随的权利要求、摘要和附图)中公开的所有特征以及如此公开的任何方法或者设备的所有过程或单元进行组合。除非另外明确陈述，本说明书(包括伴随的权利要求、摘要和附图)中公开的每个特征可以由提供相同、等同或相似目的的替代特征来代替。
此外，本领域的技术人员能够理解，尽管在此的一些实施例包括其它实施例中所包括的某些特征而不是其它特征，但是不同实施例的特征的组合意味着处于本发明的范围之内并且形成不同的实施例。例如，在权利要求书中，所要求保护的实施例的任意之一都可以以任意的组合方式来使用。
本发明的各个部件实施例可以以硬件实现，或者以在一个或者多个处理器上运行的客户端模块实现，或者以它们的组合实现。本领域的技术人员应当理解，可以在实践中使用微处理器或者数字信号处理器(DSP)来实现根据本发明实施例的软件安装包的加固保护装置中的一些或者全部部件的一些或者全部功能。本发明还可以实现为用于执行这里所描述的方法的一部分或者全部的设备或者装置程序(例如，计算机程序和计算机程序产品)。这样的实现本发明的程序可以存储在计算机可读介质上，或者可以具有一个或者多个信号的形式。这样的信号可以从因特网网站上下载得到，或者在载体信号上提供，或者以任何其他形式提供。
应该注意的是上述实施例对本发明进行说明而不是对本发明进行限制，并且本领域技术人员在不脱离所附权利要求的范围的情况下可设计出替换实施例。在权利要求中，不应将位于括号之间的任何参考符号构造成对权利要求的限制。单词“包括”不排除存在未列在权利要求中的元件或步骤。在列举了若干装置的单元权利要求中，这些装置中的若干个可以是通过同一个硬件项来具体体现。单词第一、第二、以及第三等的使用不表示任何顺序。可将这些单词解释为名称。
尽管已描述了本发明的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本发明范围的所有变更和修改。
显然，本领域的技术人员可以对本发明进行各种改动和变型而不脱离 本发明的精神和范围。这样，倘若本发明的这些修改和变型属于本发明权利要求及其等同技术的范围之内，则本发明也意图包含这些改动和变型在内。
工业实用性
本发明实施例的技术方案通过移动装置在移动过程中采集视觉特征点，提取当前时刻所采集视觉特征点的第一组特征描述子；将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行闭环检测，从而确定移动装置是否到达已经到达过的同一区域；在基于第一组特征描述子与在先提取的第二组特征描述子检测到闭环时，通过第二组特征描述子描述的视觉特征点的空间坐标，确定出移动装置在当前时刻的位姿，从而能够在移动装置再一次经过同一区域时根据先前时刻记录的视觉特征点的空间坐标重新计算移动装置当前的位姿，以修正移动装置在闭环位置的位姿，从而消除了对位姿估计的误差累计，以解决了移动装置在移动过程中积累下对自身位姿估计的误差，而严重影响定位精度的技术问题，以在不建立环境地图情况下有效提高了基于移动装置定位的精度，从而实现了在不建立环境地图情况下的准确定位，以同时确保基于移动装置定位的实时性和定位精度。

Claims (15)

  1. 一种移动装置的定位方法，包括：
    在所述移动装置采集视觉特征点的移动过程中，提取当前时刻所采集视觉特征点的第一组特征描述子；
    将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行闭环检测；
    在基于所述第一组特征描述子与第二组特征描述子检测到闭环时，通过所述第二组特征描述子所描述的视觉特征点的空间坐标，确定出所述移动装置在所述当前时刻的位姿，其中，所述第二组特征描述子为所述在先提取的多组特征描述子中的其中一组。
  2. 如权利要求1所述的移动装置的定位方法，其中，所述将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行闭环检测，包括：
    将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行相似对比，分别确定出在先提取的多组特征描述子中每组特征描述子与所述第一组特征描述子满足预设相似条件的描述子数量；
    分别判断在先提取的多组特征描述子中每组特征描述子与所述第一组特征描述子满足预设相似条件的描述子数量是否大于预设数量阈值；当满足所述预设相似条件的描述子数量大于所述预设数量阈值时，表征所述第一组特征描述子与对应的组特征描述子检测到闭环。
  3. 如权利要求2所述的移动装置的定位方法，其中，将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行相似对比，判断是否满足预设相似条件，包括：
    将所述第一组特征描述子中的每个特征描述子分别与在先提取的多组特征描述子中的每组特征描述子中的每个特征描述子进行一一对比；
    判断进行对比的特征描述子之间的向量夹角是否小于预设角度阈值，其中，在所述向量夹角小于所述预设角度阈值时表征进行对比的特征描述子满足所述预设相似条件。
  4. 如权利要求1所述的移动装置的定位方法，其中，所述通过所述第二组特征描述子所描述的视觉特征点的空间坐标，确定出所述移动装置在所述当前时刻的位姿，包括：
    确定出所述第二组特征描述子中的多个特征描述子；
    确定所述多个特征描述子在所述当前时刻采集的帧图像数据中对应的二维图像坐标；
    基于所述多个特征描述子所描述视觉特征点的空间坐标、所述二维图像坐标、以及所述移动装置内置的图像采集单元的内参数矩阵建立表示所述移动装置的位姿的转移矩阵；
    基于所述转移矩阵得到所述移动装置在所述当前时刻的位姿；
    其中，所述转移矩阵满足：
    Figure PCTCN2017096945-appb-100001
    其中，T为所述转移矩阵，Xi为所述多个特征描述子所描述视觉特征点的空间坐标，
    Figure PCTCN2017096945-appb-100002
    为所述多个特征描述子在所述当前时刻采集的帧图像数据中对应的二维图像坐标，K为所述移动装置内置的图像采集单元的内参数矩阵，R为所述移动装置的姿态，t为所述移动装置的位置。
  5. 如权利要求1所述的移动装置的定位方法，其中，所述在移动装置采集视觉特征点的移动过程中，所述方法还包括：
    采集所述移动装置在所述移动过程中的惯性数据和视觉信息；
    基于所述惯性数据和所述视觉信息估计所述移动装置在所述移动过程中的运动轨迹。
  6. 如权利要求5所述的移动装置的定位方法，其中，在所述通过所述 第二组特征描述子所描述的视觉特征点的空间坐标，确定出所述移动装置在所述当前时刻的位姿之后，所述方法还包括：
    基于确定出的所述移动装置在所述当前时刻的位姿代替基于所述惯性数据和所述视觉信息估计的对应时刻的位姿，以修正所述运动轨迹。
  7. 如权利要求1所述的移动装置的定位方法，其中，所述在先提取的多组特征描述子中的每组特征描述子包括：与所述当前时刻相比的在先时刻采集的关键帧图像数据时从所述关键帧图像数据中提取一组特征描述子，其中，所述关键帧图像数据为按照预设空间间隔依次从采集的全部帧图像数据中确定出。
  8. 一种移动装置，包括：
    提取单元，配置为在采集视觉特征点的移动过程中，提取当前时刻所采集视觉特征点的第一组特征描述子；
    检测单元，配置为将所述提取单元采集的所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行闭环检测；
    确定单元，配置为在所述检测单元基于所述第一组特征描述子与第二组特征描述子检测到闭环时，通过所述第二组特征描述子所描述的视觉特征点的空间坐标，确定出所述移动装置在所述当前时刻的位姿，其中，所述第二组特征描述子为所述在先提取的多组特征描述子中的其中一组。
  9. 如权利要求8所述的移动装置，其中，所述检测单元，包括：
    对比子单元，配置为将所述第一组特征描述子分别与在先提取的多组特征描述子中的每组特征描述子进行相似对比，分别确定出在先提取的多组特征描述子中每组特征描述子与所述第一组特征描述子满足预设相似条件的描述子数量；
    判断子单元，配置为分别判断所述对比子单元确定的在先提取的多组特征描述子中每组特征描述子与所述第一组特征描述子满足预设相似条件 的描述子数量是否大于预设数量阈值；当满足所述预设相似条件的描述子数量大于所述预设数量阈值时，表征所述第一组特征描述子与对应的组特征描述子检测到闭环。
  10. 如权利要求9所述的移动装置，其中，所述对比子单元，配置为将所述第一组特征描述子中的每个特征描述子分别与在先提取的多组特征描述子中的每组特征描述子中的每个特征描述子进行一一对比；判断进行对比的特征描述子之间的向量夹角是否小于预设角度阈值，其中，在所述向量夹角小于所述预设角度阈值时表征进行对比的特征描述子满足所述预设相似条件。
  11. 如权利要求8所述的移动装置，其中，所述确定单元，包括：
    第一确定子单元，配置为确定出所述第二组特征描述子中的多个特征描述子；
    第二确定子单元，配置为确定所述第一确定子单元确定的所述多个特征描述子在所述当前时刻采集的帧图像数据中对应的二维图像坐标；
    矩阵建立子单元，配置为基于所述多个特征描述子所描述视觉特征点的空间坐标、所述二维图像坐标、以及所述移动装置的内置图像采集单元的内参数矩阵建立表示所述移动装置的位姿的转移矩阵；
    求解子单元，配置为基于所述转移矩阵得到所述移动装置在所述当前时刻的位姿；
    其中，所述转移矩阵满足：
    Figure PCTCN2017096945-appb-100003
    其中，T为所述转移矩阵，Xi为所述多个特征描述子所描述视觉特征点的空间坐标，
    Figure PCTCN2017096945-appb-100004
    为所述多个特征描述子在所述当前时刻采集的帧图像数据中对应的二维图像坐标，K为所述移动装置的内置图像采集单元的内参数矩阵，R为所述移动装置的姿态，t为所述移动装置的位置。
  12. 如权利要求8所述移动装置，其中，所述移动装置还包括：
    采集单元，配置为采集移动过程中的惯性数据和视觉信息；
    轨迹估计单元，配置为基于所述采集单元采集的惯性数据和所述视觉信息估计在移动过程中的运动轨迹。
  13. 如权利要求12所述的移动装置，其中，所述移动装置还包括：
    修正单元，配置为基于所述确定单元确定出的所述移动装置在所述当前时刻的位姿代替基于所述惯性数据和所述视觉信息估计的对应时刻的位姿，以修正所述运动轨迹。
  14. 如权利要求8所述的移动装置，其中，所述在先提取的多组特征描述子中的每组特征描述子包括：与所述当前时刻相比的在先时刻采集的关键帧图像数据时从所述关键帧图像数据中提取一组特征描述子，其中，所述关键帧图像数据为按照预设空间间隔依次从采集的全部帧图像数据中确定出。
  15. 一种计算机存储介质，所述计算机存储介质中存储有计算机可执行指令，所述计算机可执行指令用于执行权利要求1至7任一项所述的移动装置的定位方法。
PCT/CN2017/096945 2016-08-10 2017-08-10 一种移动装置及其定位方法、计算机存储介质 WO2018028649A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610652818.0A CN106352877B (zh) 2016-08-10 2016-08-10 一种移动装置及其定位方法
CN201610652818.0 2016-08-10

Publications (1)

Publication Number Publication Date
WO2018028649A1 true WO2018028649A1 (zh) 2018-02-15

Family

ID=57843765

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/096945 WO2018028649A1 (zh) 2016-08-10 2017-08-10 一种移动装置及其定位方法、计算机存储介质

Country Status (2)

Country Link
CN (1) CN106352877B (zh)
WO (1) WO2018028649A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110293563A (zh) * 2019-06-28 2019-10-01 炬星科技（深圳）有限公司 估计机器人位姿的方法、设备及存储介质
CN110334560A (zh) * 2019-07-16 2019-10-15 济南浪潮高新科技投资发展有限公司 一种二维码定位方法和装置
CN111105459A (zh) * 2019-12-24 2020-05-05 广州视源电子科技股份有限公司 描述子地图生成方法、定位方法、装置、设备和存储介质
CN111383282A (zh) * 2018-12-29 2020-07-07 杭州海康威视数字技术股份有限公司 位姿信息确定方法及装置
CN112284399A (zh) * 2019-07-26 2021-01-29 北京初速度科技有限公司 一种基于视觉和imu的车辆定位方法及车载终端
US20210055109A1 (en) * 2018-06-01 2021-02-25 Beijing Didi Infinity Technology And Development Co., Ltd. Systems and methods for indoor positioning
CN112634360A (zh) * 2019-10-08 2021-04-09 北京京东乾石科技有限公司 一种视觉信息确定方法、装置、设备和存储介质
CN114415698A (zh) * 2022-03-31 2022-04-29 深圳市普渡科技有限公司 机器人、机器人的定位方法、装置和计算机设备

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106352877B (zh) * 2016-08-10 2019-08-23 纳恩博（北京）科技有限公司 一种移动装置及其定位方法
KR20180087994A (ko) * 2017-01-26 2018-08-03 삼성전자주식회사 스테레오 매칭 방법 및 영상 처리 장치
US10436590B2 (en) 2017-11-10 2019-10-08 Ankobot (Shanghai) Smart Technologies Co., Ltd. Localization system and method, and robot using the same
CN107907131B (zh) * 2017-11-10 2019-12-13 珊口（上海）智能科技有限公司 定位系统、方法及所适用的机器人
CN108717710B (zh) * 2018-05-18 2022-04-22 京东方科技集团股份有限公司 室内环境下的定位方法、装置及系统
CN110617821B (zh) * 2018-06-19 2021-11-02 北京嘀嘀无限科技发展有限公司 定位方法、装置及存储介质
CN110207537A (zh) * 2019-06-19 2019-09-06 赵天昊 基于计算机视觉技术的火控装置及其自动瞄准方法
WO2020258187A1 (zh) * 2019-06-27 2020-12-30 深圳市大疆创新科技有限公司 一种状态检测方法、装置及可移动平台
CN110275540A (zh) * 2019-07-01 2019-09-24 湖南海森格诺信息技术有限公司 用于扫地机器人的语义导航方法及其系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1569558A (zh) * 2003-07-22 2005-01-26 中国科学院自动化研究所 基于图像表现特征的移动机器人视觉导航方法
US20080195316A1 (en) * 2007-02-12 2008-08-14 Honeywell International Inc. System and method for motion estimation using vision sensors
CN102109348A (zh) * 2009-12-25 2011-06-29 财团法人工业技术研究院 定位载体、估测载体姿态与建地图的系统与方法
WO2014130854A1 (en) * 2013-02-21 2014-08-28 Regents Of The Univesity Of Minnesota Extrinsic parameter calibration of a vision-aided inertial navigation system
CN106352877A (zh) * 2016-08-10 2017-01-25 纳恩博（北京）科技有限公司 一种移动装置及其定位方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8913055B2 (en) * 2011-05-31 2014-12-16 Honda Motor Co., Ltd. Online environment mapping
CN103869814B (zh) * 2012-12-17 2017-04-19 联想(北京)有限公司 一种终端定位和导航方法以及可移动的终端
US9519286B2 (en) * 2013-03-19 2016-12-13 Robotic Research, Llc Delayed telop aid
CN104374395A (zh) * 2014-03-31 2015-02-25 南京邮电大学 基于图的视觉slam方法
CN104851094A (zh) * 2015-05-14 2015-08-19 西安电子科技大学 一种基于rgb-d的slam算法的改进方法
CN105783913A (zh) * 2016-03-08 2016-07-20 中山大学 一种融合车载多传感器的slam装置及其控制方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1569558A (zh) * 2003-07-22 2005-01-26 中国科学院自动化研究所 基于图像表现特征的移动机器人视觉导航方法
US20080195316A1 (en) * 2007-02-12 2008-08-14 Honeywell International Inc. System and method for motion estimation using vision sensors
CN102109348A (zh) * 2009-12-25 2011-06-29 财团法人工业技术研究院 定位载体、估测载体姿态与建地图的系统与方法
WO2014130854A1 (en) * 2013-02-21 2014-08-28 Regents Of The Univesity Of Minnesota Extrinsic parameter calibration of a vision-aided inertial navigation system
CN106352877A (zh) * 2016-08-10 2017-01-25 纳恩博（北京）科技有限公司 一种移动装置及其定位方法

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11965744B2 (en) * 2018-06-01 2024-04-23 Beijing Didi Infinity Technology And Development Co., Ltd. Systems and methods for indoor positioning
US20210055109A1 (en) * 2018-06-01 2021-02-25 Beijing Didi Infinity Technology And Development Co., Ltd. Systems and methods for indoor positioning
CN111383282A (zh) * 2018-12-29 2020-07-07 杭州海康威视数字技术股份有限公司 位姿信息确定方法及装置
CN111383282B (zh) * 2018-12-29 2023-12-01 杭州海康威视数字技术股份有限公司 位姿信息确定方法及装置
CN110293563B (zh) * 2019-06-28 2022-07-26 炬星科技（深圳）有限公司 估计机器人位姿的方法、设备及存储介质
CN110293563A (zh) * 2019-06-28 2019-10-01 炬星科技（深圳）有限公司 估计机器人位姿的方法、设备及存储介质
CN110334560A (zh) * 2019-07-16 2019-10-15 济南浪潮高新科技投资发展有限公司 一种二维码定位方法和装置
CN110334560B (zh) * 2019-07-16 2023-04-07 山东浪潮科学研究院有限公司 一种二维码定位方法和装置
CN112284399A (zh) * 2019-07-26 2021-01-29 北京初速度科技有限公司 一种基于视觉和imu的车辆定位方法及车载终端
CN112284399B (zh) * 2019-07-26 2022-12-13 北京魔门塔科技有限公司 一种基于视觉和imu的车辆定位方法及车载终端
CN112634360A (zh) * 2019-10-08 2021-04-09 北京京东乾石科技有限公司 一种视觉信息确定方法、装置、设备和存储介质
CN112634360B (zh) * 2019-10-08 2024-03-05 北京京东乾石科技有限公司 一种视觉信息确定方法、装置、设备和存储介质
CN111105459B (zh) * 2019-12-24 2023-10-20 广州视源电子科技股份有限公司 描述子地图生成方法、定位方法、装置、设备和存储介质
CN111105459A (zh) * 2019-12-24 2020-05-05 广州视源电子科技股份有限公司 描述子地图生成方法、定位方法、装置、设备和存储介质
CN114415698B (zh) * 2022-03-31 2022-11-29 深圳市普渡科技有限公司 机器人、机器人的定位方法、装置和计算机设备
CN114415698A (zh) * 2022-03-31 2022-04-29 深圳市普渡科技有限公司 机器人、机器人的定位方法、装置和计算机设备

Also Published As

Publication number Publication date
CN106352877A (zh) 2017-01-25
CN106352877B (zh) 2019-08-23

Similar Documents

Publication Publication Date Title
WO2018028649A1 (zh) 一种移动装置及其定位方法、计算机存储介质
US20210190497A1 (en) Simultaneous location and mapping (slam) using dual event cameras
US10296102B1 (en) Gesture and motion recognition using skeleton tracking
CN107833236B (zh) 一种动态环境下结合语义的视觉定位系统和方法
CN109059895B (zh) 一种基于手机摄像头和传感器的多模态室内测距及定位方法
CN108955718B (zh) 一种视觉里程计及其定位方法、机器人以及存储介质
US11199414B2 (en) Method for simultaneous localization and mapping
CN108632761B (zh) 一种基于粒子滤波算法的室内定位方法
Paul et al. An effective approach for human activity recognition on smartphone
JP6815707B2 (ja) 顔姿勢検出方法、装置及び記憶媒体
WO2018068771A1 (zh) 目标跟踪方法、系统、电子设备和计算机存储介质
JP2018522348A (ja) センサーの３次元姿勢を推定する方法及びシステム
WO2019129255A1 (zh) 一种目标跟踪方法及装置
JP2017520063A5 (zh)
TWI671740B (zh) 基於地磁訊號結合電腦視覺的室內定位系統及方法
TWI640931B (zh) 影像目標追蹤方法及裝置
TWI438702B (zh) 影像環境邊界之動態設定方法及人員活動內容之即時判定方法
WO2019196476A1 (zh) 基于激光传感器生成地图
CN111738225B (zh) 人群聚集检测方法、装置、设备及存储介质
CN111595344B (zh) 一种基于地图信息辅助的多姿态下行人行位推算方法
CN108106613B (zh) 基于视觉辅助的定位方法与系统
WO2018207426A1 (ja) 情報処理装置、情報処理方法、及びプログラム
US20170085656A1 (en) Automatic absolute orientation and position
CN109084749B (zh) 通过环境中物体进行语义定位的方法及装置
JP6922348B2 (ja) 情報処理装置、方法、及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17838779

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17838779

Country of ref document: EP

Kind code of ref document: A1