WO2022147655A1 - Positioning method and apparatus, spatial information acquisition method and apparatus, and photographing device - Google Patents

Positioning method and apparatus, spatial information acquisition method and apparatus, and photographing device Download PDF

Info

Publication number
WO2022147655A1
WO2022147655A1 PCT/CN2021/070333 CN2021070333W WO2022147655A1 WO 2022147655 A1 WO2022147655 A1 WO 2022147655A1 CN 2021070333 W CN2021070333 W CN 2021070333W WO 2022147655 A1 WO2022147655 A1 WO 2022147655A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
point
dimensional
points
photographing device
Prior art date
Application number
PCT/CN2021/070333
Other languages
French (fr)
Chinese (zh)
Inventor
赵峰
叶长春
蔡剑钊
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2021/070333 priority Critical patent/WO2022147655A1/en
Publication of WO2022147655A1 publication Critical patent/WO2022147655A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods

Definitions

  • the present application relates to the field of visual positioning, and in particular, to a positioning method, a positioning device, a spatial information acquisition method, a spatial information acquisition device, a photographing device, and a movable platform.
  • the movable platform In the absence of an external positioning system such as GPS (Global Positioning System, Global Positioning System) assisted positioning, the movable platform usually needs to be positioned according to its own related sensors, but such positioning methods exist. The problem of low precision.
  • GPS Global Positioning System, Global Positioning System
  • an embodiment of the present application provides a positioning method, including:
  • the spatial position information of the 3D point and the estimated pose reproject the 3D point to the image to obtain a projection point, and calculate the pixel position error between the projection point and the feature point;
  • the estimated pose is output as the target pose of the photographing device.
  • an embodiment of the present application provides a method for obtaining spatial information, including:
  • the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the current image, and the spatial position information of the three-dimensional points corresponding to the feature points in the current image is obtained.
  • an embodiment of the present application provides a positioning device, including:
  • memory for storing processor-executable instructions
  • processor is configured to:
  • the spatial position information of the 3D point and the estimated pose reproject the 3D point to the image to obtain a projection point, and calculate the pixel position error between the projection point and the feature point;
  • the estimated pose is output as the target pose of the photographing device.
  • an embodiment of the present application provides a device for acquiring spatial information, including:
  • memory for storing processor-executable instructions
  • processor is configured to:
  • the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the current image, and the spatial position information of the three-dimensional points corresponding to the feature points in the current image is obtained.
  • an embodiment of the present application provides a photographing device, including:
  • memory for storing processor-executable instructions
  • processor is configured to:
  • the spatial position information of the 3D point and the estimated pose reproject the 3D point to the image to obtain a projection point, and calculate the pixel position error between the projection point and the feature point;
  • the estimated pose is output as the target pose of the photographing device.
  • an embodiment of the present application provides a photographing device, including:
  • memory for storing processor-executable instructions
  • processor is configured to:
  • the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the current image, and the spatial position information of the three-dimensional points corresponding to the feature points in the current image is obtained.
  • an embodiment of the present application provides a movable platform, including the photographing device described in any of the foregoing embodiments.
  • the positioning method provided by the embodiment of the present application can correspondingly adjust the reprojection error threshold according to the change of the real-time scene, so that the reprojection error threshold used when the scene changes can ensure a small positioning error, and the position and attitude of the shooting device can be adjusted accordingly. It is output only when the corresponding reprojection error threshold in the current scene is met, to ensure that the output pose is still accurate when the scene changes.
  • FIG. 1 is a flowchart of a positioning method according to an exemplary embodiment of the present application.
  • FIG. 2 is a specific flow chart of positioning according to an exemplary embodiment of the present application.
  • FIG. 3 is a flowchart of a method for acquiring spatial information according to an exemplary embodiment of the present application.
  • FIG. 4 is a schematic diagram of a positioning device according to an exemplary embodiment of the present application.
  • FIG. 5 is a schematic diagram of an apparatus for acquiring spatial information according to an exemplary embodiment of the present application.
  • FIG. 6 is a schematic diagram of a photographing device according to an exemplary embodiment of the present application.
  • FIG. 7 is a schematic diagram of a movable platform according to an exemplary embodiment of the present application.
  • the mobile platform can be a drone, an unmanned ship, an unmanned vehicle, a robot and other equipment.
  • the mobile platform performs automated navigation or operations, such as unmanned
  • positioning is usually required to determine its own posture.
  • the autonomous return of the drone as an example, it continuously corrects its own posture and posture through positioning technology, so as to accurately return to the starting point.
  • the UAV continuously collects images of the surrounding environment to build a three-dimensional map, and records the corresponding feature points of the three-dimensional points in the physical space in the collected images.
  • the current image search in the historically captured image (called the historical image), match the similar scene between the historical image and the current image, and then extract the features of the current image point, and match the feature points with the historical image to obtain the matching pair of feature points.
  • the feature point A in the historical image and the feature point B in the current image are a matching pair
  • the spatial position information of the three-dimensional point corresponding to the feature point B can be obtained, and the pose of the camera mounted on the UAV can be calculated through the relevant algorithm, such as the pnp (perspective-n-point, multi-point perspective imaging) algorithm.
  • a certain re-projection error threshold can be set while the pose is estimated, and the estimated pose can be adjusted based on the re-projection error threshold, so that the re-projection error corresponding to the adjusted pose is smaller than the re-projection error threshold range, thereby Get the pose of the relocated shooting device, and then locate the drone.
  • the feature point can be the point on the image where the gray value changes drastically or the point with large curvature on the edge of the image
  • the reprojection error refers to reprojecting the three-dimensional point in space to the current image according to the calculated pose
  • the error between the pixel position and the corresponding feature point can be detected by setting the reprojection error threshold value when the position error is within the range of the specified threshold value, so as to determine whether the calculated pose is accurate enough .
  • the distance between the shooting device and the actual scene will affect the accuracy of relocation. Specifically, when the distance between the scene and the shooting device continues to increase, the actual physical distance represented by the same reprojection error continues to increase. This will result in that when the scene is far away, even if the set reprojection error threshold is met, the determined pose of the shooting device will have a large error, that is, the obtained positioning error will be large, resulting in inaccurate output of the shooting device. pose.
  • the embodiment of the present application first provides a new positioning method.
  • the reprojection error threshold can be correspondingly adjusted according to the change of the real-time scene, so that the reprojection error threshold used when the scene changes It can ensure a small positioning error, and only output it when the pose of the shooting device meets the corresponding re-projection error threshold in the current scene, ensuring that the output pose is still accurate when the scene changes.
  • the positioning method provided in this embodiment can be applied to a photographing device, where the photographing device may be a camera, a camera, a visual sensor, or other device with an image acquisition function, and the movable platform can carry out the image acquisition by carrying the photographing device, and perform image acquisition according to the collected image.
  • FIG. 1 is a flowchart of a positioning method shown in an exemplary embodiment of the present application, and the method includes:
  • the estimated pose when the image is captured by the shooting device can be obtained by calculation based on the pose determined at the historical moment and the real-time motion state of the shooting device.
  • the real-time motion speed predicts the motion trajectory of the shooting device, and then calculates the pose of the shooting device at the current moment;
  • the estimated pose when acquiring the image captured by the shooting device can also be obtained in the three-dimensional space corresponding to the feature points on the real-time image captured by the shooting device After the position information is calculated according to the pnp algorithm, the acquisition of the estimated pose is not listed here.
  • the distance of the shooting device relative to the three-dimensional point corresponding to the feature point on the captured image can be determined according to the estimated pose to determine the distance of the shooting device relative to the scene, wherein the relative distance of the shooting device to the scene can be determined.
  • the distance of the three-dimensional point corresponding to the feature point on the captured image may be, the average value of the distance of the shooting device relative to each three-dimensional point is taken as the distance of the shooting device relative to the scene.
  • the reprojection error threshold to be adjusted may be a threshold set in advance based on experience.
  • the reprojection error threshold may be inversely correlated with the distance of the photographing device relative to the three-dimensional point corresponding to the feature point on the photographed image, that is, the reprojection error threshold may be inversely correlated with the distance of the photographing device relative to the scene.
  • a smaller re-projection error threshold can be set accordingly to obtain higher re-projection accuracy and ensure that the obtained estimated pose can be accurately verified.
  • adjusting the reprojection error threshold according to the distance of the photographing device relative to the three-dimensional point corresponding to the feature point on the captured image may be adjusting the reprojection error threshold according to the average distance of the photographing device relative to the three-dimensional point, such as the The average distance is used as the zoom ratio to adjust the reprojection error threshold.
  • the reprojection error threshold is reduced by the average distance as the reduction ratio; when the distance is short, the reprojection error threshold may not be adjusted, or you can use This average distance magnifies the reprojection error threshold for magnification.
  • the calculation of the pixel position error between the projection point and the feature point may be to calculate the error of all the projection points, or it may be to select and calculate the error between the projection point and the feature point only.
  • the reprojection error threshold it can be judged whether the errors between all the calculated projection points and feature points are less than the reprojection error threshold, or it can be only judged whether the errors between some of the calculated projection points and the feature points are If it is less than the reprojection error threshold, for example, it may be determined that the error between the projection point and the feature point of the preset ratio is smaller than the reprojection error threshold.
  • the specific method for calculating the error and judging whether the error is smaller than the reprojection error threshold can be set by the technical personnel according to actual needs, which is not limited.
  • the following describes how to obtain the spatial position information of the three-dimensional points corresponding to the feature points in the current image captured by the photographing device.
  • the distance measurement of the shooting scene can be performed in real time when the current image is shot, and the depth information of each pixel on the image is recorded, so as to obtain the spatial position information of the three-dimensional point corresponding to each pixel, so as to measure the distance of the image.
  • the spatial position information of the three-dimensional points corresponding to the feature points can be directly obtained.
  • the feature points carry the spatial position information of three-dimensional points in the historical images captured by the photographing device, for example, during the operation of the aforementioned UAV, Create a three-dimensional map of the environment, and record the corresponding feature points of the three-dimensional points in the collected images.
  • the historical image may be an image captured by a photographing device before the current moment, which may be a single image, or an image set composed of several images.
  • determining the spatial position information of the three-dimensional point corresponding to the feature point in the current image based on the feature point carrying the spatial position information of the three-dimensional point in the historical image may be directly matching the historical image with the current image to obtain the feature point of the feature point. Matching pairs, and then according to the spatial position information of the 3D points corresponding to the feature points in the historical image, the spatial position information of the 3D points corresponding to the feature points in the current image can be obtained.
  • the relative positional relationship between the historical image and the current image can be determined, and based on the relative positional relationship, the feature points carrying the spatial position information of the three-dimensional points in the historical image can be tracked to the current image, and the corresponding feature points in the current image can be obtained.
  • the relative positional relationship between the historical image and the current image can be determined by feature point matching. Specifically, the feature points in the historical image and the feature points in the current image can be extracted, and the feature points can be matched to obtain the corresponding , and then calculate the homography matrix based on the matching pairs. For example, the homography matrix can be calculated based on the method of multi-view geometry.
  • the homography matrix represents the relative position transformation relationship between the historical image and the current image.
  • the homography matrix of can get the relative position relationship between the historical image and the image. It should be noted that since the calculation of the homography matrix does not require a large number of matching pairs of feature points, the matching process does not actually cause a large resource consumption. In addition, in order to improve the efficiency and accuracy of matching, the extraction of the feature points of the current image can be performed according to the distribution of the feature points in the historical image. The distribution of points is relatively sparse, so you can choose to extract the feature points of the area in the current image and the area with denser distribution of feature points on the historical image that have similar scenes for feature point matching, thereby improving the matching efficiency and success rate.
  • feature points on historical images extract as many feature points as possible from areas on the current image that have similar scenes to areas with denser distribution of feature points on historical images, and partially extract features on the current image that are related to historical images.
  • the area with sparse distribution of the feature points has the feature points of the area of similar scene, of course, other feature point extraction methods are also possible, which will not be exhaustive here, and the technicians can choose according to actual needs.
  • the detection of feature points before feature point extraction can be performed by using ORB (Oriented Fast and Rotated Brief) feature point detection, FAST (Features from Accelerated Segments Test) feature point detection, BRIEF (Binary Robust Independent Elementary Features) feature detection and other methods
  • the method of feature point matching can be Brute-Force Matcher (brute force matching) method, that is, traverse all candidate matching points, and take the most similar matching method as a matching pair.
  • Brute-Force Matcher brute force matching
  • the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the current image based on the relative positional relationship, and the three-dimensional points corresponding to the feature points on the current image are obtained.
  • spatial location information Specifically, the feature points that carry the spatial position information of the three-dimensional points in the historical image can be projected to the current image through the relative positional relationship, and then the spatial position information of the three-dimensional points carried in the historical image can be determined in the current image through a tracking algorithm. Since the feature points in the historical image carry the spatial position information of the 3D points, the spatial position information of the 3D points corresponding to the feature points on the current image can be obtained.
  • the tracking algorithm can use KLT (Kanade-Lucas-Tomasi Tracking) corner tracking, CNN (Convolutional Neural Network) tracking and other algorithms, of course, it is not limited to this.
  • obtaining the spatial position information of the three-dimensional point corresponding to the feature point of the current image by tracking, compared with the method of directly matching the feature point of the historical image and the current image the consumption of resources can be effectively saved.
  • it avoids the discarding of feature points that fail to match, and for the problem that many feature points in historical images that carry spatial position information corresponding to 3D points cannot be reasonably utilized, it maximizes the utilization of historical image resources.
  • the reprojection error threshold After adjusting the reprojection error threshold, reproject the three-dimensional point corresponding to the feature point of the current image captured by the shooting device to the current image according to the estimated pose to obtain the projection point, and calculate the error between the projection point and the feature point, that is, reprojection If the re-projection error is less than the re-projection error threshold obtained by the adjustment, it can be considered that the estimated pose is accurate, and the estimated pose is used as the target pose output of the shooting device; if the re-projection error is not less than the re-projection error Projection error threshold, the estimated pose can be adjusted at this time, so that the three-dimensional point is reprojected to the projection obtained by the current image according to the spatial position information of the three-dimensional point corresponding to the feature point of the current image and the adjusted estimated pose The error between the point and the feature point is less than the reprojection error threshold.
  • the preset scale can be set according to the requirements for pose accuracy.
  • the adjusted estimated pose can be output as the target pose of the photographing device.
  • the adjusted estimated pose further adjustment and optimization may be considered to improve the positioning accuracy.
  • the obtained spatial position information of the three-dimensional points corresponding to the feature points in the current image may be inaccurate, that is, some three-dimensional points may themselves be abnormal points, and these abnormal points will also affect the positioning accuracy. Therefore, for the acquired current image, the three-dimensional points whose error between the projection point and the feature point obtained by re-projecting to the current image according to the adjusted estimated pose can be excluded, so that the acquired three-dimensional point can be The outliers are eliminated, and the distance of the shooting device relative to the remaining three-dimensional points after the elimination is re-determined according to the adjusted estimated pose.
  • the re-projection error threshold may be adjusted again according to the re-determined distance of the photographing device relative to the remaining three-dimensional points after culling. For example, when the re-determined distance is small, the original re-projection error threshold may continue to be used at this time; when the re-determined distance is large, the re-projection error threshold may also be adjusted correspondingly to improve positioning accuracy.
  • the reprojection error threshold needs to be adjusted by setting a preset distance threshold. Based on the preset distance threshold, if the distance of the re-determined photographing device relative to the remaining three-dimensional points after culling is determined If it is less than the preset distance threshold, there is no need to adjust the reprojection error threshold, and the estimated pose at this time is accurate, and the adjusted estimated pose can be directly output as the target pose of the shooting device; if the re-determined shooting device If the distance relative to the remaining 3D points after culling is not less than the preset distance threshold, at least one of the following iterations can be performed until the re-determined distance of the photographing device relative to the remaining 3D points after culling is smaller than the preset distance threshold:
  • the distance of the photographing device relative to the remaining three-dimensional point after culling is re-determined according to the recalculated estimated pose.
  • abnormal points in the 3D points can be continuously removed, and the estimated pose can be iteratively optimized to improve the accuracy of relocation.
  • the adjustment of the estimated pose may be to recalculate the estimated pose of the photographing device based on the adjusted re-projection error threshold, the remaining three-dimensional points after culling and their corresponding feature points, and specifically, the pnp algorithm may be used for calculation.
  • FIG. 2 is a specific flow chart of positioning shown in an exemplary embodiment of the present application, including:
  • the spatial position information of the 3D point may include reliability information of the spatial position of the 3D point in addition to the position of the 3D point in space.
  • the reliability of the three-dimensional point is generated and saved, and the reliability information representing the reliability of the three-dimensional point is generated and saved.
  • the reprojection error threshold can be further adjusted based on the reliability information of the spatial position of the three-dimensional point. Since the reliability of the spatial position of the 3D point affects the accuracy of the final relocation, the worse the reliability of the spatial position of the 3D point, the lower the accuracy of the final relocation. Therefore, the reprojection error threshold is actually related to the spatial position of the 3D point. reliability is inversely correlated. When the reprojection error threshold is adjusted based on the reliability information of the spatial position of the 3D point, when the reliability of the spatial position of the 3D point is poor, a correspondingly smaller reprojection error threshold can be set to eliminate the unreliable 3D points .
  • FIG. 3 is a flowchart of a method for obtaining spatial information according to an exemplary embodiment of the present application, including:
  • S301 Determine the relative positional relationship between a historical image captured by a photographing device and a current image captured by the photographing device; the historical image and the current image have similar scenes, and at least some of the feature points in the historical image carry the space of three-dimensional points location information;
  • FIG. 4 is a schematic diagram of a positioning device shown in an exemplary embodiment of the present application, including:
  • memory 402 for storing instructions executable by processor 401;
  • processor 401 is configured as:
  • the spatial position information of the 3D point and the estimated pose reproject the 3D point to the image to obtain a projection point, and calculate the pixel position error between the projection point and the feature point;
  • the estimated pose is output as the target pose of the photographing device.
  • FIG. 5 is a schematic diagram of an apparatus for acquiring spatial information according to an exemplary embodiment of the present application, including:
  • memory 502 for storing instructions executable by processor 501;
  • processor 501 is configured as:
  • the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the current image, and the spatial position information of the three-dimensional points corresponding to the feature points in the current image is obtained.
  • the apparatus may also include other components necessary for normal operation according to its actual type.
  • the device in the case where the device is a processing chip or a mainboard integrated with a processing chip, it realizes the control function by being installed on the device, for example, installed on a movable platform or a control terminal.
  • the device may also include a communication interface for Perform data interaction with other devices on the device; when the device is an electronic device, the device may also include, for example, an input/output interface, a communication interface, a bus, etc., and the input/output interface can be used to connect an input/output module, for information input and output.
  • the input/output/module can be configured in the electronic device as a component (not shown in the figure), or can be externally connected to the device to provide corresponding functions.
  • Input devices may include keyboards, mice, touch screens, microphones, various types of sensors, etc.
  • output devices may include displays, speakers, vibrators, indicator lights, and the like.
  • the communication interface is used to connect a communication module (not shown in the figure) to realize communication interaction between the electronic device and other devices.
  • the communication module may implement communication through wired means (eg, USB, network cable, etc.), or may implement communication through wireless means (eg, mobile network, WIFI, Bluetooth, etc.).
  • a bus includes a path that transfers information between various components of an electronic device, such as the processor, memory, input/output interfaces, and communication interfaces.
  • FIG. 6 is a schematic diagram of a photographing device shown in an exemplary embodiment of the present application, including:
  • memory 602 for storing instructions executable by processor 601;
  • processor 601 is configured as:
  • the spatial position information of the 3D point and the estimated pose reproject the 3D point to the image to obtain a projection point, and calculate the pixel position error between the projection point and the feature point;
  • the estimated pose is output as the target pose of the photographing device.
  • the processor 601 is also configured to:
  • the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the current image, and the spatial position information of the three-dimensional points corresponding to the feature points in the current image is obtained.
  • FIG. 7 is a schematic diagram of a movable platform according to an exemplary embodiment of the present application, including the photographing device shown in FIG. 6 .
  • the embodiments of the present application further provide a computer storage medium, on which a computer program is stored, and when the computer program is executed, the method described in any of the foregoing embodiments is implemented.
  • a computer storage medium on which a computer program is stored, and when the computer program is executed, the method described in any of the foregoing embodiments is implemented.
  • the apparatus embodiments since they basically correspond to the method embodiments, reference may be made to the partial descriptions of the method embodiments for related parts.
  • the device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed over multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.
  • All the embodiments given in this application may be implemented in whole or in part by software, hardware, firmware or any combination thereof.
  • software When implemented in software, it can be implemented in whole or in part in the form of a computer program product.
  • a computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the procedures or functions according to the embodiments of the present application are generated in whole or in part.
  • the computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device.
  • Computer instructions may be stored in or transmitted over a computer-readable storage medium.
  • Computer instructions may be sent from one website site, computer, server, or data center to another website site, computer, via wire (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.) , server or data center for transmission.
  • a computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, or the like that includes an integration of one or more available media.
  • Useful media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVDs), or semiconductor media (eg, solid state disks (SSDs)), and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Analysis (AREA)

Abstract

Embodiments of the present application provide a positioning method, comprising: acquiring an estimated pose when a photographing device captures an image, and spatial position information of a three-dimensional point corresponding to a feature point in the image; determining a distance between the photographing device and the three-dimensional point according to the estimated pose of the photographing device; determining a reprojection error threshold according to the distance; according to the spatial position information of the three-dimensional point and the estimated pose, reprojecting the three-dimensional point to the image to obtain a projection point, and calculating a pixel position error between the projection point and the feature point; and if the pixel position error is less than the reprojection error threshold, outputting the estimated pose as a target pose of the photographing device. According to the present application, positioning precision is ensured. The embodiments of the present application further provide a positioning apparatus, a spatial information acquisition method, a spatial information acquisition apparatus, a photographing device, and a movable platform.

Description

定位方法、空间信息获取方法、装置、拍摄设备Positioning method, spatial information acquisition method, device, and photographing device 技术领域technical field
本申请涉及视觉定位领域，尤其涉及一种定位方法、定位装置、空间信息获取方法、空间信息获取装置、拍摄设备以及可移动平台。The present application relates to the field of visual positioning, and in particular, to a positioning method, a positioning device, a spatial information acquisition method, a spatial information acquisition device, a photographing device, and a movable platform.
背景技术Background technique
对可移动平台而言，在没有外部定位系统例如GPS(Global Positioning System，全球定位系统)辅助定位的情况下，通常需要可移动平台根据自身搭载的相关传感器进行定位，但此类定位方式存在定位精度不高的问题。For a movable platform, in the absence of an external positioning system such as GPS (Global Positioning System, Global Positioning System) assisted positioning, the movable platform usually needs to be positioned according to its own related sensors, but such positioning methods exist. The problem of low precision.
发明内容SUMMARY OF THE INVENTION
第一方面，本申请实施例提供一种定位方法，包括：In a first aspect, an embodiment of the present application provides a positioning method, including:
获取拍摄设备拍摄图像时的估计位姿，和所述图像中特征点对应的三维点的空间位置信息；Obtain the estimated pose when the image is captured by the shooting device, and the spatial position information of the three-dimensional point corresponding to the feature point in the image;
根据所述拍摄设备的估计位姿确定所述拍摄设备相对于所述三维点的距离；determining the distance of the photographing device relative to the three-dimensional point according to the estimated pose of the photographing device;
根据所述距离确定重投影误差阈值；determining a reprojection error threshold according to the distance;
根据所述三维点的空间位置信息和所述估计位姿，将所述三维点重投影至所述图像得到投影点，计算所述投影点与所述特征点的之间的像素位置误差；According to the spatial position information of the 3D point and the estimated pose, reproject the 3D point to the image to obtain a projection point, and calculate the pixel position error between the projection point and the feature point;
若所述像素位置误差小于所述重投影误差阈值，则将所述估计位姿 作为所述拍摄设备的目标位姿输出。If the pixel position error is less than the reprojection error threshold, the estimated pose is output as the target pose of the photographing device.
第二方面，本申请实施例提供一种空间信息的获取方法，包括：In a second aspect, an embodiment of the present application provides a method for obtaining spatial information, including:
确定拍摄设备拍摄的历史图像与所述拍摄设备拍摄的当前图像的相对位置关系；所述历史图像与当前图像具有相似的场景，且所述历史图像中至少部分特征点携带三维点的空间位置信息；Determine the relative positional relationship between the historical image shot by the shooting device and the current image shot by the shooting device; the historical image and the current image have similar scenes, and at least some of the feature points in the historical image carry the spatial position information of the three-dimensional point ;
基于所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点跟踪到所述当前图像，得到所述当前图像中特征点对应的三维点的空间位置信息。Based on the relative position relationship, the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the current image, and the spatial position information of the three-dimensional points corresponding to the feature points in the current image is obtained.
第三方面，本申请实施例提供一种定位装置，包括：In a third aspect, an embodiment of the present application provides a positioning device, including:
处理器；processor;
用于存储处理器可执行指令的存储器；memory for storing processor-executable instructions;
其中，所述处理器被配置为：wherein the processor is configured to:
获取拍摄设备拍摄图像时的估计位姿，和所述图像中特征点对应的三维点的空间位置信息；Obtain the estimated pose when the image is captured by the shooting device, and the spatial position information of the three-dimensional point corresponding to the feature point in the image;
根据所述拍摄设备的估计位姿确定所述拍摄设备相对于所述三维点的距离；determining the distance of the photographing device relative to the three-dimensional point according to the estimated pose of the photographing device;
根据所述距离确定重投影误差阈值；determining a reprojection error threshold according to the distance;
根据所述三维点的空间位置信息和所述估计位姿，将所述三维点重投影至所述图像得到投影点，计算所述投影点与所述特征点的之间的像素位置误差；According to the spatial position information of the 3D point and the estimated pose, reproject the 3D point to the image to obtain a projection point, and calculate the pixel position error between the projection point and the feature point;
若所述像素位置误差小于所述重投影误差阈值，则将所述估计位姿作为所述拍摄设备的目标位姿输出。If the pixel position error is smaller than the reprojection error threshold, the estimated pose is output as the target pose of the photographing device.
第四方面，本申请实施例提供一种空间信息的获取装置，包括：In a fourth aspect, an embodiment of the present application provides a device for acquiring spatial information, including:
处理器；processor;
用于存储处理器可执行指令的存储器；memory for storing processor-executable instructions;
其中，所述处理器被配置为：wherein the processor is configured to:
确定拍摄设备拍摄的历史图像与所述拍摄设备拍摄的当前图像的相对位置关系；所述历史图像与当前图像具有相似的场景，且所述历史图像中至少部分特征点携带三维点的空间位置信息；Determine the relative positional relationship between the historical image shot by the shooting device and the current image shot by the shooting device; the historical image and the current image have similar scenes, and at least some of the feature points in the historical image carry the spatial position information of the three-dimensional point ;
基于所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点跟踪到所述当前图像，得到所述当前图像中特征点对应的三维点的空间位置信息。Based on the relative position relationship, the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the current image, and the spatial position information of the three-dimensional points corresponding to the feature points in the current image is obtained.
第五方面，本申请实施例提供一种拍摄设备，包括：In a fifth aspect, an embodiment of the present application provides a photographing device, including:
处理器；processor;
用于存储处理器可执行指令的存储器；memory for storing processor-executable instructions;
其中，所述处理器被配置为：wherein the processor is configured to:
获取拍摄设备拍摄图像时的估计位姿，和所述图像中特征点对应的三维点的空间位置信息；Obtain the estimated pose when the image is captured by the shooting device, and the spatial position information of the three-dimensional point corresponding to the feature point in the image;
根据所述拍摄设备的估计位姿确定所述拍摄设备相对于所述三维点的距离；determining the distance of the photographing device relative to the three-dimensional point according to the estimated pose of the photographing device;
根据所述距离确定重投影误差阈值；determining a reprojection error threshold according to the distance;
根据所述三维点的空间位置信息和所述估计位姿，将所述三维点重投影至所述图像得到投影点，计算所述投影点与所述特征点的之间的像素位置误差；According to the spatial position information of the 3D point and the estimated pose, reproject the 3D point to the image to obtain a projection point, and calculate the pixel position error between the projection point and the feature point;
若所述像素位置误差小于所述重投影误差阈值，则将所述估计位姿作为所述拍摄设备的目标位姿输出。If the pixel position error is smaller than the reprojection error threshold, the estimated pose is output as the target pose of the photographing device.
第六方面，本申请实施例提供一种拍摄设备，包括：In a sixth aspect, an embodiment of the present application provides a photographing device, including:
处理器；processor;
用于存储处理器可执行指令的存储器；memory for storing processor-executable instructions;
其中，所述处理器被配置为：wherein the processor is configured to:
确定拍摄设备拍摄的历史图像与所述拍摄设备拍摄的当前图像的相对位置关系；所述历史图像与当前图像具有相似的场景，且所述历史图像中至少部分特征点携带三维点的空间位置信息；Determine the relative positional relationship between the historical image shot by the shooting device and the current image shot by the shooting device; the historical image and the current image have similar scenes, and at least some of the feature points in the historical image carry the spatial position information of the three-dimensional point ;
基于所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点跟踪到所述当前图像，得到所述当前图像中特征点对应的三维点的空间位置信息。Based on the relative position relationship, the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the current image, and the spatial position information of the three-dimensional points corresponding to the feature points in the current image is obtained.
第七方面，本申请实施例提供一种可移动平台，包括上述任意实施例所述的拍摄设备。In a seventh aspect, an embodiment of the present application provides a movable platform, including the photographing device described in any of the foregoing embodiments.
本申请实施例提供的定位方法，可以根据实时场景的变化对应调整重投影误差阈值，以使得在场景变化时所采用的重投影误差阈值能确保较小的定位误差，并在拍摄设备的位姿满足当前场景下对应的重投影误差阈值时才将其输出，确保在场景变化时输出的位姿依旧准确。The positioning method provided by the embodiment of the present application can correspondingly adjust the reprojection error threshold according to the change of the real-time scene, so that the reprojection error threshold used when the scene changes can ensure a small positioning error, and the position and attitude of the shooting device can be adjusted accordingly. It is output only when the corresponding reprojection error threshold in the current scene is met, to ensure that the output pose is still accurate when the scene changes.
附图说明Description of drawings
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.
图1是本申请一示例性实施例示出的一种定位方法流程图。FIG. 1 is a flowchart of a positioning method according to an exemplary embodiment of the present application.
图2是本申请一示例性实施例示出的一种定位具体流程图。FIG. 2 is a specific flow chart of positioning according to an exemplary embodiment of the present application.
图3是本申请一示例性实施例示出的一种空间信息的获取方法流程图。FIG. 3 is a flowchart of a method for acquiring spatial information according to an exemplary embodiment of the present application.
图4是本申请一示例性实施例示出的一种定位装置的示意图。FIG. 4 is a schematic diagram of a positioning device according to an exemplary embodiment of the present application.
图5是本申请一示例性实施例示出的一种空间信息的获取装置的示意图。FIG. 5 is a schematic diagram of an apparatus for acquiring spatial information according to an exemplary embodiment of the present application.
图6是本申请一示例性实施例示出的一种拍摄设备的示意图。FIG. 6 is a schematic diagram of a photographing device according to an exemplary embodiment of the present application.
图7是本申请一示例性实施例示出的一种可移动平台的示意图。FIG. 7 is a schematic diagram of a movable platform according to an exemplary embodiment of the present application.
具体实施方式Detailed ways
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.
首先介绍定位相关概念，以可移动平台为例，其中可移动平台可以是无人机、无人船、无人车、机器人等设备，可移动平台在进行自动化的航行或作业时，例如无人机自主返航、机器人仓库自动巡检、农业无人机或无人车自动化作业等情况下，通常需要进行定位来确定自身位姿。以无人机自主返航为例，其通过定位技术对自身位姿不断矫正，以便于准确返回出发点。无人机在正常作业过程中，不断对周围环境采集图像建立三维地图，并记录物理空间中的三维点在所采集的图像中对应的特征点。在返航过程中，根据当前位置拍摄的图像(称为当前图像)，在历史拍摄的图像(称为历史图像)中进行检索，匹配历史图像与当前图像中相似的场景，再对当前图像提取特征点，与历史图像进行特征点匹配，得到特征点的匹 配对，假设历史图像中的特征点A与当前图像中的特征点B为一个匹配对，则根据特征点A对应的三维点的空间位置信息，可以得到特征点B对应的三维点的空间位置信息，并通过相关算法，例如pnp(perspective-n-point，多点透视成像)算法计算无人机搭载的拍摄设备的位姿，在进行位姿估计的同时可以设置一定的重投影误差阈值，基于该重投影误差阈值对估计得到的位姿进行调整，使得调整后的位姿对应的重投影误差小于所述重投影误差阈值范围，从而得到重定位的拍摄设备的位姿，进而定位无人机。其中，特征点可以是图像上灰度值发生剧烈变化的点或者在图像边缘上曲率较大的点，重投影误差指的是将空间上的三维点根据计算出的位姿重新投影到当前图像上时，与对应的特征点之间的像素位置之间的误差，通过设定重投影误差阈值可以检测将该位置误差是否在指定阈值的范围内，以确定所计算的位姿是否已经足够准确。First, the concept of positioning is introduced, taking the mobile platform as an example. The mobile platform can be a drone, an unmanned ship, an unmanned vehicle, a robot and other equipment. When the mobile platform performs automated navigation or operations, such as unmanned In the case of autonomous return of the aircraft, automatic inspection of robot warehouses, automatic operation of agricultural drones or unmanned vehicles, etc., positioning is usually required to determine its own posture. Taking the autonomous return of the drone as an example, it continuously corrects its own posture and posture through positioning technology, so as to accurately return to the starting point. During the normal operation of the UAV, it continuously collects images of the surrounding environment to build a three-dimensional map, and records the corresponding feature points of the three-dimensional points in the physical space in the collected images. In the process of returning home, according to the image captured at the current position (called the current image), search in the historically captured image (called the historical image), match the similar scene between the historical image and the current image, and then extract the features of the current image point, and match the feature points with the historical image to obtain the matching pair of feature points. Assuming that the feature point A in the historical image and the feature point B in the current image are a matching pair, then according to the spatial position of the three-dimensional point corresponding to the feature point A information, the spatial position information of the three-dimensional point corresponding to the feature point B can be obtained, and the pose of the camera mounted on the UAV can be calculated through the relevant algorithm, such as the pnp (perspective-n-point, multi-point perspective imaging) algorithm. A certain re-projection error threshold can be set while the pose is estimated, and the estimated pose can be adjusted based on the re-projection error threshold, so that the re-projection error corresponding to the adjusted pose is smaller than the re-projection error threshold range, thereby Get the pose of the relocated shooting device, and then locate the drone. Among them, the feature point can be the point on the image where the gray value changes drastically or the point with large curvature on the edge of the image, and the reprojection error refers to reprojecting the three-dimensional point in space to the current image according to the calculated pose The error between the pixel position and the corresponding feature point can be detected by setting the reprojection error threshold value when the position error is within the range of the specified threshold value, so as to determine whether the calculated pose is accurate enough .
然而，拍摄设备与实际场景之间的距离会对重定位的精度造成影响，具体的，当场景相对拍摄设备的距离不断变大时，相同的重投影误差代表的实际物理距离在不断变大。这会导致当场景较远时，即便满足所设的重投影误差阈值，所确定的拍摄设备的位姿也存在较大误差，即求得的定位误差较大，导致输出不准确的拍摄设备的位姿。However, the distance between the shooting device and the actual scene will affect the accuracy of relocation. Specifically, when the distance between the scene and the shooting device continues to increase, the actual physical distance represented by the same reprojection error continues to increase. This will result in that when the scene is far away, even if the set reprojection error threshold is met, the determined pose of the shooting device will have a large error, that is, the obtained positioning error will be large, resulting in inaccurate output of the shooting device. pose.
针对所存在的至少以上问题，本申请实施例首先提供一种新的定位方法，作为改进，可以根据实时场景的变化对应调整重投影误差阈值，以使得在场景变化时所采用的重投影误差阈值能确保较小的定位误差，并在拍摄设备的位姿满足当前场景下对应的重投影误差阈值时才将其输出，确保在场景变化时输出的位姿依旧准确。本实施例提供的定位方法可以应用于拍摄设备上，其中拍摄设备可以是摄像头、相机、视觉传感器等具备图像采集功能的设备，可移动平台可以通过搭载拍摄设备进行图像采集，根据采集的图像进行定位确定出拍摄设备的位姿，进一步得到自身的位姿，实现定位。下面介绍本申请所提供的定位方法，参照图1，图1是本申请 一示例性实施例示出的一种定位方法流程图，该方法包括：In view of at least the above problems, the embodiment of the present application first provides a new positioning method. As an improvement, the reprojection error threshold can be correspondingly adjusted according to the change of the real-time scene, so that the reprojection error threshold used when the scene changes It can ensure a small positioning error, and only output it when the pose of the shooting device meets the corresponding re-projection error threshold in the current scene, ensuring that the output pose is still accurate when the scene changes. The positioning method provided in this embodiment can be applied to a photographing device, where the photographing device may be a camera, a camera, a visual sensor, or other device with an image acquisition function, and the movable platform can carry out the image acquisition by carrying the photographing device, and perform image acquisition according to the collected image. The positioning determines the pose of the shooting device, and further obtains its own pose to achieve positioning. The positioning method provided by the present application will be introduced below. Referring to Figure 1, Figure 1 is a flowchart of a positioning method shown in an exemplary embodiment of the present application, and the method includes:
S101，获取拍摄设备拍摄图像时的估计位姿，和所述图像中特征点对应的三维点的空间位置信息；S101, obtaining an estimated pose when a photographing device takes an image, and spatial position information of a three-dimensional point corresponding to a feature point in the image;
S102，根据所述拍摄设备的估计位姿确定所述拍摄设备相对于所述三维点的距离；S102, determining the distance of the photographing device relative to the three-dimensional point according to the estimated pose of the photographing device;
S103，根据所述距离确定重投影误差阈值；S103, determining a reprojection error threshold according to the distance;
S104，根据所述三维点的空间位置信息和所述估计位姿，将所述三维点重投影至所述图像得到投影点，计算所述投影点与所述特征点的之间的像素位置误差；S104, according to the spatial position information of the three-dimensional point and the estimated pose, reproject the three-dimensional point to the image to obtain a projected point, and calculate the pixel position error between the projected point and the feature point ;
S105，若所述像素位置误差小于所述重投影误差阈值，则将所述估计位姿作为所述拍摄设备的目标位姿输出。S105, if the pixel position error is smaller than the reprojection error threshold, output the estimated pose as the target pose of the photographing device.
其中获取拍摄设备拍摄图像时的估计位姿可以是根据历史时刻所确定的位姿以及拍摄设备的实时运动状态推算得到，具体的，可以基于拍摄设备的历史时刻的位姿，并通过拍摄设备的实时运动速度预测拍摄设备的运动轨迹，进而推算出当前时刻拍摄设备的位姿；获取拍摄设备拍摄图像时的估计位姿也可以是在获取拍摄设备拍摄的实时图像上的特征点对应的三维空间位置信息后根据pnp算法计算得到，对于估计位姿的获取，在此不一一列举。Wherein, the estimated pose when the image is captured by the shooting device can be obtained by calculation based on the pose determined at the historical moment and the real-time motion state of the shooting device. The real-time motion speed predicts the motion trajectory of the shooting device, and then calculates the pose of the shooting device at the current moment; the estimated pose when acquiring the image captured by the shooting device can also be obtained in the three-dimensional space corresponding to the feature points on the real-time image captured by the shooting device After the position information is calculated according to the pnp algorithm, the acquisition of the estimated pose is not listed here.
在获取拍摄设备的估计位姿后，可以根据估计位姿确定拍摄设备相对于所拍摄的图像上的特征点对应的三维点的距离，来确定拍摄设备相对于场景的距离，其中确定拍摄设备相对于所拍摄的图像上的特征点对应的三维点的距离可以是，将拍摄设备相对于各三维点的距离的平均值作为拍摄设备相对场景的距离。并进一步地根据该距离来调整重投影误差阈值，以使得在当前场景下所采用的重投影误差阈值能够确保较小的定位误差。其中待调整的重投影误差阈值可以是预先根据经验设定的一个阈值。After the estimated pose of the shooting device is obtained, the distance of the shooting device relative to the three-dimensional point corresponding to the feature point on the captured image can be determined according to the estimated pose to determine the distance of the shooting device relative to the scene, wherein the relative distance of the shooting device to the scene can be determined. The distance of the three-dimensional point corresponding to the feature point on the captured image may be, the average value of the distance of the shooting device relative to each three-dimensional point is taken as the distance of the shooting device relative to the scene. And further adjust the reprojection error threshold according to the distance, so that the reprojection error threshold adopted in the current scene can ensure a small positioning error. The reprojection error threshold to be adjusted may be a threshold set in advance based on experience.
示例性的，重投影误差阈值可以与拍摄设备相对于所拍摄的图像上的特征点对应的三维点的距离呈反相关，即重投影误差阈值与拍摄设备相对场景的距离呈反相关，在距离越远时，相应地可以设置较小的重投影误差阈值，以获得较高的重投影准确度，确保能够准确验证所获取的估计位姿是否准确。示例性的，根据拍摄设备相对于所拍摄的图像上的特征点对应的三维点的距离调整重投影误差阈值，可以是根据拍摄设备相对于三维点的平均距离调整重投影误差阈值，例如将该平均距离作为缩放倍率调整重投影误差阈值，在距离较远时，以该平均距离为缩小倍率对重投影误差阈值进行缩小；在距离较近时，可以不对重投影误差阈值调整，或者也可以以该平均距离为放大倍率对重投影误差阈值进行放大。当然，除了计算拍摄设备相对于三维点的平均距离来调整重投影误差阈值以外，还可以是采用其他的计算拍摄设备相对于三维点的距离的方式，例如对拍摄设备相对于各三维点的距离进行加权平均。具体采用何种计算方式，并不限定。Exemplarily, the reprojection error threshold may be inversely correlated with the distance of the photographing device relative to the three-dimensional point corresponding to the feature point on the photographed image, that is, the reprojection error threshold may be inversely correlated with the distance of the photographing device relative to the scene. When the distance is farther, a smaller re-projection error threshold can be set accordingly to obtain higher re-projection accuracy and ensure that the obtained estimated pose can be accurately verified. Exemplarily, adjusting the reprojection error threshold according to the distance of the photographing device relative to the three-dimensional point corresponding to the feature point on the captured image may be adjusting the reprojection error threshold according to the average distance of the photographing device relative to the three-dimensional point, such as the The average distance is used as the zoom ratio to adjust the reprojection error threshold. When the distance is long, the reprojection error threshold is reduced by the average distance as the reduction ratio; when the distance is short, the reprojection error threshold may not be adjusted, or you can use This average distance magnifies the reprojection error threshold for magnification. Of course, in addition to calculating the average distance of the shooting device relative to the three-dimensional point to adjust the reprojection error threshold, other methods of calculating the distance of the shooting device relative to the three-dimensional point can also be used, such as the distance of the shooting device relative to each three-dimensional point. Take a weighted average. The specific calculation method is not limited.
另外需要说明的是，计算投影点与特征点的之间的像素位置误差，可以是计算所有投影点的误差，也可以是只选择计算部分投影点与特征点的误差。判断误差是否小于重投影误差阈值，可以是判断所计算的所有投影点与特征点的误差是否都小于重投影误差阈值，也可以是只判断计算的投影点中部分投影点与特征点的误差是否小于重投影误差阈值，例如，可以是判断预设比例的投影点与特征点的误差小于重投影误差阈值。对于误差的计算以及判断误差是否小于重投影误差阈值的具体方式，可以由技术人员根据实际需求设定，对此并不限定。In addition, it should be noted that the calculation of the pixel position error between the projection point and the feature point may be to calculate the error of all the projection points, or it may be to select and calculate the error between the projection point and the feature point only. To judge whether the error is less than the reprojection error threshold, it can be judged whether the errors between all the calculated projection points and feature points are less than the reprojection error threshold, or it can be only judged whether the errors between some of the calculated projection points and the feature points are If it is less than the reprojection error threshold, for example, it may be determined that the error between the projection point and the feature point of the preset ratio is smaller than the reprojection error threshold. The specific method for calculating the error and judging whether the error is smaller than the reprojection error threshold can be set by the technical personnel according to actual needs, which is not limited.
下面介绍拍摄设备拍摄的当前图像中特征点对应的三维点的空间位置信息如何获取。The following describes how to obtain the spatial position information of the three-dimensional points corresponding to the feature points in the current image captured by the photographing device.
在一个实施例中，可以是在拍摄当前图像时实时对拍摄场景进行测距，并记录图像上各像素点的深度信息，以得到各像素点对应的三维点的空间位置信息，从而在对图像提取特征点时，可以直接得到特征点对应的 三维点的空间位置信息。In one embodiment, the distance measurement of the shooting scene can be performed in real time when the current image is shot, and the depth information of each pixel on the image is recorded, so as to obtain the spatial position information of the three-dimensional point corresponding to each pixel, so as to measure the distance of the image. When the feature points are extracted, the spatial position information of the three-dimensional points corresponding to the feature points can be directly obtained.
在另一种实施例中，在拍摄设备拍摄的历史图像中存在至少部分特征点携带三维点的空间位置信息的情况下，例如前面所提到的无人机在作业的过程中，不断对周围环境建立三维地图，并记录三维点在采集的图像中对应的特征点，此时无人机所采集的历史图像中存在较多特征点携带三维点的空间位置信息，可以是从历史图像中获取当前图像中特征点对应的三维点的空间位置信息。其中所说的历史图像可以是拍摄设备在当前时刻前拍摄的图像，其可以是单个图像，也可以是若干图像组成的图像集，由于历史图像上通常存在与当前图像上相似的场景，因此可以基于相似场景匹配获取拍摄设备拍摄的与当前图像具有相似场景的历史图像，并基于历史图像中携带三维点的空间位置信息的特征点来确定当前图像中特征点对应的三维点的空间位置信息。需要说明，为使表述简洁，下文直接将与当前图像有相似场景的历史图像简称为历史图像，除非有特殊说明的例外。In another embodiment, in the case where at least some of the feature points carry the spatial position information of three-dimensional points in the historical images captured by the photographing device, for example, during the operation of the aforementioned UAV, Create a three-dimensional map of the environment, and record the corresponding feature points of the three-dimensional points in the collected images. At this time, there are many feature points in the historical images collected by the drone that carry the spatial location information of the three-dimensional points, which can be obtained from historical images. Spatial location information of the three-dimensional points corresponding to the feature points in the current image. The historical image may be an image captured by a photographing device before the current moment, which may be a single image, or an image set composed of several images. Obtain historical images with similar scenes to the current image captured by the photographing device based on similar scene matching, and determine the spatial location information of the 3D points corresponding to the feature points in the current image based on the feature points in the historical images that carry the spatial location information of the 3D points. It should be noted that, in order to make the expression concise, the historical images with similar scenes to the current image are simply referred to as historical images hereinafter, unless there are exceptions to special instructions.
其中，基于历史图像中携带三维点的空间位置信息的特征点来确定当前图像中特征点对应的三维点的空间位置信息，可以是直接将历史图像与当前图像进行特征点匹配，得到特征点的匹配对，再根据历史图像中特征点对应的三维点的空间位置信息，可以得到当前图像中特征点对应的三维点的空间位置信息。Wherein, determining the spatial position information of the three-dimensional point corresponding to the feature point in the current image based on the feature point carrying the spatial position information of the three-dimensional point in the historical image may be directly matching the historical image with the current image to obtain the feature point of the feature point. Matching pairs, and then according to the spatial position information of the 3D points corresponding to the feature points in the historical image, the spatial position information of the 3D points corresponding to the feature points in the current image can be obtained.
但由于在进行计算位姿时，需要一定数量具有三维空间位置信息的特征点，因此要求在对历史图像与当前图像进行特征点匹配时，匹配对的数量尽可能地多。为提高匹配成功率，需要从当前图像上提取非常多的候选特征点，导致造成较大的资源消耗，占用较多系统资源。对此，还可以采用另一种方式来确定当前图像中特征点对应的三维点的空间位置信息。具体的，可以确定出历史图像与当前图像的相对位置关系，基于该相对位置关系，可以将历史图像中携带三维点的空间位置信息的特征点跟踪到当前图像上，得到当前图像中特征点对应的三维点的空间位置信息。其中历 史图像和当前图像之间的相对位置关系可以通过特征点匹配的方式确定，具体的，可以提取历史图像中的特征点，以及提取当前图像中的特征点，并进行特征点匹配，得到相应的匹配对，再基于匹配对计算出单应矩阵，例如，可以是基于多视图几何的方法来计算单应矩阵，该单应矩阵表征历史图像与当前图像的相对位置转换关系，因此通过所求的单应矩阵可以得到历史图像与图像的相对位置关系。需要说明的是，由于计算单应矩阵并不需要大量特征点的匹配对，因此实际上匹配过程中并不会造成较大资源消耗。另外，为提高匹配的效率和准确度，对于当前图像的特征点的提取可以是根据历史图像中的特征点的分布来进行提取，例如历史图像上可能部分区域特征点分布较为密集，部分区域特征点分布较为稀疏，可以选择提取当前图像上与历史图像上特征点分布较为密集的区域具有相似场景的区域的特征点进行特征点匹配，由此提高匹配效率和成功率。或者还可以根据历史图像上特征点的分布情况，尽可能多地提取当前图像上与历史图像上特征点分布较为密集的区域具有相似场景的区域的特征点，以及部分提取当前图像上与历史图像上特征点分布较为稀疏的区域具有相似场景的区域的特征点，当然，还可以是其他的特征点提取方式，在此不做穷举，技术人员可以根据实际需求选择。However, since a certain number of feature points with three-dimensional spatial position information are required when calculating the pose, it is required that the number of matching pairs should be as large as possible when matching feature points between historical images and current images. In order to improve the matching success rate, it is necessary to extract a large number of candidate feature points from the current image, resulting in large resource consumption and occupying more system resources. In this regard, another method may also be used to determine the spatial position information of the three-dimensional point corresponding to the feature point in the current image. Specifically, the relative positional relationship between the historical image and the current image can be determined, and based on the relative positional relationship, the feature points carrying the spatial position information of the three-dimensional points in the historical image can be tracked to the current image, and the corresponding feature points in the current image can be obtained. The spatial position information of the 3D point. The relative positional relationship between the historical image and the current image can be determined by feature point matching. Specifically, the feature points in the historical image and the feature points in the current image can be extracted, and the feature points can be matched to obtain the corresponding , and then calculate the homography matrix based on the matching pairs. For example, the homography matrix can be calculated based on the method of multi-view geometry. The homography matrix represents the relative position transformation relationship between the historical image and the current image. The homography matrix of can get the relative position relationship between the historical image and the image. It should be noted that since the calculation of the homography matrix does not require a large number of matching pairs of feature points, the matching process does not actually cause a large resource consumption. In addition, in order to improve the efficiency and accuracy of matching, the extraction of the feature points of the current image can be performed according to the distribution of the feature points in the historical image. The distribution of points is relatively sparse, so you can choose to extract the feature points of the area in the current image and the area with denser distribution of feature points on the historical image that have similar scenes for feature point matching, thereby improving the matching efficiency and success rate. Alternatively, according to the distribution of feature points on historical images, extract as many feature points as possible from areas on the current image that have similar scenes to areas with denser distribution of feature points on historical images, and partially extract features on the current image that are related to historical images. The area with sparse distribution of the feature points has the feature points of the area of similar scene, of course, other feature point extraction methods are also possible, which will not be exhaustive here, and the technicians can choose according to actual needs.
其中特征点提取前对于特征点的检测可以是采用ORB(Oriented Fast and Rotated Brief)特征点检测、FAST(Features from Accelerated Segments Test)特征点检测、BRIEF(Binary Robust Independent Elementary Features)特征检测等方法，特征点匹配的方式可以是Brute-Force Matcher(暴力匹配)方式，即遍历所有候选匹配点，取最相似的作为匹配对的匹配方式，当然，并不限于此，以上仅为示例，技术人员可以根据实际需求选择具体的方式。Among them, the detection of feature points before feature point extraction can be performed by using ORB (Oriented Fast and Rotated Brief) feature point detection, FAST (Features from Accelerated Segments Test) feature point detection, BRIEF (Binary Robust Independent Elementary Features) feature detection and other methods, The method of feature point matching can be Brute-Force Matcher (brute force matching) method, that is, traverse all candidate matching points, and take the most similar matching method as a matching pair. Of course, it is not limited to this. The above is only an example, and the technician can Choose a specific method according to actual needs.
进一步的，在确定历史图像和当前图像的相对位置关系后，再基于相对位置关系将历史图像中携带三维点的空间位置信息的特征点跟踪到当 前图像，得到当前图像上特征点对应的三维点的空间位置信息。具体的，可以是通过该相对位置关系将历史图像中携带三维点的空间位置信息的特征点投影到当前图像，再通过跟踪算法在当前图像中确定出与历史图像中携带三维点的空间位置信息的特征点对应的特征点，由于历史图像中特征点携带三维点的空间位置信息，进而可以得到当前图像上特征点对应的三维点的空间位置信息。其中跟踪算法可以采用KLT(Kanade-Lucas-Tomasi Tracking)角点跟踪、CNN(Convolutional Neural Network)跟踪等算法，当然并不限于此。Further, after determining the relative positional relationship between the historical image and the current image, the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the current image based on the relative positional relationship, and the three-dimensional points corresponding to the feature points on the current image are obtained. spatial location information. Specifically, the feature points that carry the spatial position information of the three-dimensional points in the historical image can be projected to the current image through the relative positional relationship, and then the spatial position information of the three-dimensional points carried in the historical image can be determined in the current image through a tracking algorithm. Since the feature points in the historical image carry the spatial position information of the 3D points, the spatial position information of the 3D points corresponding to the feature points on the current image can be obtained. The tracking algorithm can use KLT (Kanade-Lucas-Tomasi Tracking) corner tracking, CNN (Convolutional Neural Network) tracking and other algorithms, of course, it is not limited to this.
本申请实施例中，通过跟踪的方式获取当前图像的特征点对应的三维点的空间位置信息，相比直接对历史图像和当前图像进行特征点匹配的方式，可以有效节省资源的消耗。同时避免了对于匹配失败的特征点的丢弃，对于历史图像中许多携带三维点对应的空间位置信息的特征点无法合理利用的问题，最大限度地提到了对历史图像资源的利用率。In the embodiment of the present application, obtaining the spatial position information of the three-dimensional point corresponding to the feature point of the current image by tracking, compared with the method of directly matching the feature point of the historical image and the current image, the consumption of resources can be effectively saved. At the same time, it avoids the discarding of feature points that fail to match, and for the problem that many feature points in historical images that carry spatial position information corresponding to 3D points cannot be reasonably utilized, it maximizes the utilization of historical image resources.
下面在图1相关实施例的基础上进一步介绍本申请实施例提供的定位方法。The positioning method provided by the embodiment of the present application is further described below on the basis of the relevant embodiment in FIG. 1 .
在调整重投影误差阈值后，根据估计位姿对拍摄设备拍摄的当前图像的特征点对应的三维点进行重投影到当前图像上，得到投影点，计算投影点与特征点的误差，即重投影误差，如果重投影误差小于该调整得到的重投影误差阈值，则可以认为该估计位姿是准确的，并将该估计位姿作为拍摄设备的目标位姿输出；如果重投影误差不小于该重投影误差阈值，此时可以对该估计位姿进行调整，以使根据当前图像的特征点对应的三维点的空间位置信息和调整后的估计位姿，将三维点重投影至当前图像得到的投影点与特征点的误差小于重投影误差阈值。其中，考虑到实际调整估计位姿时可能难以实现重投影时所有投影点与特征点的误差都小于重投影误差阈值，因此，可选的，可以是调整估计位姿直至投影点中，预设比例的投影点与特征点的误差小于重投影误差阈值，该预设比例可以根据对位姿 精度的需求设定。After adjusting the reprojection error threshold, reproject the three-dimensional point corresponding to the feature point of the current image captured by the shooting device to the current image according to the estimated pose to obtain the projection point, and calculate the error between the projection point and the feature point, that is, reprojection If the re-projection error is less than the re-projection error threshold obtained by the adjustment, it can be considered that the estimated pose is accurate, and the estimated pose is used as the target pose output of the shooting device; if the re-projection error is not less than the re-projection error Projection error threshold, the estimated pose can be adjusted at this time, so that the three-dimensional point is reprojected to the projection obtained by the current image according to the spatial position information of the three-dimensional point corresponding to the feature point of the current image and the adjusted estimated pose The error between the point and the feature point is less than the reprojection error threshold. Among them, considering that it may be difficult to realize that the error between all projection points and feature points during re-projection is less than the re-projection error threshold when actually adjusting the estimated pose The error between the scaled projection point and the feature point is less than the reprojection error threshold, and the preset scale can be set according to the requirements for pose accuracy.
在一个实施例中，对于调整后的估计位姿，可以将其作为拍摄设备的目标位姿输出。In one embodiment, the adjusted estimated pose can be output as the target pose of the photographing device.
进一步的，对于调整后的估计位姿，还可以考虑进一步对其进行调整优化，以提高定位精度。具体的，由于所获取的当前图像中特征点对应的三维点的空间位置信息可能是不准确的，也即可能部分三维点本身便是异常点，这些异常点也将影响定位精度。因此对于所获取的当前图像，可以剔除根据调整后的估计位姿重投影至当前图像得到的投影点与特征点的误差大于重投影误差阈值的三维点，从而可以将所获取的三维点中的异常点剔除，并根据调整后的估计位姿重新确定拍摄设备相对于剔除后剩余三维点的距离。此时可以根据重新确定的拍摄设备相对于剔除后剩余三维点的距离决定是否需要再次调整重投影误差阈值。例如在重新确定的距离较小时，此时可以继续采用原来的重投影误差阈值；在重新确定的距离较大时，则还可以对应调整重投影误差阈值，以提高定位精度。Further, for the adjusted estimated pose, further adjustment and optimization may be considered to improve the positioning accuracy. Specifically, since the obtained spatial position information of the three-dimensional points corresponding to the feature points in the current image may be inaccurate, that is, some three-dimensional points may themselves be abnormal points, and these abnormal points will also affect the positioning accuracy. Therefore, for the acquired current image, the three-dimensional points whose error between the projection point and the feature point obtained by re-projecting to the current image according to the adjusted estimated pose can be excluded, so that the acquired three-dimensional point can be The outliers are eliminated, and the distance of the shooting device relative to the remaining three-dimensional points after the elimination is re-determined according to the adjusted estimated pose. At this time, it may be determined whether the re-projection error threshold needs to be adjusted again according to the re-determined distance of the photographing device relative to the remaining three-dimensional points after culling. For example, when the re-determined distance is small, the original re-projection error threshold may continue to be used at this time; when the re-determined distance is large, the re-projection error threshold may also be adjusted correspondingly to improve positioning accuracy.
具体的，在一个实施例中，可以通过设置一个预设距离阈值来判断是否需要调整重投影误差阈值，基于所预设的距离阈值，若重新确定的拍摄设备相对于剔除后剩余三维点的距离小于预设距离阈值，则可以无需调整重投影误差阈值，且此时的估计位姿已经准确，可以直接将该调整后的估计位姿作为拍摄设备的目标位姿输出；若重新确定的拍摄设备相对于剔除后剩余三维点的距离不小于预设距离阈值，则可以进行以下至少一次迭代，直至重新确定的拍摄设备相对于剔除后剩余三维点的距离小于预设距离阈值：Specifically, in one embodiment, it is possible to determine whether the reprojection error threshold needs to be adjusted by setting a preset distance threshold. Based on the preset distance threshold, if the distance of the re-determined photographing device relative to the remaining three-dimensional points after culling is determined If it is less than the preset distance threshold, there is no need to adjust the reprojection error threshold, and the estimated pose at this time is accurate, and the adjusted estimated pose can be directly output as the target pose of the shooting device; if the re-determined shooting device If the distance relative to the remaining 3D points after culling is not less than the preset distance threshold, at least one of the following iterations can be performed until the re-determined distance of the photographing device relative to the remaining 3D points after culling is smaller than the preset distance threshold:
根据重新确定的拍摄设备相对于剔除后剩余三维点的距离调整重投影误差阈值；Adjust the reprojection error threshold according to the re-determined distance of the photographing device relative to the remaining 3D points after culling;
再次调整估计位姿，以使根据三维点的空间位置信息和再次调整后 的估计位姿，将剔除后剩余三维点重投影至当前图像得到的投影点中，预设比例的投影点与特征点的误差小于调整后重投影误差阈值；Adjust the estimated pose again, so that according to the spatial position information of the three-dimensional point and the estimated pose after adjustment, the remaining three-dimensional points after culling are re-projected to the projected points obtained from the current image, and the projected points and feature points of the preset proportion are The error of is less than the adjusted reprojection error threshold;
剔除根据重新计算的估计位姿重投影至当前图像得到的投影点与特征点的误差大于调整后的重投影误差阈值的三维点；Eliminate the three-dimensional points whose error between the projection point and the feature point obtained by re-projecting the re-calculated estimated pose to the current image is greater than the adjusted re-projection error threshold;
根据重新计算的估计位姿重新确定拍摄设备相对于剔除后剩余三维点的距离。The distance of the photographing device relative to the remaining three-dimensional point after culling is re-determined according to the recalculated estimated pose.
通过以上的迭代，可以不断剔除三维点中的异常点，并且迭代优化估计位姿，提高重定位的精度。其中调整估计位姿可以是基于调整后的重投影误差阈值、剔除后剩余三维点及其对应的特征点重新计算拍摄设备的估计位姿，具体的，可以是采用pnp算法进行计算。Through the above iterations, abnormal points in the 3D points can be continuously removed, and the estimated pose can be iteratively optimized to improve the accuracy of relocation. The adjustment of the estimated pose may be to recalculate the estimated pose of the photographing device based on the adjusted re-projection error threshold, the remaining three-dimensional points after culling and their corresponding feature points, and specifically, the pnp algorithm may be used for calculation.
下面举一个实际的例子来阐述定位具体过程，其可以应用于拍摄设备上。参见图2，图2是本申请一示例性实施例示出的一种定位具体流程图，包括：An actual example is given below to illustrate the specific process of positioning, which can be applied to a photographing device. Referring to FIG. 2, FIG. 2 is a specific flow chart of positioning shown in an exemplary embodiment of the present application, including:
S201，设置初始的重投影误差阈值；S201, setting an initial reprojection error threshold;
S202，获取拍摄设备拍摄的当前图像的特征点及其对应的未剔除的三维点的空间位置信息、重投影误差阈值，并基于pnp算法进行一次计算得到估计位姿；S202, obtaining the feature points of the current image captured by the photographing device and the spatial position information and reprojection error thresholds of the corresponding unremoved three-dimensional points, and performing a calculation based on the pnp algorithm to obtain the estimated pose;
S203，剔除当前图像上特征点对应的三维点中重投影误差大于重投影误差阈值的异常三维点；S203, removing abnormal 3D points whose reprojection error is greater than the reprojection error threshold in the 3D points corresponding to the feature points on the current image;
S204，基于所得估计位姿、剔除后剩余三维点及其对应的特征点计算当前拍摄设备相对于剔除后剩余三维点的平均距离；S204, calculate the average distance of the current photographing device relative to the remaining three-dimensional points after the culling based on the obtained estimated pose, the remaining 3D points after the culling, and their corresponding feature points;
S205，在该平均距离小于预设距离阈值时，将当前拍摄设备的估计位姿作为目标位姿输出；S205, when the average distance is less than the preset distance threshold, output the estimated pose of the current photographing device as the target pose;
S206，在该平均距离不小于预设距离阈值时，基于该平均深度调整 重投影误差阈值，并执行S201。S206, when the average distance is not less than the preset distance threshold, adjust the reprojection error threshold based on the average depth, and execute S201.
对于以上步骤的详细介绍可以参照前面实施例的记载，在此不做重复介绍。For the detailed description of the above steps, reference may be made to the descriptions in the previous embodiments, and repeated descriptions are not repeated here.
在一个实施例中，三维点的空间位置信息除了包括三维点在空间上的位置以外，还可以包括三维点的空间位置的可靠性信息，例如，可以在记录三维点时根据实时场景判断所记录的三维点的可靠性，并生成和保存表征三维点的可靠程度的可靠性信息，在此基础上，可以进一步地基于三维点的空间位置的可靠性信息来调整重投影误差阈值。由于三维点的空间位置的可靠性影响最终重定位的精度，三维点的空间位置的可靠性越差时，最终重定位的精度也越低，因此实际上重投影误差阈值与三维点的空间位置的可靠性呈反相关。在基于三维点的空间位置的可靠性信息来调整重投影误差阈值时，在三维点的空间位置的可靠性较差时，可以对应地设置较小重投影误差阈值，以便剔除不可靠的三维点。In one embodiment, the spatial position information of the 3D point may include reliability information of the spatial position of the 3D point in addition to the position of the 3D point in space. The reliability of the three-dimensional point is generated and saved, and the reliability information representing the reliability of the three-dimensional point is generated and saved. On this basis, the reprojection error threshold can be further adjusted based on the reliability information of the spatial position of the three-dimensional point. Since the reliability of the spatial position of the 3D point affects the accuracy of the final relocation, the worse the reliability of the spatial position of the 3D point, the lower the accuracy of the final relocation. Therefore, the reprojection error threshold is actually related to the spatial position of the 3D point. reliability is inversely correlated. When the reprojection error threshold is adjusted based on the reliability information of the spatial position of the 3D point, when the reliability of the spatial position of the 3D point is poor, a correspondingly smaller reprojection error threshold can be set to eliminate the unreliable 3D points .
本申请实施例还提供一种空间信息的获取方法，参见图3，图3是本申请一示例性实施例示出的一种空间信息的获取方法流程图，包括：An embodiment of the present application further provides a method for obtaining spatial information. Referring to FIG. 3, FIG. 3 is a flowchart of a method for obtaining spatial information according to an exemplary embodiment of the present application, including:
S301，确定拍摄设备拍摄的历史图像与所述拍摄设备拍摄的当前图像的相对位置关系；所述历史图像与当前图像具有相似的场景，且所述历史图像中至少部分特征点携带三维点的空间位置信息；S301: Determine the relative positional relationship between a historical image captured by a photographing device and a current image captured by the photographing device; the historical image and the current image have similar scenes, and at least some of the feature points in the historical image carry the space of three-dimensional points location information;
S302，基于所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点跟踪到所述当前图像，得到所述当前图像中特征点对应的三维点的空间位置信息。S302 , based on the relative positional relationship, track the feature points carrying the spatial position information of the three-dimensional points in the historical image to the current image, and obtain the spatial position information of the three-dimensional points corresponding to the feature points in the current image.
对于以上步骤的详细介绍可以参照前面实施例的内容，在此不再重复介绍。For the detailed description of the above steps, reference may be made to the contents of the previous embodiments, and the description will not be repeated here.
本申请实施例还提供一种定位装置，参见图4，图4是本申请一示例性实施例示出的一种定位装置的示意图，包括：An embodiment of the present application further provides a positioning device. Referring to FIG. 4, FIG. 4 is a schematic diagram of a positioning device shown in an exemplary embodiment of the present application, including:
处理器401； processor 401;
用于存储处理器401可执行指令的存储器402； memory 402 for storing instructions executable by processor 401;
其中，所述处理器401被配置为：Wherein, the processor 401 is configured as:
获取拍摄设备拍摄图像时的估计位姿，和所述图像中特征点对应的三维点的空间位置信息；Obtain the estimated pose when the image is captured by the shooting device, and the spatial position information of the three-dimensional point corresponding to the feature point in the image;
根据所述拍摄设备的估计位姿确定所述拍摄设备相对于所述三维点的距离；determining the distance of the photographing device relative to the three-dimensional point according to the estimated pose of the photographing device;
根据所述距离确定重投影误差阈值；determining a reprojection error threshold according to the distance;
根据所述三维点的空间位置信息和所述估计位姿，将所述三维点重投影至所述图像得到投影点，计算所述投影点与所述特征点的之间的像素位置误差；According to the spatial position information of the 3D point and the estimated pose, reproject the 3D point to the image to obtain a projection point, and calculate the pixel position error between the projection point and the feature point;
若所述像素位置误差小于所述重投影误差阈值，则将所述估计位姿作为所述拍摄设备的目标位姿输出。If the pixel position error is smaller than the reprojection error threshold, the estimated pose is output as the target pose of the photographing device.
本申请实施例还提供一种空间信息的获取装置，参见图5，图5是本申请一示例性实施例示出的一种空间信息的获取装置的示意图，包括：An embodiment of the present application further provides an apparatus for acquiring spatial information. Referring to FIG. 5 , FIG. 5 is a schematic diagram of an apparatus for acquiring spatial information according to an exemplary embodiment of the present application, including:
处理器501； processor 501;
用于存储处理器501可执行指令的存储器502； memory 502 for storing instructions executable by processor 501;
其中，所述处理器501被配置为：Wherein, the processor 501 is configured as:
确定拍摄设备拍摄的历史图像与所述拍摄设备拍摄的当前图像的相对位置关系；所述历史图像与当前图像具有相似的场景，且所述历史图像中至少部分特征点携带三维点的空间位置信息；Determine the relative positional relationship between the historical image shot by the shooting device and the current image shot by the shooting device; the historical image and the current image have similar scenes, and at least some of the feature points in the historical image carry the spatial position information of the three-dimensional point ;
基于所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点跟踪到所述当前图像，得到所述当前图像中特征点对应的三维点的空间位置信息。Based on the relative position relationship, the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the current image, and the spatial position information of the three-dimensional points corresponding to the feature points in the current image is obtained.
可以理解，在上述装置所示出的处理器、存储器的基础上，装置根据其实际的类型还可以包括实现正常运行所必需的其他组件。例如，在装置为处理芯片或集成有处理芯片的主板的情况下，其通过安装于设备上实现控制功能，例如安装于可移动平台或控制终端上，此时装置还可以包括通信接口，用于与设备上其他器件进行数据的交互；在装置为电子设备的情况下，此时装置还可以包括例如输入/输出接口、通信接口、总线等，输入/输出接口可以用于连接输入/输出模块，以实现信息输入及输出。输入输出/模块可以作为组件配置在电子设备中(图中未示出)，也可以外接于设备以提供相应功能。输入设备可以包括键盘、鼠标、触摸屏、麦克风、各类传感器等，输出设备可以包括显示器、扬声器、振动器、指示灯等。通信接口用于连接通信模块(图中未示出)，以实现电子设备与其他设备的通信交互。其中通信模块可以通过有线方式(例如USB、网线等)实现通信，也可以通过无线方式(例如移动网络、WIFI、蓝牙等)实现通信。总线包括一通路，在电子设备的各个组件(例如处理器、存储器、输入/输出接口和通信接口)之间传输信息。It can be understood that, on the basis of the processor and memory shown in the above apparatus, the apparatus may also include other components necessary for normal operation according to its actual type. For example, in the case where the device is a processing chip or a mainboard integrated with a processing chip, it realizes the control function by being installed on the device, for example, installed on a movable platform or a control terminal. At this time, the device may also include a communication interface for Perform data interaction with other devices on the device; when the device is an electronic device, the device may also include, for example, an input/output interface, a communication interface, a bus, etc., and the input/output interface can be used to connect an input/output module, for information input and output. The input/output/module can be configured in the electronic device as a component (not shown in the figure), or can be externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various types of sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, and the like. The communication interface is used to connect a communication module (not shown in the figure) to realize communication interaction between the electronic device and other devices. The communication module may implement communication through wired means (eg, USB, network cable, etc.), or may implement communication through wireless means (eg, mobile network, WIFI, Bluetooth, etc.). A bus includes a path that transfers information between various components of an electronic device, such as the processor, memory, input/output interfaces, and communication interfaces.
本申请实施例还提供一种拍摄设备，参见图6，图6是本申请一示例性实施例示出的一种拍摄设备的示意图，包括：An embodiment of the present application further provides a photographing device. Referring to FIG. 6 , FIG. 6 is a schematic diagram of a photographing device shown in an exemplary embodiment of the present application, including:
处理器601； processor 601;
用于存储处理器601可执行指令的存储器602； memory 602 for storing instructions executable by processor 601;
输入/输出接口603；input/output interface 603;
通信接口604； communication interface 604;
总线605； bus 605;
其中，所述处理器601被配置为：Wherein, the processor 601 is configured as:
获取拍摄设备拍摄图像时的估计位姿，和所述图像中特征点对应的三维点的空间位置信息；Obtain the estimated pose when the image is captured by the shooting device, and the spatial position information of the three-dimensional point corresponding to the feature point in the image;
根据所述拍摄设备的估计位姿确定所述拍摄设备相对于所述三维点的距离；determining the distance of the photographing device relative to the three-dimensional point according to the estimated pose of the photographing device;
根据所述距离确定重投影误差阈值；determining a reprojection error threshold according to the distance;
根据所述三维点的空间位置信息和所述估计位姿，将所述三维点重投影至所述图像得到投影点，计算所述投影点与所述特征点的之间的像素位置误差；According to the spatial position information of the 3D point and the estimated pose, reproject the 3D point to the image to obtain a projection point, and calculate the pixel position error between the projection point and the feature point;
若所述像素位置误差小于所述重投影误差阈值，则将所述估计位姿作为所述拍摄设备的目标位姿输出。If the pixel position error is smaller than the reprojection error threshold, the estimated pose is output as the target pose of the photographing device.
所述处理器601还被配置为：The processor 601 is also configured to:
确定拍摄的历史图像与拍摄的当前图像的相对位置关系；所述历史图像与当前图像具有相似的场景，且所述历史图像中至少部分特征点携带三维点的空间位置信息；Determine the relative positional relationship between the photographed historical image and the photographed current image; the historical image and the current image have similar scenes, and at least some of the feature points in the historical image carry spatial position information of three-dimensional points;
基于所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点跟踪到所述当前图像，得到所述当前图像中特征点对应的三维点的空间位置信息。Based on the relative position relationship, the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the current image, and the spatial position information of the three-dimensional points corresponding to the feature points in the current image is obtained.
本申请实施例还提供一种可移动平台，参见图7，图7是本申请一示例性实施例示出的一种可移动平台的示意图，包括如图6所示的拍摄设备。An embodiment of the present application further provides a movable platform. Referring to FIG. 7 , FIG. 7 is a schematic diagram of a movable platform according to an exemplary embodiment of the present application, including the photographing device shown in FIG. 6 .
本申请实施例还提供一种计算机存储介质，其上存储有计算机程序，所述计算机程序被执行时实现上述任意实施例所述的方法。对于装置实施例而言，由于其基本对应于方法实施例，所以相关之处参见方法实施例的部分说明即可。以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部 模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。The embodiments of the present application further provide a computer storage medium, on which a computer program is stored, and when the computer program is executed, the method described in any of the foregoing embodiments is implemented. For the apparatus embodiments, since they basically correspond to the method embodiments, reference may be made to the partial descriptions of the method embodiments for related parts. The device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed over multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.
本申请所给出的所有实施例，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行计算机程序指令时，全部或部分地产生按照本申请实施例的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中，或者通过计算机可读存储介质进行传输。计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如，固态硬盘(solid state disk，SSD))等。All the embodiments given in this application may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the procedures or functions according to the embodiments of the present application are generated in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device. Computer instructions may be stored in or transmitted over a computer-readable storage medium. Computer instructions may be sent from one website site, computer, server, or data center to another website site, computer, via wire (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.) , server or data center for transmission. A computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, or the like that includes an integration of one or more available media. Useful media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVDs), or semiconductor media (eg, solid state disks (SSDs)), and the like.
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any relationship between these entities or operations. any such actual relationship or sequence exists. The terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion such that a process, method, article or device comprising a list of elements includes not only those elements, but also other not expressly listed elements, or also include elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.
以上对本申请实施例所提供的方法和装置进行了详细介绍，本文中 应用了具体个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的一般技术人员，依据本申请的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本申请的限制。The methods and devices provided by the embodiments of the present application have been described in detail above, and the principles and implementations of the present application are described with specific examples herein. At the same time, for those of ordinary skill in the art, according to the idea of the application, there will be changes in the specific implementation and application scope. In summary, the content of this specification should not be construed as a limitation to the application. .

Claims (61)

  1. 一种定位方法，其特征在于，包括：A positioning method, comprising:
    获取拍摄设备拍摄图像时的估计位姿，和所述图像中特征点对应的三维点的空间位置信息；Obtain the estimated pose when the image is captured by the shooting device, and the spatial position information of the three-dimensional point corresponding to the feature point in the image;
    根据所述拍摄设备的估计位姿确定所述拍摄设备相对于所述三维点的距离；determining the distance of the photographing device relative to the three-dimensional point according to the estimated pose of the photographing device;
    根据所述距离确定重投影误差阈值；determining a reprojection error threshold according to the distance;
    根据所述三维点的空间位置信息和所述估计位姿，将所述三维点重投影至所述图像得到投影点，计算所述投影点与所述特征点的之间的像素位置误差；According to the spatial position information of the 3D point and the estimated pose, reproject the 3D point to the image to obtain a projection point, and calculate the pixel position error between the projection point and the feature point;
    若所述像素位置误差小于所述重投影误差阈值，则将所述估计位姿作为所述拍摄设备的目标位姿输出。If the pixel position error is smaller than the reprojection error threshold, the estimated pose is output as the target pose of the photographing device.
  2. 根据权利要求1所述的方法，其特征在于，获取所述图像中特征点对应的三维点的空间位置信息之前包括：The method according to claim 1, wherein before acquiring the spatial position information of the three-dimensional points corresponding to the feature points in the image, the method comprises:
    基于相似场景匹配获取所述拍摄设备拍摄的与所述图像具有相似场景的历史图像，所述历史图像中至少部分特征点携带三维点的空间位置信息；Obtaining, based on similar scene matching, a historical image captured by the photographing device and having a similar scene to the image, at least some of the feature points in the historical image carry spatial position information of three-dimensional points;
    基于所述历史图像中携带三维点的空间位置信息的特征点确定所述图像中特征点对应的三维点的空间位置信息。The spatial position information of the three-dimensional point corresponding to the feature point in the image is determined based on the feature point carrying the spatial position information of the three-dimensional point in the historical image.
  3. 根据权利要求2所述的方法，其特征在于，所述基于所述历史图像中携带三维点的空间位置信息的特征点确定所述图像中特征点对应的三维空间位置信息，包括：The method according to claim 2, wherein the determining the three-dimensional spatial position information corresponding to the feature points in the image based on the feature points carrying the spatial position information of the three-dimensional points in the historical image comprises:
    确定所述历史图像与所述图像的相对位置关系；determining the relative positional relationship between the historical image and the image;
    基于所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点跟踪到所述图像，得到所述图像中特征点对应的三维点的空间位置信息。Based on the relative position relationship, the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the image, and the spatial position information of the three-dimensional points corresponding to the feature points in the image is obtained.
  4. 根据权利要求3所述的方法，其特征在于，所述确定所述历史图像与所述图像的相对位置关系，包括：The method according to claim 3, wherein the determining the relative positional relationship between the historical image and the image comprises:
    提取所述历史图像中的特征点，以及提取所述图像中的特征点，并进行特征点匹配确定出单应矩阵，以得到所述历史图像与所述图像的相对位置关系。Extract the feature points in the historical image, and extract the feature points in the image, and perform feature point matching to determine a homography matrix, so as to obtain the relative positional relationship between the historical image and the image.
  5. 根据权利要求4所述的方法，其特征在于，所述图像的特征点根据所述历史图像中的特征点的分布提取。The method according to claim 4, wherein the feature points of the image are extracted according to the distribution of the feature points in the historical image.
  6. 根据权利要求3所述的方法，其特征在于，所述基于所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点跟踪到所述图像，包括：The method according to claim 3, wherein the tracking of the feature points carrying the spatial position information of the three-dimensional points in the historical image to the image based on the relative position relationship comprises:
    通过所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点投影到所述图像；Projecting the feature points carrying the spatial position information of the three-dimensional points in the historical image to the image through the relative positional relationship;
    通过跟踪算法在所述图像中确定出与所述历史图像中携带三维点的空间位置信息的特征点对应的特征点。The feature points corresponding to the feature points carrying the spatial position information of the three-dimensional points in the historical image are determined in the image by the tracking algorithm.
  7. 根据权利要求1所述的方法，其特征在于，所述重投影误差阈值与所述拍摄设备相对于所述三维点的距离呈反相关。The method of claim 1, wherein the reprojection error threshold is inversely correlated with the distance of the photographing device relative to the three-dimensional point.
  8. 根据权利要求1所述的方法，其特征在于，所述方法还包括：The method according to claim 1, wherein the method further comprises:
    若所述像素位置误差不小于所述重投影误差阈值，则调整所述估计位姿，以使根据所述三维点的空间位置信息和调整后的估计位姿，将所述三维点重投影至所述图像得到的投影点中，预设比例的投影点与所述特征点的误差小于所述重投影误差阈值。If the pixel position error is not less than the reprojection error threshold, the estimated pose is adjusted so that the three-dimensional point is reprojected to a position according to the spatial position information of the three-dimensional point and the adjusted estimated pose Among the projection points obtained from the image, the error between the projection point of a preset ratio and the feature point is smaller than the re-projection error threshold.
  9. 根据权利要求8所述的方法，其特征在于，所述方法还包括：The method according to claim 8, wherein the method further comprises:
    剔除根据调整后的估计位姿重投影至所述图像得到的投影点与所述特征点的误差大于所述重投影误差阈值的三维点；Eliminating the three-dimensional point whose error between the projection point and the feature point obtained by re-projecting to the image according to the adjusted estimated pose is greater than the re-projection error threshold;
    根据调整后的估计位姿重新确定所述拍摄设备相对于剔除后剩余三维点的距离。The distance of the photographing device relative to the remaining three-dimensional point after culling is re-determined according to the adjusted estimated pose.
  10. 根据权利要求9所述的方法，其特征在于，若重新确定的所述拍摄设备相对于剔除后剩余三维点的距离小于预设距离阈值，则将所述调整后的估计位姿作为所述拍摄设备的目标位姿输出。The method according to claim 9, wherein if the re-determined distance of the shooting device relative to the remaining three-dimensional points after culling is smaller than a preset distance threshold, the adjusted estimated pose is used as the shooting The device's target pose output.
  11. 根据权利要求9所述的方法，其特征在于，若重新确定的所述拍摄设备相对于剔除后剩余三维点的距离不小于预设距离阈值，所述方法还包括以下至少一次迭代，直至重新确定的所述拍摄设备相对于剔除后剩余三维点的距离小于所述预设距离阈值：The method according to claim 9, wherein if the re-determined distance of the photographing device relative to the remaining three-dimensional points after culling is not less than a preset distance threshold, the method further comprises the following at least one iteration until the re-determination The distance of the photographing device relative to the remaining three-dimensional points after culling is less than the preset distance threshold:
    根据重新确定的所述拍摄设备相对于剔除后剩余三维点的距离调整重投影误差阈值；Adjust the reprojection error threshold according to the re-determined distance of the photographing device relative to the remaining three-dimensional points after culling;
    再次调整估计位姿，以使根据所述三维点的空间位置信息和再次调整后的估计位姿，将所述剔除后剩余三维点重投影至所述图像得到的投影点中，预设比例的投影点与所述特征点的误差小于调整后重投影误差阈值；Adjust the estimated pose again, so that according to the spatial position information of the three-dimensional point and the estimated pose after adjustment again, the remaining three-dimensional points after the culling are re-projected into the projection points obtained by the image, and the preset ratio is The error between the projection point and the feature point is less than the adjusted reprojection error threshold;
    剔除根据再次调整的估计位姿重投影至所述图像得到的投影点与所述特征点的误差大于调整后的重投影误差阈值的三维点；Eliminating the three-dimensional point whose error between the projection point and the feature point obtained by re-projecting the re-adjusted estimated pose to the image is greater than the adjusted re-projection error threshold;
    根据再次调整的估计位姿重新确定所述拍摄设备相对于剔除后剩余三维点的距离。The distance of the photographing device relative to the remaining three-dimensional point after culling is re-determined according to the readjusted estimated pose.
  12. 根据权利要求11所述的方法，其特征在于，所述再次调整估计位姿，包括：The method according to claim 11, wherein the adjusting the estimated pose again comprises:
    基于调整后的重投影误差阈值、所述剔除后剩余三维点及其对应的特征点重新计算所述拍摄设备的估计位姿。The estimated pose of the photographing device is recalculated based on the adjusted reprojection error threshold, the remaining three-dimensional points after the culling and their corresponding feature points.
  13. 根据权利要求1所述的方法，其特征在于，所述根据所述距离确定重投影误差阈值，包括：The method according to claim 1, wherein the determining a reprojection error threshold according to the distance comprises:
    根据所述拍摄设备相对于所述三维点的平均距离调整重投影误差阈值。The reprojection error threshold is adjusted according to the average distance of the photographing device relative to the three-dimensional point.
  14. 根据权利要求13所述的方法，其特征在于，所述根据所述拍摄设备相对于所述三维点的平均距离调整重投影误差阈值，包括：The method according to claim 13, wherein the adjusting the reprojection error threshold according to the average distance of the photographing device relative to the three-dimensional point comprises:
    将所述平均距离作为缩放倍率调整重投影误差阈值。The reprojection error threshold is adjusted using the average distance as the zoom factor.
  15. 根据权利要求1所述的方法，其特征在于，所述三维点的空间位置信息包括所述三维点的空间位置的可靠性信息；The method according to claim 1, wherein the spatial position information of the three-dimensional point includes reliability information of the spatial position of the three-dimensional point;
    所述方法还包括：The method also includes:
    基于所述三维点的空间位置的可靠性信息调整重投影误差阈值。The reprojection error threshold is adjusted based on the reliability information of the spatial position of the three-dimensional point.
  16. 根据权利要求15所述的方法，其特征在于，所述重投影误差阈值与所述三维点的空间位置的可靠性呈反相关。16. The method of claim 15, wherein the reprojection error threshold is inversely correlated with the reliability of the spatial position of the three-dimensional point.
  17. 一种空间信息的获取方法，其特征在于，包括：A method for obtaining spatial information, comprising:
    确定拍摄设备拍摄的历史图像与所述拍摄设备拍摄的当前图像的相对位置关系；所述历史图像与当前图像具有相似的场景，且所述历史图像中至少部分特征点携带三维点的空间位置信息；Determine the relative positional relationship between the historical image shot by the shooting device and the current image shot by the shooting device; the historical image and the current image have similar scenes, and at least some of the feature points in the historical image carry the spatial position information of the three-dimensional point ;
    基于所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点跟踪到所述当前图像，得到所述当前图像中特征点对应的三维点的空间位置信息。Based on the relative position relationship, the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the current image, and the spatial position information of the three-dimensional points corresponding to the feature points in the current image is obtained.
  18. 根据权利要求17所述的方法，其特征在于，所述确定拍摄设备拍摄的历史图像与所述拍摄设备拍摄的当前图像的相对位置关系，包括：The method according to claim 17, wherein the determining a relative positional relationship between a historical image captured by a photographing device and a current image captured by the photographing device comprises:
    提取所述历史图像中的特征点，以及提取所述当前图像中的特征点，并进行特征点匹配确定出单应矩阵，以得到所述历史图像与所述当前图像的相对位置关系。Extracting the feature points in the historical image and extracting the feature points in the current image, and performing feature point matching to determine a homography matrix, so as to obtain the relative positional relationship between the historical image and the current image.
  19. 根据权利要求18所述的方法，其特征在于，所述当前图像的特征点根据所述历史图像中的特征点的分布提取。The method according to claim 18, wherein the feature points of the current image are extracted according to the distribution of the feature points in the historical image.
  20. 根据权利要求17所述的方法，其特征在于，所述基于所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点跟踪到所述当前图像，包括：The method according to claim 17, wherein the tracking of the feature points carrying the spatial position information of the three-dimensional points in the historical image to the current image based on the relative position relationship comprises:
    通过所述相对位置关系将所述历史图像中携带三维点的空间信息的特征点投影到所述当前图像；Projecting the feature points carrying the spatial information of three-dimensional points in the historical image to the current image through the relative positional relationship;
    通过跟踪算法在所述当前图像中确定出与所述历史图像中携带三维点的空间位置信息的特征点对应的特征点。The feature points corresponding to the feature points carrying the spatial position information of the three-dimensional points in the historical image are determined in the current image by a tracking algorithm.
  21. 一种定位装置，其特征在于，包括：A positioning device, characterized in that it includes:
    处理器；processor;
    用于存储处理器可执行指令的存储器；memory for storing processor-executable instructions;
    其中，所述处理器被配置为：wherein the processor is configured to:
    获取拍摄设备拍摄图像时的估计位姿，和所述图像中特征点对应的三维点的空间位置信息；Obtain the estimated pose when the image is captured by the shooting device, and the spatial position information of the three-dimensional point corresponding to the feature point in the image;
    根据所述拍摄设备的估计位姿确定所述拍摄设备相对于所述三维点的距离；determining the distance of the photographing device relative to the three-dimensional point according to the estimated pose of the photographing device;
    根据所述距离确定重投影误差阈值；determining a reprojection error threshold according to the distance;
    根据所述三维点的空间位置信息和所述估计位姿，将所述三维点重投影至所述图像得到投影点，计算所述投影点与所述特征点的之间的像素位置误差；According to the spatial position information of the 3D point and the estimated pose, reproject the 3D point to the image to obtain a projection point, and calculate the pixel position error between the projection point and the feature point;
    若所述像素位置误差小于所述重投影误差阈值，则将所述估计位姿作为所述拍摄设备的目标位姿输出。If the pixel position error is smaller than the reprojection error threshold, the estimated pose is output as the target pose of the photographing device.
  22. 根据权利要求21所述的装置，其特征在于，所述处理器还被配置为：The apparatus of claim 21, wherein the processor is further configured to:
    基于相似场景匹配获取所述拍摄设备拍摄的与所述图像具有相似场景的历史图像，所述历史图像中至少部分特征点携带三维点的空间位置信息；Obtaining, based on similar scene matching, a historical image captured by the photographing device and having a similar scene to the image, at least some of the feature points in the historical image carry spatial position information of three-dimensional points;
    基于所述历史图像中携带三维点的空间位置信息的特征点确定所述图像中特征点对应的三维点的空间位置信息。The spatial position information of the three-dimensional point corresponding to the feature point in the image is determined based on the feature point carrying the spatial position information of the three-dimensional point in the historical image.
  23. 根据权利要求22所述的装置，其特征在于，所述处理器具体被配置为：The apparatus according to claim 22, wherein the processor is specifically configured to:
    确定所述历史图像与所述图像的相对位置关系；determining the relative positional relationship between the historical image and the image;
    基于所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点跟踪到所述图像，得到所述图像中特征点对应的三维点的空间位置信息。Based on the relative position relationship, the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the image, and the spatial position information of the three-dimensional points corresponding to the feature points in the image is obtained.
  24. 根据权利要求23所述的装置，其特征在于所述处理器具体被配置为：The apparatus of claim 23, wherein the processor is specifically configured to:
    提取所述历史图像中的特征点，以及提取所述图像中的特征点，并进行特征点匹配确定出单应矩阵，以得到所述历史图像与所述图像的相对位 置关系。Extract the feature points in the historical image, and extract the feature points in the image, and perform feature point matching to determine a homography matrix, so as to obtain the relative positional relationship between the historical image and the image.
  25. 根据权利要求24所述的装置，其特征在于，所述图像的特征点根据所述历史图像中的特征点的分布提取。The device according to claim 24, wherein the feature points of the image are extracted according to the distribution of the feature points in the historical image.
  26. 根据权利要求23所述的装置，其特征在于，所述处理器具体被配置为：The apparatus according to claim 23, wherein the processor is specifically configured to:
    通过所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点投影到所述图像；Projecting the feature points carrying the spatial position information of the three-dimensional points in the historical image to the image through the relative positional relationship;
    通过跟踪算法在所述图像中确定出与所述历史图像中携带三维点的空间位置信息的特征点对应的特征点。The feature points corresponding to the feature points carrying the spatial position information of the three-dimensional points in the historical image are determined in the image by the tracking algorithm.
  27. 根据权利要求21所述的装置，其特征在于，所述重投影误差阈值与所述拍摄设备相对于所述三维点的距离呈反相关。The apparatus of claim 21, wherein the reprojection error threshold is inversely correlated with the distance of the photographing device relative to the three-dimensional point.
  28. 根据权利要求21所述的装置，其特征在于，所述处理器还被配置为：The apparatus of claim 21, wherein the processor is further configured to:
    若所述像素位置误差不小于所述重投影误差阈值，则调整所述估计位姿，以使根据所述三维点的空间位置信息和调整后的估计位姿，将所述三维点重投影至所述图像得到的投影点中，预设比例的投影点与所述特征点的误差小于所述重投影误差阈值。If the pixel position error is not less than the reprojection error threshold, the estimated pose is adjusted so that the three-dimensional point is reprojected to a position according to the spatial position information of the three-dimensional point and the adjusted estimated pose Among the projection points obtained from the image, the error between the projection point of a preset ratio and the feature point is smaller than the re-projection error threshold.
  29. 根据权利要求28所述的装置，其特征在于，所述处理器还被配置为：The apparatus of claim 28, wherein the processor is further configured to:
    剔除根据调整后的估计位姿重投影至所述图像得到的投影点与所述特征点的误差大于所述重投影误差阈值的三维点；Eliminating the three-dimensional point whose error between the projection point and the feature point obtained by re-projecting to the image according to the adjusted estimated pose is greater than the re-projection error threshold;
    根据调整后的估计位姿重新确定所述拍摄设备相对于剔除后剩余三维点的距离。The distance of the photographing device relative to the remaining three-dimensional point after culling is re-determined according to the adjusted estimated pose.
  30. 根据权利要求29所述的装置，其特征在于，所述处理器还被配置为：The apparatus of claim 29, wherein the processor is further configured to:
    若重新确定的所述拍摄设备相对于剔除后剩余三维点的距离小于预设距离阈值，则将所述调整后的估计位姿作为所述拍摄设备的目标位姿输出。If the re-determined distance of the photographing device relative to the remaining three-dimensional points after culling is smaller than a preset distance threshold, the adjusted estimated pose is output as the target pose of the photographing device.
  31. 根据权利要求29所述的装置，其特征在于，所述处理器还被配置为：The apparatus of claim 29, wherein the processor is further configured to:
    若重新确定的所述拍摄设备相对于剔除后剩余三维点的距离不小于预设距离阈值，所述方法还包括以下至少一次迭代，直至重新确定的所述拍摄设备相对于剔除后剩余三维点的距离小于所述预设距离阈值：If the re-determined distance of the photographing device relative to the remaining 3D points after culling is not less than a preset distance threshold, the method further includes the following at least one iteration, until the re-determined distance of the photographing device relative to the remaining 3D points after culling is not less than a preset distance threshold. The distance is less than the preset distance threshold:
    根据重新确定的所述拍摄设备相对于剔除后剩余三维点的距离调整重投影误差阈值；Adjust the reprojection error threshold according to the re-determined distance of the photographing device relative to the remaining three-dimensional points after culling;
    再次调整所述估计位姿，以使根据所述三维点的空间位置信息和再次调整后的估计位姿，将所述剔除后剩余三维点重投影至所述图像得到的投影点中，预设比例的投影点与所述特征点的误差小于调整后重投影误差阈值；The estimated pose is adjusted again, so that according to the spatial position information of the three-dimensional point and the estimated pose after adjustment, the remaining three-dimensional points after the culling are re-projected into the projection points obtained by the image, and the preset The error between the proportional projection point and the feature point is less than the adjusted reprojection error threshold;
    剔除根据再次调整的估计位姿重投影至所述图像得到的投影点与所述特征点的误差大于调整后的重投影误差阈值的三维点；Eliminating the three-dimensional point whose error between the projection point and the feature point obtained by re-projecting the re-adjusted estimated pose to the image is greater than the adjusted re-projection error threshold;
    根据再次调整的估计位姿重新确定所述拍摄设备相对于剔除后剩余三维点的距离。The distance of the photographing device relative to the remaining three-dimensional point after culling is re-determined according to the readjusted estimated pose.
  32. 根据权利要求31所述的装置，其特征在于，所述处理器具体被配置为：The apparatus according to claim 31, wherein the processor is specifically configured to:
    基于调整后的重投影误差阈值、所述剔除后剩余三维点及其对应的特征点重新计算所述拍摄设备的估计位姿。The estimated pose of the photographing device is recalculated based on the adjusted reprojection error threshold, the remaining three-dimensional points after the culling and their corresponding feature points.
  33. 根据权利要求21所述的装置，其特征在于，所述处理器具体被配置为：The apparatus according to claim 21, wherein the processor is specifically configured to:
    根据所述拍摄设备相对于所述三维点的平均距离调整重投影误差阈值。The reprojection error threshold is adjusted according to the average distance of the photographing device relative to the three-dimensional point.
  34. 根据权利要求33所述的装置，其特征在于，所述处理器具体被配置为：The apparatus according to claim 33, wherein the processor is specifically configured to:
    将所述平均距离作为缩放倍率调整重投影误差阈值。The reprojection error threshold is adjusted using the average distance as the zoom factor.
  35. 根据权利要求21所述的装置，其特征在于，所述三维点的空间位 置信息包括所述三维点的空间位置的可靠性信息；The device according to claim 21, wherein the spatial position information of the three-dimensional point includes reliability information of the spatial position of the three-dimensional point;
    所述处理器还被配置为：The processor is also configured to:
    基于所述三维点的空间位置的可靠性信息调整重投影误差阈值。The reprojection error threshold is adjusted based on the reliability information of the spatial position of the three-dimensional point.
  36. 根据权利要求35所述的装置，其特征在于，所述重投影误差阈值与所述三维点的空间位置的可靠性呈反相关。The apparatus of claim 35, wherein the reprojection error threshold is inversely correlated with the reliability of the spatial position of the three-dimensional point.
  37. 一种空间信息的获取装置，其特征在于，包括：A device for obtaining spatial information, comprising:
    处理器；processor;
    用于存储处理器可执行指令的存储器；memory for storing processor-executable instructions;
    其中，所述处理器被配置为：wherein the processor is configured to:
    确定拍摄设备拍摄的历史图像与所述拍摄设备拍摄的当前图像的相对位置关系；所述历史图像与当前图像具有相似的场景，且所述历史图像中至少部分特征点携带三维点的空间位置信息；Determine the relative positional relationship between the historical image shot by the shooting device and the current image shot by the shooting device; the historical image and the current image have similar scenes, and at least some of the feature points in the historical image carry the spatial position information of the three-dimensional point ;
    基于所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点跟踪到所述当前图像，得到所述当前图像中特征点对应的三维点的空间位置信息。Based on the relative position relationship, the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the current image, and the spatial position information of the three-dimensional points corresponding to the feature points in the current image is obtained.
  38. 根据权利要求37所述的装置，其特征在于，所述处理器具体被配置为：The apparatus according to claim 37, wherein the processor is specifically configured to:
    提取所述历史图像中的特征点，以及提取所述当前图像中的特征点，并进行特征点匹配确定出单应矩阵，以得到所述历史图像与所述当前图像的相对位置关系。Extracting feature points in the historical image and extracting feature points in the current image, and performing feature point matching to determine a homography matrix, so as to obtain the relative positional relationship between the historical image and the current image.
  39. 根据权利要求38所述的装置，其特征在于，所述当前图像的特征点根据所述历史图像中的特征点的分布提取。The device according to claim 38, wherein the feature points of the current image are extracted according to the distribution of the feature points in the historical images.
  40. 根据权利要求37所述的装置，其特征在于，所述处理器具体被配置为：The apparatus of claim 37, wherein the processor is specifically configured to:
    通过所述相对位置关系将所述历史图像中携带三维点的空间信息的特征点投影到所述当前图像；Projecting the feature points carrying the spatial information of three-dimensional points in the historical image to the current image through the relative positional relationship;
    通过跟踪算法在所述当前图像中确定出与所述历史图像中携带三维点 的空间位置信息的特征点对应的特征点。The feature points corresponding to the feature points carrying the spatial position information of the three-dimensional points in the historical image are determined in the current image by the tracking algorithm.
  41. 一种拍摄设备，其特征在于，包括：A photographing device, comprising:
    处理器；processor;
    用于存储处理器可执行指令的存储器；memory for storing processor-executable instructions;
    其中，所述处理器被配置为：wherein the processor is configured to:
    获取拍摄设备拍摄图像时的估计位姿，和所述图像中特征点对应的三维点的空间位置信息；Obtain the estimated pose when the image is captured by the shooting device, and the spatial position information of the three-dimensional point corresponding to the feature point in the image;
    根据所述拍摄设备的估计位姿确定所述拍摄设备相对于所述三维点的距离；determining the distance of the photographing device relative to the three-dimensional point according to the estimated pose of the photographing device;
    根据所述距离确定重投影误差阈值；determining a reprojection error threshold according to the distance;
    根据所述三维点的空间位置信息和所述估计位姿，将所述三维点重投影至所述图像得到投影点，计算所述投影点与所述特征点的之间的像素位置误差；According to the spatial position information of the 3D point and the estimated pose, reproject the 3D point to the image to obtain a projection point, and calculate the pixel position error between the projection point and the feature point;
    若所述像素位置误差小于所述重投影误差阈值，则将所述估计位姿作为所述拍摄设备的目标位姿输出。If the pixel position error is smaller than the reprojection error threshold, the estimated pose is output as the target pose of the photographing device.
  42. 根据权利要求41所述的拍摄设备，其特征在于，所述处理器还被配置为：The photographing device of claim 41, wherein the processor is further configured to:
    基于相似场景匹配获取所述拍摄设备拍摄的与所述图像具有相似场景的历史图像，所述历史图像中至少部分特征点携带三维点的空间位置信息；Obtaining, based on similar scene matching, a historical image captured by the photographing device and having a similar scene to the image, at least some of the feature points in the historical image carry spatial position information of three-dimensional points;
    基于所述历史图像中携带三维点的空间位置信息的特征点确定所述图像中特征点对应的三维点的空间位置信息。The spatial position information of the three-dimensional point corresponding to the feature point in the image is determined based on the feature point carrying the spatial position information of the three-dimensional point in the historical image.
  43. 根据权利要求42所述的拍摄设备，其特征在于，所述处理器具体被配置为：The photographing device according to claim 42, wherein the processor is specifically configured to:
    确定所述历史图像与所述图像的相对位置关系；determining the relative positional relationship between the historical image and the image;
    基于所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点跟踪到所述图像，得到所述图像中特征点对应的三维点的空间位置信息。Based on the relative position relationship, the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the image, and the spatial position information of the three-dimensional points corresponding to the feature points in the image is obtained.
  44. 根据权利要求43所述的拍摄设备，其特征在于所述处理器具体被配置为：The photographing device according to claim 43, wherein the processor is specifically configured to:
    提取所述历史图像中的特征点，以及提取所述图像中的特征点，并进行特征点匹配确定出单应矩阵，以得到所述历史图像与所述图像的相对位置关系。Extract the feature points in the historical image, and extract the feature points in the image, and perform feature point matching to determine a homography matrix, so as to obtain the relative positional relationship between the historical image and the image.
  45. 根据权利要求44所述的拍摄设备，其特征在于，所述图像的特征点根据所述历史图像中的特征点的分布提取。The photographing device according to claim 44, wherein the feature points of the image are extracted according to the distribution of the feature points in the historical image.
  46. 根据权利要求43所述的拍摄设备，其特征在于，所述处理器具体被配置为：The photographing device according to claim 43, wherein the processor is specifically configured to:
    通过所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点投影到所述图像；Projecting the feature points carrying the spatial position information of the three-dimensional points in the historical image to the image through the relative positional relationship;
    通过跟踪算法在所述图像中确定出与所述历史图像中携带三维点的空间位置信息的特征点对应的特征点。The feature points corresponding to the feature points carrying the spatial position information of the three-dimensional points in the historical image are determined in the image by the tracking algorithm.
  47. 根据权利要求41所述的拍摄设备，其特征在于，所述重投影误差阈值与所述拍摄设备相对于所述三维点的距离呈反相关。The photographing device of claim 41, wherein the reprojection error threshold is inversely correlated with the distance of the photographing device relative to the three-dimensional point.
  48. 根据权利要求41所述的拍摄设备，其特征在于，所述处理器还被配置为：The photographing device of claim 41, wherein the processor is further configured to:
    若所述像素位置误差不小于所述重投影误差阈值，则调整所述估计位姿，以使根据所述三维点的空间位置信息和调整后的估计位姿，将所述三维点重投影至所述图像得到的投影点中，预设比例的投影点与所述特征点的误差小于所述重投影误差阈值。If the pixel position error is not less than the reprojection error threshold, the estimated pose is adjusted so that the three-dimensional point is reprojected to a position according to the spatial position information of the three-dimensional point and the adjusted estimated pose Among the projection points obtained from the image, the error between the projection point of a preset ratio and the feature point is smaller than the re-projection error threshold.
  49. 根据权利要求48所述的拍摄设备，其特征在于，所述处理器还被配置为：The photographing device of claim 48, wherein the processor is further configured to:
    剔除根据调整后的估计位姿重投影至所述图像得到的投影点与所述特征点的误差大于所述重投影误差阈值的三维点；Eliminating the three-dimensional point whose error between the projection point and the feature point obtained by re-projecting to the image according to the adjusted estimated pose is greater than the re-projection error threshold;
    根据调整后的估计位姿重新确定所述拍摄设备相对于剔除后剩余三维点的距离。The distance of the photographing device relative to the remaining three-dimensional point after culling is re-determined according to the adjusted estimated pose.
  50. 根据权利要求49所述的拍摄设备，其特征在于，所述处理器还被配置为：The photographing device of claim 49, wherein the processor is further configured to:
    若重新确定的所述拍摄设备相对于剔除后剩余三维点的距离小于预设距离阈值，则将所述调整后的估计位姿作为所述拍摄设备的目标位姿输出。If the re-determined distance of the photographing device relative to the remaining three-dimensional points after culling is smaller than a preset distance threshold, the adjusted estimated pose is output as the target pose of the photographing device.
  51. 根据权利要求49所述的拍摄设备，其特征在于，所述处理器还被配置为：The photographing device of claim 49, wherein the processor is further configured to:
    若重新确定的所述拍摄设备相对于剔除后剩余三维点的距离不小于预设距离阈值，所述方法还包括以下至少一次迭代，直至重新确定的所述拍摄设备相对于剔除后剩余三维点的距离小于所述预设距离阈值：If the re-determined distance of the photographing device relative to the remaining 3D points after culling is not less than a preset distance threshold, the method further includes the following at least one iteration, until the re-determined distance of the photographing device relative to the remaining 3D points after culling is not less than a preset distance threshold. The distance is less than the preset distance threshold:
    根据重新确定的所述拍摄设备相对于剔除后剩余三维点的距离调整重投影误差阈值；Adjust the reprojection error threshold according to the re-determined distance of the photographing device relative to the remaining three-dimensional points after culling;
    再次调整所述估计位姿，以使根据所述三维点的空间位置信息和再次调整后的估计位姿，将所述剔除后剩余三维点重投影至所述图像得到的投影点中，预设比例的投影点与所述特征点的误差小于调整后重投影误差阈值；The estimated pose is adjusted again, so that according to the spatial position information of the three-dimensional point and the estimated pose after adjustment, the remaining three-dimensional points after the culling are re-projected into the projection points obtained by the image, and the preset The error between the proportional projection point and the feature point is less than the adjusted reprojection error threshold;
    剔除根据再次调整的估计位姿重投影至所述图像得到的投影点与所述特征点的误差大于调整后的重投影误差阈值的三维点；Eliminating the three-dimensional point whose error between the projection point and the feature point obtained by re-projecting the re-adjusted estimated pose to the image is greater than the adjusted re-projection error threshold;
    根据再次调整的估计位姿重新确定所述拍摄设备相对于剔除后剩余三维点的距离。The distance of the photographing device relative to the remaining three-dimensional point after culling is re-determined according to the readjusted estimated pose.
  52. 根据权利要求51所述的拍摄设备，其特征在于，所述处理器具体被配置为：The photographing device according to claim 51, wherein the processor is specifically configured to:
    基于调整后的重投影误差阈值、所述剔除后剩余三维点及其对应的特征点重新计算所述拍摄设备的估计位姿。The estimated pose of the photographing device is recalculated based on the adjusted reprojection error threshold, the remaining three-dimensional points after the culling and their corresponding feature points.
  53. 根据权利要求51所述的拍摄设备，其特征在于，所述处理器具体被配置为：The photographing device according to claim 51, wherein the processor is specifically configured to:
    根据所述拍摄设备相对于所述三维点的平均距离调整重投影误差阈值。The reprojection error threshold is adjusted according to the average distance of the photographing device relative to the three-dimensional point.
  54. 根据权利要求53所述的拍摄设备，其特征在于，所述处理器具体被配置为：The photographing device according to claim 53, wherein the processor is specifically configured to:
    将所述平均距离作为缩放倍率调整重投影误差阈值。The reprojection error threshold is adjusted using the average distance as the zoom factor.
  55. 根据权利要求51所述的拍摄设备，其特征在于，所述三维点的空间位置信息包括所述三维点的空间位置的可靠性信息；The photographing device according to claim 51, wherein the spatial position information of the three-dimensional point includes reliability information of the spatial position of the three-dimensional point;
    所述处理器还被配置为：The processor is also configured to:
    基于所述三维点的空间位置的可靠性信息调整重投影误差阈值。The reprojection error threshold is adjusted based on the reliability information of the spatial position of the three-dimensional point.
  56. 根据权利要求55所述的拍摄设备，其特征在于，所述重投影误差阈值与所述三维点的空间位置的可靠性呈反相关。The photographing device according to claim 55, wherein the reprojection error threshold is inversely correlated with the reliability of the spatial position of the three-dimensional point.
  57. 一种拍摄设备，其特征在于，包括：A photographing device, comprising:
    处理器；processor;
    用于存储处理器可执行指令的存储器；memory for storing processor-executable instructions;
    其中，所述处理器被配置为：wherein the processor is configured to:
    确定拍摄设备拍摄的历史图像与所述拍摄设备拍摄的当前图像的相对位置关系；所述历史图像与当前图像具有相似的场景，且所述历史图像中至少部分特征点携带三维点的空间位置信息；Determine the relative positional relationship between the historical image shot by the shooting device and the current image shot by the shooting device; the historical image and the current image have similar scenes, and at least some of the feature points in the historical image carry the spatial position information of the three-dimensional point ;
    基于所述相对位置关系将所述历史图像中携带三维点的空间位置信息的特征点跟踪到所述当前图像，得到所述当前图像中特征点对应的三维点的空间位置信息。Based on the relative position relationship, the feature points carrying the spatial position information of the three-dimensional points in the historical image are tracked to the current image, and the spatial position information of the three-dimensional points corresponding to the feature points in the current image is obtained.
  58. 根据权利要求57所述的拍摄设备，其特征在于，所述处理器具体被配置为：The photographing device according to claim 57, wherein the processor is specifically configured to:
    提取所述历史图像中的特征点，以及提取所述当前图像中的特征点，并进行特征点匹配确定出单应矩阵，以得到所述历史图像与所述当前图像的相对位置关系。Extracting the feature points in the historical image and extracting the feature points in the current image, and performing feature point matching to determine a homography matrix, so as to obtain the relative positional relationship between the historical image and the current image.
  59. 根据权利要求58所述的拍摄设备，其特征在于，所述当前图像的特征点根据所述历史图像中的特征点的分布提取。The photographing device according to claim 58, wherein the feature points of the current image are extracted according to the distribution of the feature points in the historical images.
  60. 根据权利要求57所述的拍摄设备，其特征在于，所述处理器具体 被配置为：The photographing device according to claim 57, wherein the processor is specifically configured to:
    通过所述相对位置关系将所述历史图像中携带三维点的空间信息的特征点投影到所述当前图像；Projecting the feature point carrying the spatial information of the three-dimensional point in the historical image to the current image through the relative positional relationship;
    通过跟踪算法在所述当前图像中确定出与所述历史图像中携带三维点的空间位置信息的特征点对应的特征点。The feature points corresponding to the feature points carrying the spatial position information of the three-dimensional points in the historical image are determined in the current image by a tracking algorithm.
  61. 一种可移动平台，其特征在于，包括如权利要求41-60任一项所述的拍摄设备。A movable platform, characterized by comprising the photographing device according to any one of claims 41-60.
PCT/CN2021/070333 2021-01-05 2021-01-05 Positioning method and apparatus, spatial information acquisition method and apparatus, and photographing device WO2022147655A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/070333 WO2022147655A1 (en) 2021-01-05 2021-01-05 Positioning method and apparatus, spatial information acquisition method and apparatus, and photographing device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/070333 WO2022147655A1 (en) 2021-01-05 2021-01-05 Positioning method and apparatus, spatial information acquisition method and apparatus, and photographing device

Publications (1)

Publication Number Publication Date
WO2022147655A1 true WO2022147655A1 (en) 2022-07-14

Family

ID=82357076

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/070333 WO2022147655A1 (en) 2021-01-05 2021-01-05 Positioning method and apparatus, spatial information acquisition method and apparatus, and photographing device

Country Status (1)

Country Link
WO (1) WO2022147655A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115578432A (en) * 2022-09-30 2023-01-06 北京百度网讯科技有限公司 Image processing method, image processing device, electronic equipment and storage medium
CN115781092A (en) * 2023-02-08 2023-03-14 金成技术股份有限公司 Multi-angle auxiliary welding method for movable arm of excavator

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108682038A (en) * 2018-04-27 2018-10-19 腾讯科技（深圳）有限公司 Pose determines method, apparatus and storage medium
US20190206078A1 (en) * 2018-01-03 2019-07-04 Baidu Online Network Technology (Beijing) Co., Ltd . Method and device for determining pose of camera
CN111260725A (en) * 2020-01-15 2020-06-09 浙江大学 Dynamic environment-oriented wheel speed meter-assisted visual odometer method
CN111833403A (en) * 2020-07-27 2020-10-27 闪耀现实(无锡)科技有限公司 Method and apparatus for spatial localization

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190206078A1 (en) * 2018-01-03 2019-07-04 Baidu Online Network Technology (Beijing) Co., Ltd . Method and device for determining pose of camera
CN108682038A (en) * 2018-04-27 2018-10-19 腾讯科技（深圳）有限公司 Pose determines method, apparatus and storage medium
CN111260725A (en) * 2020-01-15 2020-06-09 浙江大学 Dynamic environment-oriented wheel speed meter-assisted visual odometer method
CN111833403A (en) * 2020-07-27 2020-10-27 闪耀现实(无锡)科技有限公司 Method and apparatus for spatial localization

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115578432A (en) * 2022-09-30 2023-01-06 北京百度网讯科技有限公司 Image processing method, image processing device, electronic equipment and storage medium
CN115781092A (en) * 2023-02-08 2023-03-14 金成技术股份有限公司 Multi-angle auxiliary welding method for movable arm of excavator

Similar Documents

Publication Publication Date Title
CN110322500B (en) Optimization method and device for instant positioning and map construction, medium and electronic equipment
CN111325796B (en) Method and apparatus for determining pose of vision equipment
US10984556B2 (en) Method and apparatus for calibrating relative parameters of collector, device and storage medium
CN107888828B (en) Space positioning method and device, electronic device, and storage medium
EP3627181A1 (en) Multi-sensor calibration method, multi-sensor calibration device, computer device, medium and vehicle
US9270891B2 (en) Estimation of panoramic camera orientation relative to a vehicle coordinate frame
US20200116498A1 (en) Visual assisted distance-based slam method and mobile robot using the same
CN112258567B (en) Visual positioning method and device for object grabbing point, storage medium and electronic equipment
US9466107B2 (en) Bundle adjustment based on image capture intervals
US8437501B1 (en) Using image and laser constraints to obtain consistent and improved pose estimates in vehicle pose databases
CN110766716A (en) Method and system for acquiring information of space unknown moving target
US8059154B1 (en) Systems and methods for automatic camera calibration
KR20170053007A (en) Method and apparatus for estimating pose
CN110349212B (en) Optimization method and device for instant positioning and map construction, medium and electronic equipment
WO2022147655A1 (en) Positioning method and apparatus, spatial information acquisition method and apparatus, and photographing device
CN110648363A (en) Camera posture determining method and device, storage medium and electronic equipment
CN107749069B (en) Image processing method, electronic device and image processing system
EP3633606A1 (en) Information processing device, information processing method, and program
JP2019032218A (en) Location information recording method and device
WO2022222291A1 (en) Optical axis calibration method and apparatus of optical axis detection system, terminal, system, and medium
EP3998582A1 (en) Three-dimensional model generation method and three-dimensional model generation device
CN117392241A (en) Sensor calibration method and device in automatic driving and electronic equipment
CN110706257B (en) Identification method of effective characteristic point pair, and camera state determination method and device
KR102295857B1 (en) Calibration Method for Real-Time Spherical 3D 360 Imaging and Apparatus Therefor
JP2009276073A (en) Plane estimating method, curved surface estimating method, and plane estimating device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21916713

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21916713

Country of ref document: EP

Kind code of ref document: A1