WO2020014864A1 - 位姿确定方法、设备、计算机可读存储介质 - Google Patents

位姿确定方法、设备、计算机可读存储介质 Download PDF

Info

Publication number
WO2020014864A1
WO2020014864A1 PCT/CN2018/095957 CN2018095957W WO2020014864A1 WO 2020014864 A1 WO2020014864 A1 WO 2020014864A1 CN 2018095957 W CN2018095957 W CN 2018095957W WO 2020014864 A1 WO2020014864 A1 WO 2020014864A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
image frame
pose
image
imaging device
Prior art date
Application number
PCT/CN2018/095957
Other languages
English (en)
French (fr)
Inventor
叶长春
苏坤岳
周游
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201880038859.8A priority Critical patent/CN110914867A/zh
Priority to PCT/CN2018/095957 priority patent/WO2020014864A1/zh
Publication of WO2020014864A1 publication Critical patent/WO2020014864A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras

Definitions

  • the present invention relates to the field of positioning technology, and in particular, to a pose determination method, a device, and a computer-readable storage medium.
  • Vision positioning technology can also ensure the stability of positioning in areas (indoors, high-rise buildings, etc.) without GPS (Global Positioning System). Therefore, vision positioning technology has been widely used in artificial intelligence.
  • a camera is installed on a movable platform such as a drone, a car, or a robot, and the computer analyzes the image captured by the camera to obtain the current position of the drone, car, or robot and draw a travel path.
  • the invention provides a pose determination method, a device, and a computer-readable storage medium, which can adaptively measure the pose and the environment and improve the positioning accuracy.
  • a posture determination method including:
  • the key frame sequence is selected from the original image sequence collected by the imaging device, the M is smaller than the N, and the M and the N are positive integers not less than 1.
  • an electronic device including: a memory and a processor;
  • the memory is used to store program code
  • the processor is configured to call the program code, and when the program code is executed, is configured to perform the following operations:
  • the key frame sequence is selected from the original image sequence collected by the imaging device, the M is smaller than the N, and the M and the N are positive integers not less than 1.
  • a computer-readable storage medium stores computer instructions.
  • the computer instructions When the computer instructions are executed, the bits described in the first aspect of the embodiments of the present invention are implemented. Posture determination method.
  • the image quality is poor, and only fewer key frames are used to determine the posture of the imaging device when the first image frame is collected, which can avoid The inter-frame pose error is accumulated and the error is amplified.
  • the accuracy is higher.
  • the environmental conditions are not worse than the preset imaging conditions, the image quality is better.
  • the pose error between frames is small. More frames are used to determine the pose when the imaging device collects the first image frame. Relatively fewer frames can improve the accuracy.
  • key frames with different frame numbers can be adaptively selected for measuring poses, thereby improving positioning accuracy.
  • FIG. 1 is a schematic flowchart of a pose determination method according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of selecting a key frame sequence according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram after performing edge detection on a first image frame according to an embodiment of the present invention.
  • FIG. 4 is a schematic flowchart of a pose determination method according to an embodiment of the present invention.
  • FIG. 5 is a structural block diagram of an electronic device according to an embodiment of the present invention.
  • first, second, third, etc. may be used in the present invention to describe various information, these information should not be limited to these terms. These terms are used to distinguish the same type of information from each other.
  • first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information.
  • word "if” can be interpreted as “at ", or "at !, or "in response to a determination”.
  • An embodiment of the present invention proposes a posture determination method, which can realize the real-time determination of the posture of the imaging device during the movement of the image using the images collected by the imaging device.
  • the execution subject of the pose determination method may be an electronic device, and the specific type of the electronic device is not limited, and the electronic device may be an imaging device but not limited to an imaging device.
  • the electronic device may be, for example, a device that is electrically or communicatively connected to the imaging device.
  • the imaging device in the embodiment of the present invention may include a device with an imaging function, such as a camera, a camera, and a terminal device with a camera (such as a mobile phone).
  • the imaging device can be mounted on a movable platform.
  • the imaging device can be directly mounted on a movable platform (for example, the movable platform can be a drone, a drone, an unmanned ship, a mobile robot, etc.), or The gimbal is mounted on a movable platform.
  • the imaging device is mounted on a movable platform, after the posture of the imaging device is determined, the posture of the movable platform having a determined relative relationship with the imaging device may be converted, or the imaging may be performed.
  • the pose of the device is approximately the pose of the movable platform.
  • the imaging device is not limited to this, and may also be a device such as VR / AR glasses, a mobile phone with a dual camera.
  • Drones are used in more and more fields. Human-machine positioning has put forward higher and higher requirements. In order to improve the positioning accuracy, more frames of image are usually used to determine the pose, but the image quality changes caused by the environment are ignored. Reduce positioning accuracy.
  • the camera used for positioning on the drone is a sensor that passively senses the surrounding environment. When the brightness is low at night or in a mine, the imaging quality will be greatly reduced. There are two solutions to this situation:
  • the first method is to increase the exposure time and gain of the image. This can make the camera obtain a brighter image without adding additional equipment. However, increasing the exposure time will cause motion blur in the image. Increasing the gain will introduce the image. Noise, and blur and noise have a greater impact on the positioning of the drone, so the exposure time and gain should also be limited to a specified threshold;
  • the second method is to add a supplementary light, which can illuminate the surrounding environment where the drone is located, but the power of the supplementary light carried by the drone is generally limited, and the spot formed by the supplementary light is from the center to the edge.
  • the brightness will gradually become darker, and this uneven brightness also has a greater impact on drone positioning.
  • the exposure time and gain are preferentially adjusted.
  • the fill light is turned on.
  • the image quality obtained by using the above methods of increasing the exposure time and gain and turning on the fill light is still not as bright as the environment. In this case, more frames of image are used for the bit. Attitude measurement still has the situation that the positioning accuracy is not high or even diverges, and cannot solve the problem of poor positioning accuracy.
  • the environmental conditions that cause poor positioning accuracy are not limited to brightness, but there are other environmental problems, such as the weak texture of the imaging object itself.
  • the reasons for the poor positioning accuracy caused by using more frames of image for pose measurement can include the following:
  • the image quality is not high and the accuracy of image feature tracking is not high.
  • the position of a feature point on multiple frames of image is relatively fast;
  • the environmental conditions of the imaging device when acquiring the image frame are first detected.
  • the image quality is relatively poor.
  • the pose can be adaptively adjusted according to the environment, so that no matter how the environment changes, it has a high position and positioning accuracy.
  • a pose determination method may include the following steps:
  • S200 When the environmental conditions are worse than the preset imaging conditions, select M frame key frames from the key frame sequence, and determine the posture of the imaging device when the imaging device collects the first image frame according to the M frame key frames and the first image frame;
  • S300 When the environmental conditions are not inferior to the preset imaging conditions, select N frames of key frames from the key frame sequence, and determine the pose of the imaging device when the imaging device collects the first image frames according to the N frames of key frames and the first image frame;
  • the key frame sequence is selected from the original image sequence collected by the imaging device, and M is less than N, and M and N are positive integers not less than 1.
  • the execution subject of the method may be an electronic device, and further may be a processor of the electronic device, wherein the processor may be one or more, and the processor may be a general-purpose processor or a special-purpose processor. .
  • the electronic device may be the imaging device itself, or be relatively fixed to the imaging device, or be connected to the imaging device, or include the imaging device.
  • the electronic device may be a movable platform equipped with the imaging device.
  • the electronic device is the imaging device itself Expand description.
  • the imaging device can acquire an image, and the processor can acquire the image.
  • the method may further include a step of acquiring a first image frame.
  • acquiring the first image frame is acquiring the first image frame.
  • the acquiring the first image frame may be acquiring the first image frame from the imaging device.
  • the first image frame is preferably a frame of image frames currently acquired by the imaging device, and of course, it may also be a frame of image frames previously acquired, which is not limited in particular.
  • the key frame sequence in the embodiment of the present invention is selected from the original image sequence collected by the imaging device, and can be executed by the processor of the imaging device.
  • selecting the key frame sequence from the original image sequence collected by the imaging device may include, for example, the following steps:
  • the original image can be added to the key frame sequence as a key frame.
  • the above-mentioned determination of the pose relationship between the acquired original image and the latest key frame is only one way to determine the key frame. In actual applications, it can also compare the acquired original image with respect to any previously determined key frame.
  • the pose relationship of the frames is not limited, as long as the pose relationship between two adjacent image frames in the obtained key frame sequence satisfies the preset conditions that the displacement distance is greater than the displacement threshold and the rotation angle is less than the angle threshold. .
  • the first key frame in the key frame sequence may be a designated original image.
  • the currently collected original image can be obtained as the first frame of the key frame sequence, and subsequent captured original images can be compared with the first frame to determine whether it is a key frame.
  • the subsequently collected original image can be compared with the second frame key frame, and so on, so that the original image can be compared with the latest key frame.
  • the first frame should have recognizable feature points.
  • the imaging device collects the original image. It is assumed that the imaging device is exposed at a frequency of 20 Hz and the time is fixed. Therefore, an original image is obtained every 50 ms, and the obtained original image sequence is a1, a2, a3, a4 ... an, the original image a1 is designated as the first frame of the key frame sequence, and the original image a2 and key frame a1 and the original image a3 and key frame a1 do not satisfy the above-mentioned preset conditions, so the original images a2 and a3 are not used as key frames.
  • the above-mentioned preset condition is satisfied between the original image a4 and the key frame a1, so the original image a4 can be used as a key frame.
  • the subsequent key frame an can be determined according to the above-mentioned comparison process with the latest key frame. It can be understood that as the acquisition progresses, the key frame sequence can be updated, that is, the number of frames in the key frame sequence is maintained, and the old and impossible to use key frames are removed from the key frame sequence by using the first-in, first-out principle.
  • the displacement of adjacent frames in the key frame sequence is greater than the displacement threshold, and the rotation angle of the adjacent frames is less than the angle threshold.
  • the poses between key frames include a Rotation relationship and a Translation relationship.
  • Rotation is expressed in Euler angles
  • Translation is expressed in translation in three axes:
  • ⁇ th is the angle threshold and d th is the displacement threshold.
  • the problems are as follows: first, the amount of calculation is large, and second, the possibility of errors is high, but the correct results will be biased.
  • the problems of large calculation amount and high possibility of errors can be solved.
  • the determining the pose relationship of the acquired original image relative to the latest key frame may include: determining the pose relationship of the original image relative to the latest key frame by using a visual mileage calculation method.
  • step S100 an environmental condition when the imaging device collects the first image frame is detected.
  • the processor of the imaging device can detect environmental conditions when the imaging device collects the first image frame.
  • the timing of detecting the environmental conditions may be when the imaging device collects the first image frame, or at any time after the acquisition (in this process, it is necessary to record the environmental conditions when the first image frame is collected), For example, the detection is performed when the pose and time at which the first image frame is collected needs to be determined, or it may be detected within a period of time after the collection.
  • the method for detecting the environmental conditions is also not limited.
  • the environmental conditions can be determined by detecting the first image frame itself, or the environmental conditions can be determined by detecting the state of the imaging device in response to the environment, or the environmental conditions can be determined directly by detecting the environment. Environmental conditions, etc.
  • the first image frame may be an image frame currently acquired by the imaging device, or may be an image frame acquired at another time, depending on the acquisition time at which the pose is determined.
  • step S200 when the environmental conditions are inferior to the preset imaging conditions, a M frame key frame is selected from the key frame sequence, and the posture of the imaging device when acquiring the first image frame is determined according to the M frame key frame and the first image frame. .
  • step S300 when the environmental conditions are not inferior to the preset imaging conditions, select N frames of key frames from the key frame sequence, and determine the pose of the imaging device when the imaging device collects the first image frames according to the N frame key frames and the first image frame. .
  • steps S200 and S300 does not have a sequential order, but is executed by the processor according to the conditions in which the environmental conditions are met.
  • the detected environmental conditions may be one or more than two, and the preset imaging conditions are judgment standards corresponding to the environmental conditions.
  • the detected environmental condition may reflect the image quality of the first image frame. For example, when the environmental condition reflects that the image quality of the first image frame is poor, it is determined that the environmental condition is inferior to the preset imaging condition, and step S200 is performed; and the environmental condition reflects When the image quality of the first image frame is good, it is determined that the environmental conditions are not inferior to the preset imaging conditions, and step S300 is performed.
  • M is less than N
  • M and N are positive integers not less than 1. Because the image quality is poor when the environmental conditions are worse than the preset imaging conditions, only M-frame key frames are used to determine the pose of the imaging device when it collects the first image frame, which can avoid the accumulation of inter-frame pose errors resulting in error amplification. At this time, compared with the use of more key frames to determine the pose, the accuracy is higher; and because the environmental conditions are not worse than the preset imaging conditions, the image quality is better, and the pose error between frames is smaller.
  • the use of N-frame key frames to determine the pose of the imaging device when acquiring the first image frame can improve accuracy relative to fewer frames. When the environmental conditions are inferior to the preset imaging conditions and not worse than the preset imaging conditions, a key frame with a different number of frames can be adaptively selected to measure the posture, thereby improving the positioning accuracy.
  • M is not less than 1, and N is not less than 5.
  • M is not less than 1, and N is not less than 5
  • M is not more than 4, for example, M may be 1, 2, 3, or 4.
  • M may be 1, 2, 3, or 4.
  • the selection of such frames can obtain better pose results under different environmental conditions. More preferably, M is 1 and N is 5. Of course, this is not a limitation.
  • the N used may be different, and the M used may be different, that is, N and M may have Variability and randomness can be variable.
  • determining the pose of the imaging device when acquiring the first image frame according to the M frame key frame and the first image frame may include the following steps:
  • S201 Determine the two-dimensional information of the first image of the feature points in the M-frame key frame that match the first image frame and the three-dimensional information of the feature points in the M-frame key frame;
  • S202 Determine the pose of the imaging device when collecting the first image frame by using the two-dimensional information, the three-dimensional information, and the first rotation relationship of the first image.
  • the first rotation relationship is a second image frame in the first image frame and the M frame key frame. Rotation relationship between.
  • the processor performs feature point matching on the M-frame key frame and the first image frame, and may determine a matching feature point between the M-frame key frame and the first image frame.
  • the feature point may be a point in the image where the tracked target object is imaged.
  • the processor performs feature point matching on the M-frame key frame and the first image frame to determine the feature points in the M-frame key frame and the first image frame, which may specifically include: performing a feature point tracking algorithm on the M-frame key frame and the first image frame Feature point matching is performed to determine the matched feature points in the M frame key frame and the first image frame.
  • Feature point tracking algorithms include, for example, KLT (Kanade-Lucas-Tomasi Tracking) algorithm. According to the position of feature points on one frame of image, find the position of feature points on another frame of image. Of course, there are other feature point tracking algorithms. Not specific. It can be understood that the method of matching feature points on the M-frame key frames by the processor to determine the feature points in the M-frame key frames can of course be determined in other ways, and is not limited to the KLT algorithm.
  • the coordinate position of the feature point on the corresponding image may be used as the two-dimensional information of the first image of the corresponding feature point.
  • the two-dimensional information of the first image is information that can be determined through the M frame key frame and the first image frame itself.
  • the three-dimensional information of the feature points in the M-frame key frame that matches the first image frame can be determined by, for example, a binocular vision algorithm. For example, each key frame in the M-frame key frame and another corresponding image acquired at the same time are used to calculate. For the method, refer to the existing binocular vision algorithm, which will not be repeated here.
  • the three-dimensional information of the feature points matching the first image frame in the M frame key frames can also be calculated using multi-frame monocular images. limit.
  • the 3D information of the feature points in the M frame key frame that matches the first image frame is preferably position information in the world coordinate system.
  • the 3D information can also be position information in other coordinate systems, such as the camera coordinate system. It can be converted by the coordinate conversion relationship between the two coordinate systems, so it is not limited.
  • step S202 the processor uses the two-dimensional information, the three-dimensional information, and the first rotation relationship of the first image to determine the pose when the imaging device collects the first image frame, and the first rotation relationship is the first image frame and the Rotation relationship between second image frames in M-frame key frames.
  • the processor performs pose measurement on the two-dimensional information, the three-dimensional information, and the first rotation relationship of the first image to determine the pose of the imaging device when collecting the first image frame.
  • the method of pose measurement is not limited, as long as it can use this All three types of input information can be used to determine the pose.
  • the second image frame refers to any one of the M frame key frames, and can be used to determine the rotation relationship with the first image frame.
  • the second image frame may be a newly added frame among the M frame key frames.
  • the rotation relationship between the first image frame and the second image frame is the rotation relationship between the imaging device when acquiring the first image frame and the second image frame.
  • the first rotational relationship may include determination using an inertial measurement unit (IMU).
  • IMU inertial measurement unit
  • the inertial measurement unit can be relatively fixed to the imaging device, and the first rotation relationship can be determined by the rotation data in the pose data measured by the IMU. Of course, it can also be determined by the rotation data in the pose data measured by the IMU after certain data processing. .
  • the timing of the IMU measurement is at the collection time of the first image frame and the second image frame, and the rotation relationship between the first image frame and the second image frame can be obtained through IMU integration.
  • the first rotation relationship may further include a rotation relationship between the imaging device when acquiring the first image frame and the second image frame according to the movable platform, and the imaging device and the movable
  • the relative relationship between the platforms is determined.
  • the relative relationship between the imaging device and the movable platform is preferably constant, and of course, it can also be variable (for example, the relative relationship can be optimized as the usage time increases).
  • the imaging is determined.
  • the device collects the pose during the first image frame, it only needs to estimate the displacement in the pose, so that the estimated degrees of freedom are only three and linear.
  • the attitude is Rotation relationship.
  • step S202 using the two-dimensional information, the three-dimensional information, and the first rotation relationship of the first image to determine the pose of the imaging device when collecting the first image frame may include the following steps:
  • the posture of the imaging device when acquiring the first image frame is determined according to the first displacement relationship and the first rotation relationship.
  • the two-dimensional information, the three-dimensional information, and the first rotation relationship of the first image are all used as inputs for posture calculation to obtain a first displacement relationship between the first image frame and the second image frame.
  • the posture of the imaging device when acquiring the first image frame is determined according to the first rotation relationship and the calculated first displacement relationship.
  • the first rotation relationship is an estimated value. Since the number of key frames is small, the first rotation relationship determined by the IMU may be directly used as a trust value, and the first displacement relationship may be calculated accordingly.
  • the first displacement relationship between the first image frame and the second image frame is a displacement relationship between the imaging device when acquiring the first image frame and when acquiring the second image frame.
  • the first displacement relationship may include determination using a perspective n-point positioning PnP algorithm.
  • the PnP algorithm uses a series of three-dimensional position points (three-dimensional information) of the world coordinate system and two-dimensional position points (two-dimensional information of the first image) of the corresponding pixel coordinate system in the image to estimate the camera pose, which is the required posture relationship. R1 and T1.
  • R1 can be measured by the IMU, that is, the first rotation relationship described in this embodiment, and T1, which is obtained by calculation, is the first displacement relationship.
  • T1 which is obtained by calculation, is the first displacement relationship.
  • the relationship between a key frame of the M frame key frame and the first image frame can be calculated by using the above PnP algorithm.
  • the key frames of the M frame can be calculated separately.
  • the posture relationship of each frame determines the posture when the imaging device collects the first image frame in a posture accumulation manner.
  • these feature points can be used to calculate the corresponding first displacement relationship, and finally all the first The displacement relationship is used for fusion calculation, and the result of the fusion calculation is used as the first displacement relationship between a certain key frame of the M frame key frame and the first image frame. For example, it can be averaged or weighted average. .
  • step S202 using the two-dimensional information, the three-dimensional information, and the first rotation relationship of the first image to determine the pose when the imaging device collects the first image frame may include the following steps:
  • the posture of the imaging device when acquiring the first image frame is determined according to the first displacement relationship and the first rotation relationship.
  • the first displacement relationship in this embodiment is not used as an input during posture measurement, and only the two-dimensional information and three-dimensional information of the first image are calculated to determine the first displacement relationship.
  • the first displacement relationship may also include determining using a perspective n-point positioning PnP algorithm.
  • multiple sets of feature point information can be used to solve the problem.
  • P3P four sets of feature point information are used (in the image, the four feature points are not coplanar), and three sets of feature point information are used to solve multiple solutions. 4 sets of feature point information determine the optimal solution among them; for another example, EPnP can use 3 or more sets of feature point information to solve.
  • the feature point information is two-dimensional information and three-dimensional information of the first image of the feature points. The similarities between this embodiment and the previous embodiment will not be repeated here.
  • the M-frame key frame when used to determine the posture of the imaging device when acquiring the first image frame, it can be obtained by using the first rotation relationship, the first displacement relationship, and the posture when the second image frame is acquired.
  • determining the pose of the imaging device when acquiring the first image frame according to the N frame key frames and the first image frame may include the following steps:
  • S301 Determine two-dimensional information of a second image of feature points in a first image frame that match key frames of N frames;
  • S302 Determine the pose of the imaging device when collecting the first image frame by using the two-dimensional information of the second image and the estimated pose.
  • the estimated pose is between the first image frame and the third image frame in the N-frame key frame. Estimated pose.
  • the processor determines the two-dimensional information of the second image of the feature points in the first image frame that match the key frames of the N frames.
  • the way of matching the feature points can also be determined by the feature point tracking algorithm, for example, including KLT (Kanade-Lucas-Tomasi Tracking) algorithm, of course, there are other feature point tracking algorithms, which are not limited.
  • KLT Kanade-Lucas-Tomasi Tracking
  • the processor performs feature point matching on the first image frame and the N-frame key frames to determine the feature points in the first image frame.
  • the feature points can also be determined in other ways, and is not limited to the angular KLT algorithm.
  • the two-dimensional information of the second image is the coordinates of the feature point in the first image frame, which can be directly determined by using the first image frame.
  • the estimated pose includes an estimated displacement relationship and an estimated rotation relationship between the first image frame and the third image frame, that is, when the imaging device acquires the first image frame and when it acquires the third image frame Between the estimated displacement relationship and the estimated rotation relationship.
  • the estimated pose may include determination using an inertial measurement unit (IMU).
  • IMU inertial measurement unit
  • the IMU can be relatively fixed to the imaging device, and the estimated displacement relationship and estimated rotation relationship can be determined by the pose data measured by the IMU, but of course, it can also be determined by the data processed by the IMU through certain data processing.
  • the IMU measures the poses at the first image frame acquisition time and the third image frame acquisition time, respectively, and obtains the estimated displacement relationship and the estimated rotation relationship between the first image frame and the third image frame through IMU integration.
  • the processor uses the two-dimensional information of the second image and the estimated pose to determine the pose when the imaging device collects the first image frame. Because the estimated pose is an estimated value, such as the pose data determined by the IMU, when the image quality is high, it can be used as an estimate input to update the pose, but not directly as the pose or pose Part of it to improve the accuracy of poses.
  • the estimated pose is an estimated value, such as the pose data determined by the IMU, when the image quality is high, it can be used as an estimate input to update the pose, but not directly as the pose or pose Part of it to improve the accuracy of poses.
  • using the two-dimensional information of the second image and the estimated pose to determine the pose when the imaging device collects the first image frame includes:
  • the posture of the imaging device when acquiring the first image frame is determined according to the optimized relative posture relationship.
  • the relative pose relationship includes optimization using a filtering method.
  • the filtering method can optimize the roughly estimated value to obtain a more accurate value.
  • the pose of the imaging device corresponding to the key frames of the N frames is the pose optimized by the filtering method, and can be used as a more accurate pose to calculate the accurate pose during subsequent image frame acquisition.
  • the poses used are also optimized after the PnP algorithm.
  • the filtering method includes a Kalman filtering method, which may specifically be, for example, a MSCKF (Multi-State-Constraint-Kalman-Filter, Multi-State Constrained Kalman Filter) method, of course, it is not limited to this, and may be other EKF (Extended Kalman filter) method.
  • a MSCKF Multi-State-Constraint-Kalman-Filter, Multi-State Constrained Kalman Filter
  • EKF Extended Kalman filter
  • Optimization using filtering methods includes, for example:
  • the key frames of N frames are K-4, K-3, K-2, and K-1 frames
  • the first image frame is K-th frame.
  • the two-to-two pose relationship between the K-4 frame, the K-3 frame, the K-2 frame, and the K-1 frame has been optimized.
  • any one of the K-4th, K-3, K-2, and K-1 frames can be used as the third image frame.
  • the two-dimensional information of the second image of the feature points in the K-th frame that matches the key frames in the N-frame and the estimated pose of the K-1 frame are input to the Kalman filter (because of the K-1 frame and the K-4 frames, K-3 frames, and K-2 frames have known pose relations.
  • the K-frame and The estimated poses between frames K-4, K-3, and K-2 are also known), so after the Kalman filter prediction and update steps, the optimized K-frame and The relative pose relationship between the K-1 frames, and the pose of the K-1 frame can be determined through the pose of the K-1 frame, that is, the pose when the imaging device collects the first image frame.
  • the relative pose relationship between the K-1 frame, the K-4 frame, the K-3 frame, the K-2 frame, and the pose of the K-4 frame can be used to determine The pose of the Kth frame, that is, the pose when the imaging device collects the first image frame.
  • the third image frame is any one of the N frame key frames, and can be used to determine the rotation relationship and the displacement relationship with the first image frame.
  • the third image frame may be a newly added frame among the N frame key frames.
  • the pose involved between the first image frame and the third image frame refers to a displacement relationship and a rotation relationship between the imaging device when acquiring the first image frame and when acquiring the third image frame.
  • the environmental conditions are worse than the preset imaging conditions and not worse than the preset imaging conditions, not only a different number of key frames are selected for processing, but on this basis, fewer Different information in the M-frame key frames and the more N-frame key frames are used as the pose calculation information, and the different information is calculated in different pose determination manners when the imaging device collects the first image frame, It can further improve the positioning accuracy under corresponding environmental conditions.
  • the pose when the imaging device collects the first image frame is a relative rotation relationship and a relative displacement relationship between the first image frame and other image frames (the aforementioned second image frame or the third image frame). Positioning information of other image frames and a posture of the imaging device when acquiring the first image frame, and determining positioning information of the imaging device when acquiring the first image frame.
  • the first image frame is an image frame acquired by the imaging device at the current moment, and the real-time posture of the imaging device is determined for the first image frame to ensure the real-timeness of the positioning information.
  • the selecting M frame key frames from the key frame sequence includes: selecting the newly added M frame key frames from the key frame sequence.
  • the selecting N frame key frames from the key frame sequence includes: selecting the newly added N frame key frames from the key frame sequence.
  • the pose when the imaging device collects the first image frame may include a position (for example, in a world coordinate system) and a posture of the imaging device, and may be specifically determined according to requirements.
  • the environmental conditions include at least one of an ambient light intensity and an environmental texture.
  • the pose determination method may include: if the ambient light level is lower than a preset light level, determining that the environmental condition is worse than a preset imaging condition. Ambient illumination is lower than the preset illumination, indicating that the ambient illumination is too low. At this time, the captured image will have poor quality. Therefore, it is determined that the environmental conditions are inferior to the preset imaging conditions. M-frame key frames are selected for pose processing On the other hand, it means that the ambient illumination is appropriate and the imaging quality is good. N key frames can be selected for pose processing.
  • the case where the ambient light intensity is lower than the preset light intensity includes at least one of the following:
  • Both the exposure time and the gain at the collection moment of the first image frame reach a specified threshold
  • the ambient brightness at the collection time of the first image frame is lower than a specified brightness threshold.
  • the fill light may be on an imaging device.
  • the imaging device is mounted on a movable platform, such as when mounted on a drone
  • the fill light may be mounted on the drone at the same time as the imaging device, or it may be a cloud mounted on the drone at the same time as the imaging device.
  • the platform can also be mounted on a different PTZ from the drone separately from the imaging equipment, which is not limited.
  • the fill light can be controlled by an imaging device or a movable platform. For example, it can be determined whether the fill light is turned on by detecting a control signal that controls the opening and closing of the fill light at the acquisition time of the first image frame.
  • the imaging device if the supplementary light is turned on at the collection time of the first image frame, or the exposure time and gain of the collection time of the first image frame reach the specified threshold, it means that the imaging device has sensed that the ambient light level is too low and turned on. Fill light or increase the exposure time and increase the gain can not completely improve the image quality, so at this time still need to select M frame key frames for pose processing.
  • the ambient brightness at the collection time of the first image frame can be detected by a brightness sensor, and the detected brightness value is sent to the imaging device for comparison to determine whether the ambient brightness at the collection time of the first image frame is lower than Specify the brightness threshold.
  • the above situations are only a few types of ambient light levels that are lower than the preset light levels, and can also be determined by detecting other information related to the ambient light levels.
  • the pose determination method may include: if the strength of the environment texture is lower than a preset texture intensity, determining that the environment condition is inferior to the preset imaging condition. The intensity of the environment texture is lower than the preset texture intensity, indicating that the texture of the imaged object is too weak. At this time, the captured image will have poor quality. Therefore, it is determined that the environmental conditions are worse than the preset imaging conditions. Perform pose processing; otherwise, it means that the ambient illumination is appropriate and the imaging quality is good. You can choose N-frame key frames for pose processing.
  • the case where the strength of the environment texture is lower than the preset texture strength includes at least one of the following:
  • the proportion of the size of the weakly-textured connected domain in the first image frame is greater than a specified proportion.
  • the texture information can be determined by an edge detection algorithm.
  • the texture information of interest When the texture information of interest is not detected (the texture information of interest satisfies the range of texture information determined according to needs, for example, it can have sufficient obvious texture without Exemplary objects with sufficient obvious texture may include monochrome walls, smooth glass, and / or the like), indicating that the image quality is poor, and it is determined that the environmental conditions are worse than the preset imaging conditions.
  • Feature points can be determined through feature recognition. Of course, feature points can be detected on the basis of detecting texture information of interest. When the detectable feature points are less than a specified number, it indicates that the image quality is poor and the environment is determined. Conditions are worse than preset imaging conditions.
  • the proportion of the weakly-textured connected domain in the first image frame is larger than the specified proportion, it indicates that the weakly-textured connected domain is too large, and the texture information corresponding to the first image frame is too small, which further indicates that the image quality is poor, and it can be determined
  • the environmental conditions are worse than the preset imaging conditions.
  • the weakly-textured connected region includes being determined using an edge detection algorithm.
  • the edge detection algorithm includes, for example, Sobel operator, Canny operator, and of course, it is not limited to this. Sobel operator actually obtains the gradient in the horizontal and vertical directions of the first image frame, respectively.
  • the image of the first image frame after edge detection is shown, and obvious edges are detected. Based on this, the connected domain can be detected, and the block area can be filled using the Flood fill algorithm.
  • These blocks Regions are all potentially weakly textured regions. The proportion of the block region in the image is calculated one by one, and the largest is selected for comparison with the specified proportion. When it is larger, the block region is determined to be a weakly-textured connected region.
  • the way to determine whether it is worse than or not worse than the preset imaging conditions may include:
  • the ambient light intensity is lower than the preset light intensity, or the strength of the ambient texture is lower than the preset texture intensity, it is determined that the environmental conditions are inferior to the preset imaging conditions; if the ambient light intensity is not lower than the preset Illumination, and the strength of the environment texture is not lower than the preset texture intensity, it is determined that the environment condition is not inferior to the preset imaging condition.
  • the comparison value of the ambient light intensity and the preset light intensity, and the comparison value of the environmental texture strength and the preset texture intensity can be performed by weighted sum or average, and the operation result value can be compared with the preset comparison value.
  • the comparison is performed, and when the operation result value is lower than the preset comparison value, it is determined that the environmental condition is inferior to the preset imaging condition, otherwise it is determined that the environmental condition is not inferior to the preset imaging condition.
  • the ambient light intensity is lower than the preset light intensity and the environmental texture strength is lower than the preset texture intensity, it is determined that the environmental condition is inferior to the preset imaging condition; if the ambient light intensity is not lower than the preset Illumination, or the strength of the environment texture is not lower than the preset texture intensity, it is determined that the environment condition is not inferior to the preset imaging condition.
  • the specific manner is not limited to the above two manners.
  • the detection of the strength of the environment texture below the preset texture intensity and the detection of the ambient light intensity below the preset light intensity can also be the cases described above, which will not be repeated here.
  • the pose determination method may further include the following steps:
  • S400 Control the imaging device and / or the movable platform equipped with the imaging device according to the posture of the imaging device when acquiring the first image frame.
  • the posture of the imaging device can be further adjusted according to the posture to meet the shooting requirements of other different postures.
  • other control operations can also be performed.
  • the imaging device can be turned off to To achieve the purpose of energy saving.
  • the visual positioning algorithm can be effectively solved Limitations prevent unsafe factors caused by outputting incorrect information in these special scenarios.
  • the algorithm transitions at different times in the environment can enhance the reliability and robustness of the drone's overall system. It is beneficial to achieve stable hovering and course planning of the drone, and can also maintain the stability of the drone in areas without GPS (such as indoors and high-rise buildings).
  • the posture of the first image frame collected by the imaging device can not only facilitate the control of the imaging device, but also be beneficial to the movable platform (the posture of the imaging device and the posture of the movable platform) equipped with the imaging device.
  • the control can be obtained through the corresponding relationship conversion).
  • an electronic device 100 includes a memory 101 and a processor 102 (such as one or more processors).
  • the specific type of the electronic device is not limited, and the electronic device may be an imaging device but is not limited to an imaging device.
  • the electronic device may be, for example, a device that is electrically or communicatively connected to the imaging device.
  • the device is not an imaging device, after the image is acquired by the imaging device, the image acquired by the imaging device can be acquired, and then the corresponding method can be executed.
  • the memory 101 is configured to store program code; the processor 102 is configured to call the program code, and when the program code is executed, is configured to perform the following operations:
  • an M frame key frame is selected from a key frame sequence, and the time when the imaging device acquires the first image frame is determined according to the M frame key frame and the first image frame.
  • the key frame sequence is selected from the original image sequence collected by the imaging device, the M is smaller than the N, and the M and the N are positive integers not less than 1.
  • the determining, by the processor according to the M-frame key frame and the first image frame, the posture when the imaging device collects the first image frame is specifically used to:
  • Determining the pose of the imaging device when acquiring the first image frame by using the two-dimensional information of the first image, the three-dimensional information, and a first rotation relationship, where the first rotation relationship is the first image frame and A rotation relationship between second image frames in the M-frame key frames.
  • the processor uses the two-dimensional information of the first image, the three-dimensional information, and a rotation relationship to determine a posture when the imaging device collects the first image frame
  • the processor is specifically configured to:
  • the processor uses the two-dimensional information of the first image, the three-dimensional information, and a rotation relationship to determine a posture when the imaging device collects the first image frame
  • the processor is specifically configured to:
  • the first displacement relationship includes determination using a perspective n-point positioning PnP algorithm.
  • the first rotation relationship includes determination using an inertial measurement unit.
  • the imaging device is mounted on a movable platform
  • the first rotation relationship includes a rotation relationship between the imaging platform and the movable platform when the imaging device acquires the first image frame and the second image frame according to the movable platform. The relative relationship between them is determined.
  • the determining, by the processor according to the N-frame key frame and the first image frame, the posture when the imaging device collects the first image frame is specifically used to:
  • the estimated pose is a key between the first image frame and the N frame The estimated pose between the third image frame in the frame.
  • the processor uses the two-dimensional information of the second image and the estimated pose to determine the pose when the imaging device collects the first image frame
  • the processor is specifically configured to:
  • Determining a pose when the imaging device collects the first image frame according to the optimized relative pose relationship Determining a pose when the imaging device collects the first image frame according to the optimized relative pose relationship.
  • the relative pose relationship includes optimization using a filtering method.
  • the filtering method includes a Kalman filtering method.
  • the estimated pose includes an estimated displacement relationship and an estimated rotation relationship between the first image frame and the third image frame.
  • the estimated pose includes determination using an inertial measurement unit.
  • the first image frame is an image frame currently acquired by the imaging device.
  • the processor when the processor selects M frame key frames from the key frame sequence, the processor is specifically configured to:
  • a newly added M frame key frame is selected from the key frame sequence.
  • the processor when the processor selects N key frames from the key frame sequence, the processor is specifically configured to:
  • a newly added N frame key frame is selected from the key frame sequence.
  • the environmental conditions include at least one of an ambient light intensity and an environmental texture intensity.
  • the processor is further configured to perform the following operations:
  • the environmental condition is worse than the preset imaging condition.
  • the case where the ambient light intensity is lower than the preset light intensity includes at least one of the following:
  • Both the exposure time and the gain at the collection moment of the first image frame reach a specified threshold
  • the ambient brightness at the collection time of the first image frame is lower than a specified brightness threshold.
  • the processor is further configured to perform the following operations:
  • the intensity of the environmental texture is lower than the preset texture intensity, it is determined that the environmental condition is inferior to the preset imaging condition.
  • the case where the strength of the environment texture is lower than the preset texture strength includes at least one of the following:
  • the proportion of the size of the weakly-textured connected domain in the first image frame is greater than a specified proportion.
  • the weakly-textured connected region includes being determined using an edge detection algorithm.
  • the displacement of adjacent frames in the key frame sequence is larger than the displacement threshold, and the rotation angle of the adjacent frames is smaller than the angle threshold.
  • the M is not less than 1 and the N is not less than 5.
  • the M is 1 and the N is 5.
  • the M is not greater than 4.
  • the processor is further configured to perform the following operations:
  • a computer-readable storage medium has computer instructions stored thereon, and when the computer instructions are executed, the posture determination method according to the foregoing embodiment is implemented.
  • the system, device, module, or unit described in the foregoing embodiments may be implemented by a computer chip or entity, or by a product having a certain function.
  • a typical implementation device is a computer, and the specific form of the computer may be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email sending and receiving device, and a game control Desk, tablet computer, wearable device, or a combination of any of these devices.
  • the embodiments of the present invention may be provided as a method, a system, or a computer program product. Therefore, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • these computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured article including the instruction device,
  • the instruction device implements the functions specified in a flowchart or a plurality of processes and / or a block or a block of the block diagram.
  • These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operating steps are performed on the computer or other programmable device to produce a computer-implemented process, and the instructions executed on the computer or other programmable device Provides steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of the block diagrams.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Automation & Control Theory (AREA)
  • Studio Devices (AREA)
  • Image Analysis (AREA)

Abstract

一种位姿确定方法，包括：检测成像设备采集第一图像帧时的环境条件；当所述环境条件劣于预设成像条件时，从关键帧序列中选取M帧关键帧，依据所述M帧关键帧和所述第一图像帧确定所述成像设备采集所述第一图像帧时的位姿；当所述环境条件不劣于所述预设成像条件时，从所述关键帧序列中选取N帧关键帧，依据所述N帧关键帧和所述第一图像帧确定所述成像设备采集所述第一图像帧时的位姿；其中，所述关键帧序列为从所述成像设备采集的原始图像序列中选取，所述M小于所述N，所述M和所述N为不小于1的正整数。该方法可环境自适应地测量成像设备的位姿，有利于提高成像设备的定位准确性。

Description

位姿确定方法、设备、计算机可读存储介质 技术领域
本发明涉及定位技术领域，尤其是涉及一种位姿确定方法、设备、计算机可读存储介质。
背景技术
视觉定位技术，在无GPS(Global Positioning System全球定位系统)的区域(室内，高楼间等)也能保证定位的稳定性，因而视觉定位技术在人工智能中具有广泛应用。例如，在无人机、汽车或机器人等可移动平台上安装摄像头，通过计算机对摄像头拍到的图像进行运算分析，进而得出无人机、汽车或机器人的当前位置并绘制出行进轨迹。
但是，视觉定位技术依赖于成像，而成像质量会受到环境因素的影响，在环境发生变化时，可能存在位姿的错误观测而导致定位不准的情况。
发明内容
本发明提供一种位姿确定方法、设备、计算机可读存储介质，可环境自适应地测量位姿，提高定位准确性。
本发明实施例第一方面，提供一种位姿确定方法，包括：
检测成像设备采集第一图像帧时的环境条件；
当所述环境条件劣于预设成像条件时，从关键帧序列中选取M帧关键帧，依据所述M帧关键帧和所述第一图像帧确定所述成像设备采集所述第一图像帧时的位姿；
当所述环境条件不劣于所述预设成像条件时，从所述关键帧序列中选取N帧关键帧，依据所述N帧关键帧和所述第一图像帧确定所述成像设备采集所述第一图像帧时的位姿；
其中，所述关键帧序列为从所述成像设备采集的原始图像序列中选取，所述M小于所述N，所述M和所述N为不小于1的正整数。
本发明实施例第二方面，提供一种电子设备，包括：存储器和处理器；
所述存储器，用于存储程序代码；
所述处理器，用于调用所述程序代码，当所述程序代码被执行时，用于执行以下操作：
检测成像设备采集第一图像帧时的环境条件；
当所述环境条件劣于预设成像条件时，从关键帧序列中选取M帧关键帧，依据所述M帧关键帧和所述第一图像帧确定所述成像设备采集所述第一图像帧时的位姿；
当所述环境条件不劣于所述预设成像条件时，从所述关键帧序列中选取N帧关键帧，依据所述N帧关键帧和所述第一图像帧确定所述成像设备采集所述第一图像帧时的位姿；
其中，所述关键帧序列为从所述成像设备采集的原始图像序列中选取，所述M小于所述N，所述M和所述N为不小于1的正整数。
本发明实施例第三方面，提供一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机指令，所述计算机指令被执行时，实现本发明实施例第一方面所述的位姿确定方法。
基于上述技术方案，本发明实施例中，在环境条件劣于预设成像条件时，图像质量较差，仅采用更少帧关键帧确定成像设备采集第一图像帧时的位姿，可避免将帧间位姿误差被累积而导致误差放大，此时相对于采用更多帧关键帧确定位姿而言，准确度是更高的；而环境条件不劣于预设成像条件时，图像质量较好，本身帧间位姿误差较小，采用更多帧关键帧确定成像设备采集第一图像帧时的位姿，相对更少帧而言可提高准确度；当环境条件在劣于预设成像条件和不劣于预设成像条件变化时，可适应性地选取不同帧数的关键帧进行测量位姿，提高定位准确性。
附图说明
为了更加清楚地说明本发明实施例中的技术方案，下面将对本发明实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明中记载的一些实施例，对于本领域普通技术人员来讲，还可以根据本发明实施例的这些附图获得其它的附图。
图1是本发明一实施例的位姿确定方法的流程示意图；
图2是本发明一实施例的关键帧序列选取的示意图；
图3是本发明一实施例的对第一图像帧执行边缘检测后的示意图；
图4是本发明一实施例的位姿确定方法的流程示意图；
图5是本发明一实施例的电子设备的结构框图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。另外，在不冲突的情况下，下述的实施例及实施例中的特征可以相互组合。
本发明使用的术语仅仅是出于描述特定实施例的目的，而非限制本发明。本发明和权利要求书所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其它含义。应当理解的是，本文中使用的术语“和/或”是指包含一个或多个相关联的列出项目的任何或所有可能组合。
尽管在本发明可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语用来将同一类型的信息彼此区分开。例如，在不脱离本发明范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，此外，所使用的词语“如 果”可以被解释成为“在……时”，或者，“当……时”，或者，“响应于确定”。
本发明实施例中提出一种位姿确定方法，可实现利用成像设备采集的图像实时确定在成像设备在运动过程中的位姿，当然，也可以确定成像设备在静止时的位姿。位姿确定方法的执行主体可以为电子设备，电子设备具体类型不限，电子设备可以是成像设备但不限于成像设备。电子设备例如也可以是与成像设备电连接或通信连接的设备。当设备不是成像设备时，可在成像设备采集到图像后，获取成像设备所采集的图像，进而执行相应的方法。
本发明实施例中的成像设备可以包括相机、摄像头、带摄像头的终端设备(例如手机)等具有成像功能的设备。成像设备可以搭载在可移动平台上，例如，成像设备可以直接挂载在可移动平台(例如，可移动平台可以是无人车、无人机、无人船、移动机器人等)上，或通过云台挂载在可移动平台上。此时，当成像设备搭载于可移动平台上时，可以在确定成像设备的位姿之后，再转换得到与成像设备具有已确定的相对关系的可移动平台的位姿，或者，也可以将成像设备的位姿近似为可移动平台的位姿。当然，成像设备不限于此，还可以为VR/AR眼镜、双摄像头的手机等设备。
以安装有相机的无人机为例(相机可以直接挂载在无人机上，也可以通过云台挂载在无人机上)，无人机被用在越来越多的领域，这对无人机的定位提出了越来越高的要求，为了提高定位的精度，通常会采用较多帧图像来进行位姿确定，但是却忽略了环境导致的图像质量变化，在环境较差时反而会降低定位精度。无人机上用于定位的相机是被动感知周围环境的传感器，当在夜里或者矿井等低亮度的情况下，成像质量将大大降低，针对这种情况下可以有两种解决方法：
第一种方法，增大图像的曝光时间和增益，这个可以在不增加额外设备的情况下，使相机获得比较明亮的图像，但是增大曝光时间会使图像产生运动模糊，增加增益会引入图像噪声，而模糊和噪声对无人机的定位影响比较大，所以曝光时间和增益也要限制在指定阈值以内；
第二种方法，增加补光灯，补光灯能照亮无人机所在的周围环境，但是无人机携带的补光灯功率一般受限，而且补光灯形成的光斑从中心到边缘的亮度会逐渐变暗，这种亮度的不均匀也对无人机定位影响比较大。
或者，可以综合这两种方法，在亮度较低的环境中，优先调节曝光时间和增益，当调节到指定阈值之后仍未达到期望图像亮度，再开启补光灯。
但是，在环境比较暗的情况下，使用上述的增加曝光时间和增益、以及开启补光灯的方法得到的图像质量仍不如环境比较明亮的情况，这种情况下再利用较多帧图像进行位姿测量还是会存在定位精度不高甚至发散的情况，并不能较好地解决定位精度变差的问题。当然，导致定位精度变差的环境条件也不仅限于亮度，还会存在其他环境问题，例如是成像对象本身的纹理较弱等。
在环境较差时，利用较多帧图像进行位姿测量导致定位精度变差的原因，可以包括以下几条：
1、图像质量不高导致图像特征跟踪的精度不高，一个特征点在多帧图像上位置漂移比较快；
2、图像特征跟踪的精度不高导致计算出来的深度不准确；
3、不准确的特征点用于计算后续位姿时，存在误差累积的问题，会进一步错误地计算相机的位置和姿态。
因此，在环境条件较好时，采用较多帧图像来进行位姿确定，可以提高精度；但在环境条件较差时，再采用较多帧图像来进行位姿确定，不仅不能够提高精度，反而会因图像质量过差而降低精度，适得其反。
针对上述发现，本发明实施例中，在依据成像设备采集的图像帧确定成像设备的位姿时，先检测成像设备在采集该图像帧时的环境条件，在环境条件较差时，图像质量较低，采用较少的关键帧来确定成像设备采集该图像帧时的位姿，而在环境条件较好时，图像质量较好，采用较多的关键帧来确定成像设备采集该图像帧时的位姿，可随环境自适应地调整，使得无论环境怎样变化，都具备较高的位姿定位精度。
下面结合具体实施例，对位姿确定的过程进行说明，但不作为限制。
参看图1，在一个实施例中，一种位姿确定方法可以包括以下步骤：
S100：检测成像设备采集第一图像帧时的环境条件；
S200：当环境条件劣于预设成像条件时，从关键帧序列中选取M帧关键帧，依据M帧关键帧和第一图像帧确定成像设备采集第一图像帧时的位姿；
S300：当环境条件不劣于预设成像条件时，从关键帧序列中选取N帧关键帧，依据N帧关键帧和第一图像帧确定成像设备采集第一图像帧时的位姿；
其中，关键帧序列为从成像设备采集的原始图像序列中选取，M小于N，M和N为不小于1的正整数。
具体地，所述方法的执行主体可以为电子设备，进一步地可以为电子设备的处理器，其中，所述处理器可以为一个或多个，所述处理器可以为通用处理器或者专用处理器。
电子设备可以是成像设备本身，或者和成像设备相对固定，或者和成像设备相对连接，或者包括成像设备，例如，电子设备可以为搭载有成像设备的可移动平台，下面以电子设备为成像设备本身展开描述。成像设备可以采集图像，处理器可以获取该图像。
在步骤S100之前，当然还可以包括获取第一图像帧的步骤，在本发明实施例的位姿确定执行主体为成像设备时，该获取第一图像帧即为采集第一图像帧，若执行主体是成像设备之外的电子设备，则该获取第一图像帧可以为从成像设备中获取该第一图像帧。
第一图像帧优选是成像设备当前采集的一帧图像帧，当然也可以是之前采集的一帧图像帧，具体不限。
本发明实施例的关键帧序列为从所述成像设备采集的原始图像序列中选取，可由成像设备的处理器执行。优选的，从成像设备采集的原始图像序列中选取关键帧序列例如可以包括以下步骤：
获取原始图像；
确定采集的原始图像相对最新关键帧的位姿关系；
在原始图像相对最新关键帧的位姿关系满足位移距离大于位移阈值、且旋转角度小于角度阈值的预设条件时，可以将原始图像作为关键帧加入至关键帧序列。
当然，上述确定采集的原始图像相对最新关键帧的位姿关系仅是确定关键帧的其中一种方式，在实际应用中，还可以是比较采集的原始图像相对于之前所确定的任意一帧关键帧的位姿关系，具体不限，只要在得到的关键帧序列中，相邻两帧图像帧之间的位姿关系满足位移距离大于位移阈值、且旋转角度小于角度阈值的预设条件即可。
其中，关键帧序列中的首个关键帧可以为指定原始图像。例如，在关键帧序列中为空时，可以获取当前采集的原始图像作为关键帧序列的首帧，后续采集的原始图像可以与该首帧进行比对后再确定是否为关键帧，并在确定第二帧关键帧后，可以将后续采集的原始图像与该第二帧关键帧进行比对，以此类推，以使得原始图像可以与最新关键帧进行比对。当然，该首帧中应当具有可识别的特征点。
具体的，参看图2，示出了关键帧选取的方式。成像设备采集原始图像，假设成像设备为以20Hz频率进行曝光，时间上是固定的，所以每50ms得到一张原始图像，得到的原始图像序列为a1、a2、a3、a4……an，原始图像a1被指定为关键帧序列的首帧，而原始图像a2与关键帧a1之间、原始图像a3与关键帧a1之间均不满足上述预设条件，因而原始图像a2、a3不作为关键帧，但原始图像a4与关键帧a1之间满足上述预设条件，因而原始图像a4可以作为关键帧。同理，可以按照上述与最新关键帧比对的流程确定后续的关键帧an。可以理解，随着采集的进行，关键帧序列可更新，即保持关键帧序列中的帧数量，采用先进先出的原则将旧的且不可能再采用的关键帧从关键帧序列中剔除。
优选的，当所述关键帧序列为按照时间顺序排列时，所述关键帧序列中的相邻帧的位移大于位移阈值，且所述相邻帧的旋转角度小于角度阈值。
关键帧间的位姿包括旋转(Rotation)关系和位移(Translation)关系。 这里Rotation用欧拉角表示，Translation用三轴方向的平移量表示：
Figure PCTCN2018095957-appb-000001
Translation：t＝[t x，t y，t z] T
关键帧间的位姿需要满足下述关系：
Figure PCTCN2018095957-appb-000002
Figure PCTCN2018095957-appb-000003
α th为角度阈值，d th为位移阈值，在原始图像和最新关键帧之间的位移足够大而旋转较小的情况下，原始图像就能作为新的关键帧加入关键帧序列。
在实际应用中，若将采集的所有原始图像都用来进行位姿计算，存在的问题有：一是计算量很大，二是错误的可能性很高，反而会把正确的结果带偏。然而，本发明实施例中，通过提取出关键帧作为用于位姿计算的图像序列，可以解决上述计算量大、错误的可能性高的问题。
优选的，上述确定采集的原始图像相对最新关键帧的位姿关系，可以包括：通过视觉里程计算法，确定原始图像相对最新关键帧的位姿关系。
在步骤S100中，检测成像设备采集第一图像帧时的环境条件。
具体的，成像设备的处理器可检测成像设备采集第一图像帧时的环境条件。检测环境条件的时机，可以是在成像设备采集第一图像帧的时刻进行检测，也可以在采集之后的任意时刻进行检测(在此过程中，需要记录采集第一图像帧时的环境条件)，例如是需要确定采集第一图像帧时刻的位姿时进行检测，也可以在采集之后的一段时间内检测。检测环境条件的方式也不限，例如，可以通过对第一图像帧本身的检测而确定环境条件，也可以对成像设备应对环境产生的状态检测而确定环境条件，或者直接对环境进行检测而确定环境条件等。
第一图像帧可以是成像设备当前采集的图像帧，也可以是其它时刻采集的图像帧，具体视所需确定位姿的采集时刻而定。
在步骤S200中，当环境条件劣于预设成像条件时，从关键帧序列中选取 包括M帧关键帧，依据M帧关键帧和第一图像帧确定成像设备采集第一图像帧时的位姿。
在步骤S300中，当环境条件不劣于预设成像条件时，从关键帧序列中选取N帧关键帧，依据N帧关键帧和第一图像帧确定成像设备采集第一图像帧时的位姿。
步骤S200和步骤S300的执行不具有先后顺序，而是依据环境条件所符合的情况而由处理器择一执行。
所检测的环境条件可以为一项或两项以上，预设成像条件是与环境条件对应的判断标准。所检测出的环境条件可以反映第一图像帧的图像质量，例如当环境条件反映第一图像帧的图像质量较差时，确定环境条件劣于预设成像条件，执行步骤S200；而环境条件反映第一图像帧的图像质量较好时，确定环境条件不劣于预设成像条件，执行步骤S300。
其中，M小于N，M和N为不小于1的正整数。由于环境条件劣于预设成像条件时，图像质量较差，仅采用M帧关键帧确定成像设备采集第一图像帧时的位姿，可避免将帧间位姿误差被累积而导致误差放大，此时相对于采用更多帧关键帧确定位姿而言，准确度是更高的；而由于环境条件不劣于预设成像条件时，图像质量较好，本身帧间位姿误差较小，采用N帧关键帧确定成像设备采集第一图像帧时的位姿，相对更少帧而言可提高准确度。当环境条件在劣于预设成像条件和不劣于预设成像条件变化时，可适应性地选取不同帧数的关键帧进行测量位姿，提高定位准确性。
优选的，M不小于1，N不小于5。
优选的，在M不小于1，N不小于5的基础上，M不大于4，例如M可以取1、2、3、或4。如此帧数的选取，可在不同环境条件下得到更佳的位姿结果。更优的，M为1，N为5。当然，这并非作为限制。
可以理解，电子设备在工作过程中，处理器每次执行确定成像设备采集第一图像帧时的位姿时，所采用的N可以不同，所采用的M也可以不同，即N、M可以具有可变性及随机性，可以是不固定的。
进一步的来说，步骤S200中，依据M帧关键帧和第一图像帧确定成像设备采集第一图像帧时的位姿，可以包括以下步骤：
S201：确定M帧关键帧与第一图像帧中相匹配的特征点的第一图像二维信息以及M帧关键帧中的特征点的三维信息；
S202：利用第一图像二维信息、三维信息和第一旋转关系确定成像设备采集第一图像帧时的位姿，第一旋转关系为第一图像帧与M帧关键帧中的第二图像帧之间的旋转关系。
具体的，在步骤S201中，处理器对M帧关键帧和第一图像帧进行特征点匹配，可确定在M帧关键帧与第一图像帧中之间相匹配的特征点。该特征点可以是所追踪的目标物体成像在图像中的点。
处理器对M帧关键帧和第一图像帧进行特征点匹配确定M帧关键帧和第一图像帧中的特征点，可以具体包括：通过特征点跟踪算法对M帧关键帧和第一图像帧进行特征点匹配，确定M帧关键帧和第一图像帧中所匹配出的特征点。特征点跟踪算法例如包括KLT(Kanade-Lucas-Tomasi Tracking)算法，根据特征点在一帧图像上的位置，找出特征点在另一帧图像上的位置，当然还有其他特征点跟踪算法，具体不限。可以理解，处理器对M帧关键帧进行特征点匹配，以确定M帧关键帧中特征点的方法，当然也可以通过其他方式确定，并不限制于KLT算法。
特征点在相应图像上的坐标位置可以作为对应特征点的第一图像二维信息，第一图像二维信息为通过M帧关键帧、第一图像帧本身可确定的信息。M帧关键帧中与第一图像帧匹配的特征点的三维信息可以通过诸如双目视觉算法确定，例如，利用M帧关键帧中的各关键帧及对应同时采集的另一目图像来计算，具体方式可以参看现有的双目视觉算法，在此不再赘述，当然，M帧关键帧中与第一图像帧匹配的特征点的三维信息也可以利用多帧单目图像来计算等，具体不限。
该M帧关键帧中与第一图像帧匹配的特征点的三维信息优选是在世界坐标系下的位置信息，三维信息当然也可以是其他坐标系下的位置信息，如相 机坐标系，最终都可以通过两个坐标系之间的坐标转换关系来转换得到，因而不作为限定。
具体的，在步骤S202中，处理器利用第一图像二维信息、三维信息和第一旋转关系确定成像设备采集所述第一图像帧时的位姿，第一旋转关系为第一图像帧与M帧关键帧中的第二图像帧之间的旋转关系。
处理器对第一图像二维信息、三维信息和第一旋转关系进行位姿测算便可确定成像设备采集第一图像帧时的位姿，位姿测算的方法具体不限，只要是能够利用这三种输入信息可确定位姿的均适用。
可以理解，第二图像帧是指M帧关键帧中的任意一帧，能够用于确定与第一图像帧之间的旋转关系即可。优选的，第二图像帧可以是M帧关键帧中最新加入的一帧。第一图像帧与第二图像帧之间的旋转关系是成像设备在采集第一图像帧时与采集第二图像帧时的旋转关系。
优选的，第一旋转关系可以包括使用惯性测量单元(IMU)确定。惯性测量单元可以与成像设备相对固定，第一旋转关系可以通过IMU测定的位姿数据中的旋转数据确定，当然也可通过IMU测定的位姿数据中的旋转数据经一定数据处理后的数据确定。IMU测定的时机是在第一图像帧与第二图像帧的采集时刻，可通过IMU积分得到第一图像帧与第二图像帧之间的旋转关系。
优选的，在成像设备搭载于可移动平台时，第一旋转关系还可以包括依据可移动平台在成像设备采集第一图像帧时与采集第二图像帧时的旋转关系、及成像设备与可移动平台之间的相对关系确定。其中，成像设备与可移动平台之间的相对关系优选是不变的，当然也可以是可变的(例如，该相对关系是随使用时间的延长而可优化的)。
其中，由于可以利用可移动平台在成像设备采集第一图像帧时与采集第二图像帧时的旋转关系、以及成像设备与可移动平台之间的相对关系得到第一旋转关系，因而在确定成像设备采集第一图像帧时的位姿时，只需估计位姿中的位移，这样估计的自由度就只有3个且为线性。以可移动平台为无人机为例，在无人机上一般会有一个滤波器来估计无人机的姿态，并可以使用 该无人机的姿态来确定相机的姿态，此处的姿态即为旋转关系。
在一个实施例中，步骤S202中，利用第一图像二维信息、三维信息和第一旋转关系确定成像设备采集第一图像帧时的位姿，可以包括以下步骤：
对第一图像二维信息、三维信息和第一旋转关系进行计算，得到第一图像帧与第二图像帧之间的第一位移关系；
根据第一位移关系和第一旋转关系确定成像设备采集第一图像帧时的位姿。
本实施例中，将第一图像二维信息、三维信息和第一旋转关系均作为位姿解算的输入，得到第一图像帧与第二图像帧之间的第一位移关系。根据该第一旋转关系和计算所得的第一位移关系确定成像设备采集第一图像帧时的位姿。作为输入的第一旋转关系是预估值，由于关键帧数较少，因而可直接将IMU所确定的第一旋转关系作为信任值，并由此计算第一位移关系。
可以理解，第一图像帧与第二图像帧之间的第一位移关系是成像设备在采集第一图像帧时与采集第二图像帧时的位移关系。
优选的，本实施例中，第一位移关系可以包括使用透视n点定位PnP算法确定。PnP算法是通过一系列世界坐标系的三维位置点(三维信息)以及图像中对应的像素坐标系的二维位置点(第一图像二维信息)，估算相机姿态，也就是需要的位姿关系R1和T1。
其中，R1可以由IMU测定，即本实施例中所述的第一旋转关系，而计算所得到的T1即第一位移关系。当然，利用上述PnP算法可计算出M帧关键帧中的某一帧关键帧与第一图像帧之间的关系，在M帧关键帧多于一帧时，可以分别计算得到M帧关键帧中各帧的位姿关系，以位姿累积的方式来确定成像设备采集所述第一图像帧时的位姿。
在M帧关键帧中的某一帧关键帧与第一图像帧之间存在多个匹配的特征点时，可分别利用这些特征点计算得到对应的第一位移关系，最后可以将所有的第一位移关系进行融合计算，并以融合计算的结果作为M帧关键帧中的某一帧关键帧与第一图像帧之间的第一位移关系，例如可以平均、加权平均 等，具体融合方式不限。
在另一个实施例中，步骤S202中，利用第一图像二维信息、三维信息和第一旋转关系确定成像设备采集第一图像帧时的位姿，可以包括以下步骤：
对第一图像二维信息、三维信息进行计算，得到第一图像帧与所述第二图像帧之间的第一位移关系；
根据第一位移关系和第一旋转关系确定成像设备采集第一图像帧时的位姿。
与前一实施例不同的是，本实施例的第一位移关系在进行位姿测算时不做为输入，而仅输入第一图像二维信息、三维信息计算以确定第一位移关系。
本实施例中，第一位移关系同样可以包括使用透视n点定位PnP算法确定。具体的，可以利用多组特征点信息来解算，例如P3P中，使用4组特征点信息(在图像中，4个特征点不共面)，利用3组特征点信息求解多个解，第4组特征点信息确定其中的最优解；又如，EPnP可使用大于等于3组特征点信息来解算等。特征点信息即特征点的第一图像二维信息、三维信息。本实施例与前一实施例相同之处在此不再赘述。
其中，示例性的，在利用M帧关键帧确定成像设备采集第一图像帧时的位姿时，可以利用第一旋转关系、第一位移关系及采集第二图像帧时的位姿得到。
进一步的来说，步骤S300中，依据N帧关键帧和第一图像帧确定成像设备采集第一图像帧时的位姿，可以包括以下步骤：
S301：确定第一图像帧中与N帧关键帧匹配的特征点的第二图像二维信息；
S302：利用第二图像二维信息、预估位姿确定成像设备采集第一图像帧时的位姿，预估位姿为第一图像帧与N帧关键帧中的第三图像帧之间的预估位姿。
具体的，在步骤S301中，处理器确定第一图像帧中与N帧关键帧匹配的特征点的第二图像二维信息，特征点匹配的方式同样可以通过特征点跟踪算 法确定，例如包括KLT(Kanade-Lucas-Tomasi Tracking)算法，当然还有其他特征点跟踪算法，具体不限。可以理解，处理器对第一图像帧与N帧关键帧进行特征点匹配确定第一图像帧中特征点当然也可以通过其他方式确定，并不限制于角KLT算法。第二图像二维信息是特征点在第一图像帧中的坐标，可利用第一图像帧直接确定。
优选的，预估位姿包括所述第一图像帧与所述第三图像帧之间的预估位移关系以及预估旋转关系，即成像设备采集第一图像帧时与采集第三图像帧时之间的预估位移关系、预估旋转关系。
具体的，所述预估位姿可以包括使用惯性测量单元(IMU)确定。
IMU可以与成像设备相对固定，预估位移关系以及预估旋转关系可以通过IMU测定的位姿数据确定，当然也可通过IMU测定的位姿数据经一定数据处理后的数据确定。IMU分别测定在第一图像帧采集时刻及第三图像帧采集时刻的位姿，并通过IMU积分得到第一图像帧与第三图像帧之间的预估位移关系以及预估旋转关系。
在步骤S302中，处理器利用第二图像二维信息、预估位姿确定成像设备采集第一图像帧时的位姿。由于预估位姿是预估值，例如是IMU确定的位姿数据，在图像质量较高时，可利用其作为预估输入进行位姿的更新解算，但不直接作为位姿或位姿中的一部分，提高位姿的准确性。
优选的，利用第二图像二维信息、预估位姿确定成像设备采集第一图像帧时的位姿，包括：
利用所述第二图像二维信息、预估位姿优化第一图像帧与第三图像帧之间的相对位姿关系；
根据优化后的相对位姿关系确定成像设备采集第一图像帧时的位姿。
在一个实施例中，该相对位姿关系包括使用滤波法优化。滤波法可以将粗略估计的值进行优化，得到较精确的值。
其中，N帧关键帧对应的成像设备的位姿是经滤波法优化后的位姿，可作为较准确的位姿来计算后续图像帧采集时的准确位姿。而在上述M帧关键 帧中，其采用的位姿也是通过PnP算法后优化得到。
优选的，滤波法包括卡尔曼滤波法，具体可以例如为MSCKF(Multi-State-Constraint-Kalman-Filter，多状态约束的卡尔曼滤波器)法，当然不限于此，还可以是其他EKF(扩展卡尔曼滤波器)法。可以理解，位姿确定的具体方式也不限于此，能够利用第二图像二维信息、预估位姿确定成像设备采集第一图像帧时的位姿即可。
使用滤波法优化例如包括：
假设N帧关键帧为第K-4帧、第K-3帧、第K-2帧、第K-1帧，第一图像帧为第K帧，那么按照时间顺序的关系，在相应的滤波器中，第K-4帧、第K-3帧、第K-2帧、第K-1帧之间的两两位姿关系已得到优化。当需要确定第K帧图像帧采集时的位姿时，可以将第K-4帧、第K-3帧、第K-2帧、第K-1帧中任一帧作为第三图像帧，并将第K帧中与N帧关键帧匹配的特征点的第二图像二维信息、及与第K-1帧的预估位姿输入至卡尔曼滤波器(由于第K-1帧与第K-4帧、第K-3帧、第K-2帧之间的位姿关系已知，那么在利用IMU积分预估第K帧与第K-1帧的位姿后，第K帧与第K-4帧、第K-3帧、第K-2帧之间的预估位姿也已知)，从而经过卡尔曼滤波器的预测以及更新步骤，可以得到优化后的第K帧与第K-1帧之间的相对位姿关系，并通过诸如第K-1帧的位姿，可以确定第K帧的位姿，即成像设备采集第一图像帧时的位姿。又或者，例如，可以利用第K-1帧与第K-4帧、第K-3帧、第K-2帧之间的相对位姿关系及第K-4帧的位姿，便可确定第K帧的位姿，即成像设备采集第一图像帧时的位姿。
可以理解，第三图像帧是N帧关键帧中的任意一帧，能够用于确定与第一图像帧之间的旋转关系、位移关系即可。优选的，第三图像帧可以是N帧关键帧中最新加入的一帧。第一图像帧与第三图像帧之间所涉及的位姿是指成像设备在采集第一图像帧时与采集第三图像帧时的位移关系和旋转关系。
本发明实施例中，在环境条件劣于预设成像条件时和不劣于所述预设成像条件时，不仅选取了不同数量的关键帧进行处理，在此基础上，还采用了 较少的M帧关键帧和较多的N帧关键帧中不同的信息作为位姿计算的信息，并将不同的信息用不同的位姿确定方式计算成像设备采集所述第一图像帧时的位姿，可进一步提高相应环境条件下的定位准确度。
成像设备采集所述第一图像帧时的位姿是第一图像帧与其他图像帧(前述的第二图像帧或第三图像帧)之间的相对旋转关系和相对位移关系，因而可以根据该其他图像帧的定位信息与成像设备采集所述第一图像帧时的位姿，确定成像设备采集所述第一图像帧时的定位信息。
在一个实施例中，第一图像帧是成像设备当前时刻采集的图像帧，针对该第一图像帧确定的是成像设备的实时位姿，保证定位信息的实时性。
优选的，所述从所述关键帧序列中选取M帧关键帧，包括：从所述关键帧序列中选取最新加入的M帧关键帧。
优选的，所述从所述关键帧序列中选取N帧关键帧，包括：从所述关键帧序列中选取最新加入的N帧关键帧。
可以理解，成像设备采集第一图像帧时的位姿可以包括成像设备的位置(例如，在世界坐标系下)和姿态，具体可以根据需要确定。
在一个实施例中，环境条件包括环境光照度大小和环境纹理强弱中的至少一种。
在环境条件为环境光照度大小，位姿确定方法可以包括：若环境光照度大小低于预设光照度，则确定所述环境条件劣于预设成像条件。环境光照度大小低于预设光照度说明环境光照度过低，此时采集的图像会存在质量较差的问题，因而将其确定为环境条件劣于预设成像条件，选取M帧关键帧进行位姿处理；反之，则说明环境光照度合适，成像质量好，可选取N帧关键帧进行位姿处理。
优选的，所述环境光照度大小低于预设光照度的情况包括以下至少一种：
所述第一图像帧的采集时刻开启补光灯；
所述第一图像帧的采集时刻的曝光时间及增益均达到指定阈值；
所述第一图像帧的采集时刻的环境亮度低于指定亮度阈值。
具体的，补光灯可以是在成像设备上。或者，在成像设备搭载在可移动平台上时，例如搭载在无人机上时，补光灯可以是与成像设备同时搭载在无人机上，可以是与成像设备同时搭载在无人机的一个云台上，也可以是与成像设备分别搭载在无人机的不同云台上，具体不限。补光灯可由成像设备或可移动平台控制，例如，通过在第一图像帧的采集时刻检测控制补光灯启闭的控制信号即可确定补光灯是否开启。
其中，若第一图像帧的采集时刻开启补光灯、或者第一图像帧的采集时刻的曝光时间及增益均达到指定阈值，则说明成像设备已感测到环境光照度过低的情况，而开启补光灯或者增加曝光时间并增加增益并不能完全改善图像质量，因而此时仍需要选取M帧关键帧进行位姿处理。示例性的，第一图像帧的采集时刻的环境亮度可以通过亮度传感器来检测，将检测的亮度值送入成像设备中进行比较，即可确定第一图像帧的采集时刻的环境亮度是否低于指定亮度阈值。
上述方式中，若同时存在两种以上情况，则可以将这些情况都满足时确定为环境光照度大小低于预设光照度。当然，上述情况仅是环境光照度大小低于预设光照度的几种，还可以通过检测其他与环境光照度有关的信息来确定。
环境条件为环境纹理强弱时，位姿确定方法可以包括：若所述环境纹理强弱低于预设纹理强度，则确定所述环境条件劣于预设成像条件。环境纹理强弱低于预设纹理强度说明成像对象的纹理过弱，此时采集的图像会存在质量较差的问题，因而将其确定为环境条件劣于预设成像条件，选取M帧关键帧进行位姿处理；反之，则说明环境光照度合适，成像质量好，可选取N帧关键帧进行位姿处理。
优选的，所述环境纹理强弱低于预设纹理强度的情况包括以下至少一种：
在所述第一图像帧中未检测到感兴趣的纹理信息；
在所述第一图像帧中检测到的特征点少于指定数量；
在所述第一图像帧中的弱纹理连通域的大小占比大于指定占比。
具体的，纹理信息可以通过边缘检测算法确定，当没有检测到感兴趣的纹理信息(感兴趣的纹理信息满足根据需要所确定的纹理信息范围，例如，可以是具有足够的明显的纹理，而没有足够的明显的纹理的示例性物体可以包括单色墙、光滑的玻璃和/或类似物)时，说明图像质量较差，确定所述环境条件劣于预设成像条件。特征点可以通过特征识别来确定，当然，可以在检测出感兴趣的纹理信息的基础上来检测特征点，当可检测到的特征点少于指定数量时，说明图像质量较差，确定所述环境条件劣于预设成像条件。在第一图像帧中弱纹理连通域的大小占比大于指定占比时，说明这片弱纹理连通域过大，第一图像帧对应的纹理信息过少，进而说明图像质量较差，可以确定所述环境条件劣于预设成像条件。
优选的，所述弱纹理连通域包括使用边缘检测算法确定。边缘检测算法，例如包括Sobel operator(索贝尔算子)、Canny operator(坎尼算子)，当然具体不限于此。Sobel operator实际上就是分别求取第一图像帧的水平和垂直方向上的梯度。
参看图3，示出了第一图像帧经边缘检测后的图像，检测出了明显边缘，在此基础上，可检测连通域，可以使用Flood fill(块填充)算法填充成块区域，这些块区域都是潜在的弱纹理区域，将块区域在图像上的占比一一计算出来，选取最大的与指定占比进行比较，当大于时，确定该块区域为弱纹理连通域。
在环境条件包括环境光照度大小和环境纹理强弱时，则确定是劣于还是不劣于预设成像条件的方式可以包括：
第一种，若环境光照度大小低于预设光照度、或者所述环境纹理强弱低于预设纹理强度，则确定所述环境条件劣于预设成像条件；若环境光照度大小不低于预设光照度、且所述环境纹理强弱不低于预设纹理强度，则确定所述环境条件不劣于预设成像条件。
第二种，可以将环境光照度大小与预设光照度的比较值、和环境纹理强弱与预设纹理强度的比较值，进行加权求和或取均值等运算，将运算结果值 与预设比较值进行比较，当运算结果值低于该预设比较值时则确定为环境条件劣于预设成像条件，否则确定为环境条件不劣于预设成像条件。
第三种，若环境光照度大小低于预设光照度、且所述环境纹理强弱低于预设纹理强度，则确定所述环境条件劣于预设成像条件；若环境光照度大小不低于预设光照度、或者所述环境纹理强弱不低于预设纹理强度，则确定所述环境条件不劣于预设成像条件。
当然，具体方式并不限于上述两种方式。其中，对于环境纹理强弱低于预设纹理强度的检测、环境光照度大小低于预设光照度的检测，同样可以是上述描述的几种情况，在此便不再赘述了。
在一个实施例中，参看图4，位姿确定方法还可以包括以下步骤：
S400：依据成像设备采集第一图像帧时的位姿，控制成像设备和/或搭载成像设备的可移动平台。
具体的，确定成像设备采集第一图像帧时的位姿后，可以根据该位姿进一步调控成像设备的位姿，以满足其它不同位姿的拍摄需求。当然，也可以进行其它的控制操作，例如，当第一图像帧为成像设备采集的图像帧时，若确定的成像设备采集第一图像帧时的位姿未满足需求，可以关闭成像设备，以实现节能的目的。
示例性的，当成像设备搭载在可移动平台上时，如无人机，在无人机的周围环境不利于成像时，如为低照度和/或弱纹理场景下，可以有效解决视觉定位算法的局限性，防止在这些特殊场景下输出错误的信息而引发不安全的因素，整个过程中，通过环境不同时的算法过渡切换，可以强化无人机的整机系统的可靠性与鲁棒性，有利于实现对无人机的稳定悬停及航向规划等，且在无GPS的区域(如室内、高楼间)，也能保持无人机的稳定性。
由此，通过成像设备采集第一图像帧时的位姿，不仅可以有利于对成像设备的控制，也有利于对搭载成像设备的可移动平台(成像设备的位姿与可移动平台的位姿之间可通过相应的关系换算得到)的控制。
基于与上述方法同样的构思，参看图5，一种电子设备100，包括：存储 器101和处理器102(如一个或多个处理器)。电子设备具体类型不限，电子设备可以是成像设备但不限于成像设备。电子设备例如也可以是与成像设备电连接或通信连接的设备。当设备不是成像设备时，可在成像设备采集到图像后，获取成像设备所采集的图像，进而执行相应的方法。
在一个实施例中，所述存储器101，用于存储程序代码；所述处理器102，用于调用所述程序代码，当程序代码被执行时，用于执行以下操作：
检测成像设备采集第一图像帧时的环境条件；
当所述环境条件劣于预设成像条件时，从关键帧序列中选取M帧关键帧，依据所述M帧关键帧和第一图像帧确定所述成像设备采集所述第一图像帧时的位姿；
当所述环境条件不劣于所述预设成像条件时，从所述关键帧序列中选取N帧关键帧，依据所述N帧关键帧和第一图像帧确定所述成像设备采集所述第一图像帧时的位姿；
其中，所述关键帧序列为从所述成像设备采集的原始图像序列中选取，所述M小于所述N，所述M和所述N为不小于1的正整数。
优选的，所述处理器依据所述M帧关键帧和第一图像帧确定所述成像设备采集所述第一图像帧时的位姿时具体用于：
确定所述M帧关键帧与所述第一图像帧中相匹配的特征点的第一图像二维信息以及所述M帧关键帧中的特征点的三维信息；
利用所述第一图像二维信息、所述三维信息和第一旋转关系确定所述成像设备采集所述第一图像帧时的位姿，所述第一旋转关系为所述第一图像帧与所述M帧关键帧中的第二图像帧之间的旋转关系。
优选的，所述处理器利用所述第一图像二维信息、所述三维信息和旋转关系确定所述成像设备采集所述第一图像帧时的位姿时具体用于：
对所述第一图像二维信息、所述三维信息和第一旋转关系进行计算，得到所述第一图像帧与所述第二图像帧之间的第一位移关系；
根据所述第一位移关系和所述第一旋转关系确定所述成像设备采集所述 第一图像帧时的位姿。
优选的，所述处理器利用所述第一图像二维信息、所述三维信息和旋转关系确定所述成像设备采集所述第一图像帧时的位姿时具体用于：
对所述第一图像二维信息、所述三维信息进行计算，得到所述第一图像帧与所述第二图像帧之间的第一位移关系；
根据所述第一位移关系和所述第一旋转关系确定所述成像设备采集所述第一图像帧时的位姿。
优选的，所述第一位移关系包括使用透视n点定位PnP算法确定。
优选的，所述第一旋转关系包括使用惯性测量单元确定。
优选的，所述成像设备搭载于可移动平台；
所述第一旋转关系包括依据所述可移动平台在所述成像设备采集所述第一图像帧时与采集所述第二图像帧时的旋转关系、及所述成像设备与所述可移动平台之间的相对关系确定。
优选的，所述处理器依据所述N帧关键帧和第一图像帧确定所述成像设备采集所述第一图像帧时的位姿时具体用于：
确定所述第一图像帧中与N帧关键帧匹配的特征点的第二图像二维信息；
利用所述第二图像二维信息、预估位姿确定所述成像设备采集所述第一图像帧时的位姿，所述预估位姿为所述第一图像帧与所述N帧关键帧中的第三图像帧之间的预估位姿。
优选的，所述处理器利用所述第二图像二维信息、预估位姿确定所述成像设备采集所述第一图像帧时的位姿时具体用于：
利用所述第二图像二维信息、预估位姿优化所述第一图像帧与所述第三图像帧之间的相对位姿关系；
根据优化后的所述相对位姿关系确定所述成像设备采集所述第一图像帧时的位姿。
优选的，所述相对位姿关系包括使用滤波法优化。
优选的，所述滤波法包括卡尔曼滤波法。
优选的，所述预估位姿包括所述第一图像帧与所述第三图像帧之间的预估位移关系以及预估旋转关系。
优选的，所述预估位姿包括使用惯性测量单元确定。
优选的，所述第一图像帧是所述成像设备当前采集的图像帧。
优选的，所述处理器从所述关键帧序列中选取M帧关键帧时具体用于：
从所述关键帧序列中选取最新加入的M帧关键帧。
优选的，所述处理器从所述关键帧序列中选取N帧关键帧时具体用于：
从所述关键帧序列中选取最新加入的N帧关键帧。
优选的，所述环境条件包括环境光照度大小和环境纹理强弱中的至少一种。
优选的，所述处理器还用于执行以下操作：
若所述环境光照度大小低于预设光照度，则确定所述环境条件劣于预设成像条件。
优选的，所述环境光照度大小低于预设光照度的情况包括以下至少一种：
所述第一图像帧的采集时刻开启补光灯；
所述第一图像帧的采集时刻的曝光时间及增益均达到指定阈值；
所述第一图像帧的采集时刻的环境亮度低于指定亮度阈值。
优选的，所述处理器还用于执行以下操作：
若所述环境纹理强弱低于预设纹理强度，则确定所述环境条件劣于预设成像条件。
优选的，所述环境纹理强弱低于预设纹理强度的情况包括以下至少一种：
在所述第一图像帧中未检测到感兴趣的纹理信息；
在所述第一图像帧中检测到的特征点少于指定数量；
在所述第一图像帧中的弱纹理连通域的大小占比大于指定占比。
优选的，所述弱纹理连通域包括使用边缘检测算法确定。
优选的，当所述关键帧序列为按照时间顺序排列时，所述关键帧序列中 的相邻帧的位移大于位移阈值，且所述相邻帧的旋转角度小于角度阈值。
优选的，所述M不小于1，所述N不小于5。
优选的，所述M为1，所述N为5。
优选的，所述M不大于4。
优选的，所述处理器还用于执行以下操作：
依据所述成像设备采集所述第一图像帧时的位姿，控制所述成像设备和/或搭载所述成像设备的可移动平台。
基于与上述方法同样的发明构思，一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机指令，所述计算机指令被执行时，实现前述实施例所述的位姿确定方法。
上述实施例阐明的系统、装置、模块或单元，可以由计算机芯片或实体实现，或者由具有某种功能的产品来实现。一种典型的实现设备为计算机，计算机的具体形式可以是个人计算机、膝上型计算机、蜂窝电话、相机电话、智能电话、个人数字助理、媒体播放器、导航设备、电子邮件收发设备、游戏控制台、平板计算机、可穿戴设备或者这些设备中的任意几种设备的组合。
为了描述的方便，描述以上装置时以功能分为各种单元分别描述。当然，在实施本发明时可以把各单元的功能在同一个或多个软件和/或硬件中实现。
本领域内的技术人员应明白，本发明实施例可提供为方法、系统、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明实施例可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可以由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入 式处理机或其它可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其它可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
而且，这些计算机程序指令也可以存储在能引导计算机或其它可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或者多个流程和/或方框图一个方框或者多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其它可编程数据处理设备，使得在计算机或者其它可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其它可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
以上所述仅为本发明实施例而已，并不用于限制本发明。对于本领域技术人员来说，本发明可以有各种更改和变化。凡在本发明的精神和原理之内所作的任何修改、等同替换、改进，均应包含在本发明的权利要求范围之内。

Claims (55)

  1. 一种位姿确定方法，其特征在于，包括：
    检测成像设备采集第一图像帧时的环境条件；
    当所述环境条件劣于预设成像条件时，从关键帧序列中选取M帧关键帧，依据所述M帧关键帧和所述第一图像帧确定所述成像设备采集所述第一图像帧时的位姿；
    当所述环境条件不劣于所述预设成像条件时，从所述关键帧序列中选取N帧关键帧，依据所述N帧关键帧和所述第一图像帧确定所述成像设备采集所述第一图像帧时的位姿；
    其中，所述关键帧序列为从所述成像设备采集的原始图像序列中选取，所述M小于所述N，所述M和所述N为不小于1的正整数。
  2. 根据权利要求1所述的位姿确定方法，其特征在于，所述依据所述M帧关键帧和所述第一图像帧确定所述成像设备采集所述第一图像帧时的位姿，包括：
    确定所述M帧关键帧与所述第一图像帧中相匹配的特征点的第一图像二维信息以及所述M帧关键帧中的所述特征点的三维信息；
    利用所述第一图像二维信息、所述三维信息和第一旋转关系确定所述成像设备采集所述第一图像帧时的位姿，所述第一旋转关系为所述第一图像帧与所述M帧关键帧中的第二图像帧之间的旋转关系。
  3. 根据权利要求2所述的位姿确定方法，其特征在于，所述利用所述第一图像二维信息、所述三维信息和第一旋转关系确定所述成像设备采集所述第一图像帧时的位姿，包括：
    对所述第一图像二维信息、所述三维信息和第一旋转关系进行计算，得到所述第一图像帧与所述第二图像帧之间的第一位移关系；
    根据所述第一位移关系和所述第一旋转关系确定所述成像设备采集所述第一图像帧时的位姿。
  4. 根据权利要求2所述的位姿确定方法，其特征在于，所述利用所述第 一图像二维信息、所述三维信息和第一旋转关系确定所述成像设备采集所述第一图像帧时的位姿，包括：
    对所述第一图像二维信息、所述三维信息进行计算，得到所述第一图像帧与所述第二图像帧之间的第一位移关系；
    根据所述第一位移关系和所述第一旋转关系确定所述成像设备采集所述第一图像帧时的位姿。
  5. 根据权利要求3或4所述的位姿确定方法，其特征在于，所述第一位移关系包括使用透视n点定位PnP算法确定。
  6. 根据权利要求2所述的位姿确定方法，其特征在于，所述第一旋转关系包括使用惯性测量单元确定。
  7. 根据权利要求2所述的位姿确定方法，其特征在于，所述成像设备搭载于可移动平台；
    所述第一旋转关系包括依据所述可移动平台在所述成像设备采集所述第一图像帧时与采集所述第二图像帧时的旋转关系、及所述成像设备与所述可移动平台之间的相对关系确定。
  8. 根据权利要求1所述的位姿确定方法，其特征在于，所述依据所述N帧关键帧和所述第一图像帧确定所述成像设备采集所述第一图像帧时的位姿，包括：
    确定所述第一图像帧中与所述N帧关键帧匹配的特征点的第二图像二维信息；
    利用所述第二图像二维信息、预估位姿确定所述成像设备采集所述第一图像帧时的位姿，所述预估位姿为所述第一图像帧与所述N帧关键帧中的第三图像帧之间的预估位姿。
  9. 根据权利要求8所述的位姿确定方法，其特征在于，所述利用所述第二图像二维信息、预估位姿确定所述成像设备采集所述第一图像帧时的位姿，包括：
    利用所述第二图像二维信息、预估位姿优化所述第一图像帧与所述第三 图像帧之间的相对位姿关系；
    根据优化后的所述相对位姿关系确定所述成像设备采集所述第一图像帧时的位姿。
  10. 根据权利要求9所述的位姿确定方法，其特征在于，所述相对位姿关系包括使用滤波法优化。
  11. 根据权利要求10所述的位姿确定方法，其特征在于，所述滤波法包括卡尔曼滤波法。
  12. 根据权利要求8所述的位姿确定方法，其特征在于，所述预估位姿包括所述第一图像帧与所述第三图像帧之间的预估位移关系以及预估旋转关系。
  13. 根据权利要求12所述的位姿确定方法，其特征在于，所述预估位姿包括使用惯性测量单元确定。
  14. 如权利要求1所述的位姿确定方法，其特征在于，所述第一图像帧是所述成像设备当前采集的图像帧。
  15. 如权利要求1所述的位姿确定方法，其特征在于，所述从所述关键帧序列中选取M帧关键帧，包括：
    从所述关键帧序列中选取最新加入的M帧关键帧。
  16. 如权利要求1所述的位姿确定方法，其特征在于，所述从所述关键帧序列中选取N帧关键帧，包括：
    从所述关键帧序列中选取最新加入的N帧关键帧。
  17. 如权利要求1所述的位姿确定方法，其特征在于，所述环境条件包括环境光照度大小和环境纹理强弱中的至少一种。
  18. 根据权利要求17所述的位姿确定方法，其特征在于，所述方法还包括：
    若所述环境光照度大小低于预设光照度，则确定所述环境条件劣于预设成像条件。
  19. 根据权利要求18所述的位姿确定方法，其特征在于，所述环境光照 度大小低于预设光照度的情况包括以下至少一种：
    所述第一图像帧的采集时刻开启补光灯；
    所述第一图像帧的采集时刻的曝光时间及增益均达到指定阈值；
    所述第一图像帧的采集时刻的环境亮度低于指定亮度阈值。
  20. 根据权利要求17所述的位姿确定方法，其特征在于，所述方法还包括：
    若所述环境纹理强弱低于预设纹理强度，则确定所述环境条件劣于预设成像条件。
  21. 根据权利要求20所述的位姿确定方法，其特征在于，所述环境纹理强弱低于预设纹理强度的情况包括以下至少一种：
    在所述第一图像帧中未检测到感兴趣的纹理信息；
    在所述第一图像帧中检测到的特征点少于指定数量；
    在所述第一图像帧中的弱纹理连通域的大小占比大于指定占比。
  22. 根据权利要求21所述的位姿确定方法，其特征在于，所述弱纹理连通域包括使用边缘检测算法确定。
  23. 根据权利要求1所述的位姿确定方法，其特征在于，当所述关键帧序列为按照时间顺序排列时，所述关键帧序列中的相邻帧的位移大于位移阈值，且所述相邻帧的旋转角度小于角度阈值。
  24. 根据权利要求1所述的位姿确定方法，其特征在于，所述M不小于1，所述N不小于5。
  25. 根据权利要求24所述的位姿确定方法，其特征在于，所述M为1，所述N为5。
  26. 根据权利要求24所述的位姿确定方法，其特征在于，所述M不大于4。
  27. 根据权利要求1所述的位姿确定方法，其特征在于，所述方法还包括：
    依据所述成像设备采集所述第一图像帧时的位姿，控制所述成像设备和/ 或搭载所述成像设备的可移动平台。
  28. 一种电子设备，其特征在于，包括：存储器和处理器；
    所述存储器，用于存储程序代码；
    所述处理器，用于调用所述程序代码，当所述程序代码被执行时，用于执行以下操作：
    检测成像设备采集第一图像帧时的环境条件；
    当所述环境条件劣于预设成像条件时，从关键帧序列中选取M帧关键帧，依据所述M帧关键帧和所述第一图像帧确定所述成像设备采集所述第一图像帧时的位姿；
    当所述环境条件不劣于所述预设成像条件时，从所述关键帧序列中选取N帧关键帧，依据所述N帧关键帧和所述第一图像帧确定所述成像设备采集所述第一图像帧时的位姿；
    其中，所述关键帧序列为从所述成像设备采集的原始图像序列中选取，所述M小于所述N，所述M和所述N为不小于1的正整数。
  29. 根据权利要求28所述的电子设备，其特征在于，所述处理器依据所述M帧关键帧和所述第一图像帧确定所述成像设备采集所述第一图像帧时的位姿时具体用于：
    确定所述M帧关键帧与所述第一图像帧中相匹配的特征点的第一图像二维信息以及所述M帧关键帧中的所述特征点的三维信息；
    利用所述第一图像二维信息、所述三维信息和第一旋转关系确定所述成像设备采集所述第一图像帧时的位姿，所述第一旋转关系为所述第一图像帧与所述M帧关键帧中的第二图像帧之间的旋转关系。
  30. 根据权利要求29所述的电子设备，其特征在于，所述处理器利用所述第一图像二维信息、所述三维信息和第一旋转关系确定所述成像设备采集所述第一图像帧时的位姿时具体用于：
    对所述第一图像二维信息、所述三维信息和第一旋转关系进行计算，得到所述第一图像帧与所述第二图像帧之间的第一位移关系；
    根据所述第一位移关系和所述第一旋转关系确定所述成像设备采集所述第一图像帧时的位姿。
  31. 根据权利要求29所述的电子设备，其特征在于，所述处理器利用所述第一图像二维信息、所述三维信息和第一旋转关系确定所述成像设备采集所述第一图像帧时的位姿时具体用于：
    对所述第一图像二维信息、所述三维信息进行计算，得到所述第一图像帧与所述第二图像帧之间的第一位移关系；
    根据所述第一位移关系和所述第一旋转关系确定所述成像设备采集所述第一图像帧时的位姿。
  32. 根据权利要求30或31所述的电子设备，其特征在于，所述第一位移关系包括使用透视n点定位PnP算法确定。
  33. 根据权利要求29所述的电子设备，其特征在于，所述第一旋转关系包括使用惯性测量单元确定。
  34. 根据权利要求29所述的电子设备，其特征在于，所述成像设备搭载于可移动平台；
    所述第一旋转关系包括依据所述可移动平台在所述成像设备采集所述第一图像帧时与采集所述第二图像帧时的旋转关系、及所述成像设备与所述可移动平台之间的相对关系确定。
  35. 根据权利要求28所述的电子设备，其特征在于，所述处理器依据所述N帧关键帧和所述第一图像帧确定所述成像设备采集所述第一图像帧时的位姿时具体用于：
    确定所述第一图像帧中与所述N帧关键帧匹配的特征点的第二图像二维信息；
    利用所述第二图像二维信息、预估位姿确定所述成像设备采集所述第一图像帧时的位姿，所述预估位姿为所述第一图像帧与所述N帧关键帧中的第三图像帧之间的预估位姿。
  36. 根据权利要求35所述的电子设备，其特征在于，所述利用所述第二 图像二维信息、预估位姿确定所述成像设备采集所述第一图像帧时的位姿，包括：
    利用所述第二图像二维信息、预估位姿优化所述第一图像帧与所述第三图像帧之间的相对位姿关系；
    根据优化后的所述相对位姿关系确定所述成像设备采集所述第一图像帧时的位姿。
  37. 根据权利要求35所述的电子设备，其特征在于，所述位姿包括使用滤波法优化。
  38. 根据权利要求36所述的电子设备，其特征在于，所述滤波法包括卡尔曼滤波法。
  39. 根据权利要求35所述的电子设备，其特征在于，所述预估位姿包括所述第一图像帧与所述第三图像帧之间的预估位移关系以及预估旋转关系。
  40. 根据权利要求35所述的电子设备，其特征在于，所述预估位姿包括使用惯性测量单元确定。
  41. 根据权利要求28所述的电子设备，其特征在于，所述第一图像帧是所述成像设备当前采集的图像帧。
  42. 根据权利要求28所述的电子设备，其特征在于，所述处理器从所述关键帧序列中选取M帧关键帧时具体用于：
    从所述关键帧序列中选取最新加入的M帧关键帧。
  43. 根据权利要求28所述的电子设备，其特征在于，所述处理器从所述关键帧序列中选取N帧关键帧时具体用于：
    从所述关键帧序列中选取最新加入的N帧关键帧。
  44. 根据权利要求28所述的电子设备，其特征在于，所述环境条件包括环境光照度大小和环境纹理强弱中的至少一种。
  45. 根据权利要求44所述的电子设备，其特征在于，所述处理器还用于执行以下操作：
    若所述环境光照度大小低于预设光照度，则确定所述环境条件劣于预设 成像条件。
  46. 根据权利要求45所述的电子设备，其特征在于，所述环境光照度大小低于预设光照度的情况包括以下至少一种：
    所述第一图像帧的采集时刻开启补光灯；
    所述第一图像帧的采集时刻的曝光时间及增益均达到指定阈值；
    所述第一图像帧的采集时刻的环境亮度低于指定亮度阈值。
  47. 根据权利要求44所述的电子设备，其特征在于，所述处理器还用于执行以下操作：
    若所述环境纹理强弱低于预设纹理强度，则确定所述环境条件劣于预设成像条件。
  48. 根据权利要求47所述的电子设备，其特征在于，所述环境纹理强弱低于预设纹理强度的情况包括以下至少一种：
    在所述第一图像帧中未检测到感兴趣的纹理信息；
    在所述第一图像帧中检测到的特征点少于指定数量；
    在所述第一图像帧中的弱纹理连通域的大小占比大于指定占比。
  49. 根据权利要求48所述的电子设备，其特征在于，所述弱纹理连通域包括使用边缘检测算法确定。
  50. 根据权利要求28所述的电子设备，其特征在于，当所述关键帧序列为按照时间顺序排列时，所述关键帧序列中的相邻帧的位移大于位移阈值，且所述相邻帧的旋转角度小于角度阈值。
  51. 根据权利要求28所述的电子设备，其特征在于，所述M不小于1，所述N不小于5。
  52. 根据权利要求51所述的电子设备，其特征在于，所述M为1，所述N为5。
  53. 根据权利要求51所述的电子设备，其特征在于，所述M不大于4。
  54. 根据权利要求28所述的电子设备，其特征在于，所述处理器还用于执行以下操作：
    依据所述成像设备采集所述第一图像帧时的位姿，控制所述成像设备和/或搭载所述成像设备的可移动平台。
  55. 一种计算机可读存储介质，其特征在于，
    所述计算机可读存储介质上存储有计算机指令，所述计算机指令被执行时，实现权利要求1-27任一项所述的位姿确定方法。
PCT/CN2018/095957 2018-07-17 2018-07-17 位姿确定方法、设备、计算机可读存储介质 WO2020014864A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201880038859.8A CN110914867A (zh) 2018-07-17 2018-07-17 位姿确定方法、设备、计算机可读存储介质
PCT/CN2018/095957 WO2020014864A1 (zh) 2018-07-17 2018-07-17 位姿确定方法、设备、计算机可读存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/095957 WO2020014864A1 (zh) 2018-07-17 2018-07-17 位姿确定方法、设备、计算机可读存储介质

Publications (1)

Publication Number Publication Date
WO2020014864A1 true WO2020014864A1 (zh) 2020-01-23

Family

ID=69164208

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/095957 WO2020014864A1 (zh) 2018-07-17 2018-07-17 位姿确定方法、设备、计算机可读存储介质

Country Status (2)

Country Link
CN (1) CN110914867A (zh)
WO (1) WO2020014864A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112950715A (zh) * 2021-03-04 2021-06-11 杭州迅蚁网络科技有限公司 无人机的视觉定位方法、装置、计算机设备和存储介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113286076B (zh) * 2021-04-09 2022-12-06 华为技术有限公司 拍摄方法及相关设备
CN113514058A (zh) * 2021-04-23 2021-10-19 北京华捷艾米科技有限公司 融合msckf和图优化的视觉slam定位方法及装置
CN113900439B (zh) * 2021-12-10 2022-03-11 山东理工职业学院 无人船自动进离码头的方法、系统和控制终端

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107123142A (zh) * 2017-05-09 2017-09-01 北京京东尚科信息技术有限公司 位姿估计方法和装置
CN107246868A (zh) * 2017-07-26 2017-10-13 上海舵敏智能科技有限公司 一种协同导航定位系统及导航定位方法
CN107357286A (zh) * 2016-05-09 2017-11-17 两只蚂蚁公司 视觉定位导航装置及其方法
CN108180909A (zh) * 2017-12-22 2018-06-19 北京三快在线科技有限公司 相对位置确定方法、装置及电子设备
CN108227735A (zh) * 2016-12-22 2018-06-29 Tcl集团股份有限公司 基于视觉飞行自稳定的方法、计算机可读介质和系统
CN108256574A (zh) * 2018-01-16 2018-07-06 广东省智能制造研究所 机器人定位方法及装置

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9911197B1 (en) * 2013-03-14 2018-03-06 Hrl Laboratories, Llc Moving object spotting by forward-backward motion history accumulation
WO2016065627A1 (zh) * 2014-10-31 2016-05-06 深圳市大疆创新科技有限公司 一种基于位置的控制方法、装置、可移动机器以及机器人
US10012509B2 (en) * 2015-11-12 2018-07-03 Blackberry Limited Utilizing camera to assist with indoor pedestrian navigation
CN106708048B (zh) * 2016-12-22 2023-11-28 清华大学 机器人的天花板图像定位方法和系统
CN107025668B (zh) * 2017-03-30 2020-08-18 华南理工大学 一种基于深度相机的视觉里程计的设计方法
CN107907131B (zh) * 2017-11-10 2019-12-13 珊口（上海）智能科技有限公司 定位系统、方法及所适用的机器人

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107357286A (zh) * 2016-05-09 2017-11-17 两只蚂蚁公司 视觉定位导航装置及其方法
CN108227735A (zh) * 2016-12-22 2018-06-29 Tcl集团股份有限公司 基于视觉飞行自稳定的方法、计算机可读介质和系统
CN107123142A (zh) * 2017-05-09 2017-09-01 北京京东尚科信息技术有限公司 位姿估计方法和装置
CN107246868A (zh) * 2017-07-26 2017-10-13 上海舵敏智能科技有限公司 一种协同导航定位系统及导航定位方法
CN108180909A (zh) * 2017-12-22 2018-06-19 北京三快在线科技有限公司 相对位置确定方法、装置及电子设备
CN108256574A (zh) * 2018-01-16 2018-07-06 广东省智能制造研究所 机器人定位方法及装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112950715A (zh) * 2021-03-04 2021-06-11 杭州迅蚁网络科技有限公司 无人机的视觉定位方法、装置、计算机设备和存储介质
CN112950715B (zh) * 2021-03-04 2024-04-30 杭州迅蚁网络科技有限公司 无人机的视觉定位方法、装置、计算机设备和存储介质

Also Published As

Publication number Publication date
CN110914867A (zh) 2020-03-24

Similar Documents

Publication Publication Date Title
CN110702111B (zh) 使用双事件相机的同时定位与地图创建（slam）
US11392146B2 (en) Method for detecting target object, detection apparatus and robot
CN112567201B (zh) 距离测量方法以及设备
EP3420530B1 (en) A device and method for determining a pose of a camera
WO2020014864A1 (zh) 位姿确定方法、设备、计算机可读存储介质
EP3008695B1 (en) Robust tracking using point and line features
JP6043856B2 (ja) Ｒｇｂｄカメラを用いた頭部ポーズ推定
CN108283018B (zh) 电子设备和用于电子设备的姿态识别的方法
JP2019522851A (ja) ３次元空間内の姿勢推定
WO2013119352A1 (en) Head pose tracking using a depth camera
JP2008298685A (ja) 計測装置及びプログラム
WO2020221307A1 (zh) 一种运动物体的追踪方法和装置
CN113052907B (zh) 一种动态环境移动机器人的定位方法
CN110072078A (zh) 监控摄像机、监控摄像机的控制方法和存储介质
JP6817742B2 (ja) 情報処理装置およびその制御方法
WO2023142353A1 (zh) 一种位姿预测方法及装置
CN115088244A (zh) 用于基于来自单目相机的数据进行定位的深度传感器激活
CN116468786A (zh) 一种面向动态环境的基于点线联合的语义slam方法
CN110741625B (zh) 运动估计方法及摄影器材
WO2022198508A1 (zh) 镜头异常提示方法、装置、可移动平台及可读存储介质
CN111780744B (zh) 移动机器人混合导航方法、设备及存储装置
CN111553342B (zh) 一种视觉定位方法、装置、计算机设备和存储介质
JP2015005220A (ja) 情報表示装置及び情報表示方法
KR20210050997A (ko) 포즈 추정 방법 및 장치, 컴퓨터 판독 가능한 기록 매체 및 컴퓨터 프로그램
CN111614834B (zh) 电子设备控制方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18926802

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18926802

Country of ref document: EP

Kind code of ref document: A1