WO2022188334A1 - 定位初始化方法和装置、设备、存储介质、程序产品 - Google Patents

定位初始化方法和装置、设备、存储介质、程序产品 Download PDF

Info

Publication number
WO2022188334A1
WO2022188334A1 PCT/CN2021/107630 CN2021107630W WO2022188334A1 WO 2022188334 A1 WO2022188334 A1 WO 2022188334A1 CN 2021107630 W CN2021107630 W CN 2021107630W WO 2022188334 A1 WO2022188334 A1 WO 2022188334A1
Authority
WO
WIPO (PCT)
Prior art keywords
historical image
image frame
frame
preset
positioning
Prior art date
Application number
PCT/CN2021/107630
Other languages
English (en)
French (fr)
Inventor
翟尚进
章国锋
Original Assignee
浙江商汤科技开发有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浙江商汤科技开发有限公司 filed Critical 浙江商汤科技开发有限公司
Publication of WO2022188334A1 publication Critical patent/WO2022188334A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/18—Stabilised platforms, e.g. by gyroscope
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20228—Disparity calculation for image-based rendering
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Definitions

  • the present disclosure relates to the field of positioning technology, and in particular, to a positioning initialization method and apparatus, device, storage medium, and program product.
  • visual tracking and localization methods play an important role in the fields of computer vision, robotics, unmanned aerial vehicles, 3D reconstruction and first-level augmented reality.
  • the results obtained from the initialization are directly used to track and locate the device.
  • the parameters solved in the initialization process may have large errors, such as parameters such as scale. If these parameters are directly used to track and locate the device, the accuracy of the final device tracking and positioning will be low.
  • the present disclosure provides at least a positioning initialization method and apparatus, device, storage medium and program product.
  • a first aspect of the present disclosure provides a positioning initialization method, including: after initializing positioning parameters of a device to be positioned, acquiring motion information of the device; determining a current motion state of the device based on the motion information; using the current motion state of the device , and optimize the positioning parameters to obtain the optimized positioning parameters.
  • the positioning parameters are initialized, the current motion state of the device is further obtained, and then the positioning parameters are optimized according to the current motion state, so that the subsequent movement of the device can be positioned according to the optimized parameters, which improves the positioning accuracy. Accuracy.
  • the positioning parameters are divided into first positioning parameters and second positioning parameters; according to the current motion state of the device, the positioning parameters are optimized to obtain the optimized positioning parameters, including: when the current motion state is a non-stationary state, The first positioning parameter is optimized by using a first optimization method, wherein the first positioning parameter includes a scale; or, when the current motion state is a static state, the second positioning parameter is optimized by using a second optimization method, wherein, The second positioning parameter is different from the first positioning parameter.
  • the positioning parameters are optimized, which can be applied to different motion scenarios.
  • the non-stationary state includes a first non-stationary state and a second non-stationary state; when the current motion state belongs to a non-stationary state, using the first optimization method to optimize the first positioning parameter, including: when the current motion state belongs to the non-stationary state In the case of the first non-stationary state, the re-projection error of each three-dimensional point on the current image frame and several historical image frames is used to optimize the first positioning parameter; wherein, the current image frame and the historical image frame are obtained by the device.
  • the current image frame is not used for tracking and positioning the device, and the historical image frame has been used for tracking and positioning the device; or, when the current motion state belongs to the second non-stationary state, the preset kinematic equations are used to The first positioning parameter is optimized.
  • the non-stationary state is subdivided into a variety of non-stationary states, so that when the device is in a non-stationary state, an appropriate optimization method can be selected according to the actual motion situation, and the optimization effect of the parameters can be improved.
  • determining the current motion state of the device based on the motion information includes at least one of the following steps: taking one of the historical image frames as the motion start frame; when the features between the motion start frame and the current image frame do not meet preset conditions , determine that the current motion state is the first non-stationary state; wherein, the features include one or more of the number of image frames, time difference, parallax and sensor data; the features between the motion start frame and the current image frame meet preset conditions In the case of , it is determined that the current motion state is the second non-stationary state.
  • the motion state of the device is determined through the motion start frame and subsequent historical image frames and the current image frame, which reduces the interference caused by the still image frames before the motion start frame and makes the determination of the motion state of the device more accurate.
  • the preset conditions include at least one of the following: the first time difference between the motion start frame and the current image frame is greater than or equal to the first time threshold; the number of historical image frames between the motion start frame and the current image frame is greater than or equal to the first time threshold The number threshold; the number of preset two-dimensional point pairs in the motion start frame and the current image frame is greater than or equal to the second number threshold, wherein the parallax of the preset two-dimensional point pairs is greater than or equal to the first parallax threshold; the motion start frame to The degree of dispersion of the accelerometer sensing data between the current image frames is greater than or equal to a preset degree threshold.
  • the greater the dispersion of parallax and sensing data the greater the range of motion of the device, the greater the number of historical image frames and the greater the time difference, indicating that there is enough sensing data to optimize the positioning parameters, so that the positioning parameters
  • the optimization results are more accurate.
  • using a preset kinematic equation set to optimize the first positioning parameter includes: taking one of the historical image frames as the motion start frame, and acquiring the positional relationship between adjacent image frames between the motion start frame and the current image frame, and Pre-integration information; Substitute the positional relationship and pre-integration information into the preset kinematic equations to obtain the scale and the speed of the equipment; take the obtained scale and the speed of the equipment as the optimized first positioning parameters.
  • the scale and velocity can be solved, which reduces the difficulty of solving the scale and velocity.
  • the scale and velocity obtained by solving are a set of solutions that minimize the error of the preset kinematic equations.
  • the scale and velocity obtained by solving are more accurate than other solutions, and the result of subsequent tracking and positioning is more accurate.
  • the method before using the obtained scale and the speed of the device as the optimized first positioning parameters, the method further includes: substituting the obtained scale and speed into the preset kinematic equations to obtain each of the preset kinematics equations.
  • the error of the kinematic equation if the error of each kinematic equation is less than or equal to the preset error threshold, it is determined that the scale and speed are successfully solved, and the obtained scale and the speed of the device are used as the optimized first positioning parameters.
  • taking one of the historical image frames as the motion start frame includes: sorting the second time difference between the historical image frame and the current image frame from less to more, and sequentially judging the difference between every two adjacent historical image frames. Whether the third time difference is greater than the second time threshold; in the case of judging that the third time difference between two adjacent historical image frames is greater than the second time threshold, the historical image frame with the smaller second time difference in the adjacent two historical image frames Frame as motion start.
  • the motion start frame is determined according to the time difference between two adjacent historical image frames, which makes the identification process of the motion start frame simple.
  • the method further includes: optimizing the position of each historical image frame and/or the depth of each three-dimensional point after scale optimization.
  • optimizing the position of each historical image frame after the scale optimization includes: selecting one of the historical image frames as the first reference frame; using the optimized scale to determine the distance between the first reference frame and its adjacent historical image frames distance; preliminarily adjust the position of the adjacent historical image frame of the first reference frame according to the distance; take the preliminarily adjusted adjacent historical image frame as the new first reference frame, and re-execute the optimized scale to determine the first The distance between the reference frame and its adjacent historical image frames and subsequent steps until the historical image frames within the preset time period have been preliminarily adjusted.
  • the distance relationship between the historical image frames is adjusted according to the calculated scale, and the positional relationship between the historical image frames is optimized, so that the result of subsequent tracking and positioning of the device is more accurate.
  • the method further includes: taking the historical image frame with the smallest time difference between the preset time period and the current image frame as the second reference frame; The position of the second reference frame is finally adjusted to the position before the preliminary adjustment; based on the position change of the second reference frame in the final adjustment process, the positions of the remaining historical image frames except the second reference frame within the preset time period are adjusted.
  • the position of each historical image frame before and after the preliminary adjustment changes too much, that is, there is a discontinuity between the historical image frames before and after the preliminary adjustment. Moving the position after the preliminary adjustment of the frame to the position before the preliminary adjustment can alleviate the incoherence phenomenon, so that the effect of subsequent tracking and positioning is better.
  • optimizing the depth of each 3D point after the scale optimization includes: for each 3D point, adjusting the depth of the 3D point according to the scale based on the historical image frame where the 3D point is first observed.
  • the depth of the three-dimensional point is optimized through the calculated scale, so that the device can be tracked and positioned with more accurate coordinates of the three-dimensional point subsequently, and the accuracy of the tracking and positioning can be improved.
  • the method before optimizing the position of each historical image frame after the scale optimization and/or the depth of each 3D point, the method further includes: deleting the historical image frame located before the motion start frame within a preset time period; and/or, After optimizing the position of each historical image frame and/or the depth of each three-dimensional point after the scale optimization, the method includes: increasing the uncertainty of at least one of the position, velocity and accelerometer offset of the device.
  • determining the current motion state of the device based on the motion information includes: the disparity between several historical image frames and the current image frame is all smaller than the second disparity threshold, and/or the degree of dispersion of at least one sensor data is smaller than the corresponding prediction In the case of setting the degree value, it is determined that the current motion state of the device is a static state; wherein, the second parallax threshold is fixed or dynamically adjusted based on the time difference between the historical image frame and the current image frame; at least one sensory data is in the Collected in the first time period before the current motion state, including gyroscope sensor data and/or accelerometer sensor data.
  • the identification of the static state through the combination of parallax and sensor data is more accurate than the identification of single parallax or sensor data.
  • using the second optimization manner to optimize the second positioning parameter includes: constraining the second positioning parameter with a speed of zero as a constraint.
  • the positioning parameters cannot be optimized by the reprojection error of the 3D point in the static state.
  • the pose of the current image frame will not change greatly compared with the pose of the previous historical image frame. , to improve the stability of positioning parameters.
  • the method further includes: judging whether there are a preset number of 3D points in several historical image frames before the current image frame; in the case where there is no preset number of 3D points in the several historical image frames, increasing the constraint weight of zero speed; and or, in the case that there are a preset number of three-dimensional points in several historical image frames, reduce the constraint weight of the speed to zero.
  • the localization parameters can be well optimized in various states.
  • a second aspect of the present disclosure provides a positioning initialization device, comprising: an obtaining module configured to obtain motion information of the device after initializing the positioning parameters of the device to be positioned; and a determining module configured to determine, based on the motion information, the The current motion state; the optimization module is configured to optimize the positioning parameters according to the current motion state of the equipment to obtain the optimized positioning parameters.
  • a third aspect of the present disclosure provides an electronic device including a memory and a processor, where the processor is configured to execute program instructions stored in the memory to implement the above positioning initialization method.
  • a fourth aspect of the present disclosure provides a computer-readable storage medium storing program instructions thereon, and when the program instructions are executed by a processor, the above positioning initialization method is implemented.
  • a fifth aspect of the present disclosure provides a computer program product comprising computer-readable code, when the computer-readable code is executed in an electronic device, a processor in the electronic device executes a method configured to implement the positioning initialization method .
  • the motion state of the device is further obtained, and then the positioning parameters are optimized according to the motion state, so that the subsequent movement of the device can be positioned according to the optimized positioning parameters, which improves the positioning accuracy. Accuracy.
  • FIG. 1 is a first schematic flowchart of an embodiment of a positioning initialization method of the present disclosure
  • FIG. 2 is a second schematic flowchart in an embodiment of the positioning initialization method of the present application.
  • FIG. 3 is a schematic structural diagram of an embodiment of the positioning initialization apparatus of the present disclosure
  • FIG. 4 is a schematic structural diagram of an embodiment of an electronic device of the present disclosure.
  • FIG. 5 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present disclosure.
  • FIG. 1 is a first schematic flowchart of an embodiment of a positioning initialization method of the present disclosure. The method may include the following steps:
  • Step S11 After the positioning parameters of the device to be positioned are initialized, the motion information of the device is acquired.
  • the manner of initializing the positioning parameters of the device to be positioned may include motion initialization and static initialization.
  • the way of motion initialization can be to separate the visual initialization from the inertial sensor initialization, that is, perform visual initialization first, and then align the initialization results with the inertial sensor initialization results after the visual initialization is completed, thereby completing the initialization of the positioning parameters.
  • Static initialization applies when the device is relatively stationary. Because the device is in a relatively static state, the corresponding three-dimensional point coordinates cannot be obtained from the two-dimensional points observed in each frame of images through triangulation, etc., and accurate scale information cannot be obtained.
  • the general approach is to assume a scale, and then according to the assumptions The scale determines other positioning parameters.
  • the manner of acquiring the motion information of the device may be parallax, time difference, etc. between image frames captured by the device, or sensor data between image frames, or the like.
  • the image frames collected in the process of tracking and positioning the device can be captured by any device with a camera component, of course, it can also be an image obtained from other devices, or it can be an image after frame selection, brightness, resolution, etc. .
  • the device may be the device itself that captures the image frames.
  • other devices refer to devices that can only be operated by using different central processing units.
  • the drone is the device described in the embodiments of the present disclosure.
  • the cleaning robot is at work, and the cleaning robot is also the device described in the embodiment of the present disclosure. That is, in an environment, the device realizes positioning and/or navigation of its own position in the environment.
  • Step S12 Determine the current motion state of the device based on the motion information.
  • the current motion state may be a stationary state or a non-stationary state.
  • the way to determine the current motion state may be based on the motion range of the device, for example, based on the parallax between the two-dimensional point pairs in each image frame obtained from the front and back shots, and of course, the judgment may also be based on sensor data. For example, when the disparity between the two image frames is greater than a preset value, it is determined that the current motion state of the device is a non-stationary state.
  • Step S13 According to the current motion state of the device, the positioning parameters are optimized to obtain the optimized positioning parameters.
  • each current motion state may correspond to a corresponding optimization method to optimize the positioning parameters.
  • the preset current motion states of the device include a stationary state and a non-stationary state.
  • the positioning parameters can be optimized using the optimization method corresponding to the stationary state or the non-stationary state.
  • the optimized parameters can be directly tracked and positioned, and of course, it can also be used to track and position the device after one or more optimizations are performed.
  • the positioning parameters are initialized, the current motion state of the device is still further obtained, and then the positioning parameters are optimized according to the current motion state, so that the subsequent movement of the device can be positioned according to the optimized parameters, which improves positioning. accuracy.
  • the positioning parameters are divided into first positioning parameters and second positioning parameters.
  • the first positioning parameter includes a scale
  • the second positioning parameter is different from the first positioning parameter.
  • the second positioning parameter is different from the first positioning parameter, and it can be understood that the second positioning parameter and the parameters included in the first positioning parameter are not exactly the same, for example, the second positioning parameter may not include a scale.
  • FIG. 2 is a schematic diagram of a second flow in an embodiment of the positioning initialization method of the present disclosure. As shown in FIG. 2 , the above step S13 includes step S131 and step S132.
  • step S131 when the current motion state belongs to a stationary state, use the second optimization method to optimize the second positioning parameter
  • step S132 when the current motion state belongs to a non-stationary state, use the first optimization method to The first positioning parameter is optimized, wherein the first optimization method is different from the second optimization method.
  • the manner of determining that the device is in a stationary state and a non-stationary state includes that the disparity between several historical image frames and the current image frame is smaller than the second disparity threshold, and/or the degree of dispersion of at least one sensory data is smaller than the corresponding
  • the preset degree value of it is determined that the current motion state of the device is a stationary state.
  • the historical image frames used in the still state determination may be all the image frames captured in the first time period before the current image frame. Further, the current image frame and the historical image frame are captured by the device, the current image frame is not used for the tracking and positioning of the device, and the historical image frame has been used for the tracking and positioning of the device.
  • the current image frame has only undergone image processing, that is, feature extraction and matching, and has not been tracked and positioned, while historical image frames have not only undergone image processing, but also tracked and positioned.
  • the tracking and positioning here includes using historical image frames to determine the position of the device in the world coordinate system.
  • the first time period can be set by itself according to the needs, and there is no restriction. Judge the parallax between all historical image frames in the first time period and the current image frame respectively. If the parallax between a historical image frame and the current image frame is greater than the second parallax threshold, it is determined that the current motion state of the device does not belong to stationary state.
  • the number of historical image frames in the first time period is greater than or equal to 2 frames. In the embodiment of the present disclosure, the number of historical image frames is three or more. By setting multiple frames of historical image frames to judge the parallax, the misjudgment caused by too few historical image frames can be reduced, that is, the accuracy of the static state judgment can be improved.
  • the first time period cannot be too long. If the first time period is too long, the information corresponding to the historical image frames whose shooting time is too long will bring errors to the determination of the current motion state. Wherein, whether the first time period is too long may be determined by whether it exceeds a time threshold, and if the first time period exceeds the time threshold, it is determined to be too long.
  • the time threshold can be determined according to the frequency of the device capturing images. For example, if the device can capture 200 images within 1 second (s), when the first time period is more than 1 second, it may result in more than 200 historical images The information between frames is not very accurate, which brings errors to the determination of the current motion state, so the time threshold can be 1 second, and the first time period is within 1 second, such as 200 milliseconds and so on. That is, all historical image frames captured within 1 second before capturing the current image frame are used for judging the still state.
  • the quality of the historical image frames captured in the first time period may also be selected, and if the quality of the historical image frames is too low, the determination of the still state is not involved. For example, if two-dimensional points cannot be extracted from the historical image frame, it is considered that the quality of the historical image frame is too low.
  • the second parallax threshold may be fixed, or may be dynamically adjusted based on the time difference between the historical image frame and the current image frame.
  • the dynamic adjustment based on the time difference between the historical image and the current image frame may be adjusted according to the time difference between the historical image frame and the current image frame that is farthest from the current image frame within a preset time period. For example, the larger the time difference is, the higher the second parallax threshold is, and the smaller the time difference is, the lower the second parallax threshold is.
  • the preset time period here is longer than the first time period. All historical image frames in the preset time period here are selected partial frames from all historical image frames captured by the device.
  • some historical image frames within the first time period may be simultaneously within the preset time period.
  • the actual selection method can be based on that the number of 2D points extracted from the historical image frame is greater than a certain threshold or the parallax between the previous historical image frame and the previous historical image frame is greater than the corresponding threshold, etc., select a preset from all historical images.
  • the mode of the historical image frames in the time period is not limited here. It should be noted that, except that the historical image frames used in the static state judgment are historical image frames within the first time period, the historical image frames in the remaining steps are historical image frames within a preset time period. Therefore, the following is incorrect. Excessive interpretation of each historical image frame that occurs.
  • the historical image frames used in the static state determination may also be historical image frames within a preset time period.
  • the preset time period described herein may also be referred to as a sliding window.
  • the at least one sensory data is collected within a first time period prior to the current motion state.
  • Sensing data includes gyroscope sensing data and/or accelerometer sensing data. That is, the sensory data refers to the sensory data between each historical image frame in the first time period.
  • the discrete degree of the sensing data may be the standard deviation of the gyroscope sensing data and/or the accelerometer sensing data within a preset time period.
  • other parameters may also be used to represent the degree of dispersion of the sensing data, such as variance, etc., and the actual manner of determining the degree of dispersion is not limited here.
  • the current motion state of the device if one of the standard deviations between the gyroscope sensing data and the accelerometer sensing data is greater than the corresponding preset degree value, it is determined that the current motion state of the device does not belong to the stationary state.
  • the current motion state of the device is determined to be a stationary state only when the above-mentioned parallax determinations are both smaller than the second parallax threshold and the discrete degrees are both smaller than the corresponding preset level values.
  • the identification of static state by the combination of parallax and sensor data is more accurate than the identification of parallax or sensor data alone.
  • the second positioning parameter is optimized by using the second optimization method, wherein the second positioning parameter does not include a scale.
  • the difference between the second positioning parameter and the first positioning parameter includes that the first positioning parameter needs to optimize the scale obtained by initialization, while the second parameter does not need to optimize the scale obtained by initialization.
  • the second positioning parameters include other first positioning parameters except the scale. Because when the device is in a stationary state, the coordinates of the three-dimensional point cannot be obtained according to the triangulation, and only the orientation of the three-dimensional point relative to the device can be determined, and its depth information cannot be known. Therefore, the positioning parameters cannot be optimized by means such as reprojection error.
  • the positioning parameter is optimized by the second optimization method, that is, the second positioning parameter is constrained with the speed of zero as a constraint.
  • the historical image frames within the preset time period will not change.
  • zero speed as the constraint described in the embodiments of the present disclosure does not mean that only the zero speed is used to optimize the positioning parameters, but the constraint that the current image frame speed is zero is additionally added in the general visual optimization process.
  • the positioning parameters cannot be optimized by the re-projection error of the 3D point.
  • the pose of the current image frame will not change greatly compared with the pose of the previous historical image frame. Stability of positioning parameters.
  • the process of using the second optimization method to optimize the second positioning parameter it is determined whether there are a preset number of three-dimensional points in several historical image frames before the current image frame, and in several historical image frames When there is no preset number of 3D points, increase the constraint weight of zero speed; and/or decrease the constraint weight of zero speed when there are preset number of 3D points in several historical image frames.
  • the reprojection error of the 3D points on each frame can be used to optimize the positioning parameters directly, so as to improve the optimization effect of the positioning parameters, that is, Decrease the weight of the speed to zero, and when there are no preset number of 3D points in several historical image frames before the current image frame, the positioning parameters cannot be optimized by the reprojection error, and the speed can be increased by Zero constraints on positioning parameters.
  • the localization parameters can be well optimized in various states.
  • the method of optimizing the positioning parameter may be to use a first optimization method to optimize the first positioning parameter when the current motion state belongs to a non-stationary state, wherein the first positioning parameter includes a scale.
  • the first positioning parameter may also include the speed of the device, the offset of the inertial sensor, the speed of the device, the position, and the like.
  • the non-stationary state includes a first non-stationary state and a second non-stationary state. The range of motion of the device in the first non-stationary state and the second non-stationary state is different.
  • the non-stationary state is subdivided into a first non-stationary state and a second non-stationary state according to factors such as the motion range of the device. For example, if the movement range of the device is large enough, it is determined that the device belongs to the second non-stationary state, and if the movement range is too small, it can be considered that the device belongs to the first non-stationary state.
  • the relationship between the equipment in the first non-stationary state, the second non-stationary state, and the motion amplitude in the stationary state is as follows: the motion amplitude of the equipment in the stationary state ⁇ the motion amplitude of the equipment in the first non-stationary state ⁇ the second The magnitude of the movement of the device when it is not stationary.
  • the manner of determining whether the motion state of the device belongs to the first or second non-stationary motion state includes:
  • the historical image frames used may be historical image frames within a preset time period.
  • the features include one or more of the number of image frames, time difference, parallax, and sensory data.
  • the second non-stationary state here refers to a state in which the device has a larger movement range
  • the first non-stationary state is a motion state located between the stationary state and the second non-stationary state.
  • the motion state of the device is determined by the motion start frame and subsequent historical image frames and the current image frame, which reduces the interference caused by the still image frames before the motion start frame and makes the determination of the motion state of the device more accurate.
  • the method for determining the motion start frame includes: sorting the second time difference between the historical image frame and the current image frame from least to most, and sequentially judging whether the third time difference between each adjacent two historical image frames is greater than the third time difference between the two adjacent historical image frames.
  • Two time thresholds When it is determined that the third time difference between two adjacent historical image frames is greater than the second time threshold, the historical image frame with the smaller second time difference between the two adjacent historical image frames is used as the motion start frame. That is, the second time difference refers to the time difference between each historical image frame and the current image frame, and the third time difference refers to the time difference between every two adjacent historical image frames. At this time, the third time difference does not include the time difference between the current image frame and its adjacent historical image frames.
  • the historical image frames in the determination of the motion start frame are historical image frames within a preset time period.
  • the historical image frames within the preset time period satisfy the LIFO principle. Removing, of course, removing from the preset time period does not mean that the historical image frame will be deleted from the storage space. The storage space will still retain the pose of the key frame and the relationship between it and the previous historical image frame. sensor data.
  • the time difference between the last two historical image frames captured is greater than a certain threshold, the earliest historical image frame captured within the preset time period will be removed from the preset time period.
  • the historical image frame captured at the penultimate time within the preset time period is removed from the preset time period. In this way, there will be a significant time difference between the motion start frame and the historical image frames that are adjacent to the motion start frame and taken before the motion start frame within the preset time period.
  • the time difference determines the motion start frame.
  • the motion start frame is determined according to the time difference between two adjacent historical image frames, so that the identification process of the motion start frame is simple.
  • the preset conditions include at least one of the following: 1.
  • the first time difference between the motion start frame and the current image frame is greater than or equal to a first time difference threshold. By ensuring that there is a sufficient time difference between the motion start frame and the current image frame, sufficient sensor data can be used for integration, and the accuracy of positioning parameter optimization is improved.
  • the number of historical image frames between the motion start frame and the current image frame is greater than or equal to the first number threshold. Among them, it is ensured that there are enough historical image frames between the motion start frame and the current image frame so that enough sensor data can participate in the integration, so that the optimized positioning parameters are more accurate.
  • the number of preset two-dimensional point pairs in the motion start frame and the current image frame is greater than or equal to the second number threshold.
  • the preset two-dimensional point pair refers to a two-dimensional point pair obtained by matching two-dimensional points between different historical image frames of two-dimensional points extracted from each historical image frame.
  • Two-dimensional point matching such as feature matching, that is, calculating the distance between the extracted two-dimensional points, that is, the descriptors of key points. If the distance is less than a certain threshold, the matching is considered successful, and there are two more
  • the method of dimensional point matching may be performing block matching by an optical flow method, etc., and the method for obtaining the preset two-dimensional point pair is not limited here.
  • the parallax of the preset two-dimensional point pair is greater than or equal to the first parallax threshold.
  • the number of preset two-dimensional point pairs whose parallax is greater than or equal to the first parallax threshold is greater than or equal to the second number threshold.
  • the number of preset two-dimensional point pairs in any historical image frame between the motion start frame and the current image frame and the current image frame may be greater than or equal to the second number threshold, here Not limited. 4.
  • the discrete degree of the accelerometer sensing data between the motion start frame and the current image frame is greater than or equal to a preset degree threshold.
  • a fifth judgment condition can also be added, that is, the discrete degree of the gyroscope sensor data between the motion start frame and the current image frame is greater than or equal to the corresponding Preset level threshold.
  • the current motion state of the device is determined to be the second non-stationary state, that is, if the feature does not satisfy any one of the first four conditions, the current motion state of the device is determined does not belong to the second non-stationary state.
  • the determination of whether the motion state of the device belongs to the stationary state or the non-stationary state may be performed simultaneously. If it is considered that the motion state of the device does not belong to the stationary state, nor does it satisfy any one of the above-mentioned first four preset conditions, the motion state of the device is considered to be the first non-stationary state.
  • the results are more accurate.
  • the reprojection error of each 3D point on the current image frame and the historical image frame can be used to optimize the first positioning parameter. That is, when the device is in a non-stationary state, according to the movement of the device, the current image frame captured by the device and the two-dimensional points observed in each historical image frame can be triangulated to obtain the coordinates of the three-dimensional point, and then the obtained three-dimensional point can be obtained by triangulating the coordinates of the three-dimensional point.
  • Projection points are obtained by re-projecting on the current image frame and each historical image frame, wherein the distance between the projection point corresponding to the 3D point in each image frame and the observed 2D point corresponding to the 3D point is the reprojection error.
  • the reprojection error can be optimized simultaneously for the 3D point and the pose of each image frame. By changing the pose of the camera, the reprojection error is minimized, so as to achieve the effect of optimizing the camera pose and optimizing the coordinates of the 3D points. Because the coordinates of the three-dimensional points are determined, the pose of the camera is also determined, that is, the distance and angle between adjacent frames are determined, and other positioning parameters can be determined in combination with the sensor data between adjacent image frames.
  • the first positioning parameter is optimized by using a preset kinematic equation group.
  • one of the historical image frames is used as the motion start frame, and the positional relationship and pre-integration information of adjacent image frames between the motion start frame and the current image frame are acquired.
  • the manner of determining the motion start frame is as described above. If the motion start frame has been determined in the process of determining the motion state, the step of determining the motion start frame may not be performed here.
  • the adjacent image frames between the motion start frame and the current image frame described herein are historical image frames within a preset time period.
  • the position relationship includes a distance relationship and an angle relationship.
  • the pre-integration information includes the pre-integration of the data measured by the gyroscope and the pre-integration of the data measured by the accelerometer. Substitute the positional relationship and pre-integration information into the preset kinematic equations, and solve to obtain the scale and the speed of the device. Among them, the angle in the positioning parameters, the gyroscope offset in the sensor and the accelerometer offset still use the results obtained by initialization, and substitute the results with the above positional relationship and pre-integration information into the kinematic equations to solve size and speed.
  • the positioning parameters include Wherein X k represents all the positioning parameters at time k, X C represents the currently used positioning parameters, and X S represents the positioning parameters corresponding to each historical key frame within the preset time period.
  • G represents the world coordinate system
  • C represents the camera coordinate system
  • Ck R G represents the rotation matrix from the world coordinate system to the camera coordinate system at time k
  • G p Ck represents the position of the camera in the world coordinate system at time k
  • G v Ck Represents the motion speed in the world coordinate system at time k
  • b wk represents the offset of the gyroscope at time k
  • b ak represents the offset of the accelerometer at time k
  • T represents the transposition.
  • r Ii,j represents the error between the pre-integrated value and the prior value from frame i to frame j.
  • ⁇ Ii,j represents the pre-integrated covariance matrix corresponding to r Ii,j .
  • the preset kinematic equation group is composed of a plurality of the aforementioned preset kinematic equations.
  • the scale and velocity can be solved linearly by using the pre-integration information of the sensor data and the initialized positioning parameters, wherein the initialized positioning parameters include three-dimensional points and their reprojection errors, and these two types of parameters can also be used.
  • the construction method of the preset kinematic equation is not limited to the above-mentioned kinematic equation.
  • the scale and velocity are solved, which reduces the difficulty of solving the scale and velocity.
  • the obtained scale and velocity are a set of solutions that minimize the error of the above-mentioned preset kinematic equations. That is, a set of solutions that minimize the sum of errors of each preset kinematic equation.
  • the scale and velocity obtained by solving are more accurate than other solutions, and the results of subsequent tracking and positioning are more accurate.
  • the obtained scale and speed are substituted into the preset kinematic equation system, and the error of each kinematic equation in the preset kinematic equation system is obtained. If the error of one of the equations is greater than a preset error threshold, the scale and velocity solution is deemed to have failed.
  • the current image frame is tracked and positioned according to the initialized positioning parameters. If the current image frame meets the selection requirements of historical image frames within the preset time period after tracking and positioning, the current image frame can enter the preset Set the time period as the latest historical image frame within the preset time period. Then a new image frame is acquired, the motion state of the device is determined, and subsequent steps are performed. If the errors of each kinematic equation system are less than or equal to the preset error threshold, it is determined that the scale and velocity are successfully solved. And execute the optimized first positioning parameter with the solved scale and the speed of the device. That is, the scale and velocity obtained from the original initialization are updated with the scale and velocity obtained by the solution.
  • the remaining positioning parameters including the angle, gyroscope and accelerometer offset can directly use the previously initialized convergence value, of course, can use the preset value, such as the factory calibration value or the value used in the last tracking and positioning.
  • the preset value such as the factory calibration value or the value used in the last tracking and positioning.
  • each historical image frame after scale optimization is optimized.
  • one of the historical image frames is selected as the first reference frame.
  • the historical image frame with the largest time difference between the preset time period and the current image frame is selected as the reference frame.
  • the optimized scale the distance between the first reference frame and its adjacent historical image frames is determined. That is, the optimized scale is used to replace the original scale to find the distance between two adjacent historical image frames. According to the distance, the positions of adjacent image frames of the first reference frame are preliminarily adjusted.
  • the distance between the first reference frame obtained by solving the original scale and its adjacent image frames is 1 meter
  • the distance between the first reference frame obtained by solving the optimized scale and its adjacent historical image frames is 2 m
  • move the adjacent image frames of the first reference frame so that the distance between the first reference frame and its adjacent frames is 2 m.
  • the preliminarily adjusted adjacent image frame is used as a new first reference frame
  • the optimized scale is used again to determine the distance between the first reference frame and its adjacent historical image frame and subsequent steps.
  • the historical image frames within the preset time period have been preliminarily adjusted.
  • the adjacent image frame is used as the new first reference frame, it is not necessary to continue to adjust the position of the previous first reference frame adjacent to the new first reference frame, and to continue to adjust the rest of the non-preliminarily adjusted The location of the historical image frame.
  • the historical image frame with the smallest time difference between the preset time period and the current image frame is used as the second reference frame. That is, the previous historical image frame of the current image frame.
  • the position of the second reference frame is finally adjusted to its position before the preliminary adjustment. That is, for the second reference frame, the position after the preliminary adjustment and the final adjustment does not change relative to the position before the preliminary adjustment. Then, based on the position change of the second reference frame in the final adjustment process, the positions of the remaining historical image frames except the second reference frame within the preset time period are adjusted.
  • the scale between adjacent historical image frames will not change, which is simply a translation process of the entire historical image frame within a preset time period.
  • the position of each historical image frame before and after the preliminary adjustment changes too much, that is, there is a phenomenon of incoherence between the historical image frames before and after the preliminary adjustment. Moving the position after the preliminary adjustment to the position before the preliminary adjustment can alleviate this incoherent phenomenon, so that the effect of subsequent tracking and positioning is better.
  • each 3D point after scale optimization is optimized.
  • the position of each historical image frame and the depth of each 3D point can be optimized.
  • There are many ways to optimize the depth of 3D points For example, using each historical image frame after position optimization to re-triangulate the matched 2D point pairs in each frame to obtain the coordinates of new 3D points, and use the new 3D point.
  • the point updates the original corresponding three-dimensional point, thereby realizing the update of each three-dimensional point.
  • the optimization method for the depth of the three-dimensional point can also be based on the historical image frame where the three-dimensional point is first observed, and the depth of the three-dimensional point is adjusted according to the scale.
  • the orientation of the 3D point relative to the first observation of the 3D point is known, and the depth is obtained according to the scale before optimization, it can be solved according to the orientation relationship between the original 3D point and the historical image frame where the 3D point was observed for the first time After the final adjustment, the orientation relationship between the historical image frames of the 3D point is observed for the first time, and the depth of the 3D point is obtained according to the new scale to obtain the coordinates of the new 3D point, thereby updating the coordinates of each 3D point.
  • the historical image frame located before the motion start frame within the preset time period is deleted.
  • Deleting means removing the historical image frame before the motion start frame from the second preset time period, but the pose and sensory data of the removed historical image frame may also be stored in the storage unit.
  • the uncertainty of at least one of the position, velocity, and accelerometer offset of the device is increased.
  • Spend Because the position, velocity and accelerometer bias of the device may converge with overconfidence during the previous initialization process, that is, the error of position, velocity and accelerometer bias may be large, therefore, their uncertainty needs to be increased to reduce Follow-up influence in the tracking and positioning process, and then continuously update the position, speed and accelerometer according to the motion information obtained in the subsequent tracking and positioning.
  • the motion state of the device is further obtained, and then the positioning parameters are optimized according to the motion state, so that the subsequent movement of the device can be positioned according to the optimized parameters, which improves the accuracy of positioning. Spend.
  • the execution body of the positioning initialization method may be a positioning initialization device.
  • the positioning initialization method may be executed by a terminal device or a server or other processing device, wherein the terminal device may be: a virtual reality helmet, an augmented reality glasses, a People and vehicles, mobile robots, sweeping robots, flying equipment, user equipment (UE), mobile devices, user terminals, cellular phones, cordless phones, personal digital assistants (PDA), handheld devices, computing devices, Vehicle-mounted devices, wearable devices, and other devices that have both image sensors and inertial sensors.
  • the positioning initialization method may be implemented by the processor calling computer-readable instructions stored in the memory.
  • the positioning initialization device 30 includes: an acquisition module 31 , a determination module 32 and an optimization module 33 .
  • the acquisition module 31 is configured to acquire the motion information of the equipment after the positioning parameters of the equipment to be positioned are initialized;
  • the determination module 32 is configured to determine the current motion state of the equipment based on the motion information;
  • the optimization module 33 is configured to be based on the current motion state of the equipment , and optimize the positioning parameters to obtain the optimized positioning parameters.
  • the motion state of the device is further obtained, and then the positioning parameters are optimized according to the motion state, so that the subsequent movement of the device can be positioned according to the optimized parameters, which improves the accuracy of positioning. Spend.
  • the positioning parameters are divided into a first positioning parameter and a second positioning parameter; the optimization module 33 is further configured to use the current motion state of the device to optimize the positioning parameters to obtain the optimized positioning parameters, including: When the motion state belongs to a non-stationary state, use a first optimization method to optimize the first positioning parameter, wherein the first positioning parameter includes a scale; or, when the current motion state belongs to a stationary state, use the second optimization method A second positioning parameter is optimized, wherein the second positioning parameter is different from the first positioning parameter.
  • the above solution optimizes the positioning parameters by adopting different optimization methods when the device is in a stationary state or a non-stationary state, and can be applied to different motion scenarios.
  • the non-stationary state includes a first non-stationary state and a second non-stationary state; when the current motion state belongs to the non-stationary state, the optimization module 33 uses the first optimization method to optimize the first positioning parameter, Including: when the current motion state belongs to the first non-stationary state, using the reprojection error of each three-dimensional point on the current image frame and several historical image frames to optimize the first positioning parameter; wherein, the current image frame and The historical image frame is captured by the device, the current image frame is not used for tracking and positioning the device, and the historical image frame has been used for tracking and positioning the device; or, when the current motion state belongs to the second non-stationary state, use The preset kinematic equation system optimizes the first positioning parameter.
  • the non-stationary state is subdivided into multiple non-stationary states, so that when the device is in the non-stationary state, an appropriate optimization method can be selected according to the actual motion situation, and the optimization effect of the parameters can be improved.
  • the determining module 32 is further configured to determine the current motion state of the device based on the motion information, including at least one of the following steps: taking one of the historical image frames as the motion start frame; between the motion start frame and the current image frame; In the case where the characteristics of the When the features between the image frames satisfy the preset condition, it is determined that the current motion state is the second non-stationary state.
  • the motion state of the device is determined by the motion start frame and subsequent historical image frames and the current image frame, which reduces the interference caused by the still image frames before the motion start frame and makes the determination of the motion state of the device more accurate.
  • the preset conditions include at least one of the following: the first time difference between the motion start frame and the current image frame is greater than or equal to a first time threshold; the number of historical image frames between the motion start frame and the current image frame is greater than or equal to or equal to the first number threshold; the number of preset two-dimensional point pairs in the motion start frame and the current image frame is greater than or equal to the second number threshold, wherein the parallax of the preset two-dimensional point pairs is greater than or equal to the first parallax threshold; The discrete degree of the accelerometer sensing data between the motion start frame and the current image frame is greater than or equal to a preset degree threshold.
  • the greater the degree of dispersion between parallax and sensing data the greater the range of motion of the device, the greater the number of historical image frames and the greater the time difference, indicating that there is enough sensing data to optimize the positioning parameters, so that the positioning The optimization results of the parameters are more accurate.
  • the optimization module 33 is further configured to optimize the first positioning parameter by using a preset kinematic equation set, including: taking one of the historical image frames as the motion start frame, and obtaining the distance between the motion start frame and the current image frame. The position relationship and pre-integration information of adjacent image frames between them; Substitute the position relationship and pre-integration information into the preset kinematic equations, and solve to obtain the scale and the speed of the device; take the obtained scale and the speed of the device as the optimized The first positioning parameter.
  • the above scheme solves the scale and velocity while fixing the other positioning parameters that are not coupled with the scale, which reduces the difficulty of solving the scale and velocity.
  • the obtained scale and velocity are a set of solutions that minimize the error of the preset kinematic equations.
  • the scale and velocity obtained by solving are more accurate than other solutions, and the result of subsequent tracking and positioning is more accurate.
  • the optimization module 33 before using the obtained scale and the speed of the device as the optimized first positioning parameters, is further configured to: substitute the obtained scale and speed into a preset kinematic equation system to obtain a preset kinematic equation. Set the error of each kinematic equation in the kinematic equation group; if the error of each kinematic equation is less than or equal to the preset error threshold, it is determined that the scale and speed are successfully solved, and the obtained scale and the speed of the equipment are optimized. after the first positioning parameter.
  • the scale and velocity are considered to be successfully solved, which makes the scale and velocity solution process strict, and the obtained scale and velocity are more accurate. .
  • the determining module 32 is further configured to: sort according to the second time difference between the historical image frame and the current image frame in ascending order, and sequentially determine the third time difference between each adjacent two historical image frames. Whether the time difference is greater than the second time threshold; in the case of judging that the third time difference between two adjacent historical image frames is greater than the second time threshold, the historical image frame with the smaller second time difference in the two adjacent historical image frames is used as motion Start frame.
  • the motion start frame is determined according to the time difference between two adjacent historical image frames, so that the identification process of the motion start frame is simpler.
  • the optimization module 33 is further configured to: perform the scale optimization on the position of each historical image frame and/or the depth of each three-dimensional point. optimization.
  • the optimization module 33 is further configured to: select one of the historical image frames as the first reference frame; use the optimized scale to determine the distance between the first reference frame and its adjacent historical image frames; according to the distance, Preliminarily adjust the position of the adjacent historical image frame of the first reference frame; take the preliminarily adjusted adjacent historical image frame as a new first reference frame, and re-execute using the optimized scale to determine that the first reference frame is adjacent to it The distance between the historical image frames and the subsequent steps until the historical image frames within the preset time period have been preliminarily adjusted.
  • the distance relationship between the historical image frames is adjusted according to the calculated scale, and the positional relationship between the historical image frames is optimized, so that the result of subsequent device tracking and positioning is more accurate.
  • the optimization module 33 is further configured to: select the historical image frame with the smallest time difference between the preset time period and the current image frame. As the second reference frame; finally adjust the position of the second reference frame to the position before the preliminary adjustment; based on the position change of the second reference frame in the final adjustment process, adjust the position of the second reference frame except the second reference frame within the preset time period. The position of the remaining historical image frames.
  • the position of each historical image frame before and after the preliminary adjustment changes too much, that is, there is a discontinuity between the historical image frames before and after the preliminary adjustment. Moving the position of the image frame after the preliminary adjustment to the position before the preliminary adjustment can alleviate this incoherence phenomenon, so that the effect of subsequent tracking and positioning is better.
  • the optimization module 33 is further configured to: for each 3D point, adjust the depth of the 3D point according to the scale based on the historical image frame in which the 3D point is first observed.
  • the depth of the three-dimensional point is optimized through the calculated scale, so that more accurate coordinates of the three-dimensional point can be used for subsequent tracking and positioning of the device, thereby improving the accuracy of the tracking and positioning.
  • the optimization module 33 before optimizing the position of each historical image frame and/or the depth of each 3D point after the scale optimization is performed, is further configured to: delete the historical data before the motion start frame within the preset time period. image frames; and/or, after optimizing the position of each historical image frame and/or the depth of each 3D point after the scale is optimized, including: improving at least one of the position, velocity and accelerometer offset of the device of uncertainty.
  • the optimization module 33 is further configured to: delete the historical data before the motion start frame within the preset time period. image frames; and/or, after optimizing the position of each historical image frame and/or the depth of each 3D point after the scale is optimized, including: improving at least one of the position, velocity and accelerometer offset of the device of uncertainty.
  • the determining module 32 is further configured to: determine the current motion state of the device based on the motion information, including: the disparity between several historical image frames and the current image frame is smaller than the second disparity threshold, and/or at least When the discrete degree of a sensor data is less than the corresponding preset degree value, it is determined that the current motion state of the device is a static state; wherein, the second parallax threshold is fixed or based on the historical image frame and the current image frame.
  • the time difference is dynamically adjusted; at least one sensing data is collected in a first time period before the current motion state, including gyroscope sensing data and/or accelerometer sensing data.
  • the identification of the static state through the combination of parallax and sensor data is more accurate than the identification of single parallax or sensor data.
  • the optimization module 33 is further configured to: constrain the second positioning parameter with a speed of zero as a constraint.
  • the positioning parameters cannot be optimized by the re-projection error of the 3D point in a static state.
  • the constraint of zero speed By increasing the constraint of zero speed, the pose of the current image frame will not be larger than the pose of the previous historical image frame. changes to improve the stability of positioning parameters.
  • the optimization module 33 is further configured to: determine whether there are a preset number of 3D points in several historical image frames before the current image frame; if there is no preset number of 3D points in the several historical image frames, increase the speed. A constraint weight of zero; and/or, in the case of a predetermined number of 3D points in several historical image frames, reducing a constraint weight of zero speed. In the above solution, by reducing or increasing the weight of zero speed according to the number of three-dimensional points, the positioning parameters can be well optimized in various states.
  • the motion state of the device is further obtained, and then the positioning parameters are optimized according to the motion state, so that the subsequent movement of the device can be positioned according to the optimized parameters, which improves the accuracy of positioning. Spend.
  • FIG. 4 is a schematic structural diagram of an embodiment of the electronic device of the present disclosure.
  • the electronic device 40 includes a memory 41 and a processor 42, and the processor 42 is configured to execute program instructions stored in the memory 41, so as to implement the steps in any of the foregoing positioning initialization method embodiments.
  • the electronic device 40 may include, but is not limited to, mobile robots, handheld mobile devices, sweeping robots, unmanned vehicles, virtual reality helmets, augmented reality glasses, flight devices, microcomputers, desktop computers, servers, etc.
  • the device of the image sensor and the inertial sensor module, in addition, the electronic device 40 may also include mobile devices such as a notebook computer and a tablet computer, which are not limited herein.
  • the processor 42 is configured to control itself and the memory 41 to implement the steps in any of the above-mentioned embodiments of the positioning initialization method.
  • the processor 42 may also be referred to as a central processing unit (Central Processing Unit, CPU).
  • the processor 42 may be an integrated circuit chip with signal processing capability.
  • the processor 42 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the processor 42 may be jointly implemented by an integrated circuit chip.
  • the motion state of the device is further obtained, and then the positioning parameters are optimized according to the motion state, so that the subsequent movement of the device can be positioned according to the optimized parameters, which improves the accuracy of positioning. Spend.
  • FIG. 5 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present disclosure.
  • the computer-readable storage medium 50 stores program instructions 51 that can be executed by the processor, and the program instructions 51 are used to implement the steps in any of the above-mentioned embodiments of the positioning initialization method.
  • the motion state of the device is further obtained, and then the positioning parameters are optimized according to the motion state, so that the subsequent movement of the device can be positioned according to the optimized parameters, which improves the accuracy of positioning. Spend.
  • the functions or modules included in the apparatus provided in the embodiments of the present disclosure may be configured to execute the methods described in the above method embodiments.
  • the implementation process reference may be made to the descriptions in the above method embodiments. For brevity, here No longer.
  • the disclosed method and apparatus may be implemented in other manners.
  • the device implementations described above are only illustrative.
  • the division of modules or units is only a logical function division. In actual implementation, there may be other divisions.
  • units or components may be combined or integrated. to another system, or some features can be ignored, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
  • the integrated unit, if implemented as a software functional unit and sold or used as a stand-alone product, may be stored on a computer-readable storage medium.
  • the technical solutions of the present disclosure can be embodied in the form of software products in essence, or the part that contributes to the prior art, or all or part of the technical solutions, and the computer software product is stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods of the various embodiments of the present disclosure.
  • the aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .
  • the motion information of the device is obtained; based on the motion information, the current motion state of the device is determined; according to the current motion state of the device, the positioning parameters are optimized to obtain the optimized positioning parameter.
  • the positioning parameters are optimized according to the motion state of the equipment to obtain the optimized positioning parameters, so that the subsequent positioning of the equipment is more accurate.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

一种定位初始化方法和装置、设备、存储介质、程序产品，定位初始化方法包括：在对待定位设备的定位参数进行初始化之后，获取设备的运动信息（S11）；基于运动信息，确定设备的当前运动状态（S12）；根据设备的当前运动状态，对定位参数进行优化，得到优化后的定位参数（S13）。

Description

定位初始化方法和装置、设备、存储介质、程序产品
相关申请的交叉引用
本公开基于申请号为202110272183.2、申请日为2021年03月12日、申请名称为“定位初始化方法和相关装置、设备、存储介质”的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此以全文引入的方式引入本公开。
技术领域
本公开涉及定位技术领域，特别是涉及一种定位初始化方法和装置、设备、存储介质、程序产品。
背景技术
目前，视觉跟踪定位方法在计算机视觉、机器人、无人机、三维重建一级增强现实等领域中起着重要的作用。一般情况下，对视觉跟踪定位所需参数的初始化之后，直接利用初始化得到的结果对设备进行跟踪定位，这种做法仍有问题存在，例如，当设备的运动幅度不大的情况下，在进行初始化的过程中求解的参数可能误差较大，例如，尺度等参数，若直接使用这些参数对设备进行跟踪定位，会使得最终对设备进行跟踪定位的准确度较低。
发明内容
本公开至少提供一种定位初始化方法和装置、设备、存储介质和程序产品。
本公开第一方面提供了一种定位初始化方法，包括：在对待定位的设备的定位参数进行初始化之后，获取设备的运动信息；基于运动信息，确定设备的当前运动状态；利用设备的当前运动状态，对定位参数进行优化，得到优化后的定位参数。
因此，通过在对定位参数进行初始化之后，仍进一步获取设备的当前运动状态，然后根据当前运动状态对定位参数进行优化，使得能够根据优化之后的参数对设备后续的移动进行定位，提高了定位的准确度。
其中，定位参数分为第一定位参数和第二定位参数；根据设备的当前运动状态，对定位参数进行优化，得到优化后的定位参数，包括：在当前运动状态属于非静止状态的情况下，利用第一优化方式对第一定位参数进行优化，其中，第一定位参数包括尺度；或，在当前运动状态属于静止状态的情况下，利用第二优化方式对第二定位参数进行优化，其中，第二定位参数不同于第一定位参数。
因此，通过设备处于静止状态或非静止状态时采用不同优化方式，对定位参数进行优化，能够适用不同的运动场景。
其中，非静止状态包括第一非静止状态和第二非静止状态；在当前运动状态属于非静止状态的情况下，利用第一优化方式对第一定位参数进行优化，包括：在当前运动状态属于第一非静止状态的情况下，利用各三维点在当前图像帧及若干历史图像帧上的重投影误差，对所第一定位参数进行优化；其中，当前图像帧和历史图像帧为设备拍摄得到的，当前图像帧未用于对设备的跟踪定位，历史图像帧已用于对设备的跟踪定位；或，在当前运动状态属于第二非静止状态的情况下，利用预设运动学方程组对第一定位参数进行优化。
因此，将非静止状态又细分为多种非静止状态，使得设备处于非静止状态下时又可根据实际的运动情况选择合适的优化方式，提高对参数的优化效果。
其中，基于运动信息，确定设备的当前运动状态，包括以下至少一个步骤：以其中一历史图像帧作为运动开始帧；在运动开始帧至当前图像帧之间的特征不满足预设条件 的情况下，确定当前运动状态为第一非静止状态；其中，特征包括图像帧数量、时差、视差和传感数据中的一个或多个；在运动开始帧至当前图像帧之间的特征满足预设条件的情况下，确定当前运动状态为第二非静止状态。
因此，通过运动开始帧及以后的历史图像帧以及当前图像帧对设备的运动状态进行确定，减小了运动开始帧以前的静止图像帧造成的干扰使得设备的运动状态的确定更准确。
其中，预设条件包括以下至少一个：运动开始帧与当前图像帧之间的第一时间差大于或等于第一时间阈值；运动开始帧至当前图像帧之间的历史图像帧数量大于或等于第一数量阈值；运动开始帧与当前图像帧中的预设二维点对的数量大于或等于第二数量阈值，其中，预设二维点对的视差大于或等于第一视差阈值；运动开始帧至当前图像帧之间的加速度计传感数据的离散程度大于或等于预设程度阈值。
因此，视差和传感数据的离散程度越大，则说明设备的运动幅度越大，历史图像帧的数量越多以及时差越大，说明有足够的传感数据用于优化定位参数，使得定位参数的优化结果更准确。
其中，利用预设运动学方程组对第一定位参数进行优化，包括：以其中一历史图像帧作为运动开始帧，并获取运动开始帧至当前图像帧之间的相邻图像帧的位置关系以及预积分信息；将位置关系以及预积分信息代入预设运动学方程组，求解得到尺度以及设备的速度；将求解得到的尺度以及设备的速度作为优化后的第一定位参数。
因此，在固定其余与尺度耦合不大的定位参数的情况下，求解尺度与速度，降低了对尺度以及速度的求解难度。
其中，求解得到的尺度以及速度为使得预设运动学方程组误差最小的一组解。
因此，通过将使得预设运动学方程组误差最小的一组解作为求解得到的尺度与速度，使得求解得到的尺度与速度相对其他解而言更准确，后续跟踪定位的结果更准确。
其中，在将求解得到的尺度以及设备的速度作为优化后的第一定位参数之前，方法还包括：将求解得到的尺度以及速度代入预设运动学方程组，得到预设运动学方程组中各运动学方程的误差；若各运动学方程的误差均小于或等于预设误差阈值，则确定尺度以及速度求解成功，并执行将求解得到的尺度以及设备的速度作为优化后的第一定位参数。
因此，只有在所有的运动学方程的误差均小于或等于预设误差阈值的情况下，才认为尺度和速度求解成功，使得尺度和速度的求解过程严格，从而求解得到的尺度以及速度更准确。
其中，以其中一历史图像帧作为运动开始帧，包括：按照历史图像帧与当前图像帧之间的第二时间差从少到多进行排序，并依序判断每相邻两历史图像帧之间的第三时间差是否大于第二时间阈值；在判断相邻两历史图像帧之间的第三时间差大于第二时间阈值的情况下，将相邻两历史图像帧中第二时间差更小的历史图像帧作为运动开始帧。
因此，根据相邻两历史图像帧之间的时间差确定运动开始帧，使得对运动开始帧的认定过程简便。
其中，在利用预设运动学方程组对第一定位参数进行优化之后，方法还包括：对尺度优化后的各历史图像帧的位置和/或各三维点的深度进行优化。
因此，通过将尺度优化后的各历史图像帧和三维点的深度进行优化，使得在后续跟踪定位过程中，能够有更准确的先验信息，从而跟踪定位的结果更准确。
其中，对尺度优化后的各历史图像帧的位置进行优化，包括：选择其中一历史图像帧作为第一参考帧；利用优化后的尺度，确定第一参考帧与其相邻历史图像帧之间的距离；按照距离，初步调整第一参考帧的相邻历史图像帧的位置；以初步调整后的相邻历史图像帧作为新的第一参考帧，并重新执行利用优化后的尺度，确定第一参考帧与其相 邻历史图像帧之间的距离及其后续步骤，直至预设时间段内的历史图像帧均已初步调整完成。
因此，将历史图像帧之间的距离关系根据计算得到的尺度进行调整，优化各历史图像帧之间的位置关系，使得后续对设备跟踪定位的结果更准确。
其中，在预设时间段内的历史图像帧均已初步调整完成之后，方法还包括：以预设时间段内与所当前图像帧之间的时间差最少的历史图像帧作为第二参考帧；将第二参考帧的位置最终调整为初步调整之前所在的位置；基于第二参考帧在最终调整过程中的位置变化，调整预设时间段内除第二参考帧以外的其余历史图像帧的位置。
因此，在初步调整完成之后，各历史图像帧在初步调整前后的位置变化太大，即初步调整前后历史图像帧之间出现不连贯的现象，通过将与当前图像帧之间时间差最小的历史图像帧的初步调整后的位置移动至初步调整前的位置能够缓和这种不连贯的现象，使得后续跟踪定位的效果更佳。
其中，对尺度优化后的各三维点的深度进行优化，包括：对于各三维点，以首次观测到三维点的历史图像帧为基准，按照尺度调整三维点的深度。
因此，通过计算得到的尺度对三维点的深度进行优化使得后续能够使用更准确的三维点的坐标对设备进行跟踪定位，提高跟踪定位的准确度。
其中，在对尺度优化后的各历史图像帧的位置和/或各三维点的深度进行优化之前，方法还包括：删除预设时间段内位于运动开始帧之前的历史图像帧；和/或，在对尺度优化后的各历史图像帧的位置和/或各三维点的深度进行优化之后，包括：提高设备的位置、速度以及加速度计偏置量中的至少一者的不确定度。
因此，通过删除运动开始帧以前的历史图像帧能够减少拍摄时间过长的历史图像帧对优化结果的影响。提高设备的位置、速度以及加速度计偏置量的不确定度能够减少因为收敛过度，导致后期定位结果不准确的结果出现。
其中，基于运动信息，确定设备的当前运动状态，包括：在若干历史图像帧与当前图像帧之间的视差均小于第二视差阈值，和/或至少一个传感数据的离散程度小于对应的预设程度值的情况下，确定设备的当前运动状态为静止状态；其中，第二视差阈值是固定的或基于历史图像帧与当前图像帧之间的时间差动态调整的；至少一个传感数据是在当前运动状态之前的第一时间段内采集的，包括陀螺仪传感数据和/或加速度计传感数据。
因此，通过视差和传感数据的结合对静止状态的认定，较于单一视差或传感数据的认定而言，前者的认定结果更准确。
其中，利用第二优化方式对第二定位参数进行优化，包括：以速度为零作为约束，对第二定位参数进行约束。
因此，在静止状态下无法通过三维点的重投影误差对定位参数进行优化，通过增加速度为零的约束，使得当前图像帧的位姿较上一历史图像帧的位姿不会发生较大变化，提高定位参数的稳定性。
其中，方法还包括：判断当前图像帧之前的若干历史图像帧中是否有预设数量三维点；在若干历史图像帧中没有预设数量三维点的情况下，提高速度为零的约束权重；和/或，在若干历史图像帧中有预设数量三维点的情况下，降低速度为零的约束权重。
因此，通过根据三维点的数量降低或提高速度为零的权重，使得能够在各种状态下都能够很好地对定位参数进行优化。
本公开第二方面提供了一种定位初始化装置，包括：获取模块，配置为在对待定位的设备的定位参数进行初始化之后，获取设备的运动信息；确定模块，配置为基于运动信息，确定设备的当前运动状态；优化模块，配置为根据设备的当前运动状态，对定位参数进行优化，得到优化后的定位参数。
本公开第三方面提供了一种电子设备，包括存储器和处理器，处理器配置为执行存储器中存储的程序指令，以实现上述定位初始化方法。
本公开第四方面提供了一种计算机可读存储介质，其上存储有程序指令，程序指令被处理器执行时实现上述定位初始化方法。
本公开第五方面提供了一种计算机程序产品，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行配置为实现所述定位初始化方法。
上述方案，通过在对定位参数进行初始化之后，仍进一步获取设备的运动状态，然后根据运动状态对定位参数进行优化，使得能够根据优化之后的定位参数对设备后续的移动进行定位，提高了定位的准确度。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。
图1是本公开定位初始化方法一实施例的第一流程示意图；
图2是本申请定位初始化方法一实施例中的第二流程示意图；
图3是本公开定位初始化装置一实施例的结构示意图；
图4是本公开电子设备一实施例的结构示意图；
图5是本公开计算机可读存储介质一实施例的结构示意图。
具体实施方式
以下面结合说明书附图，对本公开实施例的方案进行详细说明。
以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、接口、技术之类的实施细节，以便透彻理解本公开。
本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。此外，本文中的“多”表示两个或者多于两个。另外，本文中术语“至少一种”表示多种中的任意一种或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。
请参阅图1，图1是本公开定位初始化方法一实施例的第一流程示意图。该方法可以包括如下步骤：
步骤S11：在对待定位的设备的定位参数进行初始化之后，获取设备的运动信息。
其中，对待定位的设备的定位参数进行初始化的方式可以包括运动初始化和静止初始化。运动初始化的方式可以是将视觉初始化与惯性传感器初始化分开，即先进行视觉初始化，在视觉初始化完成之后，再将初始化结果与惯性传感器初始化结果对齐，从而完成对定位参数的初始化。静止初始化所适用的情况是，设备处于相对静止状态时。因为设备处于相对静止状态，无法通过三角化等方式从各帧图像观测到的二维点得到对应的三维点坐标，也就无法得到准确的尺度信息，一般的做法就是假定一个尺度，继而根据假定的尺度确定其他定位参数。
获取设备的运动信息的方式可以是设备拍摄得到的图像帧之间的视差、时差等，也可以是各图像帧之间的传感数据等。对设备进行跟踪定位过程中采集的图像帧可以是任意具有摄像组件的设备拍摄得到，当然还可以是从其他设备中获取到的图像，也可以是经过选帧、调整亮度、分辨率等的图像。设备可以是采集图像帧的设备本身。其中，其他设备指的是分别利用不同中央处理器才能进行操作的设备。例如，在无人机驾驶过程 中，无人机即为本公开实施例所述的设备。或者扫地机器人在工作中，扫地机器人也是本公开实施例所述的设备。即设备在一环境中，实现对自己在环境中的位置进行定位和/或导航。
步骤S12：基于运动信息，确定设备的当前运动状态。
其中，当前运动状态可以是静止状态，也可以是非静止状态。确定当前运动状态的方式可以是根据设备的运动幅度进行判断，例如根据前后拍摄得到各图像帧中二维点对之间的视差进行判断，当然，也可以根据传感数据进行判断。例如，当两图像帧之间的视差大于一个预设值时，则确定设备的当前运动状态为非静止状态。
步骤S13：根据设备的当前运动状态，对定位参数进行优化，得到优化后的定位参数。
其中，若预设的设备的当前运动状态为多种，则每种当前运动状态可以对应有相应的优化方式对定位参数进行优化，例如预设的设备的当前运动状态包括静止状态和非静止状态，则可以使用与静止状态或非静止状态对应的优化方式对定位参数进行优化。其中，优化之后的参数可以直接进行跟踪定位，当然，也可以在进行一次或多次优化之后，再使用其对设备进行跟踪定位。
上述方案，通过在对定位参数进行初始化之后，仍进一步获取设备的当前运动状态，然后根据当前运动状态对定位参数进行优化，使得能够根据优化之后的参数对设备后续的移动进行定位，提高了定位的准确度。
一些公开实施例中，定位参数分为第一定位参数和第二定位参数。其中，第一定位参数包括尺度，第二定位参数不同于第一定位参数。其中，第二定位参数不同于第一定位参数，可以理解为第二定位参数与第一定位参数所包含的参数不完全相同，例如，第二定位参数可以不包括尺度。同时参见图2，图2是本公开定位初始化方法一实施例中的第二流程示意图。如图2所示：上述步骤S13包括步骤S131和步骤S132。其中，步骤S131：在当前运动状态属于静止状态的情况下，利用第二优化方式对第二定位参数进行优化，步骤S132：在当前运动状态属于非静止状态的情况下，利用第一优化方式对第一定位参数进行优化，其中，第一优化方式与第二优化方式不同。通过设备处于静止状态或非静止状态时采用不同优化方式，对定位参数进行优化，能够适用不同的运动场景。即，根据设备的当前运动状态确定是执行步骤S131还是执行步骤S132。
在实施中，确定设备处于静止状态和非静止状态的方式包括，在若干历史图像帧与当前图像帧之间的视差均小于第二视差阈值，和/或至少一个传感数据的离散程度小于对应的预设程度值的情况下，确定设备的当前运动状态为静止状态。其中，静止状态判定中使用的历史图像帧可以是当前图像帧以前第一时间段内所有拍摄的图像帧。进一步地，当前图像帧和历史图像帧为设备拍摄得到，当前图像帧未用于设备的跟踪定位，而历史图像帧已用于对设备的跟踪定位。即，当前图像帧仅进行过图像处理，即特征提取和匹配，暂未进行过跟踪定位，而历史图像帧不仅经过图像处理，还进行过跟踪定位。其中，这里的跟踪定位包括利用历史图像帧确定过设备在世界坐标系下的位置。其中，第一时间段可以根据需求自行设定，不做限制。将第一时间段内的所有历史图像帧分别与当前图像帧进行视差的判断，若存在某一历史图像帧与当前图像帧的视差大于第二视差阈值，则认定设备的当前运动状态不属于静止状态。在一些可能的实现方式中，第一时间段内的历史图像帧的数量大于或等于2帧。本公开实施例中，历史图像帧的数量为3帧即以上。通过设置多帧历史图像帧进行视差的判断可以减小因为历史图像帧的数量过少带来的误判，即提高了静止状态判断的准确度。同时，第一时间段也不可过长，若第一时间段过长则会导致拍摄时间过长的历史图像帧对应的信息给当前运动状态的确定带来误差。其中，第一时间段是否过长可采用是否超过一时间阈值来确定，如若第一时间段超过时间阈值则确定为过长。该时间阈值可根据设备拍摄图像的频率确定，例如，设备在1 秒(s)内能够拍摄200张图像的情况下，当第一时间段为1秒以上，则可能导致200张以上的历史图像帧之间的信息不太准确，给当前运动状态的确定带来误差，故时间阈值可以为1秒，第一时间段则为1秒以内，例如200毫秒等等。即在拍摄当前图像帧以前1秒内拍摄的所有历史图像帧皆用于静止状态的判断。当然，一些公开实施例中，还可对第一时间段内拍摄的历史图像帧的质量进行选择，若历史图像帧的质量过低，则不参与静止状态的判断。例如，历史图像帧中无法提取到二维点的情况下，则认为历史图像帧的质量过低。
在一些可能的实现方式中，第二视差阈值可以是固定的，也可以是基于历史图像帧与当前图像帧之间的时间差动态调整的。其中，基于历史图像与当前图像帧之间的时间差动态调整可以是根据预设时间段内与当前图像帧相隔最远的历史图像帧与当前图像帧之间的时间差大小进行调整。例如，时间差越大，则第二视差阈值将调高，若时间差越小，则将第二视差阈值调低。其中，这里的预设时间段长于第一时间段。这里的预设时间段内的所有历史图像帧是从设备拍摄得到的所有历史图像帧中选出的部分帧。即，第一时间段内的部分历史图像帧可能同时位于预设时间段内。实际实现的选择方式可以是根据历史图像帧中提取得到的二维点数目大于某一阈值或者与其上一历史图像帧之间的视差大于对应的阈值等等，从所有的历史图像中选择预设时间段内的历史图像帧的方式，此处不做限定。需要说明的是，除了在静止状态判断中使用的历史图像帧为第一时间段内的历史图像帧，其余步骤中的历史图像帧均为预设时间段内的历史图像帧，因此，以下不对每个出现的历史图像帧进行过多解释。当然，在其他实施例中，静止状态判断中使用的历史图像帧也可以是预设时间段内的历史图像帧。在一些实施例中，本文所述的预设时间段也可以称为滑窗。
在一些可能的实现方式中，至少一个传感数据是在当前运动状态之前的第一时间段内采集的。传感数据包括陀螺仪传感数据和/或加速度计传感数据。也就是，传感数据指的是第一时间段内各历史图像帧之间的传感数据。其中，传感数据的离散程度可以是陀螺仪传感数据和/或加速度计传感数据在预设时间段内的标准差。当然，在其他实施例中，还可使用其他的参数用于表征传感数据的离散程度，例如方差等等，此处对于离散程度的实际确定方式不做限定。
本公开实施例中，若陀螺仪传感数据与加速度计传感数据的其中一个标准差大于其对应的预设程度值，则认定设备的当前运动状态不属于静止状态。在一些实施方式中，只有在上述视差判定中均小于第二视差阈值且离散程度均小于对应的预设程度值的情况下，才确定设备的当前运动状态为静止状态。
通过视差和传感数据的结合对静止状态的认定，较于单用视差或传感数据的认定而言，前者的认定结果更准确。
在当前运动状态属于静止状态的情况下，利用第二优化方式对第二定位参数进行优化，其中，第二定位参数不包括尺度。其中，第二定位参数与第一定位参数的区别包括，第一定位参数需要对初始化得到的尺度进行优化，而第二参数中不需要对初始化得到的尺度进行优化。第二定位参数中包括除尺度以外的其他第一定位参数。因为当设备属于静止状态时，无法根据三角化得到三维点的坐标，仅能确定三维点相对于设备的朝向，并不知道其深度信息。因此，无法通过重投影误差等方式对定位参数进行优化。因此，采用第二优化方式对定位参数进行优化，即以速度为零作为约束，对第二定位参数进行约束。在这种情况下，预设时间段内的历史图像帧不会发生变化。当然，本公开实施例所述的以速度为零作为约束并不是说仅用速度为零去优化定位参数，而是在一般的视觉优化过程中另外再加入当前图像帧速度为零的约束。
在静止状态下无法通过三维点的重投影误差对定位参数进行优化，通过增加速度为零的约束，使得当前图像帧的位姿较上一历史图像帧的位姿不会发生较大变化，提高定 位参数的稳定性。
在一些可能的实现方式中，在使用第二优化方式对第二定位参数进行优化的过程中，判断当前图像帧之前的若干历史图像帧中是否有预设数量三维点，在若干历史图像帧中没有预设数量三维点的情况下，提高速度为零的约束权重；和/或，在若干历史图像帧中有预设数量三维点的情况下，降低速度为零的约束权重。其中，权重越高，则说明其置信度越高，其对设备跟踪定位的影响更大；若权重越低，则说明其置信度越低，其对设备跟踪定位的影响更小。即，如果当前图像帧之前的若干历史图像帧中具有预设数量三维点，则可以直接使用三维点在各帧上的重投影误差对定位参数进行优化，提高对定位参数的优化效果，也就是降低速度为零的权重，而在当前图像帧之前的若干历史图像帧中不具有预设数量的三维点时，无法很好地通过重投影误差对定位参数进行优化，此时反而可以提高速度为零对定位参数的约束。
通过根据三维点的数量降低或提高速度为零的权重，使得能够在各种状态下都能够很好地对定位参数进行优化。
利用设备的当前运动状态，对定位参数进行优化的方式可以是在当前运动状态属于非静止状态的情况下，利用第一优化方式对第一定位参数进行优化，其中，第一定位参数包括尺度。第一定位参数还可包括设备的速度、惯性传感器的偏置量、设备的速度、位置等等。其中，本公开实施例中，非静止状态包括第一非静止状态和第二非静止状态。其中，设备在第一非静止状态和第二非静止状态下的运动幅度不同。在一些可能的实现方式中，根据设备的运动幅度等因素，将非静止状态细分为第一非静止状态和第二非静止状态。例如，设备的运动幅度够大，则认定设备属于第二非静止状态，若运动幅度太小，则可以认为设备属于第一非静止状态。其中，设备分别处于第一非静止状态、第二非静止状态以及静止状态下的运动幅度之间的关系如下：静止状态下设备的运动幅度<第一非静止状态下设备的运动幅度<第二非静止状态下设备的运动幅度。
确定设备的运动状态属于第一或第二非静止运动状态的方式包括：
以其中一历史图像帧作为运动开始帧。在运动开始帧至当前图像帧之间的特征不满足预设条件的情况下，确定当前运动状态为第一非静止状态。在运动开始帧至当前图像帧之间的特征满足预设条件的情况下，确定当前运动状态为第二非静止状态。其中，在非静止状态的判定中，使用的历史图像帧可以是预设时间段内的历史图像帧。其中，特征包括图像帧数量、时差、视差和传感数据中的一个或多个。其中，这里的第二非静止状态指的是设备的运动幅度较大的状态，而第一非静止状态则是位于静止状态和第二非静止状态之间的运动状态。因为运动开始帧以前的历史图像帧中无法获知三维点的深度信息，因此对于尺度的求解没有任何帮助，反而可能因为拍摄的时间过长而引起求解尺度等定位参数的过程中引起误差，因此，本公开实施例仅考虑运动开始帧以后的历史图像帧。
通过运动开始帧及以后的历史图像帧以及当前图像帧对设备的运动状态进行确定，减小了运动开始帧以前的静止图像帧造成的干扰使得设备的运动状态的确定更准确。
其中，运动开始帧的确定方式包括：按照历史图像帧与当前图像帧之间的第二时间差从少到多排序，并依序判断每相邻两历史图像帧之间的第三时间差是否大于第二时间阈值。在判断相邻两历史图像帧之间的第三时间差大于第二时间阈值的情况下，将相邻两历史图像帧中第二时间差更小的历史图像帧作为运动开始帧。即，第二时间差是各历史图像帧分别与当前图像帧之间的时间差，而第三时间差指的是每相邻两个历史图像帧之间的时间差。此时，第三时间差中不包含当前图像帧与其相邻历史图像帧之间的时间差。
其中，运动开始帧的确定中历史图像帧为预设时间段内的历史图像帧。本公开实施例中，若设备处于静止状态下，预设时间段内的历史图像帧满足后进先出原则，后进先 出原则指的是在后拍摄的历史图像帧会先从预设时间段内移除，当然从预设时间段内移除并不意味着将该历史图像帧会从存储空间中删除，存储空间中仍然会保留该关键帧的位姿以及与前一历史图像帧之间的传感数据。当设备处于非静止状态下，若最后拍摄的两历史图像帧之间的时差大于某一阈值，则将预设时间段内最早拍摄的一历史图像帧从预设时间段内移除，若该时差小于或等于该阈值，则将预设时间段内倒数第二拍摄的历史图像帧从预设时间段内移除。通过这种方式，预设时间段内运动开始帧与运动开始帧相邻且先于运动开始帧拍摄的历史图像帧之间会有一明显时差，由此，本公开实施例通过相邻图像帧之间的时差确定运动开始帧。根据相邻两历史图像帧之间的时间差确定运动开始帧，使得对运动开始帧的认定过程简便。
其中，预设条件包括以下至少一个：一、运动开始帧与当前图像帧之间的第一时间差大于或等于第一时间差阈值。通过保证运动开始帧与当前图像帧之间有足够的时差，使得有足够的传感数据可以用于积分，提高对定位参数优化的准确度。二、运动开始帧至当前图像帧之间的历史图像帧数量大于或等于第一数量阈值。其中，保障运动开始帧至当前图像帧之间有足够的历史图像帧使得有足够的传感数据能够参与积分使得优化之后的定位参数更加准确。三、运动开始帧与当前图像帧中的预设二维点对的数量大于或等于第二数量阈值。其中，预设二维点对指的是从各历史图像帧中提取出的二维点经过不同历史图像帧之间的二维点匹配得到的二维点对。进行二维点匹配的方式有多种，例如特征匹配，也就是计算提取得到的二维点即关键点的描述子之间的距离，若距离小于某一阈值，则认定匹配成功，还有二维点匹配的方式可以是通过光流法进行块的匹配等等，关于预设二维点对的获得方式此处不做限定。其中，预设二维点对的视差大于或等于第一视差阈值。也就是判断视差大于或等于第一视差阈值的预设二维点对的数量是否大于或等于第二数量阈值。当然，在其他实施例中，还可以根据运动开始帧与当前图像帧之间的任一历史图像帧与当前图像帧中的预设二维点对的数量大于或等于第二数量阈值，此处不做限定。四、运动开始帧至当前图像帧之间的加速度计传感数据的离散程度大于或等于预设程度阈值。当然，若根据实际需求若需要严格设置预设条件，则还可加上第五个判断条件，即运动开始帧至当前图像帧之间的陀螺仪传感数据的离散程度大于或等于与其对应的预设程度阈值。本公开实施例中，若特征同时满足前四个条件，则认定设备的当前运动状态为第二非静止状态，即若特征不满足前四个中的任一个条件，则认定设备的当前运动状态不属于第二非静止状态。其中，本公开实施例中，对设备的运动状态是属于静止状态还是非静止状态的确定可以同时进行。若认为设备的运动状态不属于静止状态，也不满足上述前四个预设条件中的任一个，则认为设备的运动状态为第一非静止状态。
视差和传感数据的离散程度越大，则说明设备的运动幅度越大，历史图像帧的数量越多以及时差越大，说明有足够的传感数据用于优化定位参数，使得定位参数的优化结果更准确。
在一些可能的实现方式中，在当前运动状态属于第一非静止状态的情况下，可利用各三维点在当前图像帧及历史图像帧上的重投影误差，对第一定位参数进行优化。即，当设备处于非静止状态，则根据设备的运动，可以将设备拍摄的当前图像帧和各历史图像帧中观测到的二维点经过三角化得到三维点的坐标，然后将得到的三维点重新投影到当前图像帧以及各历史图像帧上得到投影点，其中，各图像帧中三维点对应的投影点与该三维点对应的观测到的二维点之间距离即为重投影误差。一般地，重投影误差可以对三维点以及各图像帧的位姿同时优化。通过改变相机的位姿，使得重投影误差最小化，从而达到优化相机位姿以及优化三维点的坐标的效果。因为三维点的坐标确定，相机的位姿也确定，即相邻帧之间的距离和角度确定，结合相邻图像帧之间的传感数据可以确定其他的定位参数。
通过将非静止状态又细分为多种非静止状态，使得设备处于非静止状态下时又可根据实际的运动情况选择合适的优化方式，提高对参数的优化效果。
其中，在当前运动状态属于第二非静止状态的情况下，利用预设运动学方程组对第一定位参数进行优化。其中，以其中一历史图像帧作为运动开始帧，并获取运动开始帧至当前图像帧之间的相邻图像帧的位置关系以及预积分信息。其中，确定运动开始帧的方式如上所述，若在运动状态的确定过程中已经确定了运动开始帧，则此处可以不再执行确定运动开始帧的步骤。其中，此处所述的运动开始帧至当前图像帧之间的相邻图像帧为预设时间段内的历史图像帧。其中位置关系包括距离关系和角度关系。预积分信息包括陀螺仪测量得到的数据的预积分以及加速度计测量得到的数据的预积分。将位置关系以及预积分信息代入预设运动学方程组，求解得到尺度以及设备的速度。其中，关于定位参数中的角度、传感器中陀螺仪偏置量以及加速度计偏置量依旧采用初始化得到的结果，并将该结果与上述位置关系和预积分信息一并代入运动学方程组，求解尺度与速度。一般地，在检测到设备处于非静止状态时，定位参数包括
Figure PCTCN2021107630-appb-000001
其中X k表示在k时刻的所有定位参数，X C表示当前使用的定位参数，X S表示预设时间段内各历史关键帧对应的定位参数。其中，
Figure PCTCN2021107630-appb-000002
其中，G表示世界坐标系，C表示相机坐标系， CkR G表示从世界坐标系到k时刻相机坐标系的旋转矩阵， Gp Ck表示k时刻相机在世界坐标系下的位置， Gv Ck表示k时刻在世界坐标系下的运动速度，b wk表示k时刻陀螺仪的偏置量，b ak表示在k时刻加速度计的偏置量，T表示转置。其中，假设预设时间段中共有n帧历史图像帧，则
Figure PCTCN2021107630-appb-000003
表示从世界坐标系到初始时刻相机坐标系的旋转矩阵，
Figure PCTCN2021107630-appb-000004
表示初始时刻相机在世界坐标系下的位置，
Figure PCTCN2021107630-appb-000005
表示初始时刻在世界坐标系下的运动速度，
Figure PCTCN2021107630-appb-000006
表示从世界坐标系到n时刻相机坐标系的旋转矩阵，
Figure PCTCN2021107630-appb-000007
表示n时刻相机在世界坐标系下的位置，
Figure PCTCN2021107630-appb-000008
表示n时刻在世界坐标系下的运动速度，即预设时间段内保留了历史图像帧在世界坐标系下的位置、角度以及速度。
首先，定义一组需要求解的定位参数，
Figure PCTCN2021107630-appb-000009
其中s表示尺度，g表示重力方向，T表示转置，
Figure PCTCN2021107630-appb-000010
分别表示在k时刻设备在世界坐标系下的运动速度，
Figure PCTCN2021107630-appb-000011
表示在初始时刻相机在世界坐标系下的运动速度，
Figure PCTCN2021107630-appb-000012
表示在n时刻设备在世界坐标系下的运动速度。
其中，预设运动学方程如下公式(1)：
Figure PCTCN2021107630-appb-000013
其中，r Ii,j表示从i帧到j帧预积分值与先验值之间的误差。∑ Ii,j表示r Ii,j对应的预积分的协方差矩阵。其中，预设运动学方程组由若干个上述预设运动学方程组成。
一些公开实施例中，求解尺度和速度的方式可以采用传感数据的预积分信息与初始化的定位参数线性求解，其中，初始化的定位参数包括三维点及其重投影误差，还可以 采用这两类信息的非线性优化。因此，预设运动学方程的构建方式并不限于上述运动学方程。在固定其余与尺度耦合不大的定位参数的情况下，求解尺度与速度，降低了对尺度以及速度的求解难度。
其中，求解得到的尺度和速度为使得上述预设运动学方程组误差最小的一组解。也就是将各个预设运动学方程的误差求和最小的一组解。通过将使得预设运动学方程组误差最小的一组解作为求解得到的尺度与速度，使得求解得到的尺度与速度相对其他解而言更准确，后续跟踪定位的结果更准确。进一步地，将求解得到的尺度以及速度代入预设运动学方程组，得到预设运动学方程组中各运动学方程的误差。若其中一个方程的误差大于预设误差阈值，则认定尺度和速度求解失败。在尺度和速度求解失败之后，按照初始化的定位参数对当前图像帧进行跟踪定位，若进行跟踪定位之后，当前图像帧满足预设时间段内历史图像帧的选择要求，则当前图像帧可以进入预设时间段，作为预设时间段内最新的历史图像帧。然后获取新的图像帧，对设备的运动状态进行确定，接着执行后续的步骤。若各运动学方程组的误差均小于或等于预设误差阈值，则确定尺度和速度求解成功。并执行将求解到的尺度以及设备的速度作为优化后的第一定位参数。也就是用求解得到的尺度与速度更新原初始化得到的尺度和速度。其中，其余定位参数包括角度、陀螺仪和加速度计偏置量可以直接使用之前初始化的收敛值，当然，可以使用预设值，例如出厂标定值或上一次跟踪定位使用的值。当然，可以是将计算得到的尺度、速度以及陀螺仪偏置量的值代入传感数据的预积分方程，找到使得预积分方程误差最小的一组解。
通过在所有的运动学方程的误差均小于或等于预设误差阈值的情况下，才认为尺度和速度求解成功，使得尺度和速度的求解过程严格，从而求解得到的尺度以及速度更准确。
对尺度优化后的各历史图像帧的位置进行优化。在一些可能的实现方式中，在尺度和速度求解成功之后，选择其中一历史图像帧作为第一参考帧。例如，选择预设时间段内与当前图像帧之间的时间差最大的历史图像帧为参考帧。利用优化后的尺度，确定第一参考帧与其相邻历史图像帧之间的距离。即利用优化后的尺度代替原尺度求解相邻两历史图像帧之间的距离。按照该距离，初步调整第一参考帧的相邻图像帧的位置。例如，按照原尺度求解得到的第一参考帧与其相邻图像帧之间的距离为1米，而按照优化之后的尺度求解得到的第一参考帧与其相邻历史图像帧之间的距离为2米，则移动第一参考帧的相邻图像帧，使得第一参考帧与其相邻帧之间的距离为2米，当然，整个过程中仅移动距离不改变第一参考帧与其相邻帧之间的角度。接着，以初步调整后的该相邻图像帧作为新的第一参考帧，并重新执行利用优化后的尺度，确定第一参考帧与其相邻历史图像帧之间的距离及其后续步骤。直至预设时间段内的历史图像帧均已初步调整完成。当然，这里将该相邻图像帧作为新的第一参考帧之后，可以不用继续调整与该新的第一参考帧相邻的上一第一参考帧的位置，继续调整未经初步调整的其余历史图像帧的位置。
在预设时间段内的历史图像帧均已初步调整完成之后，以预设时间段内与当前图像帧之间的时间差最小的历史图像帧作为第二参考帧。也就是当前图像帧的前一历史图像帧。将第二参考帧的位置最终调整为初步调整之前其所在位置。即，对于第二参考帧而言，经过初步调整以及最终调整之后的位置相对于初步调整之前的位置并未发生变动。然后，基于第二参考帧在最终调整过程中的位置变化，调整预设时间段内除第二参考帧以外的其余历史图像帧的位置。其中，最终调整过程中，相邻历史图像帧之间的尺度不会发生变化，简单而言就是预设时间段内历史图像帧整体的一个平移过程。通过在初步调整完成之后，各历史图像帧在初步调整前后的位置变化太大，即初步调整前后历史图像帧之间出现不连贯的现象，通过将与当前图像帧之间时间差最小的历史图像帧的初步调整后的位置移动至初步调整前的位置能够缓和这种不连贯的现象，使得后续跟踪定位 的效果更佳。
其中，对尺度优化后的各三维点的深度进行优化。当然，对于尺度优化之后，各历史图像帧的位置和各三维点的深度可以都进行优化。其中对三维点深度的优化方式可以有多种，例如利用位置优化后的各历史图像帧重新对各帧中匹配的二维点对进行三角化得到新的三维点的坐标，并利用新的三维点对原来对应的三维点进行更新，从而实现对各三维点的更新。当然，对三维点深度的优化方式还可以是以首次观测到三维点的历史图像帧为基准，按照尺度调整三维点的深度。因为三维点相对于首次观测到该三维点的朝向是已知的，深度是按照优化前的尺度求解得到，那么可以根据该原三维点与首次观测到该三维点的历史图像帧的朝向关系求解最终调整之后首次观测到该三维点的历史图像帧之间的朝向关系并按照新的尺度求解三维点的深度以得到新三维点的坐标，由此更新各三维点的坐标。
通过将尺度优化后的各历史图像帧和三维点的深度进行优化，使得在后续跟踪定位过程中，能够有更准确的先验信息，从而跟踪定位的结果更准确。
其中，在对尺度优化后的各历史图像帧的位置和/或各三维点的深度进行优化之前，删除预设时间段内位于运动开始帧之前的历史图像帧。删除的意思是将运动开始帧之前的历史图像帧从第二预设时间段内移除，但是还可以将被移除的历史图像帧的位姿以及传感数据保存在存储单元中。
一些公开实施例中，在对尺度优化后的各历史图像帧的位置和/或各三维点的深度进行优化之后，提高设备的位置、速度以及加速度计偏置量中的至少一者的不确定度。因为设备的位置、速度以及加速度计偏置量在之前初始化过程中可能会收敛过度自信，即位置、速度以及加速度计偏置量的误差也许较大，因此，需要增加他们的不确定度以降低后续在跟踪定位过程中的影响力，然后根据后续跟踪定位中获取到的运动信息不断对位置、速度以及加速度计进行更新。
上述方案，通过在对定位参数进行初始化之后，仍进一步获取设备的运动状态，然后根据运动状态对定位参数进行优化，使得能够根据优化之后的参数对设备后续的移动进行定位，提高了定位的准确度。
需要说明的是，定位初始化方法的执行主体可以是定位初始化装置，例如，定位初始化方法可以由终端设备或服务器或其它处理设备执行，其中，终端设备可以为：虚拟现实头盔，增强现实眼镜、无人车、移动机器人、扫地机器人、飞行设备、用户设备(User Equipment，UE)、移动设备、用户终端、蜂窝电话、无绳电话、个人数字处理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等同时具有图像传感器和惯性传感器的设备。在一些可能的实现方式中，该定位初始化方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。
请参阅图3，图3是本公开定位初始化装置一实施例的结构示意图。定位初始化装置30包括：获取模块31、确定模块32以及优化模块33。获取模块31配置为在对待定位的设备的定位参数进行初始化之后，获取设备的运动信息；确定模块32配置为基于运动信息，确定设备的当前运动状态；优化模块33配置为根据设备的当前运动状态，对定位参数进行优化，得到优化后的定位参数。上述方案，通过在对定位参数进行初始化之后，仍进一步获取设备的运动状态，然后根据运动状态对定位参数进行优化，使得能够根据优化之后的参数对设备后续的移动进行定位，提高了定位的准确度。
一些公开实施例中，定位参数分为第一定位参数和第二定位参数；优化模块33还配置为利用设备的当前运动状态，对定位参数进行优化，得到优化后的定位参数，包括：在当前运动状态属于非静止状态的情况下，利用第一优化方式对第一定位参数进行优化，其中，第一定位参数包括尺度；或，在当前运动状态属于静止状态的情况下，利用第二优化方式对第二定位参数进行优化，其中，第二定位参数不同于第一定位参数。上述方 案，通过设备处于静止状态或非静止状态时采用不同优化方式，对定位参数进行优化，能够适用不同的运动场景。
一些公开实施例中，非静止状态包括第一非静止状态和第二非静止状态；优化模块33在当前运动状态属于非静止状态的情况下，利用第一优化方式对第一定位参数进行优化，包括：在当前运动状态属于第一非静止状态的情况下，利用各三维点在当前图像帧及若干历史图像帧上的重投影误差，对所第一定位参数进行优化；其中，当前图像帧和历史图像帧为设备拍摄得到的，当前图像帧未用于对设备的跟踪定位，历史图像帧已用于对设备的跟踪定位；或，在当前运动状态属于第二非静止状态的情况下，利用预设运动学方程组对第一定位参数进行优化。上述方案，将非静止状态又细分为多种非静止状态，使得设备处于非静止状态下时又可根据实际的运动情况选择合适的优化方式，提高对参数的优化效果。
一些公开实施例中，确定模块32还配置为基于运动信息，确定设备的当前运动状态，包括以下至少一个步骤：以其中一历史图像帧作为运动开始帧；在运动开始帧至当前图像帧之间的特征不满足预设条件的情况下，确定当前运动状态为第一非静止状态；其中，特征包括图像帧数量、时差、视差和传感数据中的一个或多个；在运动开始帧至当前图像帧之间的特征满足预设条件的情况下，确定当前运动状态为第二非静止状态。上述方案，通过运动开始帧及以后的历史图像帧以及当前图像帧对设备的运动状态进行确定，减小了运动开始帧以前的静止图像帧造成的干扰使得设备的运动状态的确定更准确。
一些公开实施例中，预设条件包括以下至少一个：运动开始帧与当前图像帧之间的第一时间差大于或等于第一时间阈值；运动开始帧至当前图像帧之间的历史图像帧数量大于或等于第一数量阈值；运动开始帧与当前图像帧中的预设二维点对的数量大于或等于第二数量阈值，其中，预设二维点对的视差大于或等于第一视差阈值；运动开始帧至当前图像帧之间的加速度计传感数据的离散程度大于或等于预设程度阈值。上述方案，视差和传感数据的离散程度越大，则说明设备的运动幅度越大，历史图像帧的数量越多以及时差越大，说明有足够的传感数据用于优化定位参数，使得定位参数的优化结果更准确。
一些公开实施例中，优化模块33还配置为利用预设运动学方程组对第一定位参数进行优化，包括：以其中一历史图像帧作为运动开始帧，并获取运动开始帧至当前图像帧之间的相邻图像帧的位置关系以及预积分信息；将位置关系以及预积分信息代入预设运动学方程组，求解得到尺度以及设备的速度；将求解得到的尺度以及设备的速度作为优化后的第一定位参数。上述方案，在固定其余与尺度耦合不大的定位参数的情况下，求解尺度与速度，降低了对尺度以及速度的求解难度。
一些公开实施例中，求解得到的尺度以及速度为使得预设运动学方程组误差最小的一组解。上述方案，通过将使得预设运动学方程组误差最小的一组解作为求解得到的尺度与速度，使得求解得到的尺度与速度相对其他解而言更准确，后续跟踪定位的结果更准确。
一些公开实施例中，在将求解得到的尺度以及设备的速度作为优化后的第一定位参数之前，优化模块33还配置为：将求解得到的尺度以及速度代入预设运动学方程组，得到预设运动学方程组中各运动学方程的误差；若各运动学方程的误差均小于或等于预设误差阈值，则确定尺度以及速度求解成功，并执行将求解得到的尺度以及设备的速度作为优化后的第一定位参数。上述方案，只有在所有的运动学方程的误差均小于或等于预设误差阈值的情况下，才认为尺度和速度求解成功，使得尺度和速度的求解过程严格，从而求解得到的尺度以及速度更准确。
一些公开实施例中，确定模块32还配置为：按照历史图像帧与当前图像帧之间的第二时间差从少到多进行排序，并依序判断每相邻两历史图像帧之间的第三时间差是否大 于第二时间阈值；在判断相邻两历史图像帧之间的第三时间差大于第二时间阈值的情况下，将相邻两历史图像帧中第二时间差更小的历史图像帧作为运动开始帧。上述方案，根据相邻两历史图像帧之间的时间差确定运动开始帧，使得对运动开始帧的认定过程更简便。
一些公开实施例中，在利用预设运动学方程组对第一定位参数进行优化之后，优化模块33还配置为：对尺度优化后的各历史图像帧的位置和/或各三维点的深度进行优化。
上述方案，通过将尺度优化后的各历史图像帧和三维点的深度进行优化，使得在后续跟踪定位过程中，能够有更准确的先验信息，从而跟踪定位的结果更准确。
一些公开实施例中，优化模块33还配置为：选择其中一历史图像帧作为第一参考帧；利用优化后的尺度，确定第一参考帧与其相邻历史图像帧之间的距离；按照距离，初步调整第一参考帧的相邻历史图像帧的位置；以初步调整后的相邻历史图像帧作为新的第一参考帧，并重新执行利用优化后的尺度，确定第一参考帧与其相邻历史图像帧之间的距离及其后续步骤，直至预设时间段内的历史图像帧均已初步调整完成。上述方案，将历史图像帧之间的距离关系根据计算得到的尺度进行调整，优化各历史图像帧之间的位置关系，使得后续对设备跟踪定位的结果更准确。
一些公开实施例中，在预设时间段内的历史图像帧均已初步调整完成之后，优化模块33还配置为：以预设时间段内与所当前图像帧之间的时间差最少的历史图像帧作为第二参考帧；将第二参考帧的位置最终调整为初步调整之前所在的位置；基于第二参考帧在最终调整过程中的位置变化，调整预设时间段内除第二参考帧以外的其余历史图像帧的位置。上述方案，在初步调整完成之后，各历史图像帧在初步调整前后的位置变化太大，即初步调整前后历史图像帧之间出现不连贯的现象，通过将与当前图像帧之间时间差最小的历史图像帧的初步调整后的位置移动至初步调整前的位置能够缓和这种不连贯的现象，使得后续跟踪定位的效果更佳。
一些公开实施例中，优化模块33还配置为：对于各三维点，以首次观测到三维点的历史图像帧为基准，按照尺度调整三维点的深度。上述方案，通过计算得到的尺度对三维点的深度进行优化使得后续能够使用更准确的三维点的坐标对设备进行跟踪定位，提高跟踪定位的准确度。
一些公开实施例中，在对尺度优化后的各历史图像帧的位置和/或各三维点的深度进行优化之前，优化模块33还配置为：删除预设时间段内位于运动开始帧之前的历史图像帧；和/或，在对尺度优化后的各历史图像帧的位置和/或各三维点的深度进行优化之后，包括：提高设备的位置、速度以及加速度计偏置量中的至少一者的不确定度。上述方案，通过删除运动开始帧以前的历史图像帧能够减少拍摄时间过长的历史图像帧对优化结果的影响。提高设备的位置、速度以及加速度计偏置量的不确定度能够减少因为收敛过度，导致后期定位结果不准确的结果出现。
一些公开实施例中，确定模块32还配置为：基于运动信息，确定设备的当前运动状态，包括：在若干历史图像帧与当前图像帧之间的视差均小于第二视差阈值，和/或至少一个传感数据的离散程度均小于对应的预设程度值的情况下，确定设备的当前运动状态为静止状态；其中，第二视差阈值是固定的或基于历史图像帧与当前图像帧之间的时间差动态调整的；至少一个传感数据是在当前运动状态之前的第一时间段内采集的，包括陀螺仪传感数据和/或加速度计传感数据。上述方案，通过视差和传感数据的结合对静止状态的认定，较于单一视差或传感数据的认定而言，前者的认定结果更准确。
一些公开实施例中，优化模块33还配置为：以速度为零作为约束，对第二定位参数进行约束。上述方案，在静止状态下无法通过三维点的重投影误差对定位参数进行优化，通过增加速度为零的约束，使得当前图像帧的位姿较上一历史图像帧的位姿不会发生较大变化，提高定位参数的稳定性。
一些公开实施例中，优化模块33还配置为：判断当前图像帧之前的若干历史图像帧中是否有预设数量三维点；在若干历史图像帧中没有预设数量三维点的情况下，提高速度为零的约束权重；和/或，在若干历史图像帧中有预设数量三维点的情况下，降低速度为零的约束权重。上述方案，通过根据三维点的数量降低或提高速度为零的权重，使得能够在各种状态下都能够很好地对定位参数进行优化。
上述方案，通过在对定位参数进行初始化之后，仍进一步获取设备的运动状态，然后根据运动状态对定位参数进行优化，使得能够根据优化之后的参数对设备后续的移动进行定位，提高了定位的准确度。
请参阅图4，图4是本公开电子设备一实施例的结构示意图。电子设备40包括存储器41和处理器42，处理器42配置为执行存储器41中存储的程序指令，以实现上述任一定位初始化方法实施例中的步骤。在一个实施场景中，电子设备40可以包括但不限于：移动机器人、手持移动设备、扫地机器人、无人车、虚拟现实头盔、增强现实眼镜飞行设备、微型计算机、台式电脑、服务器等其他同时具有图像传感器和惯性传感器模组的设备，此外，电子设备40还可以包括笔记本电脑、平板电脑等移动设备，在此不做限定。
处理器42配置为控制其自身以及存储器41以实现上述任一定位初始化方法实施例中的步骤。处理器42还可以称为中央处理单元(Central Processing Unit，CPU)。处理器42可能是一种集成电路芯片，具有信号的处理能力。处理器42还可以是通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。另外，处理器42可以由集成电路芯片共同实现。
上述方案，通过在对定位参数进行初始化之后，仍进一步获取设备的运动状态，然后根据运动状态对定位参数进行优化，使得能够根据优化之后的参数对设备后续的移动进行定位，提高了定位的准确度。
请参阅图5，图5是本公开计算机可读存储介质一实施例的结构示意图。计算机可读存储介质50存储有能够被处理器运行的程序指令51，程序指令51用于实现上述任一定位初始化方法实施例中的步骤。
上述方案，通过在对定位参数进行初始化之后，仍进一步获取设备的运动状态，然后根据运动状态对定位参数进行优化，使得能够根据优化之后的参数对设备后续的移动进行定位，提高了定位的准确度。
在一些实施例中，本公开实施例提供的装置具有的功能或包含的模块可以配置为执行上文方法实施例描述的方法，其实施过程可以参照上文方法实施例的描述，为了简洁，这里不再赘述。
上文对各个实施例的描述倾向于强调各个实施例之间的不同之处，其相同或相似之处可以互相参考，为了简洁，本文不再赘述。
在本公开所提供的几个实施例中，应该理解到，所揭露的方法和装置，可以通过其它的方式实现。例如，以上所描述的装置实施方式仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性、机械或其它的形式。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。集成的单元如果 以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本公开各个实施方式方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
工业实用性
本公开实施例在对待定位设备的定位参数进行初始化之后，获取设备的运动信息；基于运动信息，确定设备的当前运动状态；根据设备的当前运动状态，对定位参数进行优化，得到优化后的定位参数。上述方案，通过在对定位参数进行初始化之后，再根据设备的运动状态对定位参数进行优化得到优化后的定位参数，使得后续对设备的定位更准确。

Claims (37)

  1. 一种定位初始化方法，包括：
    在对待定位的设备的定位参数进行初始化之后，获取所述设备的运动信息；
    基于所述运动信息，确定所述设备的当前运动状态；
    根据所述设备的当前运动状态，对所述定位参数进行优化，得到优化后的定位参数。
  2. 根据权利要求1所述的方法，其中，所述定位参数分为第一定位参数和第二定位参数；所述根据所述设备的当前运动状态，对所述定位参数进行优化，得到优化后的定位参数，包括：
    在所述当前运动状态属于非静止状态的情况下，利用第一优化方式对所述第一定位参数进行优化，其中，所述第一定位参数包括尺度；
    或，在所述当前运动状态属于静止状态的情况下，利用第二优化方式对所述第二定位参数进行优化，其中，所述第二定位参数不同于所述第一定位参数。
  3. 根据权利要求2所述的方法，其中，所述非静止状态包括第一非静止状态和第二非静止状态；所述在所述当前运动状态属于非静止状态的情况下，利用第一优化方式对第一定位参数进行优化，包括：
    在所述当前运动状态属于所述第一非静止状态的情况下，利用各三维点在当前图像帧及若干历史图像帧上的重投影误差，对所第一定位参数进行优化；其中，所述当前图像帧和所述历史图像帧为所述设备拍摄得到的，所述当前图像帧未用于对所述设备的跟踪定位，所述历史图像帧已用于对所述设备的跟踪定位；
    或，在所述当前运动状态属于所述第二非静止状态的情况下，利用预设运动学方程组对所述第一定位参数进行优化。
  4. 根据权利要求3所述的方法，其中，所述基于所述运动信息，确定所述设备的当前运动状态，包括以下至少一个步骤：
    以其中一所述历史图像帧作为运动开始帧；
    在所述运动开始帧至所述当前图像帧之间的特征不满足预设条件的情况下，确定所述当前运动状态为第一非静止状态；其中，所述特征包括图像帧数量、时差、视差和传感数据中的一个或多个；
    在所述运动开始帧至所述当前图像帧之间的特征满足预设条件的情况下，确定所述当前运动状态为所述第二非静止状态。
  5. 根据权利要求4所述的方法，其中，所述预设条件包括以下至少一个：
    所述运动开始帧与所述当前图像帧之间的第一时间差大于或等于第一时间阈值；
    所述运动开始帧至所述当前图像帧之间的历史图像帧数量大于或等于第一数量阈值；
    所述运动开始帧与所述当前图像帧中的预设二维点对的数量大于或等于第二数量阈值，其中，所述预设二维点对的视差大于或等于第一视差阈值；
    所述运动开始帧至所述当前图像帧之间的加速度计传感数据的离散程度大于或等于预设程度阈值。
  6. 根据权利要求3至5任一项所述的方法，其中，所述利用预设运动学方程组对第一定位参数进行优化，包括：
    以其中一所述历史图像帧作为运动开始帧，并获取所述运动开始帧至所述当前图像帧之间的相邻图像帧的位置关系以及预积分信息；
    将所述位置关系以及预积分信息代入预设运动学方程组，求解得到所述尺度以及所 述设备的速度；
    将求解得到的所述尺度以及所述设备的速度作为优化后的第一定位参数。
  7. 根据权利要求6所述的方法，其中，所述求解得到的所述尺度以及所述速度为使得所述预设运动学方程组误差最小的一组解。
  8. 根据权利要求6或7所述的方法，其中，在所述将求解得到的所述尺度以及所述设备的速度作为优化后的第一定位参数之前，所述方法还包括：
    将求解得到的所述尺度以及速度代入所述预设运动学方程组，得到所述预设运动学方程组中各运动学方程的误差；
    若各所述运动学方程的误差均小于或等于预设误差阈值，则确定所述尺度以及速度求解成功。
  9. 根据权利要求4或6所述的方法，其中，所述以其中一所述历史图像帧作为运动开始帧，包括：
    按照所述历史图像帧与所述当前图像帧之间的第二时间差从少到多进行排序，并依序判断每相邻两历史图像帧之间的第三时间差是否大于第二时间阈值；
    在判断所述相邻两历史图像帧之间的第三时间差大于第二时间阈值的情况下，将所述相邻两历史图像帧中所述第二时间差更小的历史图像帧作为所述运动开始帧。
  10. 根据权利要求3至9任一项所述的方法，其中，在所述利用预设运动学方程组对第一定位参数进行优化之后，所述方法还包括：
    对所述尺度优化后的各历史图像帧的位置和/或各三维点的深度进行优化。
  11. 根据权利要求10所述的方法，其中，所述对所述尺度优化后的各历史图像帧的位置进行优化，包括：
    选择其中一所述历史图像帧作为第一参考帧；
    利用所述优化后的尺度，确定所述第一参考帧与其相邻历史图像帧之间的距离；
    按照所述距离，初步调整所述第一参考帧的所述相邻历史图像帧的位置；
    以初步调整后的所述相邻历史图像帧作为新的第一参考帧，并重新执行所述利用所述优化后的尺度，确定所述第一参考帧与其相邻历史图像帧之间的距离及其后续步骤，直至预设时间段内的历史图像帧均已初步调整完成。
  12. 根据权利要求11所述的方法，其中，在预设时间段内的历史图像帧均已初步调整完成之后，所述方法还包括：
    以预设时间段内与所述当前图像帧之间的时间差最少的历史图像帧作为第二参考帧；
    将所述第二参考帧的位置最终调整为所述初步调整之前所在的位置；
    基于所述第二参考帧在所述最终调整过程中的位置变化，调整所述预设时间段内除所述第二参考帧以外的其余历史图像帧的位置。
  13. 根据权利要求10至12任一项所述的方法，其中，所述对所述尺度优化后的各三维点的深度进行优化，包括：
    对于各所述三维点，以首次观测到所述三维点的历史图像帧为基准，按照所述尺度调整所述三维点的深度。
  14. 根据权利要求10所述的方法，其中，在所述对所述尺度优化后的各历史图像帧的位置和/或各三维点的深度进行优化之前，所述方法还包括：
    删除预设时间段内位于所述运动开始帧之前的历史图像帧；
    和/或，在所述对所述尺度优化后的各历史图像帧的位置和/或各三维点的深度进行优化之后，包括：
    提高所述设备的位置、速度以及加速度计偏置量中的至少一者的不确定度。
  15. 根据权利要求2至14任一项所述的方法，其中，所述基于所述运动信息，确定 所述设备的当前运动状态，包括：
    在若干历史图像帧与所述当前图像帧之间的视差均小于第二视差阈值，和/或至少一个传感数据的离散程度小于对应的预设程度值的情况下，确定所述设备的当前运动状态为所述静止状态；
    其中，所述第二视差阈值是固定的或基于所述历史图像帧与所述当前图像帧之间的时间差动态调整的；所述至少一个传感数据是在当前运动状态之前的第一时间段内采集的，包括陀螺仪传感数据和/或加速度计传感数据。
  16. 根据权利要求2至15任一项所述的方法，其中，所述利用第二优化方式对第二定位参数进行优化，包括：
    以速度为零作为约束，对所述第二定位参数进行约束。
  17. 根据权利要求16所述的方法，其中，所述方法还包括：
    判断所述当前图像帧之前的若干历史图像帧中是否有预设数量三维点；
    在所述若干历史图像帧中没有预设数量三维点的情况下，提高速度为零的约束权重；和/或，在所述若干历史图像帧中有预设数量三维点的情况下，降低速度为零的约束权重。
  18. 一种定位初始化装置，其中，包括：
    获取模块，配置为在对待定位的设备的定位参数进行初始化之后，获取所述设备的运动信息；
    确定模块，配置为基于所述运动信息，确定所述设备的当前运动状态；
    优化模块，配置为根据所述设备的当前运动状态，对所述定位参数进行优化，得到优化后的定位参数。
  19. 根据权利要求18所述的装置，其中，所述定位参数分为第一定位参数和第二定位参数；所述优化模块还配置为：
    在所述当前运动状态属于非静止状态的情况下，利用第一优化方式对所述第一定位参数进行优化，其中，所述第一定位参数包括尺度；
    或，在所述当前运动状态属于静止状态的情况下，利用第二优化方式对所述第二定位参数进行优化，其中，所述第二定位参数不同于所述第一定位参数。
  20. 根据权利19所述的装置，其中，所述非静止状态包括第一非静止状态和第二非静止状态；所述优化模块还配置为：
    在所述当前运动状态属于所述第一非静止状态的情况下，利用各三维点在当前图像帧及若干历史图像帧上的重投影误差，对所第一定位参数进行优化；其中，所述当前图像帧和所述历史图像帧为所述设备拍摄得到的，所述当前图像帧未用于对所述设备的跟踪定位，所述历史图像帧已用于对所述设备的跟踪定位；
    或，在所述当前运动状态属于所述第二非静止状态的情况下，利用预设运动学方程组对所述第一定位参数进行优化。
  21. 根据权利要求20所述的装置，其中，所述确定模块32还配置为执行以下至少一个步骤：
    以其中一所述历史图像帧作为运动开始帧；
    在所述运动开始帧至所述当前图像帧之间的特征不满足预设条件的情况下，确定所述当前运动状态为第一非静止状态；其中，所述特征包括图像帧数量、时差、视差和传感数据中的一个或多个；
    在所述运动开始帧至所述当前图像帧之间的特征满足预设条件的情况下，确定所述当前运动状态为所述第二非静止状态。
  22. 根据权利要求21所述的装置，其中，所述预设条件包括以下至少一个：
    所述运动开始帧与所述当前图像帧之间的第一时间差大于或等于第一时间阈值；
    所述运动开始帧至所述当前图像帧之间的历史图像帧数量大于或等于第一数量阈 值；
    所述运动开始帧与所述当前图像帧中的预设二维点对的数量大于或等于第二数量阈值，其中，所述预设二维点对的视差大于或等于第一视差阈值；
    所述运动开始帧至所述当前图像帧之间的加速度计传感数据的离散程度大于或等于预设程度阈值。
  23. 根据权利要求20至22任一项所述的装置，其中，所述优化模块还配置为：
    以其中一所述历史图像帧作为运动开始帧，并获取所述运动开始帧至所述当前图像帧之间的相邻图像帧的位置关系以及预积分信息；
    将所述位置关系以及预积分信息代入预设运动学方程组，求解得到所述尺度以及所述设备的速度；
    将求解得到的所述尺度以及所述设备的速度作为优化后的第一定位参数。
  24. 根据权利要求23所述的装置，其中，所述优化模块还配置为：所述求解得到的所述尺度以及所述速度为使得所述预设运动学方程组误差最小的一组解。
  25. 根据权利要求23或24所述的装置，其中，在所述将求解得到的所述尺度以及所述设备的速度作为优化后的第一定位参数之前，所述优化模块还配置为：
    将求解得到的所述尺度以及速度代入所述预设运动学方程组，得到所述预设运动学方程组中各运动学方程的误差；
    若各所述运动学方程的误差均小于或等于预设误差阈值，则确定所述尺度以及速度求解成功。
  26. 根据权利要求21或23所述的装置，其中，所述确定模块还配置为：
    按照所述历史图像帧与当前图像帧之间的第二时间差从少到多进行排序，并依序判断每相邻两历史图像帧之间的第三时间差是否大于第二时间阈值；
    在判断所述相邻两历史图像帧之间的第三时间差大于第二时间阈值的情况下，将所述相邻两历史图像帧中所述第二时间差更小的历史图像帧作为所述运动开始帧。
  27. 根据权利要求20至26任一项所述的装置，其中，在所述利用预设运动学方程组对第一定位参数进行优化之后，所述优化模块还配置为：
    对所述尺度优化后的各历史图像帧的位置和/或各三维点的深度进行优化。
  28. 根据权利要求27所述的装置，其中，所述优化模块还配置为：
    选择其中一所述历史图像帧作为第一参考帧；
    利用所述优化后的尺度，确定所述第一参考帧与其相邻历史图像帧之间的距离；
    按照所述距离，初步调整所述第一参考帧的所述相邻历史图像帧的位置；
    以初步调整后的所述相邻历史图像帧作为新的第一参考帧，并重新执行所述利用所述优化后的尺度，确定所述第一参考帧与其相邻历史图像帧之间的距离及其后续步骤，直至预设时间段内的历史图像帧均已初步调整完成。
  29. 根据权利要求28所述的装置，其中，在预设时间段内的历史图像帧均已初步调整完成之后，所述优化模块还配置为：
    以预设时间段内与所述当前图像帧之间的时间差最少的历史图像帧作为第二参考帧；将所述第二参考帧的位置最终调整为所述初步调整之前所在的位置；
    基于所述第二参考帧在所述最终调整过程中的位置变化，调整所述预设时间段内除所述第二参考帧以外的其余历史图像帧的位置。
  30. 根据权利要求27至29任一项所述的装置，其中，所述优化装置还配置为：
    对于各所述三维点，以首次观测到所述三维点的历史图像帧为基准，按照所述尺度调整所述三维点的深度。
  31. 根据权利要求27所述的装置，其中，在所述对所述尺度优化后的各历史图像帧的位置和/或各三维点的深度进行优化之前，所述优化装置还配置为：
    删除预设时间段内位于所述运动开始帧之前的历史图像帧；
    和/或，在所述对所述尺度优化后的各历史图像帧的位置和/或各三维点的深度进行优化之后，包括：
    提高所述设备的位置、速度以及加速度计偏置量中的至少一者的不确定度。
  32. 根据权利要求19至31任一项所述的装置，其中，所述确定模块还配置为：
    在若干历史图像帧与当前图像帧之间的视差均小于第二视差阈值，和/或至少一个传感数据的离散程度小于对应的预设程度值的情况下，确定所述设备的当前运动状态为所述静止状态；
    其中，所述第二视差阈值是固定的或基于所述历史图像帧与所述当前图像帧之间的时间差动态调整的；所述至少一个传感数据是在当前运动状态之前的第一时间段内采集的，包括陀螺仪传感数据和/或加速度计传感数据。
  33. 根据权利要求19至32任一项所述的装置，其中，所述优化模块还配置为：
    以速度为零作为约束，对所述第二定位参数进行约束。
  34. 根据权利要求33所述的装置，其中，所述优化模块还配置为：
    判断所述当前图像帧之前的若干历史图像帧中是否有预设数量三维点；
    在所述若干历史图像帧中没有预设数量三维点的情况下，提高速度为零的约束权重；和/或，在所述若干历史图像帧中有预设数量三维点的情况下，降低速度为零的约束权重。
  35. 一种电子设备，其中，包括存储器和处理器，所述处理器配置为执行所述存储器中存储的程序指令，以实现权利要求1至17任一项所述的方法。
  36. 一种计算机可读存储介质，其上存储有程序指令，其中，所述程序指令被处理器执行时实现权利要求1至17任一项所述的方法。
  37. 一种计算机程序产品，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行配置为实现权利要求1至17中任意一项所述的方法。
PCT/CN2021/107630 2021-03-12 2021-07-21 定位初始化方法和装置、设备、存储介质、程序产品 WO2022188334A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110272183.2A CN113012224B (zh) 2021-03-12 2021-03-12 定位初始化方法和相关装置、设备、存储介质
CN202110272183.2 2021-03-12

Publications (1)

Publication Number Publication Date
WO2022188334A1 true WO2022188334A1 (zh) 2022-09-15

Family

ID=76406498

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/107630 WO2022188334A1 (zh) 2021-03-12 2021-07-21 定位初始化方法和装置、设备、存储介质、程序产品

Country Status (2)

Country Link
CN (1) CN113012224B (zh)
WO (1) WO2022188334A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113012224B (zh) * 2021-03-12 2022-06-03 浙江商汤科技开发有限公司 定位初始化方法和相关装置、设备、存储介质
CN113899364B (zh) * 2021-09-29 2022-12-27 深圳市慧鲤科技有限公司 定位方法及装置、设备、存储介质
CN113847907A (zh) * 2021-09-29 2021-12-28 深圳市慧鲤科技有限公司 定位方法及装置、设备、存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170050318A1 (en) * 2009-11-06 2017-02-23 Irobot Corporation Localization by Learning of Wave-Signal Distributions
CN108804161A (zh) * 2018-06-21 2018-11-13 北京字节跳动网络技术有限公司 应用的初始化方法、装置、终端和存储介质
CN109767470A (zh) * 2019-01-07 2019-05-17 浙江商汤科技开发有限公司 一种跟踪系统初始化方法及终端设备
CN109855617A (zh) * 2019-02-28 2019-06-07 深圳市元征科技股份有限公司 一种车辆定位方法、车辆定位装置及终端设备
CN110084832A (zh) * 2019-04-25 2019-08-02 亮风台（上海）信息科技有限公司 相机位姿的纠正方法、装置、系统、设备和存储介质
CN113012224A (zh) * 2021-03-12 2021-06-22 浙江商汤科技开发有限公司 定位初始化方法和相关装置、设备、存储介质

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1278501C (zh) * 2002-12-06 2006-10-04 国际商业机器公司 移动目标位置数据的压缩方法和压缩设备
CN101074965A (zh) * 2007-06-26 2007-11-21 南京航空航天大学 基于图像识别定位的转速测量方法及其装置
CN101576384B (zh) * 2009-06-18 2011-01-05 北京航空航天大学 一种基于视觉信息校正的室内移动机器人实时导航方法
CN101789126B (zh) * 2010-01-26 2012-12-26 北京航空航天大学 基于体素的三维人体运动跟踪方法
CN103424114B (zh) * 2012-05-22 2016-01-20 同济大学 一种视觉导航/惯性导航的全组合方法
CN103047999B (zh) * 2012-12-18 2015-09-30 东南大学 一种舰载主/子惯导传递对准过程中的陀螺误差快速估计方法
US9182237B2 (en) * 2013-12-06 2015-11-10 Novatel Inc. Navigation system with rapid GNSS and inertial initialization
CN104915965A (zh) * 2014-03-14 2015-09-16 华为技术有限公司 一种摄像机跟踪方法及装置
CN105021192B (zh) * 2015-07-30 2018-10-09 华南理工大学 一种基于零速校正的组合导航系统的实现方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170050318A1 (en) * 2009-11-06 2017-02-23 Irobot Corporation Localization by Learning of Wave-Signal Distributions
CN108804161A (zh) * 2018-06-21 2018-11-13 北京字节跳动网络技术有限公司 应用的初始化方法、装置、终端和存储介质
CN109767470A (zh) * 2019-01-07 2019-05-17 浙江商汤科技开发有限公司 一种跟踪系统初始化方法及终端设备
CN109855617A (zh) * 2019-02-28 2019-06-07 深圳市元征科技股份有限公司 一种车辆定位方法、车辆定位装置及终端设备
CN110084832A (zh) * 2019-04-25 2019-08-02 亮风台（上海）信息科技有限公司 相机位姿的纠正方法、装置、系统、设备和存储介质
CN113012224A (zh) * 2021-03-12 2021-06-22 浙江商汤科技开发有限公司 定位初始化方法和相关装置、设备、存储介质

Also Published As

Publication number Publication date
CN113012224A (zh) 2021-06-22
CN113012224B (zh) 2022-06-03

Similar Documents

Publication Publication Date Title
US20210190497A1 (en) Simultaneous location and mapping (slam) using dual event cameras
WO2022188334A1 (zh) 定位初始化方法和装置、设备、存储介质、程序产品
CN110322500B (zh) 即时定位与地图构建的优化方法及装置、介质和电子设备
US11199414B2 (en) Method for simultaneous localization and mapping
CN110490900B (zh) 动态环境下的双目视觉定位方法及系统
US9111351B2 (en) Minimizing drift using depth camera images
US20210183100A1 (en) Data processing method and apparatus
US10636190B2 (en) Methods and systems for exploiting per-pixel motion conflicts to extract primary and secondary motions in augmented reality systems
US20220051031A1 (en) Moving object tracking method and apparatus
JP7280385B2 (ja) 視覚的ポジショニング方法および関連装置、機器並びにコンピュータ可読記憶媒体
CN113551665A (zh) 一种用于运动载体的高动态运动状态感知系统及感知方法
CN111307146A (zh) 一种基于双目相机和imu的虚拟现实头戴显示设备定位系统
TWI722738B (zh) 擴增實境裝置與定位方法
WO2023050634A1 (zh) 定位方法及装置、设备、存储介质及计算机程序产品
CN114022556A (zh) 定位初始化方法、装置和计算机可读存储介质
CN112731503A (zh) 一种基于前端紧耦合的位姿估计方法及系统
WO2023030062A1 (zh) 一种无人机的飞行控制方法、装置、设备、介质及程序
CN115914841A (zh) 一种被动式光学运动捕捉方法
US11158119B2 (en) Systems and methods for reconstructing a three-dimensional object
CN113847907A (zh) 定位方法及装置、设备、存储介质
JPH1137736A (ja) ３次元形状計測方法及び３次元形状計測装置
CN110660134B (zh) 三维地图构建方法、三维地图构建装置及终端设备
US10636205B2 (en) Systems and methods for outlier edge rejection
JP2023554001A (ja) エピポーラ制約を使用したバンドル調節
CN116698023A (zh) 状态区分方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21929803

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21929803

Country of ref document: EP

Kind code of ref document: A1