WO2023087681A1 - Procédé et appareil d'initialisation de positionnement, et support de stockage lisible par ordinateur et produit programme informatique - Google Patents

Procédé et appareil d'initialisation de positionnement, et support de stockage lisible par ordinateur et produit programme informatique Download PDF

Info

Publication number
WO2023087681A1
WO2023087681A1 PCT/CN2022/098214 CN2022098214W WO2023087681A1 WO 2023087681 A1 WO2023087681 A1 WO 2023087681A1 CN 2022098214 W CN2022098214 W CN 2022098214W WO 2023087681 A1 WO2023087681 A1 WO 2023087681A1
Authority
WO
WIPO (PCT)
Prior art keywords
state variable
positioning
value
state
inertial
Prior art date
Application number
PCT/CN2022/098214
Other languages
English (en)
Chinese (zh)
Inventor
翟尚进
陈丹鹏
王楠
章国锋
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Publication of WO2023087681A1 publication Critical patent/WO2023087681A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Definitions

  • the embodiment of the present disclosure is based on the Chinese patent application with the application number 202111356493.9, the application date is November 16, 2021, and the application name is "positioning initialization method, device and computer-readable storage medium", and the priority of the Chinese patent application is required Right, the entire content of this Chinese patent application is hereby incorporated into this disclosure as a reference.
  • the present disclosure relates to the technical field of positioning, and in particular, to a positioning initialization method, device, computer-readable storage medium, and computer program product.
  • the visual-inertial tracking and positioning system is an important underlying technology in the fields of computer vision, robotics, unmanned vehicles, 3D reconstruction, and augmented reality.
  • the rapid initialization method of the visual-inertial tracking and positioning system has important practical value in the fields of augmented reality and virtual reality.
  • initialization method of the usual visual-inertial tracking and positioning system is relatively complicated, and users need to receive more professional technical guidance to use it normally.
  • initialization solutions separate vision initialization from inertial sensor initialization, and there are some drawbacks in the initialization process in the related art.
  • Embodiments of the present disclosure provide at least one positioning initialization method, device, computer-readable storage medium, and computer program product.
  • the first aspect of the embodiments of the present disclosure provides a positioning initialization method, the method comprising: acquiring prior information of a state variable of a positioning device in a static state; determining a priori information of the state variable based on the prior information of the state variable Initial value; wherein, the state variable is used for positioning, and includes a visual state variable and an inertial state variable; during the running positioning process, the state variable is updated; after it is determined that the updated state variable satisfies the condition of convergence Next, confirm that the positioning initialization is complete.
  • initial values are set for the state variables, where the state variables are used for localization, and include visual state variables and inertial state variables, and during the running localization process, when determining When the updated state variable satisfies the convergence, it is determined that the positioning initialization is completed, that is, the visual and inertial systems are initialized at the same time when the device is close to a static state by using prior information, making the positioning initialization process fast and stable, and easy for users to operate.
  • the second aspect of the embodiments of the present disclosure provides a positioning initialization device, including: an acquisition module configured to acquire the prior information of the state variables of the positioning device in a static state; a setting module configured to The prior information of the variable determines the initial value of the state variable; wherein the state variable is used for positioning and includes a visual state variable and an inertial state variable; an update module is configured to update the state during the positioning process. The variable is updated; the determination module is configured to determine that the positioning initialization is completed when it is determined that the updated state variable satisfies convergence.
  • the third aspect of the embodiments of the present disclosure provides a positioning initialization device, including a memory and a processor coupled to each other, and the processor is used to execute the program instructions stored in the memory, so as to realize the above first aspect Positioning initialization method in .
  • a fourth aspect of the embodiments of the present disclosure provides a computer-readable storage medium on which program instructions are stored, and when the program instructions are executed by a processor, the positioning initialization method in the first aspect above is implemented.
  • the fifth aspect of the embodiments of the present disclosure provides a computer program product, including a computer-readable storage medium storing program codes.
  • the instructions included in the program codes are executed by the processor of the computer device, the above-mentioned The positioning initialization method in the first aspect.
  • the above scheme sets initial values for the state variables by using the prior information of the state variables acquired in the static state, wherein the state variables are used for positioning, and include visual state variables and inertial state variables, and during the running positioning process, in When it is determined that the updated state variables satisfy the convergence, it is determined that the positioning initialization is completed, that is, the use of prior information is used to initialize the visual and inertial systems at the same time when the device is close to a static state, making the initialization process of positioning fast and stable, and easy for users to operate .
  • FIG. 1 is a schematic flow diagram of an embodiment of the positioning initialization method of the present disclosure
  • Fig. 2 is a schematic flow chart of an embodiment of step S13 in Fig. 1;
  • FIG. 3 is a schematic flowchart of another embodiment of the positioning initialization method of the present disclosure.
  • FIG. 4 is a schematic flowchart of an application scenario of the positioning initialization method of the present disclosure
  • Fig. 5 is a schematic frame diagram of an embodiment of the positioning initialization device of the present disclosure.
  • Fig. 6 is a schematic frame diagram of another embodiment of the positioning initialization device of the present disclosure.
  • FIG. 7 is a block diagram of an embodiment of a computer-readable storage medium of the present disclosure.
  • system and “network” are often used interchangeably herein.
  • the term “and/or” in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations.
  • the character "/” in this article generally indicates that the contextual objects are an “or” relationship.
  • “many” herein means two or more than two.
  • the initialization process includes: first, the vision needs enough parallax to reconstruct the existing scene, and after the vision initialization is completed, the result of the vision initialization is aligned with the result of the inertial sensor to complete the initialization of the inertial system.
  • Such an initialization process has some disadvantages: on the one hand, when the device is stationary or rotating, it cannot provide enough parallax to complete the visual initialization; There is a certain probability that it will not succeed; at the same time, for scenes such as not rich textures or large-scale outdoor environments, the success rate of initialization is low.
  • FIG. 1 is a schematic flowchart of an embodiment of the positioning initialization method of the present disclosure. The method may include the following steps:
  • Step S11 Obtain prior information of state variables of the positioning device in a static state.
  • Step S12 Determine the initial value of the state variable based on the prior information of the state variable.
  • the state variables are used for positioning, and include visual state variables and inertial state variables.
  • the visual-inertial tracking and positioning system is an algorithm that fuses camera and inertial measurement unit (Inertial Measurement Unit, IMU) data to realize simultaneous positioning and map construction (Simultaneous Localization and Mapping, SLAM). Due to the nonlinearity of the visual-inertial tracking and positioning system, whether it is based on Filtering or graph-based optimization methods, the performance of the sensor is heavily dependent on the accuracy of the initial value. Once poorly initialized, it will not only reduce the convergence speed, but even lead to wrong estimation. Therefore, a robust initialization method is crucial.
  • IMU Inertial Measurement Unit
  • the visual-only initialization is first performed to calculate the relative pose of the camera; and then the alignment with the IMU pre-integration is performed to solve the initialization parameters to complete the visual-inertial joint initialization process.
  • the visual-inertial tracking and positioning system it is difficult for the visual-inertial tracking and positioning system to obtain an accurate initial state.
  • the scale information of the camera cannot be directly observed.
  • non-zero acceleration motion is required to initialize the scale information. At rest, this initialization method fails.
  • the execution subject of the positioning initialization method of the present disclosure may be a positioning initialization device.
  • the positioning initialization method may be executed by a positioning device or a server or other processing device, wherein the positioning device may be a mobile device such as a robot, an unmanned vehicle, or a drone.
  • the positioning device may be a mobile device such as a robot, an unmanned vehicle, or a drone.
  • the positioning device can also be user equipment (User Equipment, UE), user terminal, cordless phone, personal digital assistant (Personal Digital Assistant, PDA), handheld device, computing device, vehicle-mounted device, wearable device, etc.
  • the positioning initialization method may be implemented by a processor invoking computer-readable instructions stored in a memory.
  • the positioning initialization method performs initialization when the positioning device is in a static state, and prior information on the state variables of the positioning device in a static state needs to be obtained before initialization.
  • IMU data can be used as prior information of inertial state variables
  • camera data can be used as prior information of visual state variables.
  • IMU data can be used as prior information, because the bias of the gyroscope and accelerometer can be estimated, and the gravity vector can also be observed. At this time, the IMU has some small measurement errors and random walks of the device.
  • the stationary state in the present disclosure may be a state in which the positioning device moves very slowly, or a state in which the positioning device is completely stationary.
  • the inertial state variable includes at least one of gravity, position, angle, velocity, scale, and inertial system offset; the visual state variable includes at least one of three-dimensional point inverse depth, position, and angle. Therefore, since the inertial state variable includes at least one of gravity, position, angle, velocity, scale, and inertial system offset, and the visual state variable includes at least one of three-dimensional point inverse depth, position, and angle, so by using The prior information of each state variable obtained in the static state can be used to set the initial value for each state variable of the system, thereby realizing the initialization of the entire system.
  • the above step S12 may include: using the prior information of the inertial state variable as the initial value of the inertial state variable. It can be understood that, since the present disclosure is used for initialization when the positioning device is in a static state, the inertial state variable does not change when the positioning device is in a static state, therefore, the prior information of the inertial state variable can be used as the inertial state variable initial value.
  • the above step S11 includes: when the state variable includes speed, determining the speed as a rest speed value, and using the rest speed value as prior information of the inertia state variable.
  • the speed in the static state is the static speed value. Since this embodiment is used for initialization when the positioning device is in the static state, the speed of the positioning device is the static speed value, and the static speed value is used as the inertial state variable The prior information of the velocity variable has great confidence, which makes the initialization process of the system more accurate.
  • the above step S11 includes: in the case that the state variable includes an inertial system offset, acquiring an offset or a preset calibration value during the positioning process before this positioning initialization, and converting the The offset amount or the preset calibration value in the positioning process before the current positioning initialization is used as the prior information of the inertial system offset amount.
  • the IMU will have a certain offset.
  • the offset is generally related to temperature, factory accuracy, etc.
  • the positioning device is used for the first time, its inertial system offset can be a preset calibration value.
  • the offset during the positioning process before this positioning initialization can also be used as this positioning Initialized inertial system offset.
  • the above step S11 includes: using an inertial complementary filter to acquire the current value of the inertial state variable of the positioning device in the static state, and using the acquired current value of the inertial state variable as the inertia Prior information on state variables.
  • an inertial complementary filter to acquire the current value of the inertial state variable of the positioning device in the static state, and using the acquired current value of the inertial state variable as the inertia Prior information on state variables.
  • the proportion of the accelerometer is increased, and different weights are given to the gyroscope and accelerometer.
  • the positioning device itself can calculate its gravity direction, and the calculated gravity direction is reliable enough, the gravity direction calculated by the positioning device itself can be used instead of the gravity direction obtained by the above inertial complementary filter as Prior information about the direction of gravity.
  • the accuracy of the system initialization process can be made higher.
  • Step S13 During the running positioning process, update the state variable.
  • Step S14 In a case where it is determined that the updated state variable satisfies convergence, it is determined that the positioning initialization is completed.
  • the positioning device may be in a static state or in a moving state.
  • the time positioning device is in the process of running positioning.
  • each state variable will update data in real time, and the degree of convergence of the state variables will continue to change. Therefore, the state variables can be updated. , and when it is determined that the updated state variables satisfy the convergence, it is determined that the positioning initialization is completed; it should be noted that the positioning initialization of the entire visual-inertial tracking and positioning system is completed only when all the state variables satisfy the convergence.
  • the above scheme sets initial values for the state variables by using the prior information of the state variables acquired in the static state, wherein the state variables are used for positioning, and include visual state variables and inertial state variables, and during the running positioning process, in When it is determined that the updated state variables satisfy the convergence, it is determined that the positioning initialization is completed, that is, the use of prior information is used to initialize the visual and inertial systems at the same time when the device is close to a static state, making the initialization process of positioning fast and stable, and easy for users to operate .
  • FIG. 2 is a schematic flowchart of an embodiment of step S13 in FIG. 1 .
  • the above step S13 may include the following steps:
  • Step S131 Obtain the current value of the state variable during the running positioning process.
  • Step S132 Obtain an updated value of the state variable based on the current value of the state variable.
  • the positioning initialization method before the updating of the state variables in the above step S13, the positioning initialization method further includes: performing decoupling processing on the updating of several state variables.
  • the above-mentioned step of decoupling the updates of several state variables includes: determining the initial value confidence of each of the state variables, wherein the initial confidence is used to represent the initial How much the value affects the updated state variable.
  • the above step S132 includes:
  • Step S1321 Obtain an updated value of the state variable based on the initial value of the state variable, the confidence degree of the initial value, and the current value.
  • the initial value confidence degree can be set for each state variable based on experience, and the initial confidence degree can represent: after setting the initial value of the state variable of the visual-inertial tracking and positioning system, how big is it in the process of subsequent tracking and observation? Weight to believe in the initial value or the current value in the tracking state; in the process of running positioning, using a filter optimizer or a nonlinear optimizer, it can be based on the initial value of the state variable, the confidence of the initial value, and the state obtained at each moment The observed value of the variable is calculated to obtain the update value corresponding to this moment. Therefore, for the current moment, the value and confidence of the state variable at the previous moment have been obtained, that is, the historical value of the state variable and the corresponding confidence degree have been obtained.
  • a filter optimizer or a nonlinear optimizer is used for calculation to obtain an updated value of the state variable at the current moment. Therefore, after the updated value of the state variable is obtained, the confidence of the updated value can be obtained, thereby weakening the influence of the system cumulative error or observation error.
  • the initial confidence degree is used to indicate the degree of influence of the initial value on the updated state variable, so that the mutual influence between each state variable is reduced, and each state variable It has independent observability, so that the system can converge to the optimal value, and the fluctuation is small during the convergence process, which can improve the stability of static initialization and the convergence speed of the system.
  • the above step S13 may include: when the state variable includes scale information, during the running positioning process, perform motion initialization on the positioning device, and use the scale information obtained after the motion initialization is completed , to get the current value of the scale information. It is understandable that since the pure visual positioning system cannot solve the scale information problem, and the IMU can estimate the scale information through its own inertial measurement value, the scale information is obtained by adding the IMU. In the existing motion initialization method, the scale information of the visual-inertial tracking and positioning system can be obtained more accurately.
  • the motion initialization method can be run in the background at the same time, and after the motion initialization is stable, the system information of motion initialization, such as relatively accurate scale information, can be applied to In the current visual-inertial tracking and positioning system, it can facilitate the fast convergence of the scale information in the positioning initialization of the present disclosure.
  • the above step S13 may include: when the state variable includes the inverse depth of the three-dimensional point, during the running positioning process, acquiring the second scene image collected by the positioning device, along the depth direction of the point Determine the random inverse depth value of the corresponding inverse depth three-dimensional point for the two-dimensional point in the second scene image as the current value of the inverse depth of the three-dimensional point, that is, the random inverse depth value of the inverse depth three-dimensional point is used as The current value of the inverse depth of the 3D point.
  • the positioning device when the positioning device is in a static state, the 3D information in the scene cannot be obtained correctly, and in the absence of 3D information of the scene, the visual-inertial tracking and positioning system has no constraints, and the system is prone to deviation. Therefore, for the feature points in the scene image, a random inverse depth value can be given along the depth direction of the point, forming an inverse depth three-dimensional point. Therefore, during the running positioning process, the current value of the inverse depth of the three-dimensional point of the inverse depth three-dimensional point can be obtained at any time.
  • the inverse depth value When the positioning device moves normally, the inverse depth value will converge towards the correct direction, and when the positioning device does not move or a small amount of movement occurs, the inverse depth value can be used as prior information to constrain the system from causing large deviations, that is, Improve the stability of initialization of visual inertial tracking and positioning system in scenes with unknown depth.
  • the integration of discontinuous information in the inertial system will quickly accumulate errors, and the inverse depth 3D point as the 3D information of the environment can constrain the current frame information and historical frame information at the same time, thereby reducing the cumulative error; for example,
  • the inverse depth 3D point can be converted into a global Euclidean space 3D point, that is, the coordinates of the point can be decoupled from the pose information of the positioning device, thereby further reducing the accumulation of errors, and can more effectively improve the positioning system in the scene of small and fast shaking down stability.
  • the scale information obtained after the motion initialization of the positioning device is used as the current value of the scale information, so that the scale information can quickly converge after the static initialization is completed;
  • observing the inverse depth value of the inverse depth 3D point can improve the stability of the system.
  • the random inverse depth value of the above inverse depth 3D point can be replaced by the depth value acquired by the depth sensor.
  • the positioning initialization method further includes: acquiring at least two frames The device orientation information corresponding to the second scene image; according to the two-dimensional points in the at least two frames of the second scene image and the device orientation information, removing outliers in the second scene image. Therefore, according to the two-dimensional points in at least two frames of the second scene image and the device orientation information, outliers in the second scene image can be eliminated, thereby optimizing the scene image and improving the stability of system initialization.
  • the device orientation information of at least two frames of scene images is known. Random sample consistency detection method to remove obvious outliers in scene images.
  • the orientation information of the device can be obtained by calculating the inertial complementary filter. Therefore, according to the two-dimensional points in at least two frames of the scene image and the orientation information of the device, the outliers in the scene image can be eliminated, so that the scene image can be Optimization to improve the stability of the positioning initialization of the positioning device in dynamic scenes.
  • the above step S13 may also include: when the state variable includes speed, during the running positioning process, judging whether the positioning device is in a stationary state; Next, the static speed value is used as the current value of the speed; based on the current value of the speed, an updated value of the speed is obtained.
  • the static speed value is used as the current value of the speed
  • the update value of the speed is calculated by using a filter optimizer or a nonlinear optimizer
  • the positioning initialization method may further include: obtaining the uncertainty corresponding to the update value of the state variable, and when the uncertainty of the state variable is lower than the second In the case of a preset threshold, it is determined that the state variable satisfies convergence.
  • a preset threshold it is determined that the state variable satisfies convergence.
  • the positioning initialization method may further include: using the updated value of the state variable to determine the pose of the positioning device. It can be understood that, according to the visual state variable and the inertial state variable, the pose information of the positioning device can be obtained, so that the positioning of the device can be realized.
  • the pose information of the positioning device can be obtained to realize the positioning of the device.
  • FIG. 3 is a schematic flowchart of another embodiment of a positioning initialization method of the present disclosure.
  • the method may include the steps of:
  • Step S31 Obtain two consecutive frames of first scene images collected by the positioning device.
  • Step S32 If it is determined that the disparity values between all feature points in the first scene images of the two consecutive frames are less than a first preset threshold, then determine that the positioning device is in a static state.
  • visual information may be used to determine whether the positioning device is in a static state. For example, by acquiring two consecutive frames of the first scene image collected by the positioning device, if it is determined that the disparity values between all feature points in the first scene image of the two consecutive frames are less than the first preset threshold, then Determining that the positioning device is in a static state can accurately determine whether the positioning device is in a static state, thereby providing a suitable initialization moment for the initialization process.
  • Step S33 Obtain prior information of state variables of the positioning device in a static state.
  • Step S34 Based on the prior information of the state variable, determine the initial value of the state variable; wherein, the state variable is used for positioning, and includes a visual state variable and an inertial state variable.
  • Step S35 During the running positioning process, update the state variables.
  • Step S36 In a case where it is determined that the updated state variable satisfies convergence, it is determined that the positioning initialization is completed.
  • steps S33 to S36 are basically similar to steps S11 to S14 in the above embodiments of the present disclosure.
  • FIG. 4 is a schematic flowchart of an application scenario of the positioning initialization method of the present disclosure.
  • the positioning initialization method of the positioning device is divided into three stages, including an initialization preparation stage, an initialization stage, and an initialization completion stage.
  • the initialization preparation stage it is mainly to obtain the prior information of positioning initialization and select the appropriate initialization time.
  • the inertial complementary filter can be used to estimate the gravity direction as the prior information of the gravity direction, and then the inertial filter converges
  • the visual information is used to judge whether the positioning device is in a static state, and if it is confirmed that the positioning device is in a static state, the initialization phase can be entered.
  • the system initialization is mainly carried out, that is, the initial value of each state variable of the system is set by using prior information, and at the same time, the state variables in the system are decoupled by setting the confidence of the state variable; for example, setting each state variable of the system
  • the initial value of the state variable includes: setting the gravity vector, at this time, the gravity direction estimated in the initialization preparation stage can be used as the gravity direction in the gravity vector; setting the velocity value, which can be initialized with zero velocity; setting the inertial system bias You can use the saved value or the preset calibration value during the previous stable operation of the system; before the entire system converges, state variables such as scale, gravity direction, and inertial sensor bias will be coupled together, which may prompt the system to move towards a non-linear state.
  • the direction of the optimal value converges or there is a large fluctuation during the convergence process. Therefore, the corresponding confidence can be added to the initial value of each state variable to reduce the mutual influence of each state variable.
  • the confidence level When a state variable When the confidence level is larger, the convergence speed of the single state variable can be accelerated.
  • a variety of strategies are mainly used to increase the system stability from the initialization stage to before the system converges; for example, to eliminate outliers in non-continuous frames, from the initial value setting of the system state variables to the system before the system converges
  • the feature detection method is used to eliminate obvious outliers in the scene image; for example, to add and maintain the inverse depth 3D point, for the feature point in the scene image, a random inverse depth value can be given along the depth direction of the point, forming an inverse Depth 3D point, through the inverse depth value of the inverse depth 3D point, when the positioning device moves normally, the system can converge in the correct direction, and when the positioning device does not move or a small amount of movement occurs, it can be used as a priori information
  • the inverse depth 3D point as the 3D information of the environment can constrain the current frame information and historical frame information at the same time, thereby reducing the cumulative error; for example, when performing zero-speed update, before the system converges,
  • the static speed value can be used as the update value of the speed to directly constrain the speed information of the system, which can effectively suppress the large offset of the system and improve the stability of the system initialization;
  • the pose information of the positioning device can be output according to each state variable, so that the positioning of the device can be realized.
  • FIG. 5 is a schematic frame diagram of an embodiment of a positioning initialization device of the present disclosure.
  • the positioning initialization device 50 includes: an acquisition module 500 configured to acquire prior information of a state variable of the positioning device in a static state; a setting module 502 configured to determine an initial value of the state variable based on the prior information of the state variable ; Wherein, the state variable is used for positioning, and includes a visual state variable and an inertial state variable; the update module 504 is configured to update the state variable during the running positioning process; the determination module 506 is configured to determine the update In a case where the state variables satisfy the convergence requirements, it is determined that the positioning initialization is completed.
  • the setting module 502 uses the prior information of the state variable acquired by the acquisition module 500 in the static state to set the initial value for the state variable, wherein the state variable is used for positioning, and includes the visual state variable and the inertial state variable.
  • the determination module 506 determines that the positioning initialization is completed when it is determined that the updated state variables meet the convergence requirements, that is, the use of prior information is used to initialize the visual and inertial systems at the same time when the device is close to a static state, so that the positioning initialization process Fast and stable, yet user-friendly.
  • the setting module 502 is configured to use the prior information of the inertial state variable as the initial value of the inertial state variable.
  • the obtaining module 500 may be configured to: if the state variable includes a speed, determine that the speed is a static speed value; and/or, if the state variable includes an inertial system bias, obtain The offset amount or the preset calibration value in the positioning process before the current positioning initialization is used as the inertial system offset value or the preset calibration value in the positioning process before the current positioning initialization and/or, use an inertial complementary filter to acquire the current value of the inertial state variable of the positioning device in the static state, and use the acquired current value of the inertial state variable as the inertia Prior information on state variables.
  • the positioning initialization device 50 further includes: a judging module configured to acquire two consecutive frames of the first scene image collected by the positioning device; determine all the feature points in the two consecutive frames of the first scene image If the disparity values between are smaller than the first preset threshold, it is determined that the positioning device is in a static state.
  • the update module 504 is configured to obtain the current value of the state variable during the positioning process, and obtain an updated value of the state variable based on the current value of the state variable.
  • the setting module 502 is further configured to decouple the update of several state variables; in some embodiments, the setting module 502 is configured to determine the initial value confidence of each of the state variables, wherein , the initial confidence is used to represent the degree of influence of the initial value on the updated state variable. At this time, the updating module 504 is configured to obtain an updated value of the state variable based on the initial value of the state variable, the confidence degree of the initial value, and the current value.
  • the update module 504 executes the step of obtaining the current value of the state variable during the running positioning process, which may include at least one of the following steps: when the state variable includes scale information, during the running positioning process In this method, the motion initialization is performed on the positioning device, and the current value of the scale information is obtained by using the scale information obtained after the motion initialization is completed; when the state variable includes the three-dimensional point inverse depth, the running positioning process In the method, the second scene image collected by the positioning device is obtained, and the random inverse depth value of the corresponding inverse depth three-dimensional point is determined for the two-dimensional point in the second scene image along the depth direction of the point, and the inverse depth The random inverse depth value of the 3D point is used as the current value of the inverse depth of the 3D point.
  • the update module 504 is further configured to acquire device orientation information corresponding to at least two frames of the second scene image; according to the two-dimensional points in the at least two frames of the second scene image and the device orientation information, and remove outliers in the second scene image.
  • the update module 504 is further configured to determine whether the positioning device is in a static state during the running positioning process if the state variable includes a speed; if the positioning device is in a static state , using the rest velocity value as the update value for the velocity.
  • the determination module 506 may also be configured to obtain the uncertainty corresponding to the update value of the state variable, and determine that if the uncertainty of the state variable is lower than a second preset threshold The state variable satisfies convergence; using the update value of the state variable to determine the pose of the positioning device.
  • FIG. 6 is a schematic frame diagram of another embodiment of the positioning initialization device of the present disclosure.
  • the location initialization device 60 includes a memory 61 and a processor 62 coupled to each other.
  • the processor 62 is configured to execute program instructions stored in the memory 61 to implement the steps of any one of the location initialization method embodiments above.
  • the positioning initialization device 60 may include, but not limited to: a microcomputer and a server.
  • the processor 62 is configured to control itself and the memory 61 to implement the steps in any of the above embodiments of the location initialization method.
  • the processor 62 may also be called a central processing unit (Central Processing Unit, CPU).
  • the processor 62 may be an integrated circuit chip with signal processing capabilities.
  • the processor 62 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the processor 62 may be jointly implemented by an integrated circuit chip.
  • the processor uses the prior information of the state variables obtained in the static state to set the initial value for the state variables, wherein the state variables are used for positioning, and include visual state variables and inertial state variables, and during the running positioning process,
  • the positioning initialization is completed, that is, the use of prior information is used to initialize the visual and inertial systems at the same time when the device is close to a static state, making the initialization process of positioning fast and stable, and at the same time easy for users operate.
  • FIG. 7 is a schematic diagram of an embodiment of a computer-readable storage medium of the present disclosure.
  • the computer-readable storage medium 70 stores program instructions 700 that can be executed by the processor, and the program instructions 700 are used to implement the steps in any of the above embodiments of the positioning initialization method.
  • An embodiment of the present disclosure further provides a computer program product, the computer program product carries a program code, and instructions included in the program code can be used to implement the steps in any of the above embodiments of the positioning initialization method, and reference can be made to the above method embodiments.
  • the above-mentioned computer program product may be implemented by means of hardware, software or a combination thereof.
  • the computer program product may be embodied as a computer storage medium, and in another optional embodiment, the computer program product may be embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. wait.
  • a software development kit Software Development Kit, SDK
  • This disclosure relates to the field of augmented reality.
  • Augmented Reality By acquiring the image information of the target object in the real environment, and then using various visual correlation algorithms to detect or identify the relevant features, states and attributes of the target object, and thus obtain the image information that matches the specific application.
  • Augmented Reality effect combining virtual and reality.
  • the target object may involve faces, limbs, gestures, actions, etc. related to the human body, or markers and markers related to objects, or sand tables, display areas or display items related to venues or places.
  • Vision-related algorithms can involve visual positioning, SLAM, 3D reconstruction, image registration, background segmentation, object key point extraction and tracking, object pose or depth detection, etc.
  • Specific applications can not only involve interactive scenes such as guided tours, navigation, explanations, reconstructions, virtual effect overlays and display related to real scenes or objects, but also special effects processing related to people, such as makeup beautification, body beautification, special effect display, virtual Interactive scenarios such as model display.
  • the relevant features, states and attributes of the target object can be detected or identified through the convolutional neural network.
  • the above-mentioned convolutional neural network is a network model obtained by performing model training based on a deep learning framework.
  • the disclosed methods and devices may be implemented in other ways.
  • the device implementations described above are only illustrative.
  • the division of modules or units is only a logical function division. In actual implementation, there may be other division methods.
  • units or components can be combined or integrated. to another system, or some features may be ignored, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • a unit described as a separate component may or may not be physically separated, and a component shown as a unit may or may not be a physical unit, that is, it may be located in one place, or may also be distributed to network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the technical solution of the present disclosure is essentially or part of the contribution to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) execute all or part of the steps of the methods in various embodiments of the present disclosure.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .
  • the embodiment of the present disclosure discloses a positioning initialization method, device, computer-readable storage medium, and computer program product, wherein the positioning initialization method includes: acquiring prior information of a state variable of a positioning device in a static state; based on the state variable The prior information of the state variable is used to determine the initial value of the state variable; wherein the state variable is used for positioning and includes a visual state variable and an inertial state variable; during the running positioning process, the state variable is updated; when determining In a case where the updated state variable satisfies convergence, it is determined that the positioning initialization is completed.
  • the embodiments of the present disclosure set initial values for the state variables by using the prior information of the state variables obtained in the static state, where the state variables are used for positioning, and include visual state variables and inertial state variables, and during the running positioning process,
  • the positioning initialization is completed, that is, the use of prior information is used to initialize the visual and inertial systems at the same time when the device is close to a static state, making the initialization process of positioning fast and stable, and at the same time easy for users operate.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Navigation (AREA)
  • Image Analysis (AREA)

Abstract

La présente invention concerne un procédé et un appareil d'initialisation de positionnement, ainsi qu'un support de stockage lisible par ordinateur et un produit programme informatique. Le procédé d'initialisation de positionnement selon l'invention consiste : à acquérir des informations a priori de variables d'état d'un dispositif de positionnement dans un état statique ; à déterminer des valeurs initiales des variables d'état en fonction des informations a priori des variables d'état, les variables d'état servant à effectuer un positionnement et comprenant une variable d'état visuelle et une variable d'état inertielle ; à mettre à jour les variables d'état pendant l'exécution d'un processus de positionnement ; et, lorsqu'il est déterminé que les variables d'état mises à jour respectent la convergence, à déterminer que l'initialisation de positionnement est achevée. Cette solution permet d'obtenir un procédé d'initialisation pouvant présenter une stabilité relativement élevée.
PCT/CN2022/098214 2021-11-16 2022-06-10 Procédé et appareil d'initialisation de positionnement, et support de stockage lisible par ordinateur et produit programme informatique WO2023087681A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111356493.9A CN114022556A (zh) 2021-11-16 2021-11-16 定位初始化方法、装置和计算机可读存储介质
CN202111356493.9 2021-11-16

Publications (1)

Publication Number Publication Date
WO2023087681A1 true WO2023087681A1 (fr) 2023-05-25

Family

ID=80064583

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/098214 WO2023087681A1 (fr) 2021-11-16 2022-06-10 Procédé et appareil d'initialisation de positionnement, et support de stockage lisible par ordinateur et produit programme informatique

Country Status (2)

Country Link
CN (1) CN114022556A (fr)
WO (1) WO2023087681A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114022556A (zh) * 2021-11-16 2022-02-08 浙江商汤科技开发有限公司 定位初始化方法、装置和计算机可读存储介质
CN114993304A (zh) * 2022-06-29 2022-09-02 北京石头创新科技有限公司 一种机器人位置的确定方法、装置、介质和电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10043076B1 (en) * 2016-08-29 2018-08-07 PerceptIn, Inc. Visual-inertial positional awareness for autonomous and non-autonomous tracking
CN109238277A (zh) * 2018-08-01 2019-01-18 清华大学 视觉惯性数据深度融合的定位方法及装置
CN112798010A (zh) * 2019-11-13 2021-05-14 北京三快在线科技有限公司 一种视觉惯性里程计vio系统的初始化方法、装置
CN113052897A (zh) * 2021-03-25 2021-06-29 浙江商汤科技开发有限公司 定位初始化方法和相关装置、设备、存储介质
CN114022556A (zh) * 2021-11-16 2022-02-08 浙江商汤科技开发有限公司 定位初始化方法、装置和计算机可读存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10043076B1 (en) * 2016-08-29 2018-08-07 PerceptIn, Inc. Visual-inertial positional awareness for autonomous and non-autonomous tracking
CN109238277A (zh) * 2018-08-01 2019-01-18 清华大学 视觉惯性数据深度融合的定位方法及装置
CN112798010A (zh) * 2019-11-13 2021-05-14 北京三快在线科技有限公司 一种视觉惯性里程计vio系统的初始化方法、装置
CN113052897A (zh) * 2021-03-25 2021-06-29 浙江商汤科技开发有限公司 定位初始化方法和相关装置、设备、存储介质
CN114022556A (zh) * 2021-11-16 2022-02-08 浙江商汤科技开发有限公司 定位初始化方法、装置和计算机可读存储介质

Also Published As

Publication number Publication date
CN114022556A (zh) 2022-02-08

Similar Documents

Publication Publication Date Title
CN109307508B (zh) 一种基于多关键帧的全景惯导slam方法
JP6198230B2 (ja) 深度カメラを使用した頭部姿勢トラッキング
WO2023087681A1 (fr) Procédé et appareil d'initialisation de positionnement, et support de stockage lisible par ordinateur et produit programme informatique
US10247556B2 (en) Method for processing feature measurements in vision-aided inertial navigation
CN107748569B (zh) 用于无人机的运动控制方法、装置及无人机系统
Kneip et al. Closed-form solution for absolute scale velocity determination combining inertial measurements and a single feature correspondence
US10636190B2 (en) Methods and systems for exploiting per-pixel motion conflicts to extract primary and secondary motions in augmented reality systems
US20210183100A1 (en) Data processing method and apparatus
WO2020221307A1 (fr) Procédé et dispositif pour suivre un objet mobile
EP2904417A1 (fr) Procédé de détermination de la position et de l'orientation d'un dispositif associé à un dispositif de capture pour capturer au moins une image
JP7131994B2 (ja) 自己位置推定装置、自己位置推定方法、自己位置推定プログラム、学習装置、学習方法及び学習プログラム
Michot et al. Bi-objective bundle adjustment with application to multi-sensor slam
WO2023050634A1 (fr) Procédé et appareil de positionnement, dispositif, support de stockage et produit de programme informatique
Tomažič et al. Fusion of visual odometry and inertial navigation system on a smartphone
CN110749308B (zh) 使用消费级gps和2.5d建筑物模型的面向slam的室外定位方法
CN110660098A (zh) 基于单目视觉的定位方法和装置
WO2023087758A1 (fr) Procédé et appareil de positionnement, support de stockage lisible par ordinateur et produit-programme informatique
CN114565728A (zh) 地图构建方法、位姿确定方法及相关装置、设备
JP2023021994A (ja) 自動運転車両に対するデータ処理方法及び装置、電子機器、記憶媒体、コンピュータプログラム、ならびに自動運転車両
CN112731503B (zh) 一种基于前端紧耦合的位姿估计方法及系统
KR20220004604A (ko) 장애물의 검출 방법, 전자 기기, 노변 기기 및 클라우드 컨트롤 플랫폼
JP2022132063A (ja) 拡張現実提供装置のポーズ決定方法及び装置
WO2023140990A1 (fr) Odométrie inertielle visuelle avec profondeur d'apprentissage automatique
CN115727871A (zh) 一种轨迹质量检测方法、装置、电子设备和存储介质
Ullah et al. EMoVI-SLAM: Embedded monocular visual inertial SLAM with scale update for large scale mapping and localization

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22894224

Country of ref document: EP

Kind code of ref document: A1