WO2022193508A1 - Procédé et appareil d'optimisation de posture, dispositif électronique, support de stockage lisible par ordinateur, programme d'ordinateur et produit-programme - Google Patents

Procédé et appareil d'optimisation de posture, dispositif électronique, support de stockage lisible par ordinateur, programme d'ordinateur et produit-programme Download PDF

Info

Publication number
WO2022193508A1
WO2022193508A1 PCT/CN2021/106997 CN2021106997W WO2022193508A1 WO 2022193508 A1 WO2022193508 A1 WO 2022193508A1 CN 2021106997 W CN2021106997 W CN 2021106997W WO 2022193508 A1 WO2022193508 A1 WO 2022193508A1
Authority
WO
WIPO (PCT)
Prior art keywords
pose
current
movement
mobile device
information
Prior art date
Application number
PCT/CN2021/106997
Other languages
English (en)
Chinese (zh)
Inventor
章国锋
鲍虎军
叶智超
刘浩敏
Original Assignee
浙江商汤科技开发有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浙江商汤科技开发有限公司 filed Critical 浙江商汤科技开发有限公司
Publication of WO2022193508A1 publication Critical patent/WO2022193508A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/97—Determining parameters from multiple pictures

Definitions

  • the present disclosure relates to the field of computer vision technology, and in particular, to a pose optimization method, apparatus, electronic device, computer-readable storage medium, computer program, and program product.
  • the positioning of unmanned equipment through vision is an important part of applications in the fields of robots, unmanned aerial vehicles, automatic navigation and augmented reality. has been widely used.
  • the image data collected by the monocular camera cannot provide the real three-dimensional size information of these objects, so the scale of the monocular visual odometry cannot be determined, which leads to certain errors in the pose calculation.
  • This embodiment provides at least one pose optimization method, apparatus, electronic device, computer-readable storage medium, computer program, and program product.
  • an embodiment of the present disclosure provides a pose optimization method, including:
  • the first image includes the current frame image captured when the mobile device is at the current position and before the mobile device is at the current position The starting frame image taken when the starting position is ;
  • the pieces of estimated pose information include estimated pose information corresponding to the starting frame image, estimated pose information corresponding to the current frame image, and the starting frame image.
  • the current predicted pose is optimized based on the movement parameter information and the plurality of estimated pose information to obtain a current optimized pose.
  • the scale information of the visual odometer is recovered by acquiring the movement parameter information corresponding to the target movement process experienced by the mobile device when it moves to the current position, and the current predicted pose is optimized based on the scale information. , which can improve the calculation accuracy of pose.
  • this method uses the acquired movement parameter information as a scale constraint, and then obtains an optimized pose information.
  • the computational complexity is low, which reduces the performance requirements of the device (for example, it is applied to some low-end mobile phones), so The adaptability of the method is improved.
  • the movement parameter information includes a movement distance
  • the determining of the movement parameter information corresponding to the target movement process experienced by the mobile device when it moves to the current position includes:
  • the current predicted pose is optimized based on the movement parameter information and the plurality of estimated pose information to obtain the current optimized pose, including:
  • the current predicted pose is optimized based on the moving distance and the plurality of estimated pose information to obtain the current optimized pose.
  • the moving distance corresponding to the moving process of the target is determined based on the pedestrian dead reckoning algorithm, which can simplify the process of determining the moving distance, improve the efficiency of determining the moving distance, and further improve the efficiency of the pose optimization method. Operational efficiency.
  • the current predicted pose is optimized based on the moving distance and the plurality of estimated pose information to obtain the current optimized pose, including: :
  • the initial moving frame image corresponding to the moving start time and the moving ending frame image corresponding to the moving ending time are determined, and the moving initial frame image corresponding to the moving initial frame image and the moving ending frame image are respectively determined.
  • the movement initial estimated pose and the movement end estimated position corresponding to the movement initial frame image and the movement end frame image are determined
  • the movement distance is determined as the distance between the initial estimated pose of the movement and the estimated pose of the end of the movement, so that the distance constraint is obtained between the initial estimated pose of the movement and the estimated pose of the end of the movement, and then It can realize the optimization of the current predicted pose and improve the accuracy of visual positioning.
  • the moving distance corresponding to the moving process of the target experienced by the mobile device when moving to the current position is determined based on a pedestrian dead reckoning algorithm, include:
  • the moving distance is determined based on the number of walking steps and the step length of the user.
  • the moving distance is determined by using the user's walking steps and step length
  • only an Inertial Measurement Unit (Inertial Measurement Unit) with lower precision can be used to achieve this, and thus the positioning accuracy can be improved at the same time.
  • the requirement for the accuracy of the IMU is reduced, so that the method can perform indoor positioning in scenarios with limited computing power (for example, on the applet of a low-end mobile phone).
  • the movement parameter information further includes a movement speed
  • the determining of the movement parameter information corresponding to the target movement process experienced by the mobile device when it moves to the current position further includes:
  • the optimizing the current predicted pose based on the moving distance and the plurality of estimated pose information to obtain the current optimized pose includes:
  • the current predicted pose is optimized based on the moving distance, the moving speed, and the plurality of estimated pose information to obtain the current optimized pose.
  • the moving speed is also determined, so that the corresponding estimated pose information of the multi-frame images corresponding to the distance that the user walks one step is also constrained by the distance, thereby further improving the optimization of the current predicted pose precision.
  • the determining of the moving speed corresponding to the target moving process experienced by the mobile device when moving to the current position includes:
  • the movement speed is predicted based on a movement model and the historical movement speed.
  • the moving speed of the target process is predicted by the motion model and the historical motion speed, which can improve the accuracy of parameter acquisition compared to the user's walking speed measured directly based on the IMU.
  • the method further includes:
  • real-time pose information of the mobile device is determined.
  • the current predicted speed and the current orientation can be directly used to continue tracking for a period of time until the visual tracking Track recovery. In this way, even in the scene where the visual odometry is difficult to locate, the navigation and positioning results can still be obtained through the pedestrian dead position information, which improves the applicability of the method.
  • the method further includes:
  • a navigation route is planned, and the navigation route is displayed on the map.
  • the current route planning is performed by using the optimized current predicted pose, which can improve the accuracy of the planned route.
  • the navigation route is planned based on the current optimized pose, the destination input by the user, and the stored map, and the navigation route is displayed on the map, include:
  • the at least one connection line is displayed in the map.
  • the destination information input by the user is first determined in the map, and then at least one connection route is planned according to the current optimized pose and the location information of the destination, and is The at least one connection line is displayed on the map, so that multiple choices can be provided for the user, so that the user can select one of the multiple lines for navigation according to the actual situation, thereby improving the applicability of the navigation line.
  • an embodiment of the present disclosure provides a pose optimization device, including:
  • the pose estimation part is configured to determine the current predicted pose of the mobile device based on the first image captured by the mobile device; the first image includes the current frame image captured when the mobile device is in the current position and the movement The starting frame image taken when the device is at the starting position before the current position;
  • a parameter determination part configured to determine the movement parameter information corresponding to the target movement process experienced by the mobile device when moving to the current position; the target movement process is located between the starting position and the current position ;
  • a pose acquisition part configured to obtain a plurality of estimated pose information of the mobile device, the plurality of estimated pose information including the estimated pose information corresponding to the initial frame image, the estimated pose information corresponding to the current frame image pose information and estimated pose information respectively corresponding to the multi-frame images between the initial frame image and the current frame image;
  • the pose optimization part is configured to optimize the current predicted pose based on the movement parameter information and the plurality of estimated pose information to obtain the current optimized pose.
  • the movement parameter information includes a movement distance
  • the parameter determination part is configured as:
  • the pose optimization part is configured as:
  • the current predicted pose is optimized based on the moving distance and the plurality of estimated pose information to obtain the current optimized pose.
  • the pose optimization part is configured as:
  • the initial moving frame image corresponding to the moving start time and the moving ending frame image corresponding to the moving ending time are determined, and the moving initial frame image corresponding to the moving initial frame image and the moving ending frame image are respectively determined.
  • the parameter determining part is configured to:
  • the moving distance is determined based on the number of walking steps and the step length of the user.
  • the movement parameter information further includes a movement speed
  • the parameter determination part is also configured as:
  • the pose optimization part is configured as:
  • the current predicted pose is optimized based on the moving distance, the moving speed, and the plurality of estimated pose information to obtain the current optimized pose.
  • the parameter determining part is configured to:
  • the movement speed is predicted based on a movement model and the historical movement speed.
  • the pose optimization part is further configured to:
  • real-time pose information of the mobile device is determined.
  • the pose optimization device further includes a route planning part, and the route planning part is configured to:
  • a navigation route is planned, and the navigation route is displayed on the map.
  • the route planning part is configured as:
  • the at least one connection line is displayed in the map.
  • embodiments of the present disclosure provide an electronic device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the processing The processor and the memory communicate through a bus, and the machine-readable instructions are executed by the processor to execute the pose optimization method according to the first aspect.
  • an embodiment of the present disclosure provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to execute the pose optimization method according to the first aspect .
  • an embodiment of the present disclosure provides a computer program, including computer-readable code, where the computer-readable code runs in an electronic device and is executed by a processor in the electronic device, and executes the following steps: The pose optimization method described in the first aspect.
  • an embodiment of the present disclosure provides a computer program product that, when executed on a computer, causes the computer to execute the pose optimization method described in the first aspect.
  • FIG. 1 shows a flowchart of a pose optimization method provided by an embodiment of the present disclosure
  • FIG. 2 shows a flowchart of a method for optimizing a current predicted pose based on a moving distance provided by an embodiment of the present disclosure
  • FIG. 3 shows a flowchart of a method for obtaining a moving distance provided by an embodiment of the present disclosure
  • FIG. 4 shows a flowchart of a method for obtaining a moving speed provided by an embodiment of the present disclosure
  • FIG. 5 shows a flowchart of another pose optimization method provided by an embodiment of the present disclosure
  • FIG. 6 shows a schematic structural diagram of a pose optimization apparatus provided by an embodiment of the present disclosure
  • FIG. 7 shows a schematic structural diagram of another pose optimization apparatus provided by an embodiment of the present disclosure.
  • FIG. 8 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.
  • the positioning of unmanned equipment through vision is an important part of applications in the fields of robots, unmanned aerial vehicles, automatic navigation, and augmented reality.
  • the vision method only relies on the image information collected by the camera, based on the camera model and visual geometry Learning models, etc. to calculate the six-degree-of-freedom motion of the machine itself.
  • This self-localization orientation method based on image information is similar to traditional odometer dead reckoning, so it is called visual odometry.
  • One type of solution is to combine additional accurate sensors to obtain more information and restore scales, such as visual odometry combined with high-precision GPS and IMU; however, this method requires the device to be equipped with additional high-precision sensing devices and to the computing power of the device. Accuracy is required.
  • Another type of solution is to build in advance through the navigation scene, and restore the scale of the visual odometry by continuously positioning in the map.
  • this method has high requirements on the frequency of pre-mapping and positioning.
  • the present disclosure provides a pose optimization method, which determines the current predicted pose of the mobile device based on the first image captured by the mobile device; the first image includes the current frame image captured when the mobile device The initial frame image taken when the device is in the initial position before the current position; determine the movement parameter information corresponding to the target movement process experienced by the mobile device when it moves to the current position; the target movement process is located at the initial position and the current position; obtain multiple estimated pose information of the mobile device, the multiple estimated pose information includes the estimated pose information corresponding to the starting frame image and the current frame image respectively, as well as the starting frame image and the current frame image The estimated pose information corresponding to the multi-frame images between them respectively; the current predicted pose is optimized based on the movement parameter information and the multiple estimated pose information to obtain the current optimized pose.
  • the scale information of the visual odometer is recovered by acquiring the movement parameter information corresponding to the movement process of the target experienced by the mobile device when it moves to the current position. , and optimize the current predicted pose based on the scale information, which can improve the accuracy of pose calculation.
  • the execution subject of the pose optimization method provided by the embodiment of the present disclosure is generally an electronic device with a certain computing capability.
  • the electronic device includes, for example, a terminal device or a server or other processing device, and the terminal device may be a mobile device, a user terminal, a terminal, a handheld device, a computing device, a vehicle-mounted device, a wearable device, and the like.
  • the pose optimization method may be implemented by the processor calling computer-readable instructions stored in the memory.
  • the pose optimization method includes the following S101-S104:
  • S101 Determine the current predicted pose of the mobile device based on a first image captured by the mobile device; the first image includes a current frame image captured when the mobile device is at the current position and a start image captured when the mobile device is at a starting position before the current position Start frame image.
  • the process of visual localization it usually includes the process of visual tracking, estimating pose and optimizing.
  • the correlation between the feature points between the images can be obtained by visual tracking, and then the rough pose of the camera is estimated according to the correlation between the feature points.
  • some representative points should be selected first, called feature points; after that, only the motion of the camera is estimated for these feature points, and the spatial position of the feature points and the information of other non-feature points in the image are estimated at the same time. , is discarded.
  • common feature point extraction methods including corner points, color blocks, etc. in the image.
  • a mobile device refers to a device that can be mounted on a carrier (such as a human body) and can move with the carrier.
  • the mobile device may specifically include a mobile phone, a tablet computer, augmented reality (Augmented Reality, AR) glasses, and the like. That is, the mobile device may be a mobile device or a handheld device among the aforementioned electronic devices having a certain computing capability.
  • the mobile device can have a built-in image acquisition component or an external image acquisition component. After the mobile device enters the working state, the image acquisition component can be used to capture images of the real scene to obtain several images.
  • the first image is a part of the several images, that is, the first image includes the current frame image taken when the mobile device is at the current position and the starting position before the mobile device is at the current position The starting frame image taken at the time.
  • the starting position can be set according to the complexity of the specific real scene environment. For example, if the current real scene is relatively empty and simple, the position far from the current position before the current position can be used as the starting position; If the real scene environment is more complex, the position that is closer to the current position before the current position can be used as the starting position, which is not limited here.
  • the image acquisition component in the embodiment of the present disclosure is a monocular camera, there is a problem of uncertain scale, that is, only the relative position of the current position relative to the starting position can be obtained by using the feature point method based on the first image, Therefore, the obtained current predicted pose is only a relative pose, which is not accurate, and the current predicted pose needs to be optimized.
  • S102 Determine the movement parameter information corresponding to the target movement process experienced by the mobile device when it moves to the current position; the target movement process is located between the initial position and the current position.
  • the target moving process may be a process with the starting position as the starting point and the current position as the ending point, or may be any process between the starting position and the current position, here Not limited.
  • the movement parameter information may only include the movement distance; in other embodiments, the movement parameter information includes the movement distance and the movement speed, which will be described later for details.
  • S103 Acquire multiple estimated pose information of the mobile device, where the multiple estimated pose information includes estimated pose information corresponding to the initial frame image, estimated pose information corresponding to the current frame image, and the difference between the initial frame image and the current frame image The estimated pose information corresponding to the multi-frame images in between.
  • each image has estimated pose information of the camera (mobile device) corresponding to the image. Specifically, after matching the feature points, two one-to-one corresponding pixel point sets can be obtained, and then the motion of the camera is estimated according to the two sets of matched point sets.
  • the current predicted pose may be optimized based on the light velocity adjustment method (Bundle Adjustment).
  • the beam method adjustment method uses the camera attitude and the three-dimensional coordinates of the measurement point as unknown parameters, and uses the coordinates of the feature points detected on the image for forward intersection as the observation data to adjust to obtain the optimal camera parameters and World point coordinates.
  • the scale information of the visual odometer is recovered by acquiring the movement parameter information corresponding to the target movement process experienced by the mobile device when it moves to the current position, and based on the scale information
  • the information optimizes the current predicted pose, which in turn can improve the accuracy of pose calculation.
  • this method uses the acquired movement parameter information as a constraint, and then obtains an optimized pose information, the computational complexity is low, and the performance requirements of the device are reduced (for example, it is applied to some low-end mobile phones), thereby improving the adaptability of the method.
  • the movement parameter information includes the movement distance. Therefore, for the above S102, when determining the movement parameter information corresponding to the target movement process experienced by the mobile device when moving to the current position, the movement parameter information may include: Based on the pedestrian dead reckoning algorithm, determine the moving distance corresponding to the target moving process experienced by the mobile device when moving to the current position; for the above S104, in the based on the moving parameter information and The multiple estimated pose information optimizes the current predicted pose, and when the current optimized pose is obtained, the method may include: performing an optimization on the current predicted pose based on the moving distance and the multiple estimated pose information. Perform optimization to obtain the current optimized pose.
  • dead reckoning is a typical process of calculating the user's current position based on a previously determined position, estimated speed and elapsed time period, which can be determined based on sensor data generated by inertial sensors.
  • the scale information of the visual odometry is recovered by obtaining the moving distance corresponding to the moving process of the target experienced by the mobile device when it moves to the current position, thereby optimizing the pose and improving the accuracy of visual positioning.
  • determining the moving distance corresponding to the target moving process based on the pedestrian dead reckoning algorithm can simplify the process of determining the moving distance, improve the efficiency of determining the moving distance, and further improve the computing efficiency of the pose optimization method.
  • the flow chart of the method for optimizing the current predicted pose based on the moving distance includes the following S1041-1043:
  • the movement initial estimated pose and the movement end estimated position corresponding to the movement initial frame image and the movement end frame image are determined
  • the movement distance is determined as the distance between the initial estimated pose of the movement and the estimated pose of the end of the movement, so that the distance constraint is obtained between the initial estimated pose of the movement and the estimated pose of the end of the movement, and then The optimization of the current predicted pose can be achieved.
  • S1021 Acquire the number of steps taken by the user of the mobile device during the movement of the target.
  • the number of steps taken by the user of the mobile device during the movement of the target may be acquired based on the IMU on the mobile device.
  • onboard inertial sensors such as accelerometers and magnetometers enable mobile computing devices to count a user's steps and calculate the distance the user has moved in one step, ie, step length, through pedestrian dead reckoning.
  • the pedestrian dead reckoning algorithm is used to determine the moving distance of the target moving process, only an IMU with lower precision can be used to realize the realization, thereby improving the positioning accuracy and reducing the requirement for the accuracy of the IMU, so that the This method can perform indoor navigation in scenarios with limited computing power (for example, on small programs on low-end mobile phones).
  • the movement parameter further includes a movement speed.
  • the determining of the movement parameter information corresponding to the target movement process experienced by the mobile device when the mobile device moves to the current position further includes: determining that the mobile device is moving to the current location. position, the moving speed corresponding to the experienced target movement process; for the above S104, the current predicted pose is optimized based on the moving distance and the plurality of estimated pose information, and the obtained When optimizing the current pose, it may include: optimizing the current predicted pose based on the moving distance, the moving speed, and the plurality of estimated pose information to obtain the current optimized pose.
  • the moving speed is also determined, so that the corresponding estimated pose information of the multi-frame images corresponding to the distance that the user walks one step is also constrained by the distance, thereby further improving the optimization of the current predicted pose precision.
  • the historical motion speed of the user of the mobile device may be acquired based on the IMU.
  • the motion model may be a uniform motion model, a uniform acceleration motion model, or a more complex motion model, which is not limited herein.
  • the movement speed of the user during the movement of the target can be predicted, and the predicted movement speed is added to the optimization algorithm of the visual odometry to calculate the optimal pose information of the current position. That is, according to the predicted moving speed combined with the user's walking time, the scale information corresponding to any two frames of images during the movement of the target can be obtained, thereby further improving the accuracy of visual positioning.
  • the historical motion speed refers to the average motion speed of the user for a period of time before the target moving process.
  • the error will be relatively large.
  • the moving speed of the target process is predicted by using the motion model and the historical motion speed, which can improve the accuracy of parameter acquisition.
  • the pose optimization method further includes the following S105 after step S104 ⁇ S108:
  • step S105 determine whether the error of the current optimized pose is less than a preset threshold; if yes, go to step S108; if not, go to step S106.
  • step S106 it may be determined based on the reprojection error whether the error of the optimized pose exceeds a preset threshold, and if the error of the current optimized pose information exceeds the preset threshold, step S106 is performed.
  • the prediction method of the current predicted speed is the same as the prediction method of the aforementioned movement speed.
  • the historical movement speed at this time is the average speed of the user's movement for a period of time before the current moment, that is, although the prediction method The same, but because the historical movement speed of the user in different time periods is different, the obtained predicted speed is also different.
  • the current heading can be obtained through IMU measurement.
  • S107 Determine real-time pose information of the mobile device based on the current optimized pose, the current predicted speed, and the current orientation.
  • the current predicted speed and the current orientation can be directly used based on the current optimized pose to continue tracking for a period of time until The error of the current optimized pose is less than the preset threshold.
  • the location information of the destination in response to a user's destination input request, may be determined in the map; and then the current optimized pose and location information of the destination may be determined based on the current At least one connecting line between the pose and the destination is optimized; finally, the at least one connecting line is displayed in the map.
  • route planning can be performed based on the current optimized pose, the destination input by the user, and the map, so that even in an indoor environment, a more accurate navigation function can be provided for the user, and the user experience can be improved.
  • the destination information input by the user is first determined in the map, and then at least one connecting route is planned according to the current optimized pose and the location information of the destination, and all the routes are displayed on the map.
  • the at least one connection line is described, so that more choices can be provided for the user, so that the user can select one of the multiple lines for navigation according to the actual situation, thereby improving the applicability of the navigation line.
  • the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.
  • the embodiments of the present disclosure also provide a pose optimization device corresponding to the pose optimization method.
  • a pose optimization device corresponding to the pose optimization method.
  • the implementation of the apparatus reference may be made to the implementation of the method, and the repetition will not be repeated.
  • the pose optimization apparatus 500 includes:
  • the pose estimation part 501 is configured to determine the current predicted pose of the mobile device based on a first image captured by the mobile device; the first image includes the current frame image captured when the mobile device is in the current position and the The initial frame image taken when the mobile device is at the initial position before the current position;
  • the parameter determination part 502 is configured to determine the movement parameter information corresponding to the target movement process experienced by the mobile device when it moves to the current position; the target movement process is located between the starting position and the current position. between;
  • the pose obtaining part 503 is configured to obtain a plurality of estimated pose information of the mobile device, the plurality of estimated pose information includes the estimated pose information corresponding to the starting frame image, the estimated pose information corresponding to the current frame image Estimated pose information and estimated pose information respectively corresponding to multiple frames between the initial frame image and the current frame image;
  • the pose optimization part 504 is configured to optimize the current predicted pose based on the movement parameter information and the plurality of estimated pose information to obtain a current optimized pose.
  • the movement parameter information includes movement distance
  • the parameter determination part 502 is configured as:
  • the pose optimization part 504 is configured as:
  • the current predicted pose is optimized based on the moving distance and the plurality of estimated pose information to obtain the current optimized pose.
  • the pose optimization section 504 is configured to:
  • the initial moving frame image corresponding to the moving start time and the moving ending frame image corresponding to the moving ending time are determined, and the moving initial frame image corresponding to the moving initial frame image and the moving ending frame image are respectively determined.
  • the parameter determination section 502 is configured to:
  • the moving distance is determined based on the number of walking steps and the step length of the user.
  • the movement parameter information further includes movement speed
  • the parameter determination part 502 is further configured to:
  • the pose optimization part 504 is configured as:
  • the current predicted pose is optimized based on the moving distance, the moving speed, and the plurality of estimated pose information to obtain the current optimized pose.
  • the parameter determination section 502 is configured to:
  • the movement speed is predicted based on a movement model and the historical movement speed.
  • the pose optimization part 504 is further configured to:
  • real-time pose information of the mobile device is determined.
  • the pose optimization apparatus 500 further includes a route planning part 505, and the route planning part 505 is configured as:
  • a navigation route is planned, and the navigation route is displayed on the map.
  • the route planning section 505 is configured to:
  • the at least one connection line is displayed in the map.
  • a schematic structural diagram of an electronic device 700 provided by an embodiment of the present disclosure includes a processor 701 , a memory 702 , and a bus 703 .
  • the memory 702 is used to store the execution instructions, including the memory 7021 and the external memory 7022; the memory 7021 here is also called the internal memory, which is used to temporarily store the operation data in the processor 701 and the data exchanged with the external memory 7022 such as the hard disk,
  • the processor 701 exchanges data with the external memory 7022 through the memory 7021 .
  • the memory 702 is specifically used to store the application program code for executing the solution of the present application, and the execution is controlled by the processor 701 . That is, when the electronic device 700 is running, the processor 701 communicates with the memory 702 through the bus 703, so that the processor 701 executes the application code stored in the memory 702, thereby executing the method described in any of the foregoing embodiments.
  • the memory 702 may be, but not limited to, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), programmable read only memory (Programmable Read-Only Memory, PROM), or Erasable Programmable Read-Only Memory (EPROM), Electrical Erasable Programmable Read-Only Memory (EEPROM), etc.
  • RAM Random Access Memory
  • ROM read only memory
  • PROM programmable read only memory
  • EPROM Erasable Programmable Read-Only Memory
  • EEPROM Electrical Erasable Programmable Read-Only Memory
  • the processor 701 may be an integrated circuit chip with signal processing capability.
  • the above-mentioned processor can be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC) , Field Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • DSP digital signal processor
  • ASIC application-specific integrated circuit
  • FPGA Field Programmable Gate Array
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 700 .
  • the electronic device 700 may include more or less components than shown, or combine some components, or separate some components, or arrange different components.
  • the illustrated components may be implemented in hardware, software, or a combination of software and hardware.
  • Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the pose optimization method in the foregoing method embodiments are executed.
  • the storage medium may be a volatile or non-volatile computer-readable storage medium.
  • Embodiments of the present disclosure also provide a computer program, including computer-readable code, when the computer-readable code is executed in an electronic device, a processor in the electronic device executes the steps configured to implement the above-mentioned pose optimization method .
  • Embodiments of the present disclosure further provide a computer program product, where the computer program product carries program codes, and the instructions included in the program codes can be used to execute the steps of the pose optimization method in the foregoing method embodiments.
  • the computer program product carries program codes
  • the instructions included in the program codes can be used to execute the steps of the pose optimization method in the foregoing method embodiments.
  • the above-mentioned computer program product can be specifically implemented by means of hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium.
  • the computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .
  • the current predicted pose of the mobile device is determined based on a first image captured by the mobile device; the first image includes a current frame image captured when the mobile device is at the current position and a start before the mobile device is at the current position
  • the initial frame image captured when the mobile device moves to the current position determine the movement parameter information corresponding to the target movement process experienced by the mobile device when it moves to the current position; obtain multiple estimated pose information of the mobile device, and the multiple estimated pose information includes the starting frame.
  • the predicted pose is optimized to obtain the current optimized pose, which further improves the accuracy of visual positioning.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Navigation (AREA)

Abstract

La présente divulgation concerne un procédé et un appareil d'optimisation de posture, un dispositif électronique, un support de stockage lisible par ordinateur, un programme d'ordinateur et un produit-programme. Le procédé d'optimisation de posture consiste : à déterminer une posture prédite actuelle d'un dispositif mobile sur la base de premières images capturées par le dispositif mobile, les premières images comprenant une trame actuelle d'image capturée par le dispositif mobile lorsque ce dernier se trouve à la position actuelle et une trame initiale d'image capturée par le dispositif mobile lorsque ce dernier se trouve à une position initiale avant la position actuelle ; à déterminer des informations de paramètre de mouvement correspondant à un processus de mouvement cible auquel a été soumis le dispositif mobile lorsque ce dernier se déplace vers la position actuelle ; à acquérir de multiples éléments d'informations de posture estimée du dispositif mobile, les multiples éléments d'informations de posture estimée comprenant des informations de posture estimée correspondant respectivement à la trame initiale d'image et à la trame actuelle d'image et des informations de posture estimée correspondant respectivement à de multiples trames d'images entre la trame initiale d'image et la trame actuelle d'image ; et à optimiser la posture prédite actuelle sur la base des informations de paramètre de mouvement et des multiples éléments d'informations de posture estimée pour produire une posture optimisée actuelle.
PCT/CN2021/106997 2021-03-16 2021-07-19 Procédé et appareil d'optimisation de posture, dispositif électronique, support de stockage lisible par ordinateur, programme d'ordinateur et produit-programme WO2022193508A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110279737.1A CN113034594A (zh) 2021-03-16 2021-03-16 位姿优化方法、装置、电子设备及存储介质
CN202110279737.1 2021-03-16

Publications (1)

Publication Number Publication Date
WO2022193508A1 true WO2022193508A1 (fr) 2022-09-22

Family

ID=76470723

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/106997 WO2022193508A1 (fr) 2021-03-16 2021-07-19 Procédé et appareil d'optimisation de posture, dispositif électronique, support de stockage lisible par ordinateur, programme d'ordinateur et produit-programme

Country Status (2)

Country Link
CN (1) CN113034594A (fr)
WO (1) WO2022193508A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117109603A (zh) * 2023-02-22 2023-11-24 荣耀终端有限公司 一种poi更新方法和导航服务器
CN117471513A (zh) * 2023-12-26 2024-01-30 合众新能源汽车股份有限公司 一种车辆定位方法、定位装置、电子设备及存储介质

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113034594A (zh) * 2021-03-16 2021-06-25 浙江商汤科技开发有限公司 位姿优化方法、装置、电子设备及存储介质
CN113342055A (zh) * 2021-06-30 2021-09-03 广州极飞科技股份有限公司 无人机飞行控制方法、装置、电子设备及存储介质
CN113658260A (zh) * 2021-07-12 2021-11-16 南方科技大学 机器人位姿计算方法、系统、机器人及存储介质
CN114565728A (zh) * 2022-02-09 2022-05-31 浙江商汤科技开发有限公司 地图构建方法、位姿确定方法及相关装置、设备
CN115937305A (zh) * 2022-06-28 2023-04-07 北京字跳网络技术有限公司 图像处理方法、装置及电子设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109544630A (zh) * 2018-11-30 2019-03-29 南京人工智能高等研究院有限公司 位姿信息确定方法和装置、视觉点云构建方法和装置
CN110084832A (zh) * 2019-04-25 2019-08-02 亮风台（上海）信息科技有限公司 相机位姿的纠正方法、装置、系统、设备和存储介质
CN110264509A (zh) * 2018-04-27 2019-09-20 腾讯科技（深圳）有限公司 确定图像捕捉设备的位姿的方法、装置及其存储介质
CN110322500A (zh) * 2019-06-28 2019-10-11 Oppo广东移动通信有限公司 即时定位与地图构建的优化方法及装置、介质和电子设备
CN112204946A (zh) * 2019-10-28 2021-01-08 深圳市大疆创新科技有限公司 数据处理方法、装置、可移动平台及计算机可读存储介质
CN113034594A (zh) * 2021-03-16 2021-06-25 浙江商汤科技开发有限公司 位姿优化方法、装置、电子设备及存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110631554B (zh) * 2018-06-22 2021-11-30 北京京东乾石科技有限公司 机器人位姿的确定方法、装置、机器人和可读存储介质
CN111489393B (zh) * 2019-01-28 2023-06-02 速感科技（北京）有限公司 Vslam方法、控制器和可移动设备
CN110246147B (zh) * 2019-05-14 2023-04-07 中国科学院深圳先进技术研究院 视觉惯性里程计方法、视觉惯性里程计装置及移动设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110264509A (zh) * 2018-04-27 2019-09-20 腾讯科技（深圳）有限公司 确定图像捕捉设备的位姿的方法、装置及其存储介质
CN109544630A (zh) * 2018-11-30 2019-03-29 南京人工智能高等研究院有限公司 位姿信息确定方法和装置、视觉点云构建方法和装置
CN110084832A (zh) * 2019-04-25 2019-08-02 亮风台（上海）信息科技有限公司 相机位姿的纠正方法、装置、系统、设备和存储介质
CN110322500A (zh) * 2019-06-28 2019-10-11 Oppo广东移动通信有限公司 即时定位与地图构建的优化方法及装置、介质和电子设备
CN112204946A (zh) * 2019-10-28 2021-01-08 深圳市大疆创新科技有限公司 数据处理方法、装置、可移动平台及计算机可读存储介质
CN113034594A (zh) * 2021-03-16 2021-06-25 浙江商汤科技开发有限公司 位姿优化方法、装置、电子设备及存储介质

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117109603A (zh) * 2023-02-22 2023-11-24 荣耀终端有限公司 一种poi更新方法和导航服务器
CN117471513A (zh) * 2023-12-26 2024-01-30 合众新能源汽车股份有限公司 一种车辆定位方法、定位装置、电子设备及存储介质
CN117471513B (zh) * 2023-12-26 2024-03-15 合众新能源汽车股份有限公司 一种车辆定位方法、定位装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN113034594A (zh) 2021-06-25

Similar Documents

Publication Publication Date Title
WO2022193508A1 (fr) Procédé et appareil d'optimisation de posture, dispositif électronique, support de stockage lisible par ordinateur, programme d'ordinateur et produit-programme
US20210012520A1 (en) Distance measuring method and device
US11668571B2 (en) Simultaneous localization and mapping (SLAM) using dual event cameras
CN107990899B (zh) 一种基于slam的定位方法和系统
CN109887057B (zh) 生成高精度地图的方法和装置
CN110084832B (zh) 相机位姿的纠正方法、装置、系统、设备和存储介质
JP6434513B2 (ja) 視覚に基づく慣性ナビゲーション
CN112652016B (zh) 点云预测模型的生成方法、位姿估计方法及其装置
JP5987823B2 (ja) 画像センサおよび運動または位置センサから生じたデータを融合するための方法およびシステム
CN107748569B (zh) 用于无人机的运动控制方法、装置及无人机系统
KR20220028042A (ko) 포즈 결정 방법, 장치, 전자 기기, 저장 매체 및 프로그램
US20210183100A1 (en) Data processing method and apparatus
WO2015134795A2 (fr) Procédé et système de capture 3d sur la base d'une structure à partir d'un mouvement avec outil de détection de pose
CN110553648A (zh) 一种用于室内导航的方法和系统
JP6321202B2 (ja) モバイルプラットフォームの運動を決定する方法、装置、及びシステム
US20180075614A1 (en) Method of Depth Estimation Using a Camera and Inertial Sensor
WO2022062480A1 (fr) Procédé de positionnement et appareil de positionnement de dispositif mobile
CN113610702B (zh) 一种建图方法、装置、电子设备及存储介质
Irmisch et al. Simulation framework for a visual-inertial navigation system
Huai et al. Segway DRIVE benchmark: Place recognition and SLAM data collected by a fleet of delivery robots
CN116958452A (zh) 三维重建方法和系统
EP3392748B1 (fr) Systeme et procede de suivi de position dans un systeme de realite virtuelle
JP2021047024A (ja) 推定装置、推定方法及びプログラム
CN113379850B (zh) 移动机器人控制方法、装置、移动机器人及存储介质
WO2022193193A1 (fr) Procédé et dispositif de traitement de données

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21931086

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21931086

Country of ref document: EP

Kind code of ref document: A1