WO2023160445A1

WO2023160445A1 - Simultaneous localization and mapping method and apparatus, electronic device, and readable storage medium

Info

Publication number: WO2023160445A1
Application number: PCT/CN2023/076247
Authority: WO
Inventors: 向学勤
Original assignee: 维沃移动通信有限公司
Priority date: 2022-02-22
Filing date: 2023-02-15
Publication date: 2023-08-31
Also published as: CN115205419A

Abstract

The application discloses a simultaneous localization and mapping method and apparatus, an electronic device, and a readable storage medium, belonging to the technical field of communications. The method comprises: according to final pose information of an acquired (i-1)th frame of image, determining initial pose information of an acquired i-th frame of image, wherein i is an integer greater than 1; fusing the initial pose information of the i-th frame of image with a first interpolation variable, so as to obtain final pose information of the i-th frame of image, wherein the first interpolation variable is the last interpolation variable obtained before the i-th frame of image is acquired, the first interpolation variable is an interpolation variable between the initial pose information and target pose information of a first image, the first image is an image which is the key frame in the images acquired before the i-th frame of image, and the target pose information is pose information after optimizing the initial pose information of the first image; and performing simultaneous localization and mapping on the basis of the final pose information of the i-th frame of image, and the i-th frame of image.

Description

Instant positioning and map construction method, device, electronic device and readable storage medium

Cross References to Related Applications

This application claims priority to Chinese Patent Application No. 202210163295.9 filed in China on February 22, 2022, the entire contents of which are hereby incorporated by reference.

technical field

The application belongs to the technical field of communication, and in particular relates to a real-time positioning and map construction method, device, electronic equipment and readable storage medium.

Background technique

With the continuous development of communication technology, the functions of electronic devices are becoming more and more abundant. For example, electronic devices can perform real-time positioning and map construction by tracking the pose of each image frame of the current scene.

Among them, in related technologies, the electronic device can process the pose of the image frame of the current scene acquired by the electronic device immediately based on a filtering method or an optimization method, and output the processed pose, so that each pose tracking of image frames.

However, according to the above method, on the one hand, the filter-based method cannot correct the pose of the image frame, resulting in poor accuracy of electronic equipment tracking pose; on the other hand, the optimization-based method requires a large amount of calculation , so that it takes a long time for the electronic device to process the pose of a single image frame, which in turn leads to a long delay for the electronic device to track the pose; therefore, it may cause the electronic device to perform poor real-time positioning and map construction.

Contents of the invention

The purpose of the embodiment of the present application is to provide a real-time positioning and map construction method, device, electronic device and readable storage medium, which can solve the problem that the electronic device performs poor real-time positioning and map construction.

In the first aspect, the embodiment of the present application provides a real-time positioning and map construction method, the method includes: determining the initial pose information of the i-th frame image collected according to the final pose information of the i-1th frame image collected , i is an integer greater than 1; the initial pose information of the i-th frame image is fused with the first interpolation variable to obtain the final pose information of the i-th frame image, and the first interpolation variable is obtained before collecting the i-th frame image The last interpolation variable, the first interpolation variable is the interpolation variable between the initial pose information of the first image and the target pose information, the first image is the image of the key frame in the image collected before the i-th frame image, and the target The pose information is the pose information after optimizing the initial pose information of the first image; based on the final pose information of the i-th frame image and the i-th frame image, real-time positioning and map construction are performed.

In the second aspect, the embodiment of the present application provides a real-time positioning and map construction device, the device includes a collection module, a determination module, a fusion module and a processing module; the determination module is used to collect the i-1th frame image according to the collection module The final pose information of the i-th frame image is determined to determine the initial pose information of the i-th frame image collected by the acquisition module, and i is an integer greater than 1; the fusion module is used to fuse the initial pose information of the i-th frame image with the first interpolation variable, get i The final pose information of the frame image, the first interpolation variable is the last interpolation variable obtained before the acquisition module collects the i-th frame image, the first interpolation variable is the initial pose information of the first image and the target pose information Interpolation variables, the first image is the image of the key frame in the image collected by the acquisition module before the i-th frame image, and the target pose information is the pose information after optimizing the initial pose information of the first image; the processing module, It is used for real-time positioning and map construction based on the final pose information of the i-th frame image and the i-th frame image.

In the third aspect, the embodiment of the present application provides an electronic device, the electronic device includes a processor and a memory, the memory stores programs or instructions that can run on the processor, and the programs or instructions are processed by the The steps of the method described in the first aspect are realized when the controller is executed.

In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or an instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented .

In the fifth aspect, the embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions, so as to implement the first aspect the method described.

In a sixth aspect, an embodiment of the present application provides a computer program product, the program product is stored in a storage medium, and the program product is executed by at least one processor to implement the method described in the first aspect.

In the embodiment of the present application, the initial pose information of the i-th frame image collected can be determined according to the final pose information of the i-1th frame image collected, where i is an integer greater than 1; and the i-th frame image's The initial pose information is fused with the first interpolation variable to obtain the final pose information of the i-th frame image, the first interpolation variable is the last interpolation variable obtained before the i-th frame image is collected, and the first interpolation variable is the first image's The interpolation variable between the initial pose information and the target pose information, the first image is the image of the key frame in the image collected before the i-th frame image, and the target pose information is the initial pose information of the first image. Optimized pose information; and based on the final pose information of the i-th frame image and the i-th frame image, real-time positioning and map construction are performed. Through this solution, since the electronic device can be based on the i-th frame image, and the pose information after the fusion of the first interpolation variable and the initial pose information of the i-th frame image determined according to the final pose information of the i-1th frame image , to perform real-time positioning and map construction, and the first interpolation variable is the interpolation variable between the initial pose information before and after optimization of the key frame image in the image collected by the electronic device before the i-th frame image, therefore, a On the one hand, the electronic device can correct the initial pose of the image in the i-th frame, thereby improving the accuracy of the tracking pose; delay. In this way, it is possible to ensure that the electronic device outputs high-frequency and high-precision pose information, thereby improving the effect of the electronic device's real-time positioning and map construction.

Description of drawings

Fig. 1 is a flow chart of the real-time positioning and map construction method provided by the embodiment of the present application;

Fig. 2 is a schematic diagram of the real-time positioning and map construction device provided by the embodiment of the present application;

Fig. 3 is a schematic diagram of an electronic device provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of hardware of an electronic device provided by an embodiment of the present application.

Detailed ways

The following will clearly describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in this application belong to the protection scope of this application.

The terms "first", "second" and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific sequence or sequence. It should be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application can be practiced in sequences other than those illustrated or described herein, and that references to "first," "second," etc. distinguish Objects are generally of one type, and the number of objects is not limited. For example, there may be one or more first objects. In addition, "and/or" in the specification and claims means at least one of the connected objects, and the character "/" generally means that the related objects are an "or" relationship.

The real-time positioning and map construction method, device, electronic device and readable storage medium provided by the embodiments of the present application will be described in detail below through specific embodiments and application scenarios with reference to the accompanying drawings.

Real-time positioning and map construction refers to the process in which a moving object calculates its own position and builds an environmental map based on sensor information. At present, real-time positioning and map construction have been applied in the fields of robotics, virtual reality and augmented reality, and its uses include the positioning of the sensor itself, as well as subsequent path planning and scene understanding. At present, the mainstream real-time positioning and map construction methods are generally divided into two types based on filtering and optimization. The filter-based method uses the values of various states at the previous moment to estimate the next moment; while the optimization-based method regards all states as variables, regards the equations of motion and observation equations as constraints between variables, and constructs errors function that minimizes the quadratic form of this error.

However, the filter-based method represented by the Kalman filter MSCKF (Multi-State Constraint Kalman Filter, MSCKF) under multi-state constraints has the characteristics of low algorithm power consumption, high algorithm output frequency, and good real-time positioning accuracy. , which is more suitable for application scenarios of mobile terminals. However, due to its incomplete theoretical framework, it lacks the function of map construction; moreover, the error of real-time spatial positioning will gradually increase with the increase of the running time of the method; and, in practical applications, once the positioning of electronic equipment fails In this case, the entire MSCKF system will not be able to continue to operate, which may lead to poor robustness of the existing MSCKF system. The method based on optimization has the characteristics of complete system modules (including real-time positioning module and map building module), high real-time positioning accuracy, and strong robustness of the whole system. However, since this method has a relatively large amount of calculation during operation, the corresponding algorithm consumes a lot of power, and the output frequency of the algorithm is low, so this method is not suitable for the application needs of high output rate and low calculation power consumption of mobile terminals. In this way, the effect of real-time positioning and map construction by the electronic device is poor.

In order to solve the above problems, in the embodiment of the present application, the electronic device can determine the initial pose information of the i-th frame image collected according to the final pose information of the i-1th frame image collected, where i is an integer greater than 1; and The initial pose information of the i-th frame image is fused with the first interpolation variable to obtain the final pose information of the i-th frame image, the first interpolation variable is the last interpolation variable obtained before the i-th frame image is collected, the first interpolation The variable is the interpolation variable between the initial pose information of the first image and the target pose information, and the first image is collected before the i-th frame image The image is the image of the key frame, and the target pose information is the pose information after optimizing the initial pose information of the first image; and based on the final pose information of the i-th frame image and the i-th frame image, perform real-time positioning with map build. Through this method, on the one hand, the electronic device can correct the initial pose of the i-th frame image, so the accuracy of the tracking pose can be improved; on the other hand, since the electronic device only needs to calculate the interpolation variable for the key frame image , so the delay of tracking pose can be shortened. In this way, it is possible to ensure that the electronic device outputs high-frequency and high-precision pose information, thereby improving the effect of the electronic device's real-time positioning and map construction.

An embodiment of the present application provides a method for real-time positioning and map construction, and FIG. 1 shows a flow chart of the method for real-time positioning and map construction provided by the embodiment of the present application. As shown in FIG. 1 , the real-time positioning and map construction method provided by the embodiment of the present application may include the following steps 101 to 103 .

Step 101. The electronic device determines the initial pose information of the collected i-th frame image according to the collected final pose information of the i-1th frame image.

In the embodiment of the present application, the above i is an integer greater than 1.

Optionally, in this embodiment of the present application, during the instant positioning and map building process of the electronic device, the sensor in the electronic device may collect images of the scene where the electronic device is located in real time.

In the embodiment of the present application, the pose information of the image may indicate the position of the image in the three-dimensional space.

Optionally, in this embodiment of the present application, the pose information may include rotation coordinates and displacement coordinates.

For example, pose information T={R, P}, where R is the rotation coordinate, including the rotation coordinate centered on the X axis, the rotation coordinate centered on the Y axis, and the rotation centered on the Z axis in three-dimensional space. Coordinates; P is the displacement coordinates, including the coordinates on the X axis, the Y axis and the Z axis in the three-dimensional space.

The specific method for the electronic device to determine the initial pose information of the captured i-th frame image will be described in detail below.

Optionally, in the embodiment of the present application, the above step 101 can be specifically implemented through the following step 101a, and step A or step B.

Step 101a, the electronic device uses a filtering algorithm to process the final pose information of the i-1th frame image to obtain the first pose information.

Optionally, in this embodiment of the application, the principle of the filtering algorithm may be:

x=f(x _i-1 )+n

Among them, x is the first pose information, x _i-1 is the final pose information of the i-1th frame image, f is the transfer matrix, and n is the noise term.

It can be seen that the electronic device can calculate the first pose information according to the final pose information of the i-1 frame image.

Optionally, in the embodiment of the present application, after the electronic device obtains the first pose information, it can judge the matching situation between the first pose information and the final pose information of the i-1th frame image, and determine according to the matching situation Do step A, or do step B.

Step A. The matching degree of the first pose information of the electronic device and the final pose information of the i-1th frame image is less than or If it is equal to the preset matching degree, the final pose information of the i-1th frame image is determined as the initial pose information of the i-th frame image.

Optionally, in this embodiment of the present application, the preset matching degree may be set by default by the system or set by the user according to actual usage requirements.

It can be understood that the matching degree between the first pose information and the final pose information of the i-1th frame image is less than or equal to the preset matching degree, that is, the difference between the first pose information and the final pose information of the i-1th frame image is too big.

For example, if the electronic device fails to locate during the process of capturing the image of the current scene, for example, the electronic device collects the image of a white wall for a long time, when the electronic device uses the filtering algorithm to perform calculations, due to the accumulation of errors, There will be an abnormality in the calculated pose information, that is, the matching degree between the pose information (that is, the first pose information) and the final pose information of the previous frame image (that is, the i-1th frame image) is less than or equal to the preset suitability.

Optionally, in the embodiment of the present application, the electronic device may save the historical information of the entire positioning process, when the matching degree of the first pose information and the final pose information of the i-1th frame image is less than or equal to the preset matching degree , determine the final pose information of the i-1th frame image as the initial pose information of the i-th frame image, thereby reducing errors and ensuring the accuracy of the initial pose information of the i-th frame image.

Step B. When the matching degree of the first pose information and the final pose information of the i-1th frame image is greater than the preset matching degree, the electronic device determines the first pose information as the initial position of the i-th frame image Posture information.

It can be understood that the matching degree between the first pose information and the final pose information of the i-1th frame image is greater than the preset matching degree, that is, the first pose information is not much different from the final pose information of the i-1th frame image , at this time, the electronic device can determine the first pose information as the initial pose information of the i-th frame image, that is, the initial pose information of the i-th frame image is that the electronic device adopts a filtering algorithm, and the i-1th frame image The pose information obtained by the final pose information processing.

In the embodiment of the present application, since the electronic device can judge the final pose information of the i-1th frame image, and use the filtering algorithm to process the final pose information of the i-1th frame image (that is, the first pose information), the final pose information of the i-1th frame image, or the first pose information determines the initial pose information of the i-th frame image, so the initial pose information of the i-th frame image can be ensured The accuracy of posture information.

Step 102, the electronic device fuses the initial pose information of the i-th frame image with the first interpolation variable to obtain the final pose information of the i-th frame image.

In the embodiment of the present application, the first interpolation variable is the last interpolation variable obtained before the i-th frame of image is collected.

It should be noted that the electronic device can obtain an interpolation variable by calculating the pose information of the key frame image, and the electronic device no longer calculates the pose information of the newly acquired key frame image during the calculation process. Carry out calculations until the calculation ends; if the electronic device captures an image that is a key frame after the calculation, the electronic device can obtain a new interpolation variable by calculating the pose information of the image .

In the embodiment of the present application, the first interpolation variable is an interpolation variable between initial pose information and target pose information of the first image.

In the embodiment of the present application, the target pose information may be pose information optimized from the initial pose information of the first image.

In the embodiment of the present application, the first image is an image of a key frame among the images collected before the image of the i-th frame.

Optionally, in this embodiment of the present application, the image collected by the electronic device is a key frame image, which may be a frame of image collected by the electronic device for a long time, or may include elements that have not been collected in the image collected by the electronic device. images etc.

Optionally, in this embodiment of the present application, the electronic device may fuse the initial pose information of the i-th frame image with the first interpolation variable through coordinate calculation, and obtain the final pose information of the i-th frame image.

For example, if the initial pose information of the i-th frame image is The first interpolation variable is Then the electronic equipment can be calculated by coordinates The initial pose information of the i-th frame image is fused with the first interpolation variable, and the final pose information T _out ={R _out , P _out } of the i-th frame image is obtained.

Optionally, in this embodiment of the present application, the electronic device may fuse the initial pose information of the i-th frame image with the first interpolation variable by using a Kalman filter method to obtain the final pose information of the i-th frame image.

The method for real-time positioning and map construction provided by the embodiment of the present application is exemplarily described below.

For example, assuming that the a-th frame image is an image of a key frame among the images collected before the i-th frame image, the electronic device can calculate the initial pose information of the a-th frame image and the initial pose information of the a-th frame image The interpolation variable between the optimized pose information (that is, the target pose information), if the i-th frame image is the image collected after the electronic device calculates the interpolation variable, the electronic device can use the initial position of the i-th frame image The pose information is fused with the interpolation variable (i.e. the first interpolation variable) to obtain the final pose information of the i-th frame image; if the i-th frame image is an image collected before the electronic device calculates the interpolation variable, the electronic device can use the The initial pose information of the i-frame image is fused with the interpolation variable (ie, the first interpolation variable) calculated last time to obtain the final pose information of the i-th frame image. In this way, the accuracy of the final pose information of the obtained i-th frame image can be ensured.

The specific method for determining the first interpolation variable by the electronic device and optimizing the initial pose information of the first image will be described in detail in the following embodiments, and will not be repeated here to avoid repetition.

Step 103, the electronic device performs real-time positioning and map construction based on the final pose information of the i-th frame of image and the i-th frame of image.

In the embodiment of the present application, the electronic device may perform real-time positioning and map construction based on the final pose information of the i-th frame image and the i-th frame image, so as to construct a three-dimensional map corresponding to the i-th frame image.

Optionally, in this embodiment of the present application, the electronic device may superimpose the constructed three-dimensional map corresponding to the i-th frame image with the three-dimensional map corresponding to each frame image constructed before the i-th frame image, so that the current Real-time positioning and map construction of the scene.

It should be noted that if the electronic device does not obtain any interpolation variable before collecting the i-th frame image, the electronic device can directly perform real-time positioning and map construction based on the initial pose information of the i-th frame image and the i-th frame image ; Or, the electronic device can optimize the initial pose information of the i-th frame image, and determine a difference variable, and fuse the initial pose information of the i-th frame image with the interpolation variable to obtain the i-th frame image. The final pose information, so that the electronic device can perform real-time Positioning and Mapping.

In the real-time positioning and map construction method provided by the embodiment of the present application, since the electronic device can be based on the i-th frame image, and the first interpolation variable and the i-th frame image determined according to the final pose information of the i-1th frame image The pose information after the fusion of the initial pose information is used for real-time positioning and map construction, and the first interpolation variable is the initial position before and after optimization of the image that is the key frame in the image collected by the electronic device before the i-th frame image Therefore, on the one hand, the electronic device can correct the initial pose of the i-th frame image, thereby improving the accuracy of tracking pose; on the other hand, the electronic device only needs to correct the initial pose of the key frame The image computes interpolation variables, which can reduce the latency of tracking poses. In this way, it is possible to ensure that the electronic device outputs high-frequency and high-precision pose information, thereby improving the effect of the electronic device's real-time positioning and map construction.

Optionally, in the embodiment of the present application, before the above step 101, the instant positioning and map construction method provided in the embodiment of the present application may further include the following steps 104 and 105.

Step 104: The electronic device optimizes the initial pose information of the first image based on the M pieces of pose information, the M sets of offset information, and the initial pose information of the first image to obtain target pose information.

In the embodiment of the present application, the above-mentioned M pose information is the pose information after the latest optimization of M frames of images, and the M frames of images are images that are key frames in the images collected before the first image,

In the embodiment of the present application, each set of offset information in the above M sets of offset information is an offset amount of the feature points of the first image relative to the feature points of one frame of the above M frames of images.

Optionally, in this embodiment of the present application, the feature point of the image may be any possible point such as a vertex, a corner point, or a center point in the image.

Optionally, in this embodiment of the present application, the number of feature points in the image may be one or multiple, and specifically may be determined by the electronic device according to the collected image.

Optionally, in this embodiment of the present application, the number of feature points of different images may be the same or different.

Optionally, in this embodiment of the present application, the number of feature points corresponding between the first image and one frame of the above-mentioned M frames of images may be N, and N is an integer greater than or equal to 0; it can be understood that this At this time, the set of offset information includes N offsets.

The specific method for the electronic device to determine the above M sets of offset information will be described in detail in the following embodiments, and will not be repeated here to avoid repetition.

The specific method for the electronic device to optimize the initial pose information of the first image will be described in detail below.

Optionally, in this embodiment of the present application, the foregoing step 104 may be specifically implemented through the following steps 104a and 104b.

Step 104a, the electronic device determines M sets of three-dimensional position information according to the above M sets of offset information.

In the embodiment of the present application, the above M sets of offset information are in one-to-one correspondence with the above M sets of three-dimensional position information, and each set of three-dimensional position information can be used to indicate the feature points in the three-dimensional map constructed based on one frame of the above M frames of images .

Optionally, in this embodiment of the present application, when a set of offset information in the above M sets of offset information includes N offset When shifting, the set of three-dimensional position information determined by the electronic device according to the set of offset information may indicate the N feature points in the constructed three-dimensional map.

Optionally, in this embodiment of the present application, for each set of offset information in the above M sets of offset information, the electronic device may, according to the The offset is used to determine a set of three-dimensional position information used to indicate the feature points in the three-dimensional map constructed based on the frame image.

Step 104b: The electronic device uses a preset beam adjustment algorithm to process the above M pieces of pose information, the initial pose information of the first image, and the above M sets of three-dimensional position information to obtain target pose information.

Optionally, in this embodiment of the application, the principle of the preset beam adjustment algorithm can be:

Among them, T is the initial pose information of the image, P is the three-dimensional position information, Z is the two-dimensional observation, π is the projection equation, M is the number of images that are key frames before the first image, and N is the M group of three-dimensional position information instructions The number of feature points in the 3D map.

It can be seen that the electronic device can process the above M pieces of pose information, the initial pose information of the first image, and the above M sets of three-dimensional position information, so as to obtain target pose information.

It should be noted that, in the embodiment of the present application, the electronic device adopts the preset beam adjustment algorithm, and at the same time of obtaining the target pose information, it can optimize the above M pose information, so that the electronic device will use the preset beam adjustment algorithm next time. When the beam adjustment algorithm obtains the target pose information of the new first image, it can be based on the optimized M pose information, thereby improving the accuracy of the instant positioning of the electronic device.

In the embodiment of the present application, since the electronic device can determine the M sets of three-dimensional position information corresponding to the M sets of offset information according to the M sets of offset information, and can indicate the feature points in the constructed three-dimensional map, and use The preset beam adjustment algorithm calculates the pose information optimized for the initial pose information of the first image, so it can improve the accuracy of the electronic device in optimizing the initial pose information of the first image, so that the electronic device can instantly When positioning and map construction, the accuracy of map construction can be improved.

Step 105, the electronic device determines a first interpolation variable according to the first rotation coordinate and the first displacement coordinate in the initial pose information of the first image, and the second rotation coordinate and second displacement coordinate in the target pose information.

For detailed descriptions of the rotation coordinates and the displacement coordinates, specific reference may be made to relevant descriptions in the foregoing embodiments, and details are not repeated here to avoid repetition.

The specific method for the electronic device to determine the first interpolation variable will be described in detail below.

Optionally, in the embodiment of the present application, the foregoing step 105 may be specifically implemented through the following steps 105a to 105c.

Step 105a, the electronic device determines the target rotation coordinates according to the first rotation coordinates and the second rotation coordinates.

Optionally, in this embodiment of the present application, the electronic device may perform a multiplication operation on the transposition of the first rotation coordinate and the second rotation coordinate to obtain the target rotation coordinate.

Step 105b, the electronic device determines the target position according to the target rotation coordinate, the first displacement coordinate, and the second displacement coordinate. Mark displacement coordinates.

Optionally, in the embodiment of the present application, the electronic device may multiply the target rotation coordinates and the first displacement coordinates to obtain the intermediate displacement coordinates; and perform subtraction operation on the second displacement coordinates and the intermediate displacement coordinates to obtain the target displacement coordinates.

In step 105c, the electronic device determines the target rotation coordinates and target displacement coordinates as first interpolation variables.

Exemplarily, it is assumed that the first rotation coordinate in the initial pose information of the first image is The first displacement coordinate is The second rotation coordinate in the target pose information of the first image is The second displacement coordinate is Then, the electronic device can determine the target rotation coordinates according to the first rotation coordinates and the second rotation coordinates The electronic device can determine the target displacement coordinates according to the target rotation coordinates, the first displacement coordinates, and the second displacement coordinates In this way, the electronic device can rotate the coordinates of the target and target displacement coordinates Determined as the first interpolation variable.

In the embodiment of the present application, the electronic device can determine the target rotation coordinates according to the first rotation coordinates in the initial pose information of the first image and the second rotation coordinates in the target pose information of the first image, and according to the target rotation coordinates, the first displacement coordinates in the initial pose information of the first image, and the second displacement coordinates in the target pose information of the first image to determine the target displacement coordinates. Therefore, the electronic device can determine the target rotation coordinates and the target displacement coordinates as the first interpolation variables, so as to facilitate further fusion processing.

In the embodiment of the present application, since the electronic device can be based on the latest optimized pose information, M sets of offset information, and the initial pose information of the first image based on M frames of the image captured before the first image as a key frame , optimize the initial pose information of the first image, and determine the first The interpolation variable, that is, the electronic device can determine the first interpolation variable based on the coordinates in the pose information of the first image before and after optimization, so the accuracy of determining the first interpolation variable by the electronic device can be improved.

The specific method for the electronic device to determine the above M sets of offset information will be described in detail below.

Optionally, in the embodiment of the present application, before the above step 104, the instant positioning and map construction method provided in the embodiment of the present application may further include the following step 106.

Step 106: The electronic device determines the M sets of offset information according to the two-dimensional position information of the feature points of the first image and the two-dimensional position information of the feature points of the M frames of images.

Optionally, in this embodiment of the present application, the two-dimensional position information of the feature points of the first image may be determined by an electronic device through a filtering method.

Optionally, in this embodiment of the present application, the two-dimensional position information may be used to indicate the position of the feature point of the first image in the first image.

Optionally, in the embodiment of the present application, the principle of determining the two-dimensional position information of the feature points of the first image by the electronic device is as follows:

z _k ＝h(x _k )+r _k

Among them, z _k is the two-dimensional position information of the feature points of the first image (ie two-dimensional observation), x _k is the initial pose information of the first image, r _k is the noise item, and h is the observation matrix.

It can be seen that the electronic device may determine the two-dimensional position information of the feature points of the first image through a filtering method according to the initial pose information of the first image.

Optionally, in this embodiment of the present application, each set of offset information in the M sets of offset information may be an image coordinate difference between a feature point of the first image and a feature point of one frame of the above-mentioned M frames of images.

For example, the offset of the feature point A (x1, y1) of the first image relative to the feature point A' (x2, y2) of one frame of the above-mentioned M frames of images is x=|x1-x2|, y =|y1-y2|.

It should be noted that, in the embodiment of the present application, each set of offset information includes offsets of all the feature points corresponding to the feature points of the first image and one frame of the above-mentioned M frames of images.

For example, assuming that the first image includes feature point 1, feature point 2, and feature point 3, one frame of image a in the above M frames of images includes feature point 1' corresponding to feature point 1, and feature point 3 corresponding to feature point 3 ' and feature point 4, then the set of offset information may include the image coordinate difference between feature point 1 and feature point 1', and the image coordinate difference between feature point 3 and feature point 3'.

In the embodiment of the present application, since the electronic device can determine M sets of offset information based on the two-dimensional position information of the feature points of the first image and the two-dimensional position information of the feature points of M frames of images, the electronic device can acquire the first image The offset information between each image that is a key frame collected before the first image is collected, so that the electronic device can obtain accurate target pose information based on the offset information, and then can improve the real-time positioning of the electronic device and map construction. precision.

The real-time positioning and map building method provided in the embodiment of the present application may be executed by a real-time positioning and map building device. In the embodiment of the present application, the real-time positioning and map construction device performed by the real-time positioning and map construction device is taken as an example to illustrate the real-time positioning and map construction device provided in the embodiment of the present application.

Referring to FIG. 2 , the embodiment of the present application provides an instant positioning and map construction device 20 , which may include: a collection module 21 , a determination module 22 , a fusion module 23 and a processing module 24 . The determination module 22 can be used to determine the initial pose information of the i-th frame image collected by the acquisition module 21 according to the final pose information of the i-1th frame image collected by the acquisition module 21, where i is an integer greater than 1. The fusion module 23 can be used to fuse the initial pose information of the i-th frame image with the first interpolation variable to obtain the final pose information of the i-th frame image. The first interpolation variable is obtained before the acquisition module collects the i-th frame image The last interpolation variable of , the first interpolation variable is the interpolation variable between the initial pose information of the first image and the target pose information, and the first image is a key frame in the image collected by the acquisition module before the ith frame image image, the target pose information is the pose information after optimizing the initial pose information of the first image. The processing module 24 can be used for real-time positioning and map construction based on the final pose information of the i-th frame image and the i-th frame image.

In a possible implementation manner, the device 20 for real-time positioning and map construction may further include an optimization module. The optimization module can be used to determine the initial pose information of the i-th frame image collected by the acquisition module 21 according to the final pose information of the i-1th frame image collected by the acquisition module 21, based on M poses information, M sets of offset information and the initial pose information of the first image, optimize the initial pose information of the first image, and obtain the target pose information, and the M pose information is the latest optimized pose information of M frames of images , M frames of images are images of key frames in the images collected by the acquisition module 21 before the first image, and each set of offset information is the offset of the feature points of the first image relative to the feature points of one frame of images in the M frames of images displacement. The determining module 22 can also be used to determine the first interpolation value according to the first rotation coordinate and the first displacement coordinate in the initial pose information of the first image, and the second rotation coordinate and the second displacement coordinate in the target pose information variable.

In a possible implementation manner, the determination module 22 can specifically be used to determine M sets of three-dimensional position information according to M sets of offset information, where M sets of offset information correspond to M sets of three-dimensional position information, and each set of three-dimensional position information It is used to indicate the feature points in the 3D map constructed based on one frame of M images. The optimization module can specifically be used to process the M pieces of pose information, the initial pose information of the first image, and the M groups of three-dimensional position information by using the preset beam adjustment algorithm to obtain the target pose information.

In a possible implementation, the determination module 22 can also be used to determine the initial pose information of the first image in the optimization module based on M pose information, M sets of offset information, and the initial pose information of the first image. After optimization, before obtaining the target pose information, M sets of offset information are determined according to the two-dimensional position information of the feature points of the first image and the two-dimensional position information of the feature points of the M frames of images.

In a possible implementation manner, the determining module 22 may specifically be configured to determine the target rotation coordinates according to the first rotation coordinates and the second rotation coordinates. The determination module 22 may be specifically configured to determine the target displacement coordinates according to the target rotation coordinates, the first displacement coordinates, and the second displacement coordinates. The determining module 22 may be specifically configured to determine the target rotation coordinates and the target displacement coordinates as the first interpolation variables.

In a possible implementation manner, the processing module 24 may be specifically configured to use a filtering algorithm to process the final pose information of the i-1th frame image to obtain the first pose information. The determination module 22 can specifically be used to set the final position of the i-1th frame image to The pose information is determined as the initial pose information of the i-th frame image. The determining module 22 can specifically be used to determine the first pose information as the i-th frame image when the matching degree between the first pose information and the final pose information of the i-1th frame image is greater than the preset matching degree initial pose information.

In the real-time positioning and map construction device provided in the embodiment of the present application, since the real-time positioning and map construction device can be based on the i-th frame image, and the first interpolation variable and the final pose information determined according to the i-1th frame image The pose information after the fusion of the initial pose information of the i-th frame image is used for real-time positioning and map construction, and the first interpolation variable is in the image collected by the real-time positioning and map construction device before the i-th frame image is The interpolation variable between the initial pose information before and after the optimization of the image of the key frame, therefore, on the one hand, the instant positioning and map construction device can correct the initial pose of the i-th frame image, thereby improving the tracking pose Accuracy, on the other hand, the real-time positioning and map construction device only needs to calculate the interpolation variable for the key frame image, so that the delay of tracking pose can be shortened. In this way, it can ensure that the real-time positioning and mapping device outputs high-frequency and high-precision pose information, thereby improving the effect of real-time positioning and map building by the real-time positioning and mapping device.

For the beneficial effects of the various implementations in this embodiment, refer to the beneficial effects of the corresponding implementations in the foregoing method embodiments. To avoid repetition, details are not repeated here.

The device for real-time positioning and map construction in the embodiment of the present application may be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal, or other devices other than the terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a notebook computer, a handheld computer, a vehicle electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR)/virtual reality (virtual reality, VR) ) equipment, robots, wearable devices, ultra-mobile personal computer (ultra-mobile personal computer, UMPC), netbook or personal digital assistant (personal digital assistant, PDA), etc., can also serve as server, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (television, TV), teller machine, or self-service machine, etc., which are not specifically limited in this embodiment of the present application.

The device for real-time positioning and map construction in the embodiment of the present application may be a device with an operating system. The operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in this embodiment of the present application.

The real-time positioning and map construction device provided by the embodiment of the present application can realize various processes realized by the method embodiment in FIG. 1 , and details are not repeated here to avoid repetition.

Optionally, as shown in FIG. 3 , the embodiment of the present application also provides an electronic device 300, including a processor 301 and a memory 302. The memory 302 stores programs or instructions that can run on the processor 301. The When the programs or instructions are executed by the processor 301, the various steps in the embodiment of the real-time positioning and map construction method described above can be achieved, and the same technical effect can be achieved. To avoid repetition, details are not repeated here.

It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

FIG. 4 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.

The electronic device 1000 includes, but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010, etc. part.

Those skilled in the art can understand that the electronic device 1000 can also include a power supply (such as a battery) for supplying power to various components, and the power supply can be logically connected to the processor 1010 through the power management system, so that the management of charging, discharging, and function can be realized through the power management system. Consumption management and other functions. The structure of the electronic device shown in FIG. 4 does not constitute a limitation to the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine some components, or arrange different components, which will not be repeated here. .

Wherein, the processor 1010 can be configured to determine the initial pose information of the i-th frame image collected by the sensor 1005 according to the final pose information of the i-1th frame image collected by the sensor 1005, where i is an integer greater than 1. The processor 1010 can also be used to fuse the initial pose information of the i-th frame image with the first interpolation variable to obtain the final pose information of the i-th frame image, the first interpolation variable is before the acquisition module collects the i-th frame image get the last interpolation Variable, the first interpolation variable is the interpolation variable between the initial pose information of the first image and the target pose information, the first image is the image of the key frame in the image collected by the acquisition module before the ith frame image, and the target position The pose information is pose information after optimizing the initial pose information of the first image. The processor 1010 can also be used for real-time positioning and map construction based on the final pose information of the i-th frame image and the i-th frame image.

In a possible implementation manner, the processor 1010 may also be configured to determine the initial pose information of the i-th frame image collected by the sensor 1005 according to the final pose information of the i-1th frame image collected by the sensor 1005, Based on the M pose information, M sets of offset information and the initial pose information of the first image, optimize the initial pose information of the first image to obtain the target pose information, and the M pose information is the closest M frame image Posture information after one optimization, M frames of images are images of key frames in the images collected by the first image front sensor 1005, and each set of offset information is the feature point of the first image relative to one frame in the M frames of images The offset of the feature points of the image. The processor 1010 may also be configured to determine the first interpolation value according to the first rotation coordinate and the first displacement coordinate in the initial pose information of the first image, and the second rotation coordinate and the second displacement coordinate in the target pose information variable.

In a possible implementation manner, the processor 1010 may be specifically configured to determine M sets of three-dimensional position information according to M sets of offset information, where M sets of offset information correspond to M sets of three-dimensional position information, and each set of three-dimensional position information It is used to indicate the feature points in the 3D map constructed based on one frame of M images. The processor 1010 may be specifically configured to use a preset beam adjustment algorithm to process M pieces of pose information, initial pose information of the first image, and M sets of three-dimensional position information to obtain target pose information.

In a possible implementation, the processor 1010 may also be configured to optimize the initial pose information of the first image based on the M pieces of pose information, M sets of offset information, and the initial pose information of the first image , before obtaining the target pose information, M sets of offset information are determined according to the two-dimensional position information of the feature points of the first image and the two-dimensional position information of the feature points of the M frames of images.

In a possible implementation manner, the processor 1010 may specifically be configured to determine the target rotation coordinates according to the first rotation coordinates and the second rotation coordinates. The processor 1010 may specifically be configured to determine the target displacement coordinates according to the target rotation coordinates, the first displacement coordinates, and the second displacement coordinates. The processor 1010 may be specifically configured to determine the target rotation coordinates and the target displacement coordinates as the first interpolation variables.

In a possible implementation manner, the processor 1010 may be specifically configured to use a filtering algorithm to process the final pose information of the i-1th frame image to obtain the first pose information. The processor 1010 can be specifically configured to set the final position of the i-1th frame image to The pose information is determined as the initial pose information of the i-th frame image. The processor 1010 may specifically be configured to determine the first pose information as the i-th frame image when the matching degree between the first pose information and the final pose information of the i-1th frame image is greater than a preset matching degree initial pose information.

In the electronic device provided in the embodiment of the present application, since the electronic device can be based on the i-th frame image, and the first interpolation variable and the initial pose of the i-th frame image determined according to the final pose information of the i-1th frame image The pose information after information fusion is used for real-time positioning and map construction, and the first interpolation variable is between the initial pose information before and after optimization of the image that is a key frame in the image collected by the electronic device before the i-th frame image the interpolation variable, Therefore, on the one hand, the electronic device can correct the initial pose of the i-th frame image, thereby improving the accuracy of tracking pose; on the other hand, the electronic device only needs to calculate the interpolation variable for the key frame image, which can shorten Delay in tracking pose. In this way, it is possible to ensure that the electronic device outputs high-frequency and high-precision pose information, thereby improving the effect of the electronic device's real-time positioning and map construction.

It should be understood that, in the embodiment of the present application, the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 10041 and a microphone 10042, and the graphics processor 10041 is used for the image capture device ( Such as the image data of the still picture or video obtained by the camera) for processing. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072 . The touch panel 10071 is also called a touch screen. The touch panel 10071 may include two parts, a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, and joysticks, which will not be repeated here.

The memory 1009 can be used to store software programs as well as various data. The memory 1009 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required by at least one function (such as a sound playing function, image playback function, etc.), etc. Furthermore, memory 1009 may include volatile memory or nonvolatile memory, or, memory 1009 may include both volatile and nonvolatile memory. Among them, the non-volatile memory can be read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable ROM, PROM), erasable programmable read-only memory (Erasable PROM, EPROM), electronically programmable Erase Programmable Read-Only Memory (Electrically EPROM, EEPROM) or Flash. Volatile memory can be random access memory (Random Access Memory, RAM), static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory (Synch link DRAM , SLDRAM) and Direct Memory Bus Random Access Memory (Direct Rambus RAM, DRRAM). The memory 1009 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., Modem processors mainly process wireless communication signals, such as baseband processors. It can be understood that the foregoing modem processor may not be integrated into the processor 1010 .

The embodiment of the present application also provides a readable storage medium, where programs or instructions are stored on the readable storage medium, When the program or instruction is executed by the processor, it realizes the various processes in the embodiment of the real-time positioning and map construction method described above, and can achieve the same technical effect. In order to avoid repetition, details are not repeated here.

Wherein, the processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk, and the like.

The embodiment of the present application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to realize real-time positioning and map construction as described above Each process of the method embodiment can achieve the same technical effect, and will not be repeated here to avoid repetition.

It should be understood that the chips mentioned in the embodiments of the present application may also be called system-on-chip, system-on-chip, system-on-a-chip, or system-on-a-chip.

The embodiment of the present application provides a computer program product, the program product is stored in a storage medium, and the program product is executed by at least one processor to realize the various processes in the above embodiments of the real-time positioning and map construction method, and can achieve the same To avoid repetition, the technical effects will not be repeated here.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions are performed, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of computer software products, which are stored in a storage medium (such as ROM/RAM, magnetic disk, etc.) , optical disc), including several instructions to enable a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Under the inspiration of this application, without departing from the purpose of this application and the scope of protection of the claims, many forms can also be made, all of which belong to the protection of this application.

Claims

A method for instant positioning and map construction, the method comprising:

According to the final pose information of the collected i-1th frame image, determine the initial pose information of the i-th frame image collected, where i is an integer greater than 1;

Fusing the initial pose information of the i-th frame image with a first interpolation variable to obtain the final pose information of the i-th frame image, the first interpolation variable is obtained before collecting the i-th frame image The last interpolation variable, the first interpolation variable is the interpolation variable between the initial pose information of the first image and the target pose information, and the first image is in the image collected before the i-th frame image is The image of the key frame, the target pose information is the pose information after optimizing the initial pose information of the first image;

Real-time positioning and map construction are performed based on the final pose information of the i-th frame image and the i-th frame image.
The method according to claim 1, wherein said method further comprises:

Based on the M pieces of pose information, M sets of offset information, and the initial pose information of the first image, optimize the initial pose information of the first image to obtain the target pose information, and the M The pose information is the last optimized pose information of M frames of images, the M frames of images are images of key frames in the images collected before the first image, and each group of offset information is the first image The offset of the feature points relative to the feature points of a frame of images in the M frames of images;

The first interpolation variable is determined according to the first rotation coordinate and the first displacement coordinate in the initial pose information of the first image, and the second rotation coordinate and second displacement coordinate in the target pose information.
The method according to claim 2, wherein the initial pose information of the first image is optimized based on the M pieces of pose information, M sets of offset information, and the initial pose information of the first image , to obtain the target pose information, including:

According to the M sets of offset information, determine M sets of three-dimensional position information, the M sets of offset information correspond to the M sets of three-dimensional position information one by one, and each set of three-dimensional position information is used to indicate the The feature points in the three-dimensional map constructed from a frame of image;

The M pieces of pose information, the initial pose information of the first image, and the M sets of three-dimensional position information are processed by using a preset beam adjustment algorithm to obtain the target pose information.
The method according to claim 3, wherein the initial pose information of the first image is optimized based on the M pose information, M sets of offset information, and the initial pose information of the first image , before obtaining the target pose information, the method also includes:

The M sets of offset information are determined according to the two-dimensional position information of the feature points of the first image and the two-dimensional position information of the feature points of the M frames of images.
The method according to any one of claims 2 to 4, wherein, the first rotation coordinate and the first displacement coordinate in the initial pose information according to the first image, and the target pose information in the The second rotation coordinate and the second displacement coordinate are used to determine the first interpolation variable, including:

determining a target rotation coordinate according to the first rotation coordinate and the second rotation coordinate;

determining target displacement coordinates according to the target rotation coordinates, the first displacement coordinates, and the second displacement coordinates;

The target rotation coordinates and the target displacement coordinates are determined as the first interpolation variables.
The method according to claim 1, wherein said determining the initial pose information of the i-th frame image collected according to the final pose information of the i-1 frame image collected comprises:

Using a filtering algorithm to process the final pose information of the i-1th frame image to obtain the first pose information;

When the matching degree between the first pose information and the final pose information of the i-1th frame image is less than or equal to the preset matching degree, the final pose information of the i-1th frame image Determined as the initial pose information of the i-th frame image;

When the matching degree between the first pose information and the final pose information of the i-1th frame image is greater than a preset matching degree, determining the first pose information as the i-th frame image initial pose information.
A real-time positioning and map construction device, the device includes a collection module, a determination module, a fusion module and a processing module;

The determination module is configured to determine the initial pose information of the i-th frame image collected by the acquisition module according to the final pose information of the i-1th frame image collected by the acquisition module, where i is an integer greater than 1;

The fusion module is configured to fuse the initial pose information of the i-th frame image with a first interpolation variable to obtain the final pose information of the i-th frame image, and the first interpolation variable is The last interpolation variable obtained before the module collects the i-th frame image, the first interpolation variable is the interpolation variable between the initial pose information of the first image and the target pose information, and the first image is the Among the images collected by the acquisition module before the i-th frame image, it is an image of a key frame, and the target pose information is the pose information after optimizing the initial pose information of the first image;

The processing module is configured to perform real-time positioning and map construction based on the final pose information of the i-th frame image and the i-th frame image.
The device according to claim 7, wherein the device further comprises an optimization module;

The optimization module is configured to, in the determination module, determine the initial pose of the i-th frame image collected by the collection module according to the final pose information of the i-1th frame image collected by the collection module Before information, based on the M pose information, M groups of offset information, and the initial pose information of the first image, the initial pose information of the first image is optimized to obtain the target pose information, so The M pose information is the last optimized pose information of the M frames of images, and the M frames of images are images that are key frames in the images collected by the acquisition module before the first image, and each group of offsets The information is the offset of the feature points of the first image relative to the feature points of one frame of images in the M frames of images;

The determination module is further configured to, according to the first rotation coordinate and the first displacement coordinate in the initial pose information of the first image, and the second rotation coordinate and second displacement coordinate in the target pose information, The first interpolation variable is determined.
The device according to claim 8, wherein the optimization module is specifically configured to determine M sets of three-dimensional position information according to the M sets of offset information, the M sets of offset information and the M sets of three-dimensional position information One-to-one correspondence, each set of three-dimensional position information is used to indicate the feature points in the three-dimensional map constructed based on one frame of the M frame images; and the preset beam adjustment algorithm is used to calculate the M pose information , the initial pose information of the first image and the M sets of three-dimensional position information are processed to obtain the target pose information.
The device according to claim 9, wherein the determining module is further configured to, in the optimization module, based on the M pieces of pose information, the M sets of offset information, and the initial pose of the first image Information, optimize the initial pose information of the first image, before obtaining the target pose information, according to the two-dimensional position information of the feature points of the first image and the two-dimensional position information of the feature points of the M frame images dimensional position information, and determine the M sets of offset information.
The device according to any one of claims 8 to 10, wherein the determination module is specifically configured to determine a target rotation coordinate according to the first rotation coordinate and the second rotation coordinate; and according to the target The rotation coordinate, the first displacement coordinate, and the second displacement coordinate determine a target displacement coordinate; and determine the target rotation coordinate and the target displacement coordinate as the first interpolation variable.
The device according to claim 7, wherein the determining module is specifically configured to use a filtering algorithm to process the final pose information of the i-1th frame image to obtain first pose information; and in the When the matching degree between the first pose information and the final pose information of the i-1th frame image is less than or equal to the preset matching degree, determine the final pose information of the i-1th frame image as the The initial pose information of the i-th frame image; and when the matching degree of the first pose information and the final pose information of the i-1th frame image is greater than the preset matching degree, the first A piece of pose information is determined as the initial pose information of the i-th frame image.
An electronic device, comprising a processor and a memory, the memory stores programs or instructions that can run on the processor, and when the programs or instructions are executed by the processor, any one of claims 1-6 is implemented Steps of the instant positioning and map construction method described in the item.
A readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the real-time positioning and map construction method according to any one of claims 1-6 is realized step.
A computer software product, the computer software product is executed by at least one processor to realize the real-time positioning and map construction method according to any one of claims 1-6.
An electronic device, comprising the electronic device configured to execute the instant positioning and map construction method according to any one of claims 1-6.
A chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, the processor is used to run programs or instructions, and realize the real-time Localization and Mapping Methods.