TW202242806A - Processing method and interface system equipment and computer readable storage medium of pose date - Google Patents

Abstract

The embodiments of the present disclosure relate to a processing method and interface, system, equipment and medium of pose data. The processing method of the pose data includes: acquiring the current pose data of the camera and the identification information of the current pose data; in the pose data set, storing the current pose data and the current pose data Identification information, wherein any item of pose data in the pose data set can be updated based on the identification information of any item of pose data; and the pose data set is sent to the preset module.

Description

Pose data processing method and interface, system, device and computer-readable storage medium

The embodiments of the present invention relate to the technical field of computer vision, in particular to a method, interface, system, device and medium for processing pose data.

With the continuous maturity of vision localization technology, more and more applications of vision-based localization methods have been applied. Visual localization is an important problem in the field of computer vision and robotics. In visual positioning, positioning can be realized only by surrounding visual information, which is a low-cost positioning method. In addition, visual positioning technology can not only locate location information, but also locate attitude information, so that the positioning results can not only serve the needs of conventional location information acquisition, but also realize more intelligent applications, such as AR (Augmented Reality, augmented reality), etc.

An embodiment of the present invention provides a technical solution for processing pose data.

According to an aspect of an embodiment of the present invention, a method for processing pose data is provided, including: Obtaining the current pose data of the camera and the identification information of the current pose data; In the pose data set, store the current pose data and the identification information of the current pose data, wherein any pose data in the pose data set can be based on the any pose The identification information of the data is updated; Send the set of pose data to a preset module.

By acquiring the current pose data of the camera and the identification information of the current pose data, storing the current pose data and the identification information of the current pose data in the pose data set, wherein the pose Any one of the pose data in the data set can be updated based on the identification information of any one of the pose data, and the pose data set is sent to the default module, thereby maintaining a current pose data and The pose data set of the pose data can be updated based on the identification information, and the pose data set is provided to the default module, so that the preset module can satisfy the accuracy requirements of the pose data. The real-time requirements of the preset module for pose data.

In some embodiments, the method further includes: acquiring the optimized pose data and the identification information of the optimized pose data; corresponding the pose data set with the identification information of the optimized pose data The pose data of is updated to the optimized pose data. In this implementation, by obtaining the optimized pose data and the identification information of the optimized pose data, and combining the pose data corresponding to the identification information of the optimized pose data in the pose data set , update to the optimized pose data, thereby improving the accuracy of the pose data in the pose data set, thereby providing more accurate pose data to the preset module.

In some embodiments, the sending the pose data set to the preset module includes: sending the pose data set to the preset module at a preset frequency, wherein the preset frequency is high Based on the frequency of obtaining optimized pose data; and/or, in response to an update of the pose data set, sending the pose data set to a default module. In this implementation, by sending the pose data set to the preset module in response to the update of the pose data set, the preset module can obtain the latest pose data set in time, thereby enabling Further meet the real-time requirements of the preset modules for pose data.

In some embodiments, the method further includes: acquiring an image corresponding to the current pose data, wherein the image corresponding to the current pose data is collected by the camera; , storing images corresponding to the current pose data. In this implementation, by acquiring the image corresponding to the current pose data and storing the image corresponding to the current pose data in the pose data set, it is possible to While the group provides pose data, it also provides the image collected by the camera to the preset module, so that it can facilitate the preset module to perform operations such as AR measurement or 3D scene reconstruction.

In some embodiments, the method further includes: when an image corresponding to any pose data in the pose data set has been sent to the preset module, from the pose data Delete images corresponding to the pose data from the collection. In this implementation, when an image corresponding to any pose data in the pose data set has been sent to the preset module, delete the image corresponding to the pose data from the pose data set The image corresponding to the above pose data can save the storage space required for the pose data collection.

In some embodiments, the acquiring the current pose data of the camera and the identification information of the current pose data includes: acquiring the current pose data of the camera and the current position data from the front end of the synchronous positioning and mapping SLAM system The identification information of the pose data; the identification information of the obtained optimized pose data and the optimized pose data, including: obtaining the optimized pose data and the optimized pose data from the back end of the SLAM system identifying information for . In this implementation, by obtaining the current pose data of the camera and the identification information of the current pose data from the front end of the SLAM system, the optimized pose data and the optimized pose data are obtained from the back end of the SLAM system. The identification information of the pose data, so that accurate current pose data and optimized pose data can be obtained.

In some embodiments, the current pose data includes current pose data processed by a sliding window. In this implementation manner, by storing the current pose data processed by the sliding window in the pose data set, the accuracy of the current pose data in the pose data set can be improved.

According to an aspect of an embodiment of the present invention, a visual positioning method is provided, including: acquiring the current pose data of the camera and the identification information of the current pose data; storing the current pose data in the pose data set and the identification information of the current pose data, wherein any item of pose data in the set of pose data can be updated based on the identification information of any item of pose data; according to the set of pose data , to get the visual positioning result. By acquiring the current pose data of the camera and the identification information of the current pose data, storing the current pose data and the identification information of the current pose data in the pose data set, wherein the pose Any item of pose data in the data set can be updated based on the identification information of any one of the pose data, and a visual positioning result can be obtained based on the pose data set, so that it can be based on the pose data set in the same time The pose data that meets the accuracy and immediacy requirements are used for visual positioning, so that accurate visual positioning results can be obtained quickly.

In some embodiments, the obtaining the visual positioning result according to the pose data set includes: obtaining the position information of at least one anchor point according to the pose data set; according to the position information of the at least one anchor point , to get the visual positioning result. In this implementation, the location information of at least one anchor point is obtained according to the pose data set, and the visual positioning result is obtained according to the location information of the at least one anchor point, so that the location information based on the anchor point can be Accurate visual positioning results are obtained.

In some embodiments, the visual positioning result includes a measurement result; the obtaining the visual positioning result according to the position information of the at least one anchor point includes: obtaining the position information of multiple anchor points on the object to be measured The measurement result of the object to be measured. In this implementation, the location information of multiple anchor points on the object to be measured is obtained according to the set of pose data, and the object to be measured is obtained according to the location information of multiple anchor points on the object to be measured The measurement results of the object to be measured can be obtained quickly and accurately.

In some embodiments, the visual positioning result includes the reconstruction result of the 3D scene; the obtaining the visual positioning result according to the position information of the at least one anchor point includes: according to the position information of multiple anchor points in the 3D scene , to obtain the reconstruction result of the 3D scene. In this implementation, the location information of multiple anchor points in the 3D scene is obtained according to the set of pose data, and the reconstruction result of the 3D scene is obtained according to the location information of the multiple anchor points in the 3D scene , so that an accurate three-dimensional scene reconstruction result can be quickly obtained.

In some embodiments, the method further includes: acquiring the optimized pose data and the identification information of the optimized pose data; corresponding the pose data set with the identification information of the optimized pose data The pose data of is updated to the optimized pose data. In this implementation, by obtaining the optimized pose data and the identification information of the optimized pose data, and combining the pose data corresponding to the identification information of the optimized pose data in the pose data set , updated to the optimized pose data, thereby improving the accuracy of the pose data in the pose data set.

In some embodiments, the method further includes: acquiring an image corresponding to the current pose data, wherein the image corresponding to the current pose data is collected by the camera; , storing images corresponding to the current pose data. In this implementation, in this implementation, by acquiring the image corresponding to the current pose data, and storing the image corresponding to the current pose data in the pose data set, thereby Operations such as AR measurement or three-dimensional scene reconstruction can be facilitated by using the image.

In some embodiments, the method further includes: when an image corresponding to any pose data in the pose data set has been sent to a preset module, from the pose data set Delete images corresponding to the pose data. In this implementation, when the image corresponding to any pose data in the pose data set has been sent to the preset module, delete the image corresponding to the pose data from the pose data set The image corresponding to the pose data can save the storage space required for the pose data collection.

For the description of the effects of the following devices, electronic equipment, etc., please refer to the above-mentioned description of the processing method of pose data.

According to an aspect of the embodiments of the present invention, an interface is provided, including: The first acquisition module is configured to acquire the current pose data of the camera and the identification information of the current pose data; The first storage module is configured to store the current pose data and the identification information of the current pose data in the pose data set, wherein any item of pose data in the pose data set can be updating based on the identification information of any one of the pose data; The sending module is configured to send the set of pose data to the preset module.

In some embodiments, the interface also includes: The third obtaining module is configured to obtain optimized pose data and identification information of the optimized pose data; The first update module is configured to update the pose data corresponding to the identification information of the optimized pose data in the pose data set to the optimized pose data.

In some embodiments, the sending module is configured to: Sending the set of pose data to the preset module at a preset frequency, wherein the preset frequency is higher than the frequency of obtaining optimized pose data; and/or, In response to an update of the pose data set, the pose data set is sent to a default module.

In some embodiments, the interface also includes: The fourth acquisition module is configured to acquire the image corresponding to the current pose data, wherein the image corresponding to the current pose data is collected by the camera; The third storage module is configured to store an image corresponding to the current pose data in the pose data set.

In some embodiments, the interface also includes: The first deletion module is configured to delete an image corresponding to any pose data in the pose data set has been sent to the preset module, from the pose data set to delete the image corresponding to the pose data set The image corresponding to the pose data.

In some embodiments, The first acquisition module is configured to: acquire the current pose data of the camera and the identification information of the current pose data from the front end of the synchronous positioning and mapping SLAM system; The third acquisition module is configured to: acquire optimized pose data and identification information of the optimized pose data from the back end of the SLAM system.

In some embodiments, the current pose data includes current pose data processed by a sliding window.

According to an aspect of the embodiments of the present invention, a visual positioning device is provided, including: The second acquisition module is configured to acquire the current pose data of the camera and the identification information of the current pose data; The second storage module is configured to store the current pose data and the identification information of the current pose data in the pose data set, wherein any item of pose data in the pose data set can be updating based on the identification information of any one of the pose data; The visual positioning module is configured to obtain a visual positioning result according to the set of pose data.

In some embodiments, the visual positioning module is configured to: Obtain position information of at least one anchor point according to the set of pose data; According to the position information of the at least one anchor point, a visual positioning result is obtained.

In some embodiments, The visual positioning results include measurement results; The visual positioning module is configured to: obtain a measurement result of the object to be measured according to position information of a plurality of anchor points on the object to be measured.

In some embodiments, The visual positioning result includes a reconstruction result of a three-dimensional scene; The visual positioning module is configured to: obtain a reconstruction result of the 3D scene according to position information of multiple anchor points in the 3D scene.

In some embodiments, the device also includes: The fifth obtaining module is configured to obtain optimized pose data and identification information of the optimized pose data; The second update module is configured to update the pose data corresponding to the identification information of the optimized pose data in the pose data set to the optimized pose data.

In some embodiments, the device also includes: The sixth acquisition module is configured to acquire the image corresponding to the current pose data, wherein the image corresponding to the current pose data is collected by the camera; The fourth storage module is configured to store an image corresponding to the current pose data in the pose data set.

In some embodiments, the device also includes: The second deletion module is configured to delete the image corresponding to any pose data in the pose data set from the pose data set when the image corresponding to the pose data has been sent to the preset module. The image corresponding to the pose data.

According to an aspect of the embodiments of the present invention, a visual positioning system is provided, including: The front end is configured to obtain the current pose data of the camera, and send the current pose data and the identification information of the current pose data to the interface; The interface is configured to store the current pose data and the identification information of the current pose data in the pose data set, and send the pose data set to a default module, wherein the pose Any item of pose data in the pose data set can be updated based on the identification information of any one of the pose data; The preset module is configured to obtain a visual positioning result according to the pose data set.

Send the current pose data and the identification information of the current pose data to the interface through the front end, and the interface stores the current pose data and the identification information of the current pose data in the pose data set , and send the pose data set to the preset module, wherein any pose data in the pose data set can be updated based on the identification information of any pose data, and the preset Assume that the module obtains a visual positioning result based on the pose data set, thereby maintaining a pose data set in the interface that contains the current pose data and can update the pose data based on the identification information, and stores the pose data The data set is provided to the preset module, so that the requirement of the preset module for the immediacy of the pose data can be met on the premise of meeting the requirement of the preset module for the accuracy of the pose data. The preset module can perform visual positioning based on pose data that meets both accuracy and immediacy requirements, so that accurate visual positioning results can be quickly obtained.

In some embodiments, the current pose data includes current pose data processed by a sliding window.

In some embodiments, the visual positioning system also includes: The backend is configured to obtain optimized pose data, and send the optimized pose data and identification information of the optimized pose data to the interface; The interface is further configured to: update the pose data corresponding to the identification information of the optimized pose data in the pose data set to the optimized pose data.

According to an aspect of the embodiments of the present invention, there is provided an electronic device, including: one or more processors; a memory configured to store executable instructions; wherein, the one or more processors are configured to call the Executable instructions stored in the memory to execute the above method.

According to an aspect of the embodiments of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored, and the above-mentioned method is implemented when the computer program instructions are executed by a processor.

An embodiment of the present invention also provides a computer program, the computer program includes computer-readable codes, and when the computer-readable codes run in an electronic device, the processor of the electronic device executes the processing of the above-mentioned pose data. Approach.

In the embodiment of the present invention, by acquiring the current pose data of the camera and the identification information of the current pose data, the current pose data and the identification information of the current pose data are stored in the pose data set , wherein, any one of the pose data in the pose data set can be updated based on the identification information of any one of the pose data, and the pose data set is sent to the default module, thereby maintaining A pose data set that contains the current pose data and can update the pose data based on the identification information, and provides the pose data set to the default module, so that the accuracy of the pose data can be satisfied by the preset module Under the premise of the requirements, the real-time requirements of the preset module for pose data are met.

It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, rather than limiting the embodiments of the present invention.

Other features and aspects of embodiments of the present invention will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

In order to make the above-mentioned objects, features and advantages of the embodiments of the present invention more comprehensible, preferred embodiments will be described below in detail together with the accompanying drawings.

Various exemplary embodiments, features, and aspects of embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures indicate functionally identical or similar elements. While various aspects of the embodiments are shown in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

The term "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist at the same time, and B exists alone. three conditions. In addition, the term "at least one" herein means any one or any combination of at least two of the plurality, for example, including at least one of A, B, and C, may mean including the composition consisting of A, B, and C Any one or more elements selected in the collection.

In addition, in order to better illustrate the embodiments of the present invention, numerous specific details are given in the following specific implementation manners. It will be understood by those skilled in the art that embodiments of the invention may be practiced without certain specific details. In some instances, methods, means, components and circuits well known to those skilled in the art are not described in detail so as to highlight the gist of the embodiments of the present invention.

With the continuous development of information technology, positioning technology is becoming more and more important to people's life. Traditional positioning technologies mainly include GPS (Global Position System, Global Positioning System)-based positioning technology, wireless area network or Bluetooth-based positioning technology, ultra-wideband-based positioning technology, and the like. These traditional positioning technologies all have certain limitations. Among them, GPS signal penetration is poor, and it is difficult to achieve effective and accurate positioning in an environment with dense buildings or in an indoor environment. In addition, even in an open scene, to achieve high-precision positioning, high-cost professional GPS equipment is required, so it is difficult to implement consumer-grade applications. The positioning technology based on wireless local area network or bluetooth needs to pre-arrange relevant equipment in the area to be located. The arrangement process is cumbersome, the reliability and accuracy are poor, and the positioning range is small. Ultra-wideband-based positioning technology can achieve relatively high accuracy, but it requires at least three receivers, and the space between the transmitter and receiver needs to be kept open, which limits the application scenarios of ultra-wideband-based positioning technology . For large scenarios, positioning technology based on ultra-wideband often needs to double the number of receivers, and the system reliability is poor. In addition, the above-mentioned positioning technologies usually only obtain location information, and it is difficult to obtain reliable attitude information.

The SLAM (Simultaneous Localization And Mapping) system plays an important role in application scenarios such as AR measurement, environment reconstruction, and AR games. SLAM systems can be divided into front-end and back-end. Among them, the front end can be equivalent to a visual odometer (Visual Odometry, VO), which can be used to study the transformation relationship between frames. For example, the front end can extract the feature points of each frame of image, use adjacent frame images for feature point matching, and can use RANSAC (RANdom Sample Consensus, random sampling consensus) algorithm to remove large noise, and then perform matching, Get pose information. At the same time, the front end can use the information provided by the IMU (Intertial Measurement Unit, inertial measurement unit) to perform filter fusion. The backend can be used to optimize the pose data output by the frontend. For example, the backend can use filtering theory or optimization theory to optimize trees or graphs to obtain optimized pose data.

Wherein, the front end can be used to provide the current pose data of the camera, for example, the current pose data provided by the front end can be used for real-time user interaction. The backend can be used to provide optimized pose data of the camera, that is, the backend can provide more accurate pose data. For example, the pose data provided by the backend can be used to obtain accurate measurement results in the application scenario of AR measurement, and obtain accurate reconstruction results of the 3D scene in the application scenario of 3D scene reconstruction.

In related technologies, there is a large delay between the front-end and the back-end. After the front-end outputs the current pose data, it usually takes a long time before the back-end can output the corresponding optimized pose data, which cannot satisfy AR measurement, Immediate requirements for application scenarios such as 3D scene reconstruction.

In order to solve the technical problems similar to those described above, an embodiment of the present invention provides a method for processing pose data. By acquiring the current pose data of the camera and the identification information of the current pose data, in the pose data set, storing the current pose data and identification information of the current pose data, wherein any item of pose data in the set of pose data can be updated based on the identification information of any item of pose data, And send the pose data set to the preset module, thereby maintaining a pose data set that contains the current pose data and can update the pose data based on the identification information, and provide the pose data set to the preset model group, so as to satisfy the immediacy requirement of the preset module for the pose data on the premise of satisfying the accuracy requirement of the preset module for the pose data.

The following describes the embodiment of the present invention through a specific application scenario. Fig. 1 shows a schematic diagram of a visual positioning system provided by an embodiment of the present invention. As shown in FIG. 1 , the visual positioning system includes: a front end 100 of the SLAM system, a back end 200 of the SLAM system, an interface 300 and a preset module 400 .

The front end 100 of the SLAM system can perform visual positioning according to the image collected by the camera and the three-axis attitude angle and acceleration provided by the IMU, and obtain the current pose data of the camera. The front end 100 of the SLAM system can perform optical flow tracking and IMU integration on the current pose data through a sliding window to obtain the current pose data processed by the sliding window. The front end 100 of the SLAM system can send the image collected by the camera, the current pose data processed by the sliding window and its identification information (such as time stamp) to The interface 300 . The interface 300 can maintain a pose data set, in which the image collected by the camera, the current pose data processed by the sliding window and its identification information are stored.

The backend 200 of the SLAM system can optimize the current pose data of the camera to obtain optimized pose data. For example, the backend 200 of the SLAM system may use filtering theory or optimization theory to perform tree or graph optimization on the current pose data of the camera to obtain optimized pose data. Among them, EKF (Extended Kalman Filter, extended Kalman filter), UKF (Unscented Kalman Filter, lossless Kalman filter) can be used for filtering, and TORO (Tree-based netwORk Optimizer, tree-based network optimizer) can be used for optimization ), G2O (General Graph Optimization, general graph optimization), etc. The backend 200 of the SLAM system can send the optimized pose data and its identification information to the interface 300 after obtaining the optimized pose data. After receiving the optimized pose data and its identification information, the interface 300 may update the pose data corresponding to the identification information of the optimized pose data in the pose data set to the Optimized pose data, thereby improving the accuracy of the pose data in the pose data set.

Table 1 shows a schematic table of pose data sets. As shown in Table 1, the pose data set may include multiple elements, for example, the pose data set may include up to 200 elements. Any element in the pose data set can include identification information and pose data. For example, in Table 1, the pose data corresponding to the logo information T1 is P1, the pose data corresponding to the logo information T2 is P2, the pose data corresponding to the logo information T3 is P3, and the pose data corresponding to the logo information T4 is P4 , the pose data corresponding to the logo information T5 is P5, and the pose data corresponding to the logo information T6 is P6. Elements in the pose data set may also include images. For example, in Table 1, the image corresponding to the identification information T5 is F5, and the image corresponding to the identification information T6 is F6. If any image in the pose data set has been sent to the preset module 400, the image can be deleted from the pose data set. For example, the images corresponding to the identification information T1, the identification information T2, the identification information T3, and the identification information T4 have been deleted from the pose data set because they have been sent to the default module 400 . After the image F5 corresponding to the identification information T5 and the image F6 corresponding to the identification information T6 are sent to the default module 400, they may also be deleted from the pose data set.

As shown in Table 1, the elements in the pose data set can also include labels. Wherein, a label of 1 indicates that the pose data in this element is an optimized pose data, for example, the pose data in this element is an optimized pose data from the back end 200 of the SLAM system; the label is 0 Indicates that the pose data in this element is unoptimized pose data, for example, the pose data in this element is unoptimized pose data from the front end 100 of the SLAM system that has been processed through a sliding window. For example, in Table 1, the pose data P1 corresponding to the logo information T1, the pose data P2 corresponding to the logo information T2, and the pose data P3 corresponding to the logo information T3 are optimized pose data, and the pose data corresponding to the logo information T4 The data P4, the pose data P5 corresponding to the logo information T5, and the pose data P6 corresponding to the logo information T6 are unoptimized pose data. Table 1 Correspondence between identification information and location data Logo information T1 T2 T3 T4 T5 T6 pose data P1 P2 P3 P4 P5 P6 image none none none none F5 F6 Label 1 1 1 0 0 0

The method for processing pose data and the method for visual positioning provided by the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

FIG. 2 shows a flowchart of a method for processing pose data provided by an embodiment of the present invention. The execution body of the pose data processing method may be an interface for processing pose data or a pose data processing device. For example, the method for processing pose data may be executed by a terminal device or a server or other processing devices. Among them, the terminal device may be user equipment (User Equipment, UE), mobile device, user terminal, terminal, cellular phone, wireless phone, personal digital assistant (Personal Digital Assistant, PDA), handheld device, computing device, vehicle-mounted device Or wearable devices, etc. In some possible implementation manners, the method for processing the pose data may be realized by calling a computer-readable instruction stored in a memory by a processor. As shown in FIG. 2 , the method for processing the pose data includes steps S21 to S23.

In step S21, the current pose data of the camera and identification information of the current pose data are acquired.

The camera in the embodiment of the present invention may be a camera of any electronic device, for example, it may be a camera of electronic devices such as a mobile phone, AR glasses, a tablet computer, a smart watch, and a robot. The current pose data of the camera may represent the current pose data of the camera, that is, the real-time pose data of the camera. Wherein, the pose data may include one or both of position data and attitude data. The location data can be represented by coordinates, for example, it can be represented by three-dimensional coordinates in the world coordinate system. Attitude data may be expressed in terms of angles.

In the embodiment of the present invention, the current pose data of the camera can be obtained through processing such as SLAM system, visual odometry, visual inertial odometry (Visual Inertial Odometry, VIO), etc., which is not limited here. For example, SLAM systems, visual odometry, etc. can perform visual positioning based on the images collected by the camera to obtain the current pose data of the camera. As another example, the SLAM system, visual odometry, visual inertial odometry, etc. can perform visual positioning based on the images collected by the camera and the three-axis attitude angle and acceleration output by the IMU to obtain the current pose data of the camera.

In the embodiment of the present invention, the identification information of the current pose data may be information capable of uniquely identifying the current pose data. For example, the identification information of the current pose data may be the time stamp, serial number, etc. of the current pose data.

In some embodiments, the current pose data includes current pose data processed by a sliding window. In this implementation manner, by storing the current pose data processed by the sliding window in the pose data set, the accuracy of the current pose data in the pose data set can be improved. As an example of this implementation, optical flow tracking and IMU integration can be performed on the current pose data through a sliding window to obtain the current pose data processed by the sliding window.

In another possible implementation manner, the current pose data includes current pose data that has not been processed by a sliding window. In this implementation manner, the current pose data that has not been processed by the sliding window may be stored in the pose data set.

In step S22, the current pose data and the identification information of the current pose data are stored in the pose data set, wherein any pose data in the pose data set can be based on the The identification information of any pose data is updated.

In the embodiment of the present invention, the pose data set may be in the form of a list, a set, etc., which is not limited here. In the embodiment of the present invention, the current pose data and the identification information of the current pose data can be associated and stored in the pose data set, that is, in the pose data set, through the associated data The structure stores the current pose data and the identification information of the current pose data to record the corresponding relationship between the current pose data and the identification information of the current pose data.

In the embodiment of the present invention, since any one of the pose data in the pose data set can be updated based on the identification information of any one of the pose data, it can meet the requirements of the preset model for pose data. accuracy requirements.

In step S23, the pose data set is sent to a preset module.

In the embodiment of the present invention, the preset module may refer to a preset module that needs to use a set of pose data. For example, the preset modules may include at least one of a detection module, a measurement module, an environment reconstruction module, and the like. For example, the detection module can be used for AR measurement and/or 3D scene reconstruction, the measurement module can be used for AR measurement, and the environment reconstruction module can be used for 3D scene reconstruction.

In the embodiment of the present invention, by acquiring the current pose data of the camera and the identification information of the current pose data, the current pose data and the identification information of the current pose data are stored in the pose data set , wherein, any one of the pose data in the pose data set can be updated based on the identification information of any one of the pose data, and the pose data set is sent to the default module, thereby maintaining A pose data set that contains the current pose data and can update the pose data based on the identification information, and provides the pose data set to the default module, so that the accuracy of the pose data can be satisfied by the preset module Under the premise of the requirements, the real-time requirements of the preset module for pose data are met.

In the embodiment of the present invention, by providing the set of pose data to the preset module, there is no need for the preset module to actively obtain the pose data from the backend, for example, there is no need for the preset module to actively obtain the pose data from the backend of the SLAM system. The terminal obtains the pose data. In some cases of related technologies, the slow initialization speed of the backend leads to the fact that the preset module cannot obtain pose data from the backend. By adopting the embodiment of the present invention, the situation that the preset module cannot obtain pose data can be greatly reduced .

In addition, in the embodiment of the present invention, by providing the pose data set to the preset module, the preset module does not need to save too much redundant data, thereby saving the storage space of the preset module.

In some embodiments, the method further includes: acquiring the optimized pose data and the identification information of the optimized pose data; corresponding the pose data set with the identification information of the optimized pose data The pose data of is updated to the optimized pose data. In this implementation, by obtaining the optimized pose data and the identification information of the optimized pose data, and combining the pose data corresponding to the identification information of the optimized pose data in the pose data set , update to the optimized pose data, thereby improving the accuracy of the pose data in the pose data set, thereby providing more accurate pose data to the preset module.

As an example of this implementation, the acquisition of the current pose data of the camera and the identification information of the current pose data includes: obtaining the current pose data of the camera and the identification information of the current pose data from the front end of the synchronous positioning and mapping SLAM system The identification information of the current pose data; the acquisition of the optimized pose data and the identification information of the optimized pose data includes: obtaining the optimized pose data and the optimized pose data from the back end of the SLAM system Identification information of posture data.

In this example, the SLAM system includes a front end and a back end, the front end is used to obtain the current pose data of the camera, and the back end is used to obtain the optimized pose data. In an example, the front end can perform visual positioning according to images collected by the camera to obtain current pose data of the camera. In another example, the front end can perform visual positioning according to the image collected by the camera and the three-axis attitude angle and acceleration output by the IMU, so as to obtain the current pose data of the camera. For example, the frequency at which the camera collects images may be 30 Hz, that is, the camera may collect 30 images per second. As another example, the frequency at which the IMU outputs the three-axis attitude angle and acceleration may be 200 Hz, that is, the IMU may output the three-axis attitude angle and acceleration 200 times per second. The frequency at which the camera collects images and the frequency at which the IMU outputs the three-axis attitude angle and acceleration can be flexibly set according to the requirements of the actual application scenario, and is not limited here.

In this example, by obtaining the current pose data of the camera and the identification information of the current pose data from the front end of the SLAM system, the optimized pose data and the optimized pose data are obtained from the back end of the SLAM system The identification information of the data, so that accurate current pose data and optimized pose data can be obtained.

In other examples, the current pose data of the camera, identification information of the current pose data, optimized pose data, and identification information of the optimized pose data may also be obtained from other modules or devices. For example, the current pose data of the camera and the identification information of the current pose data can be obtained from other modules or devices that can process the current pose data of the camera; and/or can be optimized from other processing Obtain the optimized pose data and the identification information of the optimized pose data in the module or device of the pose data. The embodiment of the present invention does not limit the source of the current pose data of the camera, the identification information of the current pose data, the optimized pose data, and the identification information of the optimized pose data.

In some embodiments, the sending the pose data set to the preset module includes: sending the pose data set to the preset module at a preset frequency, wherein the preset frequency is high Based on the frequency of obtaining optimized pose data; and/or, in response to an update of the pose data set, sending the pose data set to a default module.

As an example of this implementation manner, the pose data set may be sent to the preset module at a preset frequency. For example, the preset frequency may be 3 Hz, that is, the pose data set may be sent to the preset module 3 times per second. In this example, the preset frequency can be flexibly set according to the requirements of the actual application scenario, as long as the preset frequency is higher than the frequency for obtaining optimized pose data.

In one example, the preset frequency is less than or equal to the frequency of obtaining the current pose data. In this example, by setting the preset frequency to be less than or equal to the frequency of obtaining current pose data, the possibility of sending repeated pose data sets can be greatly reduced, and the efficiency of sending pose data sets can be improved.

As an example of this implementation, the pose data set may be sent to a preset module in response to an update of the pose data set. Wherein, the pose data set may be updated when the current pose data is acquired, and/or the pose data set may be updated when optimized pose data is obtained. By sending the pose data set to the preset module in response to the update of the pose data set, the preset module can obtain the latest pose data set in time, thereby further satisfying the preset module Immediate requirements for pose data.

In some embodiments, the method further includes: acquiring an image corresponding to the current pose data, wherein the image corresponding to the current pose data is collected by the camera; , storing images corresponding to the current pose data. In this implementation, by acquiring the image corresponding to the current pose data and storing the image corresponding to the current pose data in the pose data set, it is possible to While the group provides pose data, it also provides the image collected by the camera to the preset module, so that it can facilitate the preset module to perform operations such as AR measurement or 3D scene reconstruction.

As an example of this implementation, the method further includes: when an image corresponding to any pose data in the pose data set has been sent to the preset module, from the position Delete the image corresponding to the pose data from the pose data set. In this example, when an image corresponding to any pose data in the pose data set has been sent to the preset module, delete the image corresponding to the pose data from the pose data set The image corresponding to the pose data can save the storage space required for the pose data collection.

In some embodiments, the pose data set may set a maximum number of elements, for example, the maximum number of elements of the pose data set may be a preset value, for example, the preset value may be 200 and so on. Of course, those skilled in the art can also flexibly set the value of the maximum number of elements of the pose data set according to the requirements of actual application scenarios, or the length of the pose data set may not be limited.

FIG. 3 shows a schematic diagram of a method for processing pose data provided by an embodiment of the present invention. As shown in Fig. 3, the input of the front end of the SLAM system may include IMU input and image input. Among them, the IMU can output the three-axis attitude angle and acceleration to the front end of the SLAM system. For example, the IMU can output the three-axis attitude angle and acceleration to the front end of the SLAM system 200 times per second. The camera can output images to the front end of the SLAM system. For example, a camera can output 30 images per second to the front end of a SLAM system. The front end of the SLAM system can perform visual positioning based on the image collected by the camera and the three-axis attitude angle and acceleration output by the IMU, and obtain the current pose data of the camera. Wherein, the current pose data of the camera may be in one-to-one correspondence with the images, that is, one image collected by the camera may correspond to one piece of current pose data. The front end of the SLAM system can also perform optical flow tracking and IMU integration on the current pose data of the camera through the sliding window to obtain the current pose data processed by the sliding window. The back end of the SLAM system can optimize the current pose data of the camera or the current pose data processed by the sliding window to obtain the optimized pose data. As shown in FIG. 3 , the identification information can be used to record the corresponding relationship between the image, the unoptimized pose data, the unoptimized pose data processed by the sliding window, and the optimized pose data. Fig. 3 shows the image corresponding to the identification information T1, the unoptimized pose data, the unoptimized pose data processed by the sliding window, and the optimized pose data.

Fig. 4 shows a flowchart of a visual positioning method provided by an embodiment of the present invention. The execution subject of the visual positioning method may be a visual positioning device. For example, the visual positioning method may be executed by a terminal device or a server or other processing devices. Wherein, the terminal device may be a user device, a mobile device, a user terminal, a terminal, a cellular phone, a wireless phone, a personal digital assistant, a handheld device, a computing device, a vehicle device, or a wearable device. In some possible implementation manners, the visual positioning method may be implemented by a processor invoking computer-readable instructions stored in a memory. As shown in FIG. 4 , the visual positioning method includes steps S41 to S43.

In step S41, the current pose data of the camera and identification information of the current pose data are acquired.

The camera in the embodiment of the present invention may be a camera of any electronic device, for example, it may be a camera of electronic devices such as a mobile phone, AR glasses, a tablet computer, a smart watch, and a robot. The current pose data of the camera may represent the real-time pose data of the camera. Wherein, the pose data may include one or both of position data and attitude data. The location data can be represented by coordinates, for example, it can be represented by three-dimensional coordinates in the world coordinate system. Attitude data may be expressed in terms of angles.

In the embodiment of the present invention, the current pose data of the camera can be obtained through processing such as SLAM system, visual odometry, visual-inertial odometry, etc., which is not limited here. For example, SLAM systems, visual odometry, etc. can perform visual positioning based on the images collected by the camera to obtain the current pose data of the camera. As another example, the SLAM system, visual odometry, visual inertial odometry, etc. can perform visual positioning based on the images collected by the camera and the three-axis attitude angle and acceleration output by the IMU to obtain the current pose data of the camera.

In the embodiment of the present invention, the identification information of the current pose data may be information capable of uniquely identifying the current pose data. For example, the identification information of the current pose data may be the time stamp, serial number, etc. of the current pose data.

In some embodiments, the current pose data includes current pose data processed by a sliding window. In this implementation manner, by storing the current pose data processed by the sliding window in the pose data set, the accuracy of the current pose data in the pose data set can be improved. As an example of this implementation, optical flow tracking and IMU integration can be performed on the current pose data through a sliding window to obtain the current pose data processed by the sliding window.

In another possible implementation manner, the current pose data includes current pose data that has not been processed by a sliding window. In this implementation manner, the current pose data that has not been processed by the sliding window may be stored in the pose data set.

In step S42, the current pose data and the identification information of the current pose data are stored in the pose data set, wherein any pose data in the pose data set can be based on the The identification information of any pose data is updated.

In the embodiment of the present invention, the pose data set may be in the form of a list, a set, etc., which is not limited here. In the embodiment of the present invention, the current pose data and the identification information of the current pose data can be associated and stored in the pose data set, that is, in the pose data set, through the associated data The structure stores the current pose data and the identification information of the current pose data to record the corresponding relationship between the current pose data and the identification information of the current pose data.

In the embodiment of the present invention, since any one of the pose data in the pose data set can be updated based on the identification information of any one of the pose data, it can meet the requirements of the preset model for pose data. accuracy requirements.

In step S43, a visual positioning result is obtained according to the set of pose data.

In the embodiment of the present invention, by acquiring the current pose data of the camera and the identification information of the current pose data, the current pose data and the identification information of the current pose data are stored in the pose data set , wherein any one of the pose data in the set of pose data can be updated based on the identification information of any one of the pose data, and a visual positioning result can be obtained according to the set of pose data, thereby being able to Visual positioning is performed based on the pose data in the pose data set that meets the requirements of both accuracy and immediacy, so that accurate visual positioning results can be quickly obtained.

The visual positioning method provided by the embodiments of the present invention can be applied to application scenarios such as AR measurement and/or three-dimensional scene reconstruction.

In some embodiments, the obtaining the visual positioning result according to the pose data set includes: obtaining the position information of at least one anchor point according to the pose data set; according to the position information of the at least one anchor point , to get the visual positioning result.

In this implementation manner, the anchor point may represent a point to be located. As an example of this implementation manner, the anchor point may be a point selected by the user. For example, in an application scenario of AR measurement, a user may select one or more points in space as anchor points. For example, if the user points to a certain point on the table and clicks on the interactive interface, this point can be used as an anchor point. As another example of this implementation manner, the anchor point may be an automatically selected point (such as a corner point, a key point, etc.). For example, the 8 corner points of the box can be used as anchor points respectively. In this implementation, the location information of the anchor point may include the three-dimensional coordinates of the anchor point in the world coordinate system.

In this implementation, the location information of at least one anchor point is obtained according to the pose data set, and the visual positioning result is obtained according to the location information of the at least one anchor point, so that the location information based on the anchor point can be Accurate visual positioning results are obtained.

As an example of this implementation, the visual positioning result includes a measurement result; the obtaining the visual positioning result according to the position information of the at least one anchor point includes: according to the position information of multiple anchor points on the object to be measured , to obtain the measurement result of the object to be measured. In this example, the location information of multiple anchor points on the object to be measured is obtained according to the set of pose data, and the location information of the object to be measured is obtained according to the location information of multiple anchor points on the object to be measured. The measurement result, thus the accurate measurement result of the object to be measured can be obtained quickly.

Fig. 5 shows a schematic diagram of an application scenario of AR measurement. As shown in FIG. 5 , in the application scenario of AR measurement, the anchor frame of the object to be measured can be displayed according to the location information of the anchor point of the object to be measured (eg, a box). Wherein, the anchor point frame of the object to be measured may represent a frame obtained by connecting the anchor points of the object to be measured. For example, the anchor points of the object to be measured (box) in FIG. 5 may include 8 corner points of the object to be measured, and connecting the 8 corner points of the object to be measured can obtain the anchor point frame of the object to be measured. In the measurement result, the distance between the anchor points may be displayed. For example, in the example shown in FIG. 5 , it shows that the length, width and height of the object to be measured are 31.5 cm, 24.6 cm and 10.7 cm, respectively.

As another example of this implementation, the visual positioning result includes a reconstruction result of the 3D scene; the obtaining the visual positioning result according to the position information of the at least one anchor point includes: according to multiple anchor points in the 3D scene The location information of the 3D scene is obtained to obtain the reconstruction result of the 3D scene. In this example, the location information of multiple anchor points in the 3D scene is obtained according to the set of pose data, and the reconstruction result of the 3D scene is obtained according to the location information of the multiple anchor points in the 3D scene, In this way, an accurate three-dimensional scene reconstruction result can be quickly obtained.

In an example, in an application scenario of real-time 3D reconstruction, the reconstructed 3D point cloud may be displayed according to the current pose data. In an example, in an application scenario of offline 3D reconstruction, it is not necessary to display the reconstructed 3D point cloud according to the current pose data.

In some embodiments, the method further includes: acquiring the optimized pose data and the identification information of the optimized pose data; corresponding the pose data set with the identification information of the optimized pose data The pose data of is updated to the optimized pose data. In this implementation, by obtaining the optimized pose data and the identification information of the optimized pose data, and combining the pose data corresponding to the identification information of the optimized pose data in the pose data set , updated to the optimized pose data, thereby improving the accuracy of the pose data in the pose data set.

As an example of this implementation, the acquisition of the current pose data of the camera and the identification information of the current pose data includes: obtaining the current pose data of the camera and the identification information of the current pose data from the front end of the synchronous positioning and mapping SLAM system The identification information of the current pose data; the acquisition of the optimized pose data and the identification information of the optimized pose data includes: obtaining the optimized pose data and the optimized pose data from the back end of the SLAM system Identification information of posture data.

In this example, the SLAM system includes a front end and a back end, the front end is used to obtain the current pose data of the camera, and the back end is used to obtain the optimized pose data. In an example, the front end can perform visual positioning according to images collected by the camera to obtain current pose data of the camera. In another example, the front end can perform visual positioning according to the image collected by the camera and the three-axis attitude angle and acceleration output by the IMU, so as to obtain the current pose data of the camera. For example, the frequency at which the camera collects images may be 30 Hz, that is, the camera may collect 30 images per second. As another example, the frequency at which the IMU outputs the three-axis attitude angle and acceleration may be 200 Hz, that is, the IMU may output the three-axis attitude angle and acceleration 200 times per second. The frequency at which the camera collects images and the frequency at which the IMU outputs the three-axis attitude angle and acceleration can be flexibly set according to the requirements of the actual application scenario, and is not limited here.

In this example, by obtaining the current pose data of the camera and the identification information of the current pose data from the front end of the SLAM system, the optimized pose data and the optimized pose data are obtained from the back end of the SLAM system The identification information of the data, so that accurate current pose data and optimized pose data can be obtained.

In other examples, the current pose data of the camera, identification information of the current pose data, optimized pose data, and identification information of the optimized pose data may also be obtained from other modules or devices. For example, the current pose data of the camera and the identification information of the current pose data can be obtained from other modules or devices that can process the current pose data of the camera; and/or can be optimized from other processing Obtain the optimized pose data and the identification information of the optimized pose data in the module or device of the pose data. The embodiment of the present invention does not limit the source of the current pose data of the camera, the identification information of the current pose data, the optimized pose data, and the identification information of the optimized pose data.

In some embodiments, the method further includes: acquiring an image corresponding to the current pose data, wherein the image corresponding to the current pose data is collected by the camera; , storing images corresponding to the current pose data. In this implementation, by acquiring the image corresponding to the current pose data and storing the image corresponding to the current pose data in the pose data set, it is possible to facilitate the use of the image Operations such as AR measurement or 3D scene reconstruction.

As an example of this implementation, the method further includes: when an image corresponding to any pose data in the pose data set has been sent to a preset module, from the pose data Delete images corresponding to the pose data from the collection. In this example, when the image corresponding to any pose data in the pose data set has been sent to the preset module, delete the image corresponding to the pose data from the pose data set The image corresponding to the data can save the storage space required for the pose data collection.

In some embodiments, the pose data set may set a maximum number of elements, for example, the maximum number of elements of the pose data set may be a preset value, for example, the preset value may be 200 and so on. Of course, those skilled in the art can also flexibly set the value of the maximum number of elements of the pose data set according to the requirements of actual application scenarios, or the length of the pose data set may not be limited.

It can be understood that the above-mentioned method embodiments mentioned in the embodiments of the present invention can all be combined with each other to form a combined embodiment without violating the principle and logic. Due to space limitations, the embodiments of the present invention will not be repeated here. Those skilled in the art can understand that, in the above method in some embodiments, the execution sequence of each step should be determined by its function and possible internal logic.

In addition, the embodiment of the present invention also provides an interface, a visual positioning device, a visual positioning system, electronic equipment, a computer-readable storage medium, and a program, all of which can be used to implement any processing method or posture data provided by the embodiment of the present invention. For the visual positioning method, corresponding technical solutions and technical effects, please refer to the corresponding records in the method part, and will not repeat them here.

FIG. 6 shows a block diagram of an interface provided by an embodiment of the present invention. As shown in Figure 6, the interface includes: The first acquiring module 61 is configured to acquire the current pose data of the camera and the identification information of the current pose data; The first storage module 62 is configured to store the current pose data and the identification information of the current pose data in the pose data set, wherein any item of pose data in the pose data set It can be updated based on the identification information of any one of the pose data; The sending module 63 is configured to send the set of pose data to the preset module.

In some embodiments, the interface also includes: The third obtaining module is configured to obtain optimized pose data and identification information of the optimized pose data; The first update module is configured to update the pose data corresponding to the identification information of the optimized pose data in the pose data set to the optimized pose data.

In some embodiments, the sending module 63 is configured to: Sending the set of pose data to the preset module at a preset frequency, wherein the preset frequency is higher than the frequency of obtaining optimized pose data; and/or, In response to an update of the pose data set, the pose data set is sent to a default module.

In some embodiments, the interface also includes: The fourth acquisition module is configured to acquire the image corresponding to the current pose data, wherein the image corresponding to the current pose data is collected by the camera; The third storage module is configured to store an image corresponding to the current pose data in the pose data set.

In some embodiments, the interface also includes: The first deletion module is configured to delete an image corresponding to any pose data in the pose data set has been sent to the preset module, from the pose data set to delete the image corresponding to the pose data set The image corresponding to the pose data.

In some embodiments, The first acquisition module 61 is configured to: acquire the current pose data of the camera and the identification information of the current pose data from the front end of the synchronous positioning and mapping SLAM system; The third acquisition module is configured to: acquire optimized pose data and identification information of the optimized pose data from the back end of the SLAM system.

In some embodiments, the current pose data includes current pose data processed by a sliding window.

In the embodiment of the present invention, by acquiring the current pose data of the camera and the identification information of the current pose data, the current pose data and the identification information of the current pose data are stored in the pose data set , wherein, any one of the pose data in the pose data set can be updated based on the identification information of any one of the pose data, and the pose data set is sent to the default module, thereby maintaining A pose data set that contains the current pose data and can update the pose data based on the identification information, and provides the pose data set to the default module, so that the accuracy of the pose data can be satisfied by the preset module Under the premise of the requirements, the real-time requirements of the preset module for pose data are met.

The embodiment of the present invention also provides another interface. The interface can obtain current pose data from the front end, and send the current pose data to the display module for real-time drawing and display by the display module.

Fig. 7 shows a block diagram of a visual positioning device provided by an embodiment of the present invention. As shown in Figure 7, the visual positioning device includes: The second acquiring module 71 is configured to acquire the current pose data of the camera and the identification information of the current pose data; The second storage module 72 is configured to store the current pose data and the identification information of the current pose data in the pose data set, wherein any item of pose data in the pose data set It can be updated based on the identification information of any one of the pose data; The visual positioning module 73 is configured to obtain a visual positioning result according to the set of pose data.

In some embodiments, the visual positioning module 73 is configured to: Obtain position information of at least one anchor point according to the set of pose data; According to the position information of the at least one anchor point, a visual positioning result is obtained.

In some embodiments, The visual positioning results include measurement results; The visual positioning module 73 is configured to: obtain the measurement result of the object to be measured according to the location information of multiple anchor points on the object to be measured.

In some embodiments, The visual positioning result includes a reconstruction result of a three-dimensional scene; The visual positioning module 73 is configured to obtain a reconstruction result of the 3D scene according to the position information of multiple anchor points in the 3D scene.

In some embodiments, the device also includes: The fifth obtaining module is configured to obtain optimized pose data and identification information of the optimized pose data; The second update module is configured to update the pose data corresponding to the identification information of the optimized pose data in the pose data set to the optimized pose data.

In some embodiments, the device also includes: The sixth acquisition module is configured to acquire the image corresponding to the current pose data, wherein the image corresponding to the current pose data is collected by the camera; The fourth storage module is configured to store an image corresponding to the current pose data in the pose data set.

In some embodiments, the device also includes: The second deletion module is configured to delete the image corresponding to any pose data in the pose data set from the pose data set when the image corresponding to the pose data has been sent to the preset module. The image corresponding to the pose data.

In the embodiment of the present invention, by acquiring the current pose data of the camera and the identification information of the current pose data, the current pose data and the identification information of the current pose data are stored in the pose data set , wherein any one of the pose data in the set of pose data can be updated based on the identification information of any one of the pose data, and a visual positioning result can be obtained according to the set of pose data, thereby being able to Visual positioning is performed based on the pose data in the pose data set that meets the requirements of both accuracy and immediacy, so that accurate visual positioning results can be quickly obtained.

Fig. 8 shows a block diagram of a visual positioning system provided by an embodiment of the present invention. As shown in FIG. 8 , the visual positioning system includes: a front end 500 configured to obtain the current pose data of the camera, and send the current pose data and the identification information of the current pose data to the interface 300; The interface 300 is configured to store the current pose data and the identification information of the current pose data in the pose data set, and send the pose data set to the preset module 400, wherein the Any one of the pose data in the pose data set can be updated based on the identification information of any one of the pose data; the preset module 400 is configured to obtain a visual positioning result according to the pose data set .

In the embodiment of the present invention, the front end 500 sends the current pose data and the identification information of the current pose data to the interface 300, and the interface 300 stores the current pose data in the pose data set and the identification information of the current pose data, and send the pose data set to the preset module 400, wherein any pose data in the pose data set can be based on any one of the pose data The identification information of the pose data is updated, and the preset module 400 obtains the visual positioning result according to the set of pose data, thereby maintaining a position in the interface 300 that contains the current pose data and can update the position based on the identification information. The pose data set of the pose data, and the pose data set is provided to the preset module 400, so as to satisfy the preset module 400 under the premise of meeting the accuracy requirements of the pose data of the preset module 400. 400 Requirements for immediacy of pose data. The preset module 400 can perform visual positioning based on pose data satisfying both accuracy and immediacy requirements, so that accurate visual positioning results can be quickly obtained.

In the embodiment of the present invention, the pose data set is provided to the preset module 400 through the interface 300, so that the preset module 400 is not required to actively obtain pose data from the backend 600, for example, no preset model The group 400 actively acquires pose data from the backend 600 of the SLAM system. In some cases of related technologies, the initialization speed of the backend 600 is slow, so that the preset module 400 cannot obtain pose data from the backend 600. By adopting the embodiment of the present invention, it is possible to greatly reduce the failure of the preset module 400 to acquire posture data.

In addition, in the embodiment of the present invention, the pose data set is provided to the preset module 400 through the interface 300, so that the preset module 400 does not need to save too much redundant data, thereby saving preset modules. Storage space for groups of 400.

In some embodiments, the current pose data includes current pose data processed by a sliding window. For example, the front end 500 may perform optical flow tracking and IMU integration on the current pose data through a sliding window to obtain the current pose data processed by the sliding window.

As an example of this implementation, the front end 500 may send the new current pose data processed by the sliding window and the new current pose data processed by the sliding window to the interface 300 in response to obtaining the new current pose data processed by the sliding window and the The identification information of the current pose data processed by the new sliding window. In this example, the front end 500 can send the new current pose data to the interface 300 after each processing of the sliding window to obtain the new current pose data, so that the interface 300 can obtain the new current pose data in time material.

As an example of this implementation, the front end 500 may perform optical flow tracking and IMU integration on the real-time position data through a sliding window to obtain the current pose data processed by the sliding window.

In some embodiments, the visual positioning system further includes: a backend 600 configured to obtain optimized pose data, and send the optimized pose data and identification information of the optimized pose data to the The interface 300; the interface 300 is further configured to: update the pose data corresponding to the identification information of the optimized pose data in the pose data set to the optimized pose data.

As an example of this implementation, the backend 600 may respond to obtaining the optimized pose data, and send the optimized pose data and the identification information of the optimized pose data to the interface 300, by The rear end 600 can send the optimized pose data to the interface 300 in time, so that the interface 300 can obtain the optimized pose data in time.

In some embodiments, the functions or included modules of the device provided by the embodiments of the present invention can be configured to execute the methods described in the above method embodiments, and its implementation and technical effects can refer to the descriptions of the above method embodiments, for It is concise and will not be repeated here.

An embodiment of the present invention also provides a computer-readable storage medium, on which computer program instructions are stored, and the above-mentioned method is implemented when the computer program instructions are executed by a processor. Wherein, the computer-readable storage medium may be a non-volatile computer-readable storage medium, or may be a volatile computer-readable storage medium.

The embodiment of the present invention also proposes a computer program, including computer readable codes, and when the computer readable codes are run in the electronic device, the processor in the electronic device executes to implement the above method.

The embodiment of the present invention also provides another computer program product, which is used for storing computer-readable instructions. When the instructions are executed, the computer executes the operation of the method provided by any of the above-mentioned embodiments.

An embodiment of the present invention also provides an electronic device, including: one or more processors; a memory for storing executable instructions; wherein, the one or more processors are configured to call the executable instructions stored in the memory Execute instructions to implement the above method.

Electronic devices may be provided as terminals, servers, or other types of devices.

FIG. 9 shows a block diagram of an electronic device 800 provided by an embodiment of the present invention. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant and other terminals.

Referring to FIG. 9, the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor The implementer component 814, and the communication component 816.

The processing component 802 generally controls the overall operations of the electronic device 800, such as those associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802 .

The memory 804 is configured to store various types of data to support operations of the electronic device 800 . Examples of such data include instructions for any application or method operating on the electronic device 800, contact information, phonebook information, messages, pictures, videos, and the like. Memory 804 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic Disk or Optical Disk.

The power supply component 806 provides power to various components of the electronic device 800 . Power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for electronic device 800 .

The multimedia component 808 includes a screen providing an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or slide action, but also detect a duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive an external audio signal when the electronic device 800 is in an operation mode, such as a calling mode, a recording mode and a voice recognition mode. The received audio signal may be further stored in the memory 804 or sent via the communication component 816 . In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing unit 802 and peripheral interface modules, such as a keyboard, a click wheel, and buttons. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.

Sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for electronic device 800 . For example, the sensor assembly 814 can detect the open/closed state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the electronic device 800 or The position of a component of the electronic device 800 changes, the presence or absence of user contact with the electronic device 800 , the orientation or acceleration/deceleration of the electronic device 800 and the temperature of the electronic device 800 change. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. The sensor assembly 814 may also include a light sensor, such as a complementary metal-oxide-semiconductor or charge-coupled device image sensor, for use in imaging applications. In some embodiments, the sensor component 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access wireless networks based on communication standards, such as wireless networks, second-generation mobile communication technologies, third-generation mobile communication technologies, fourth-generation mobile communication technologies/long-term evolution of universal mobile communication technologies, and fifth-generation mobile communication technologies. Generation mobile communication technology or their combination. In an exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication module to facilitate short-range communication. For example, the near-field communication module can be implemented based on radio frequency identification technology, infrared data association technology, ultra-wideband technology, bluetooth technology and other technologies.

In an exemplary embodiment, electronic device 800 may be implemented by one or more application specific integrated circuits, digital signal processors, digital signal processing devices, programmable logic devices, field programmable gate arrays, controllers, microcontrollers Implemented by a processor, microprocessor or other electronic components for performing the above method.

In an exemplary embodiment, there is also provided a non-volatile computer-readable storage medium, such as the memory 804 including computer program instructions, which can be executed by the processor 820 of the electronic device 800 to implement the above method.

FIG. 10 shows a block diagram of an electronic device 1900 provided by an embodiment of the present invention. For example, electronic device 1900 may be provided as a server. Referring to FIG. 10 , the electronic device 1900 includes a processing component 1922 , which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922 , such as application programs. The application programs stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input/output (I/O ) interface 1958. The electronic device 1900 can operate based on the operating system stored in the memory 1932, such as Microsoft server operating system (Windows ServerTM), Apple's operating system based on a graphical user interface (Mac OS XTM), multi-user and multi-process Computer operating system (UnixTM), free and open source Unix-like operating system (LinuxTM), open source Unix-like operating system (FreeBSDTM) or similar.

In an exemplary embodiment, there is also provided a non-volatile computer-readable storage medium, such as the memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to implement the above method.

Embodiments of the invention may be systems, methods and/or computer program products. The computer program product may include a computer-readable storage medium loaded with computer-readable program instructions for causing a processor to implement various aspects of the embodiments of the present invention.

A computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer readable storage media include: portable computer disk, hard disk, random access memory, read only memory, erasable programmable read only memory ( EPROM or flash memory), static random access memory, portable compact disk read-only memory, digital versatile disks, memory sticks, floppy disks, mechanically encoded devices such as punch holes with instructions stored thereon The protruding structure in the card or groove, and any suitable combination of the above. A computer-readable storage medium as used herein is not to be construed as a transient signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other Electrical signals transmitted by wires.

The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in computer-readable storage in each computing/processing device medium.

The computer program instructions for performing the operations of the embodiments of the present invention may be assembly instructions, instruction set architecture instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or any combination of one or more programming languages Source or object code written in programming languages including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as “C” or similar programming languages. Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone package, partly on the user's computer and partly on a remote computer, or entirely on a remote computer. Execute on the terminal computer or server. In cases involving a remote computer, the remote computer can be connected to the user computer via any kind of network, including a local area network or wide area network, or it can be connected to an external computer (such as through an Internet connection). In some embodiments, electronic circuits, such as programmable logic circuits, field programmable gate arrays, or programmable logic arrays, can be customized by utilizing state information of computer-readable program instructions, which can execute computer-programmable Program instructions are read, so as to realize various aspects of the embodiments of the present invention.

Aspects of embodiments of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart and/or block diagram and combinations of blocks in the flowchart and/or block diagram can be implemented by computer readable program instructions.

These computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data-processing device, thereby producing a machine such that the instructions are executed by the processor of the computer or other programmable data-processing device When, means for realizing the function/action specified in one or more blocks in the flowchart and/or block diagram are produced. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause a computer, a programmable data processing device and/or other equipment to operate in a specific manner, so that the computer-readable medium storing the instructions is An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks in the flowchart and/or block diagrams is included.

It is also possible to load computer-readable program instructions into a computer, other programmable data processing device, or other equipment, so that a series of operation steps are executed on the computer, other programmable data processing device, or other equipment to produce a computer The process of implementation, so that instructions executed on computers, other programmable data processing devices, or other equipment implement the functions/actions specified in one or more blocks in the flowchart and/or block diagram.

The flowchart and block diagrams in the figures show the architecture, functions and operations of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction that contains one or more logic for implementing the specified Executable instructions for a function. In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two consecutive blocks may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified function or action. implemented, or may be implemented using a combination of dedicated hardware and computer instructions.

The computer program product can be realized by hardware, software or a combination thereof. In some embodiments, the computer program product is embodied as a computer storage medium. In another embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

Having described various embodiments of the embodiments of the present invention above, the foregoing description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and alterations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principle of each embodiment, practical application or improvement of technology in the market, or to enable other ordinary skilled in the art to understand each embodiment disclosed herein.

Industrial Applicability Embodiments of the present invention relate to a pose data processing method, interface, device, system, equipment and medium. The processing method of the pose data includes: acquiring the current pose data of the camera and identification information of the current pose data; storing the current pose data and the current pose data in the pose data set Identification information, wherein any item of pose data in the set of pose data can be updated based on the identification information of any item of pose data; sending the set of pose data to a preset module.

100: Front end of SLAM system 200: Backend of SLAM system 300: interface 400: Default module 500: front end 600: Backend 61: The first acquisition module 62: The first storage module 63:Send module 71: The second acquisition module 72:Second storage module 73: Visual positioning module 800: Electronic equipment 802: processing components 804: Memory 806: Power components 808:Multimedia component 810:Audio components 812: input/output interface 814: Sensor component 816:Communication component 820: Processor 900: Electronic equipment 922: Processing components 926: Power components 932:Memory 950: Network interface 958: Input/Output Interface S21~S23, S41~S43: steps

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the accompanying drawings used in the embodiments will be briefly introduced below, and the drawings here are incorporated into the specification and constitute a part of the specification. The drawings show embodiments consistent with the embodiments of the present invention, and are used together with the description to illustrate the technical solutions of the embodiments of the present invention. It should be understood that the following drawings only show some embodiments of the embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without paying creative work, Other related drawings can also be obtained from these drawings. The accompanying drawings here are incorporated into the specification and constitute a part of the specification. These drawings show embodiments consistent with the embodiments of the present invention, and are used together with the description to illustrate the technical solutions of the embodiments of the present invention. Fig. 1 shows a schematic diagram of a visual positioning system provided by an embodiment of the present invention. FIG. 2 shows a flowchart of a method for processing pose data provided by an embodiment of the present invention. FIG. 3 shows a schematic diagram of a method for processing pose data provided by an embodiment of the present invention. Fig. 4 shows a flowchart of a visual positioning method provided by an embodiment of the present invention. Fig. 5 shows a schematic diagram of an application scenario of AR measurement. FIG. 6 shows a block diagram of an interface provided by an embodiment of the present invention. Fig. 7 shows a block diagram of a visual positioning device provided by an embodiment of the present invention. Fig. 8 shows a block diagram of a visual positioning system provided by an embodiment of the present invention. FIG. 9 shows a block diagram of an electronic device 800 provided by an embodiment of the present invention. FIG. 10 shows a block diagram of an electronic device 1900 provided by an embodiment of the present invention.

S21~S23: Steps

Claims (18)

A method for processing pose data, the method being executed by an electronic device, the method comprising: Obtaining the current pose data of the camera and the identification information of the current pose data; In the pose data set, store the current pose data and the identification information of the current pose data, wherein any pose data in the pose data set can be based on the any pose The identification information of the data is updated; Send the set of pose data to a preset module. According to the method described in claim 1, the method also includes: Obtaining optimized pose data and identification information of the optimized pose data; Updating the pose data corresponding to the identification information of the optimized pose data in the pose data set to the optimized pose data. The method according to claim 1 or 2, wherein the sending the pose data set to the preset module includes at least one of the following: sending the set of pose data to the preset module at a preset frequency, wherein the preset frequency is higher than the frequency of obtaining optimized pose data; In response to an update of the pose data set, the pose data set is sent to a default module. According to the method described in claim 1 or 2, the method also includes: Acquiring an image corresponding to the current pose data, wherein the image corresponding to the current pose data is collected by the camera; In the pose data set, images corresponding to the current pose data are stored. According to the method described in claim 4, the method also includes: In the case that an image corresponding to any pose data in the pose data set has been sent to the preset module, delete the image corresponding to the pose data from the pose data set picture. According to the method described in claim 2, wherein, The acquiring the current pose data of the camera and the identification information of the current pose data includes: acquiring the current pose data of the camera and the identification information of the current pose data from the front end of the synchronous positioning and mapping SLAM system; The obtaining the optimized pose data and the identification information of the optimized pose data includes: obtaining the optimized pose data and the identification information of the optimized pose data from the back end of the SLAM system. The method according to claim 1 or 2, wherein the current pose data includes current pose data processed through a sliding window. A visual positioning method, comprising: Obtaining the current pose data of the camera and the identification information of the current pose data; In the pose data set, store the current pose data and the identification information of the current pose data, wherein any pose data in the pose data set can be based on the any pose The identification information of the data is updated; According to the set of pose data, a visual positioning result is obtained. According to the method described in claim 8, wherein said obtaining a visual positioning result according to the set of pose data includes: Obtain position information of at least one anchor point according to the set of pose data; According to the position information of the at least one anchor point, a visual positioning result is obtained. The method according to claim 9, wherein, The visual positioning results include measurement results; The obtaining the visual positioning result according to the position information of the at least one anchor point includes: obtaining the measurement result of the object to be measured according to the position information of multiple anchor points on the object to be measured. The method according to claim 9, wherein, The visual positioning result includes a reconstruction result of a three-dimensional scene; The obtaining the visual positioning result according to the position information of the at least one anchor point includes: obtaining the reconstruction result of the 3D scene according to the position information of multiple anchor points in the 3D scene. According to the method described in any one of claims 8 to 11, the method further comprises: Obtaining optimized pose data and identification information of the optimized pose data; Updating the pose data corresponding to the identification information of the optimized pose data in the pose data set to the optimized pose data. According to the method described in any one of claims 8 to 11, the method further comprises: Acquiring an image corresponding to the current pose data, wherein the image corresponding to the current pose data is collected by the camera; In the pose data set, images corresponding to the current pose data are stored. According to the method described in claim 13, the method further comprises: In the case that an image corresponding to any pose data in the pose data set has been sent to a preset module, delete the image corresponding to the pose data from the pose data set A visual positioning system comprising: The front end is configured to obtain the current pose data of the camera, and send the current pose data and the identification information of the current pose data to the interface; The interface is configured to store the current pose data and the identification information of the current pose data in the pose data set, and send the pose data set to a default module, wherein the pose Any item of pose data in the pose data set can be updated based on the identification information of any one of the pose data; The preset module is configured to obtain a visual positioning result according to the pose data set. According to the visual positioning system described in claim 15, it also includes: The backend is configured to obtain optimized pose data, and send the optimized pose data and identification information of the optimized pose data to the interface; The interface is further configured to: update the pose data corresponding to the identification information of the optimized pose data in the pose data set to the optimized pose data. An electronic device comprising: one or more processors; memory configured to store executable instructions; Wherein, the one or more processors are configured to call the executable instructions stored in the memory to perform the method described in any one of claim items 1 to 7; or, to perform any one of claim items 8 to 14 one of the methods described. A computer-readable storage medium, on which computer program instructions are stored, wherein, when the computer program instructions are executed by a processor, the method described in any one of claims 1 to 7 is implemented; or, the computer program instructions are executed by The processor realizes the method described in any one of Claims 8 to 14 when executed.

Images