WO2022016909A1 - 获取Wi-Fi指纹空间分布的方法、装置和电子设备 - Google Patents

获取Wi-Fi指纹空间分布的方法、装置和电子设备 Download PDF

Info

Publication number
WO2022016909A1
WO2022016909A1 PCT/CN2021/085972 CN2021085972W WO2022016909A1 WO 2022016909 A1 WO2022016909 A1 WO 2022016909A1 CN 2021085972 W CN2021085972 W CN 2021085972W WO 2022016909 A1 WO2022016909 A1 WO 2022016909A1
Authority
WO
WIPO (PCT)
Prior art keywords
terminal device
video frame
pose
frame
target area
Prior art date
Application number
PCT/CN2021/085972
Other languages
English (en)
French (fr)
Inventor
王民航
薛景涛
邓宇
贺亚农
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2022016909A1 publication Critical patent/WO2022016909A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/30—Services specially adapted for particular environments, situations or purposes
    • H04W4/33—Services specially adapted for particular environments, situations or purposes for indoor environments, e.g. buildings
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/02—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves
    • G01S5/0252—Radio frequency fingerprinting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/02—Services making use of location information
    • H04W4/021—Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W84/00—Network topologies
    • H04W84/02—Hierarchically pre-organised networks, e.g. paging networks, cellular networks, WLAN [Wireless Local Area Network] or WLL [Wireless Local Loop]
    • H04W84/10—Small scale networks; Flat hierarchical networks
    • H04W84/12—WLAN [Wireless Local Area Networks]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance

Definitions

  • the embodiments of the present application relate to computer vision technologies, and in particular, to a method, apparatus, and electronic device for obtaining the spatial distribution of Wi-Fi fingerprints.
  • the Wi-Fi fingerprint is the attribute information of the Wi-Fi signal, such as the strength and frequency band of the Wi-Fi signal.
  • the router As a device that connects electronic devices to the Wi-Fi network, the router is located in the indoor space. If the layout of the router is reasonable, the Wi-Fi signal can stably cover every location in the room. Therefore, it is very important to obtain the spatial distribution of indoor Wi-Fi fingerprints, that is, the Wi-Fi fingerprints at various indoor locations.
  • the room in order to obtain the spatial distribution of indoor Wi-Fi fingerprints, the room can be pre-divided into grids with multiple known locations. The location and the Wi-Fi fingerprint at the known location are stored correspondingly, and the spatial distribution of the Wi-Fi fingerprint is obtained.
  • a known location needs to be pre-determined, and then the Wi-Fi fingerprint at the known location is collected and stored, which has low acquisition efficiency, and currently requires special equipment to collect Wi-Fi fingerprints, which has low applicability.
  • the embodiments of the present application provide a method, apparatus, and electronic device for obtaining the spatial distribution of Wi-Fi fingerprints, which can not only obtain the spatial distribution of Wi-Fi fingerprints by using terminal equipment, and have high applicability, but also can improve the efficiency of the spatial distribution of Wi-Fi fingerprints. Gain efficiency and accuracy.
  • an embodiment of the present application provides a method for obtaining the spatial distribution of Wi-Fi fingerprints, and the method can be applied to a terminal device or a chip in the terminal device.
  • the method is described below by taking the application to a terminal device as an example.
  • the terminal device collects video frames and Wi-Fi fingerprints of the target area within the current time window. and the inertial measurement unit imu data of the terminal device.
  • the terminal device may acquire video frames, Wi-Fi fingerprints, and imu data of the target area every preset time window, and may obtain video frames, Wi-Fi fingerprints, and imu data of the target area according to the current time window.
  • the terminal device may be positioned based on the video frame and the imu data to obtain the pose of the terminal device during the movement process. That is to say, in this embodiment of the present application, the pose of the terminal device acquired in the historical time window may be corrected or processed according to the pose of the terminal device in the current time window, so as to obtain the movement of the terminal device in the target area. pose during the process.
  • the pose of the terminal device during the movement of the target area and the Wi-Fi fingerprint collected by the terminal device during the movement of the target area can be aligned according to time , to obtain the spatial distribution of Wi-Fi fingerprints in the target area.
  • the spatial distribution of Wi-Fi fingerprints can be understood as the correspondence between the pose of the terminal device and the Wi-Fi fingerprint when it moves in the target area.
  • users can obtain the spatial distribution of Wi-Fi fingerprints through their own terminal equipment, which has high applicability and avoids the problem of using special equipment.
  • the terminal device itself is used to obtain the pose of the terminal device.
  • the terminal device can not only accurately locate the position, but also can Obtaining accurate device pose improves the accuracy of Wi-Fi fingerprint spatial distribution.
  • the relative movement information of the terminal device when collecting the video frame may be acquired according to the imu data. It should be understood that the terminal device can collect multiple video frames in the process of moving the target area, and the terminal device has a corresponding pose when collecting each video frame, because the imu data can represent the movement posture of the terminal device during the movement process, Then, the terminal device can obtain the relative movement information of the terminal device when collecting the video frame through the imu data.
  • the relative movement information may be obtained according to the imu data collected by the terminal device between the video frame and the previous video frame, and the relative movement information represents the comparison between the terminal device and the video frame when the video frame is collected.
  • the movement situation when the previous video frame is collected the movement situation can represent the movement of the position of the terminal device. That is to say, when the terminal device collects two adjacent video frames, it can collect imu data, and then according to the imu data collected by the terminal device between the two adjacent video frames, obtain a video frame after the terminal device collects , relative movement information relative to the previous video frame.
  • the terminal device may acquire the feature of the video frame, where the feature of the video frame includes point features and/or line features of the video frame.
  • the terminal device may acquire point features and/or line features of the video frame, and then acquire the data collected by the terminal device according to the relative movement information, the point features and/or the line features (features of the video frame).
  • the pose of the video frame is obtained to obtain the pose of the terminal device during the movement of the target area. It should be understood that, in the embodiment of the present application, the point feature and line feature of the video frame are used, compared with the method of using the point feature or the line feature, because the features in the video frame are increased, the accuracy of the positioning of the terminal device (that is, the acquisition of The accuracy of the pose when the terminal device collects the video frame).
  • the terminal device may acquire point features in a video frame, wherein, if there are too few point features collected in a video frame, the accuracy of the point features in describing the features of the video frame may be low, but If there are too many point features collected in a video frame, the calculation amount of the terminal device will be increased. Therefore, in this embodiment of the present application, the first number threshold may be preset, which can ensure the accuracy of the description of the features of the video frame. On the basis of performance, the calculation amount of terminal equipment is reduced. Wherein, the first video frame satisfies the above-mentioned preset point feature detection parameters.
  • the first video frame satisfies the following conditions: the number of corner points in the first video frame is less than or equal to the second number threshold, and the number of point features in the first video frame is greater than or equal to the first number threshold , and the distance between the corner points in the first video frame is greater than or equal to the first distance threshold, and the quality level of the corner points in the first video frame is greater than a preset level.
  • the point feature tracking method may be used instead of the point feature matching method to obtain the point feature in each video frame, which can reduce the amount of calculation and improve the speed of obtaining the point feature.
  • the point feature of the previous video frame may be tracked in the video frame, wherein, if the number of point features tracked in the video frame is less than the first number threshold, the A new point feature is extracted from the video frame to obtain the point feature of the first threshold value, and then the point feature obtained by tracking in the video frame and the new point feature are used as the point feature of the video frame. It should be noted that, if the number of point features tracked in the video frame is equal to the first number threshold, the point features in the video frame are tracked in the next video frame. Wherein, in order to improve the accuracy of the point feature tracked in the video frame, the wrongly tracked point feature may also be acquired, and the wrongly tracked point feature may be deleted in the video frame.
  • the line feature matching method is adopted to obtain and match the line feature of the video frame with the line feature of the previous video frame, and then establish an initial line pair, wherein the initial line pair indicates that in the Two line features that are successfully matched in the video frame and the previous video frame.
  • the wrong line pair may be deleted in the initial line pair to obtain the video Line features in the frame.
  • the distance between the two line features, the difference in the length of the line segment, and the included angle between the line segments can be obtained, and then the wrong line pair can be determined according to the distance between the two line features, the difference in the length of the line segment, and the included angle between the line segments.
  • the wrong line pair has: a line pair whose distance between the two line features is greater than the second distance threshold, or a line pair whose line segment length difference between the two line features is greater than the length difference threshold, or the line segment angle between the two line features is greater than The line pair for the second angle threshold.
  • the manner of acquiring the pose of the terminal device during the movement process may include the following two methods Way.
  • the relative movement information includes: relative displacement and relative angle, and the key frames in the embodiments of the present application are introduced here:
  • the key frame may be obtained according to the relative displacement, the relative angle, or the time difference between the video frame and the previous key frame.
  • the key frame satisfies at least one of the following conditions: the relative displacement is greater than a first displacement threshold, or the relative angle is greater than a first angle threshold, or the time difference is greater than a time difference threshold.
  • the terminal device can determine whether the video frame is a key frame according to the above conditions, and then select the following corresponding method to obtain the pose of the terminal device in the moving process. It should be understood that, in the embodiment of the present application, which of the following two methods is adopted for the pose of the terminal device during the movement process, there may be user-defined settings.
  • the first way obtain the pose of the terminal device when collecting each video frame during the moving process.
  • the pose of the terminal device during the movement process includes: the pose of the terminal device when the video frame is collected within the current time window.
  • the pose of the terminal device when collecting the video frame may be acquired according to the relative displacement information between the video frame and the adjacent key frames before the video frame. It should be understood that, in this manner, the pose when the terminal device captures the previous key frame of the video frame may be acquired in the following second manner.
  • the relative displacement and the relative angle may be added on the basis of the pose when the terminal device collects the adjacent key frames before the video frame, so as to obtain the pose of the terminal device when the video frame is collected
  • the second method obtain the pose of the terminal device when the key frame is collected during the movement.
  • the key frame is the current key frame in the current time window as an example for description.
  • the terminal device can use the relative movement information of the key frame in the current time window, the point feature of the key frame, and the line feature of the key frame as parameters to construct a state vector to be estimated, and establish a maximum a posteriori probability objective function, and Substitute the state vector to be estimated into the maximum a posteriori probability objective function, solve the maximum a posteriori probability objective function, and obtain the time when the terminal device collects the key frame in the current time window.
  • the initial pose and then, according to the initial pose of the terminal device, obtain the pose when the terminal device collects the key frame, so as to obtain the pose during the movement of the terminal device in the target area.
  • the terminal device can perform closed-loop detection according to the point features and line features of the current key frame in the current time window, and the point features and line features of historical key frames, and perform spatiotemporal consistency correction on the current key frame. test.
  • the historical key frame is: a key frame in a time window before the current time window when the terminal device moves in the target area. Further, according to the initial pose of the terminal device, the closed-loop detection result, and the verification result of the current key frame, the pose of the terminal device during the movement process is acquired.
  • the closed-loop detection in the embodiment of the present application can significantly reduce the accumulated drift error caused by continuous integration, thereby improving the accuracy of the pose.
  • the terminal device moves rapidly, there will be a problem that insufficient feature points cannot be extracted due to the motion blur of the video frame. The motion of moving objects leads to inaccurate pose estimation or even failure. Therefore, if the space-time consistency check is not performed after the closed-loop detection is successful, the above method can only be applied to static scenes with slow and steady motion, not fast Scenes with rotating and dynamic objects have poor robustness.
  • the key frame can be checked for space-time consistency, so that the method in the embodiment of the present application rotates rapidly in the terminal device, the target area contains dynamic objects, and the target area has weak textures , illumination changes and other complex environments, it has good robustness.
  • the initial pose of the terminal device and the pose of the terminal device when collecting the historical key frame are performed. Correction is performed to obtain the pose of the terminal device during the movement of the target area.
  • the first current vector and the second current vector corresponding to the current key frame are stored, which can then be used as the historical key frame of the next time window.
  • the initial pose of the terminal device and the position of the terminal device when the historical key frame is collected are compared.
  • the pose is taken as the pose of the terminal device during the movement of the target area.
  • a first current vector may be constructed according to the descriptor of the point feature of the current key frame, where the current key frame is a key frame in the current time window, and the first current vector and the first current vector are obtained.
  • the similarity of the history vector, and the key frame corresponding to the first history vector whose similarity is greater than the similarity threshold is used as the first candidate frame, and the first history vector is constructed according to the descriptor of the point feature of the historical key frame.
  • a second current vector is constructed; the similarity between the second current vector and the second history vector is obtained, and the similarity is greater than the similarity threshold corresponding to the second history vector
  • the key frame is used as the second candidate frame, and the second historical vector is constructed according to the descriptor of the line feature of the historical key frame.
  • a closed loop detection is performed, and a spatiotemporal consistency check is performed on the current key frame.
  • the terminal device may perform weighting processing on the similarity of the first candidate frame and the similarity of the second candidate frame to obtain a weighted score; if the weighted score is greater than or equal to the score threshold, closed-loop detection Pass; if the weighted score is less than the score threshold, the closed-loop detection fails.
  • the space-time consistency check also includes a space consistency check and a time consistency check.
  • the terminal device can acquire the relative displacement and relative angle between the first candidate frame and the current key frame; if the relative displacement is less than the second displacement threshold and the relative angle is less than the third angle threshold, then determine the relative displacement and relative angle between the first candidate frame and the current key frame.
  • the current key frame has passed the spatial consistency check. Wherein, if the weighted scores of the first N key frames of the current key frame are all greater than the score threshold, it is determined that the current key frame has passed the time consistency check, and the N is an integer greater than or equal to 1 .
  • the terminal device can also output the pose of the terminal device during the movement of the target area, the time corresponding to the pose, and the location of the terminal device. video frames captured at the time described.
  • the terminal device can output the pose of the terminal device during the movement of the target area, the time corresponding to the pose, and the schematic diagram corresponding to the video frame collected by the terminal device at the time, so that the user can The pose of the terminal device can be visually observed in real time.
  • the embodiment of the present application obtains the above-mentioned Wi-Fi fingerprint spatial distribution for application in a specific scenario.
  • the application scenario can be adjustment of router position, optimization of router antenna pointing, improvement of terminal device positioning accuracy, or router layout position recommendation.
  • the Wi-Fi fingerprints may be filtered according to the application scenario of the spatial distribution of the Wi-Fi fingerprints in the target area, thereby reducing the storage space of the terminal device.
  • the Wi-Fi fingerprint spatial distribution map of the target area may also be obtained according to the Wi-Fi fingerprint spatial distribution of the target area, and then the Wi-Fi fingerprint spatial distribution map of the target area is displayed. , so that users can more intuitively obtain the spatial distribution of Wi-Fi fingerprints in the target area. It should be noted that, if the application scenario of the spatial distribution of Wi-Fi fingerprints in the target area is to obtain the installation position of the router, then in this embodiment of the present application, all the information can be displayed in the spatial distribution map of Wi-Fi fingerprints in the target area. the location where the router is to be installed.
  • the terminal device may also display the adjusted position of the router, or the recommended router antenna pointing, etc.
  • the application scenario of the terminal device and the display and processing process of the terminal device after application are not limited.
  • an embodiment of the present application provides a device for acquiring the spatial distribution of Wi-Fi fingerprints, the device comprising:
  • the acquisition module is used to collect the video frame, Wi-Fi fingerprint of the target area within the current time window, and the inertial measurement unit imu data of the terminal device.
  • the processing module is used to locate the terminal device based on the video frame and the imu data, obtain the pose of the terminal device in the moving process of the target area, and move the terminal device in the target area.
  • the pose in the target area and the Wi-Fi fingerprint collected by the terminal device during the movement of the target area are aligned according to time, and the spatial distribution of the Wi-Fi fingerprint of the target area is obtained.
  • the processing module is specifically configured to acquire, according to the imu data, the relative movement information of the terminal device when acquiring the video frame, and acquire the characteristics of the video frame, and According to the relative movement information and the characteristic features of the video frame, the pose when the terminal device collects the video frame is acquired, so as to obtain the pose of the terminal device during the movement of the target area.
  • the features of the video frame may include point features and/or line features.
  • the processing module is specifically configured to obtain the relative movement information according to the imu data collected by the terminal device between the video frame and the previous video frame, where the relative movement information represents the terminal The movement of the device when the video frame is captured compared to when the previous video frame was captured.
  • the feature of the video frame includes the point feature of the video frame
  • the processing module is specifically configured to track the point feature of the previous video frame in the video frame, if the The number of point features tracked in the video frame is less than the first number threshold, then extract new point features in the video frame to obtain the point features of the first number threshold, and track the video frame.
  • the obtained point feature and the new point feature are used as the point feature of the video frame.
  • the first video frame satisfies the following conditions:
  • the number of corner points in the first video frame is less than or equal to a second number threshold
  • the number of point features of the first video frame is greater than or equal to the first number threshold
  • the distance between the corner points in the first video frame is greater than or equal to a first distance threshold
  • the quality level of the corners in the first video frame is greater than a preset level.
  • the processing module is further configured to acquire the point feature of the tracking error, and delete the point feature of the tracking error in the video frame.
  • the feature of the video frame includes the line feature of the video frame
  • the processing module is specifically configured to match the line feature of the video frame with the line feature of the previous video frame , establish an initial line pair indicating two line features that are successfully matched in the video frame and the previous video frame; in the initial line pair, delete the wrong line pair to obtain the describe the line features in the video frame.
  • the processing module is specifically configured to obtain the distance between the two line features in the line pair, the difference in the lengths of the line segments, and the included angle between the line segments; the distance for deleting the two line features is greater than the second and, deleting the line pair whose line segment length difference of the two line features is greater than the length difference threshold; and, deleting the line pair whose line segment angle of the two line features is greater than the second angle threshold.
  • the relative movement information includes: relative displacement and relative angle.
  • the processing module is further configured to obtain a key frame according to the relative displacement, the relative angle or the time difference between the video frame and a previous key frame.
  • the key frame satisfies at least one of the following conditions: the relative displacement is greater than a first displacement threshold; the relative angle is greater than a first angle threshold; and the time difference is greater than a time difference threshold.
  • the pose of the terminal device during the movement of the target area includes: the pose of the terminal device when the video frame is collected within the current time window.
  • the processing module is specifically configured to obtain the pose of the adjacent key frame before the video frame; according to the pose of the adjacent key frame before the video frame, and the video frame and the video frame The relative displacement information between adjacent key frames is used to obtain the pose of the terminal device when the video frame is collected.
  • the processing module is specifically configured to use the relative movement information of the key frame, the point feature of the key frame and the line feature of the key frame as parameters to construct the state vector to be estimated; establishing a maximum a posteriori probability objective function, and substituting the state vector to be estimated into the maximum a posteriori probability objective function, solving the maximum a posteriori probability objective function, and obtaining the terminal device at the current time
  • the initial pose when the key frame is collected in the window according to the initial pose of the terminal device, obtain the pose when the terminal device collects the key frame, so as to obtain the pose of the terminal device during the movement of the target area. pose.
  • the key frame is a current key frame in the current time window, and the features of the video frame include point features and line features of the video frame;
  • the processing module specifically, uses In order to perform closed-loop detection according to the point feature and line feature of the current key frame in the current time window and the point feature and line feature of the historical key frame, and perform a spatiotemporal consistency check on the current key frame, the The historical key frame is: the key frame in the time window before the current time window when the terminal device moves in the target area; according to the initial pose of the terminal device, the closed-loop detection result, and the current key frame The verification result is obtained, and the pose when the terminal device collects the current key frame is obtained, so as to obtain the pose of the terminal device during the movement of the target area.
  • the processing module is specifically configured to construct a first current vector according to the descriptor of the point feature of the current key frame, where the current key frame is a key in the current time window frame; obtain the similarity between the first current vector and the first history vector, and use the key frame corresponding to the first history vector whose similarity is greater than the similarity threshold as the first candidate frame, and the first history vector is based on the history It is constructed by the descriptor of the point feature of the key frame; the second current vector is constructed according to the descriptor of the line feature of the current key frame; the similarity between the second current vector and the second history vector is obtained, and the similarity is greater than the The key frame corresponding to the second history vector of the similarity threshold is used as the second candidate frame, and the second history vector is constructed according to the descriptor of the line feature of the historical key frame; according to the similarity of the first candidate frame and the similarity between the second candidate frame and the second candidate frame, perform closed-loop detection, and perform a spatiotempo
  • the processing module is specifically configured to perform weighting processing on the similarity of the first candidate frame and the similarity of the second candidate frame to obtain a weighted score; if the weighted If the score is greater than or equal to the score threshold, the closed-loop detection passes; if the weighted score is less than the score threshold, the closed-loop detection fails.
  • the space-time consistency check includes spatial consistency check.
  • the processing module is specifically configured to obtain the relative displacement and relative angle between the first candidate frame and the current key frame; if the relative displacement is less than the second displacement threshold, and the relative angle is less than the third angle threshold, it is determined that the current key frame has passed the spatial consistency check.
  • the spatiotemporal consistency check further includes a time consistency check.
  • the processing module is specifically configured to determine that the current key frame has passed the time consistency check if the weighted scores of the first N key frames of the current key frame are all greater than the score threshold, and the N is Integer greater than or equal to 1.
  • the processing module is specifically configured to fuse the point feature of the current key frame and the first key frame if the closed-loop detection passes and the current key frame passes the spatiotemporal consistency check.
  • the point feature of the candidate frame; the line feature of the current key frame and the line feature of the second candidate frame are fused; through the fused point feature and the fused line feature, the initial pose and all Correct the pose of the terminal device when collecting the historical key frame, and obtain the pose of the terminal device during the movement of the target area.
  • the processing module is specifically configured to, if the closed-loop detection fails, or if the closed-loop detection passes and the current key frame fails the spatiotemporal consistency check, the initial bit of the terminal device The pose and the pose when the terminal device collects the historical key frame is taken as the pose of the terminal device during the movement of the target area.
  • the processing module is further configured to store the first current vector and the second current vector corresponding to the current key frame if the current key frame passes the spatiotemporal consistency check.
  • the processing module is further configured to output the pose of the terminal device during the movement process, the time corresponding to the pose, and the video frame collected by the terminal device at the time.
  • the processing module is further configured to filter the Wi-Fi fingerprints according to the application scenario of the spatial distribution of the Wi-Fi fingerprints in the target area.
  • the processing module is further configured to acquire a Wi-Fi fingerprint spatial distribution map of the target area according to the Wi-Fi fingerprint spatial distribution of the target area.
  • the display module is used for displaying the Wi-Fi fingerprint spatial distribution map of the target area.
  • the display module is further configured to display the Wi-Fi fingerprints in the target area if the application scenario of the spatial distribution of the Wi-Fi fingerprints in the target area is to obtain the installation location of the router. The location where the router is to be installed is displayed in the spatial distribution diagram.
  • the apparatus for obtaining the spatial distribution of Wi-Fi fingerprints provided in the embodiments of the present application can perform the actions of the terminal device in the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and are not repeated here.
  • an embodiment of the present application provides an electronic device, where the electronic device may be the terminal device in the above-mentioned first aspect.
  • the electronic device includes: a processor, a memory, and a transceiver; the transceiver is coupled to the processor, and the processor controls the transceiver's transceiving action; wherein the memory is used for storing computer-executable program codes,
  • the program code includes instructions; when executed by the processor, the instructions cause the electronic device to perform the method provided by the first aspect.
  • inventions of the present application provide an electronic device, including a unit, a module, or a circuit for executing the method provided by the possible designs of the first aspect above.
  • the electronic device may be a terminal device, or may be a module applied to the terminal device, for example, may be a chip applied to the terminal device.
  • an embodiment of the present application provides a chip, where a computer program is stored on the chip, and when the computer program is executed by the chip, the method provided in the first aspect is implemented.
  • embodiments of the present application provide a computer program product containing instructions, which, when executed on a computer, cause the computer to execute the method in the first aspect.
  • an embodiment of the present application provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, when the computer-readable storage medium runs on a computer, the computer executes the method in the first aspect.
  • Embodiments of the present application provide a method, apparatus, and electronic device for obtaining the spatial distribution of Wi-Fi fingerprints.
  • the method includes: collecting video frames and Wi-Fi fingerprints of a target area within a current time window, and an inertial measurement unit of a terminal device. imu data; based on the video frame and imu data, obtain the pose of the terminal device during the movement of the target area; the pose of the terminal device during the movement of the target area and the Wi-Fi fingerprint collected by the terminal device during the movement of the target area According to time alignment, the spatial distribution of Wi-Fi fingerprints in the target area is obtained.
  • the embodiment of the present application can avoid the problem of low collection efficiency caused by pre-setting the position of the external anchor point.
  • the terminal device can not only accurately locate the position, but also obtain the accurate device posture, which improves the spatial distribution of Wi-Fi fingerprints. accuracy.
  • FIG. 1 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of an embodiment of a method for obtaining a Wi-Fi fingerprint spatial distribution provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of an interface change of a terminal device provided by an embodiment of the present application.
  • FIG. 4 is a schematic flowchart of another embodiment of a method for obtaining a Wi-Fi fingerprint spatial distribution provided by an embodiment of the present application
  • FIG. 5 is a schematic diagram of a pose and a point-line feature map provided by an embodiment of the present application.
  • FIG. 6 is a schematic interface diagram of a terminal device provided by an embodiment of the present application.
  • FIG. 7 is a schematic flowchart of another embodiment of the method for obtaining the spatial distribution of Wi-Fi fingerprints provided by an embodiment of the present application.
  • FIG. 8 is a schematic flowchart of another embodiment of a method for obtaining a Wi-Fi fingerprint spatial distribution provided by an embodiment of the present application.
  • FIG. 9 is a Wi-Fi fingerprint spatial distribution diagram 1 provided by an embodiment of the present application.
  • FIG. 10 is a second Wi-Fi fingerprint spatial distribution diagram provided by an embodiment of the present application.
  • FIG. 11 is a third spatial distribution diagram of Wi-Fi fingerprints provided by an embodiment of the present application.
  • FIG. 12 is a schematic structural diagram of an apparatus for obtaining the spatial distribution of Wi-Fi fingerprints according to an embodiment of the present application.
  • the present application provides a method for obtaining the spatial distribution of Wi-Fi fingerprints.
  • the device collects Wi-Fi fingerprints, and the terminal device itself accurately determines the pose during the movement process to obtain the spatial distribution of Wi-Fi fingerprints in the target area.
  • users can obtain the spatial distribution of Wi-Fi fingerprints through their own terminal equipment, which has high applicability and avoids the need to use special equipment.
  • the terminal device can not only accurately locate the position, but also obtain an accurate posture, thereby improving the accuracy of the spatial distribution of Wi-Fi fingerprints.
  • the terminal device in the embodiments of the present application may refer to user equipment, access terminal, subscriber unit, subscriber station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent, or user equipment.
  • the terminal equipment can be a mobile phone (mobile phone), a tablet computer (pad), a computer with wireless transceiver function, a personal digital assistant (PDA), a handheld device with wireless communication function, a computer or other processing equipment, a wearable equipment, virtual reality (VR) terminal equipment, augmented reality (AR) terminal equipment, wireless terminal in smart home (smart home), intelligent robot, terminal equipment in 5G network or future evolution of public land Terminal equipment in a mobile communication network (public land mobile network, PLMN), etc.
  • VR virtual reality
  • AR augmented reality
  • FIG. 1 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • the terminal device 100 in this embodiment of the present application may include: a processor 101 , a memory 102 , a wireless communication module 103 , a sensor 104 , a camera 105 , a display screen 106 , and the like. It can be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the terminal device 100 . In other embodiments of the present application, the terminal device 100 may include more or less components than those shown in the drawings, or combine some components, or separate some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
  • the processor 101 may include one or more processing units, for example, the processor 101 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (DSP), baseband processor, display process unit (DPU), and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • the terminal device 100 may also include one or more processors 101 .
  • the processor may be the nerve center and command center of the terminal device 100 .
  • the controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.
  • a memory may also be provided in the processor 101 for storing instructions and data.
  • the memory in processor 101 is a cache memory.
  • the memory may hold instructions or data that have just been used or recycled by the processor 101 . If the processor 101 needs to use the instruction or data again, it can be called directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 101, and thus improves the efficiency of the terminal device 100.
  • the processor 101 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transceiver (universal asynchronous transmitter) receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and / or universal serial bus (universal serial bus, USB) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous transceiver
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • SIM subscriber identity module
  • USB universal serial bus
  • the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration, and does not constitute a structural limitation of the terminal device 100 .
  • the terminal device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.
  • Memory 102 may be used to store one or more computer programs including instructions.
  • the processor 101 may execute the instructions stored in the memory 102, thereby causing the terminal device 100 to perform the relevant actions in the following embodiments.
  • the memory 102 may include a stored program area and a stored data area. Wherein, the stored program area may store the operating system; the stored program area may also store one or more application programs (such as gallery, contacts, etc.) and the like.
  • the storage data area may store data (such as photos, contacts, etc.) created during the use of the terminal device 100 and the like.
  • the memory 102 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.
  • the processor 101 may cause the terminal device 100 to perform various functional applications and data processing by executing instructions stored in the memory 102 and/or instructions stored in a memory provided in the processor 101 .
  • the wireless communication function of the terminal device 100 may be implemented by the wireless communication module 103 .
  • the wireless communication module 103 can provide applications on the terminal device 100 including wireless local area networks (WLAN), Bluetooth, global navigation satellite system (GNSS), frequency modulation (FM), NFC, Infrared technology (infrared, IR) and other wireless communication solutions.
  • the wireless communication module 103 in this embodiment of the present application may be a wireless network card in a terminal device.
  • the wireless communication module 103 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 103 in the embodiment of the present application is used to implement the sending and receiving function of the electronic device, for example, to implement the communication with the server in FIG. 1 above.
  • the sensors 104 may include a pressure sensor 104A, a gyro sensor 104B, an acceleration sensor 104C, and the like.
  • the pressure sensor 104A is used to sense pressure signals, and can convert the pressure signals into electrical signals.
  • the pressure sensor 104A may be disposed on the display screen 106, and the terminal device 100 detects the intensity of the touch operation according to the pressure sensor 104A.
  • the gyro sensor 104B can be used to determine the motion attitude of the terminal device 100 .
  • the angular velocity of end device 100 about three axes ie, x, y, and z axes) may be determined by gyro sensor 104B.
  • the gyro sensor 104B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyroscope sensor 104B detects the angle at which the terminal device 100 shakes, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to offset the shake of the terminal device 100 through reverse motion to achieve anti-shake.
  • the acceleration sensor 104C can detect the magnitude of the acceleration of the terminal device 100 in various directions (generally three axes). The magnitude and direction of gravity can be detected when the terminal device 100 is stationary. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.
  • the terminal device 100 can realize the shooting function through one or more cameras 105 .
  • the terminal device 100 can implement a display function through the display screen 106 .
  • Display screen 106 is used to display images, videos, and the like.
  • Display screen 106 includes a display panel.
  • the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light).
  • AMOLED flexible light-emitting diode
  • FLED flexible light-emitting diode
  • Miniled MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on.
  • the terminal device 100 in this embodiment of the present application may further include: an audio module 407 and a microphone 408 .
  • the terminal device 100 can implement audio functions, such as music playback, recording, and voice interaction with the user, through the audio module 407 and the microphone 408 .
  • the audio module 407 is used for converting digital audio information into analog audio signal output, and also for converting analog audio input into digital audio signal. Audio module 407 may also be used to encode and decode audio signals.
  • the audio module 407 may be provided in the processor 101 , or some functional modules of the audio module 407 may be provided in the processor 101 .
  • the terminal device 100 may be provided with at least one microphone 408 .
  • the terminal device 100 may be provided with two microphones 408 , which can implement a noise reduction function in addition to collecting sound signals.
  • the terminal device 100 may further be provided with three, four or more microphones 408 to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions.
  • the camera 105 may collect video of the target area, and the gyro sensor 104B and the acceleration sensor 104C may collect inertial measurement unit (imu) data of the terminal device.
  • the imu data may include the acceleration and angular velocity of the terminal device, the gyro sensor 104B may collect the angular velocity of the terminal device, and the acceleration sensor 104C may collect the acceleration of the terminal device.
  • the processor 401 is configured to execute the actions shown in FIG. 2 , FIG. 4 , FIG. 7 or FIG. 8 in the following embodiments. Based on the terminal device shown in FIG. 1 , the method for obtaining the spatial distribution of Wi-Fi fingerprints in the following embodiments can be implemented.
  • FIG. 1 the structure in FIG. 1 is described as an example of a terminal device.
  • the execution subject of the method for obtaining the spatial distribution of Wi-Fi fingerprints is described as a terminal device.
  • Each module can realize the functions of the above modules.
  • FIG. 2 is a schematic flowchart of an embodiment of a method for obtaining a Wi-Fi fingerprint spatial distribution provided by an embodiment of the present application. As shown in FIG. 2 , the method for obtaining the spatial distribution of Wi-Fi fingerprints provided by this embodiment of the present application may include:
  • S201 collect video frames, Wi-Fi fingerprints, and inertial measurement unit imu data of a target area within a current time window.
  • the terminal device may collect video frames, Wi-Fi fingerprints of the target area within the current time window, and inertial measurement unit imu data of the terminal device during the movement of the target area.
  • the target area can be, but not limited to, an indoor space.
  • the movement mode of the terminal device in the target area may be: the user holds the terminal device and moves in the target area, or the terminal device itself moves in the target area.
  • the terminal device is a smartphone
  • the user can hold the smartphone and move in the target area, so that the terminal device moves in the target area; or when the terminal device is a robot, the robot can move in the target area.
  • the terminal device will continue to collect video frames, Wi-Fi fingerprints of the target area, and imu data of the terminal device during the movement of the target area.
  • the pose of the terminal device may be obtained according to the video frame, Wi-Fi fingerprint of the target area in the current time window, and imu data of the terminal device.
  • the time window in this embodiment of the present application may be understood as a preset duration. Illustratively, for example, taking the terminal device starting to collect video frames of the target area as an example, every preset time period is a time window. If the time window is 5s, the video frame of the target area is collected from the terminal device, 0-5s is a time window, 5s-10s is a time window, and so on.
  • the time window is 5s.
  • the terminal device After the terminal device starts to move, it will collect video frames, Wi-Fi fingerprints of the target area, and imu data of the terminal device.
  • the video and Wi-Fi fingerprints of the target area collected within 0-5s, and the imu data of the terminal device can be used as the video frames, Wi-Fi fingerprints of the target area in the current time window, and the imu data of the terminal device.
  • the video frame of the target area in this embodiment of the present application may be the video frame included in the video of the target area collected by the terminal device, or the video frame collected by the terminal device at preset time intervals, and the video frame may also be referred to as an image frame.
  • the Wi-Fi fingerprint refers to the attribute information of the Wi-Fi signal, such as the strength of the Wi-Fi signal, frequency band, Wi-Fi name, etc.
  • the collection of Wi-Fi fingerprints at various indoor locations is the indoor Wi-Fi Fingerprint spatial distribution.
  • the imu data of the terminal device may be data collected by an inertial measurement unit (inertial measurement unit, IMU) set in the terminal device.
  • IMU inertial measurement unit
  • the user can trigger the terminal device to start collecting video frames, Wi-Fi fingerprints of the target area, and imu data of the terminal device.
  • the terminal device is provided with a control (such as a function in an application) to obtain the spatial distribution of Wi-Fi fingerprints, or the terminal device is provided with an application to obtain the spatial distribution of Wi-Fi fingerprints, the user clicks
  • the control or opening the application can trigger the terminal device to collect the video frame, Wi-Fi fingerprint of the target area, and imu data of the terminal device.
  • FIG. 3 is a schematic diagram of an interface change of a terminal device provided by an embodiment of the present application.
  • the interface 301 displays an icon of an application program of “Wi-Fi fingerprint spatial distribution”. The user clicks on the icon, and the interface 301 jumps to the interface 302.
  • the interface 302 can be similar to a smart Mobile phone recording interface. It is worth noting that after interface 301 jumps to interface 302, the terminal device is triggered to start shooting video (ie, capture video frames), read imu data of the terminal device, and obtain Wi-Fi fingerprints at the same time.
  • a video recording control is displayed on the interface 302, and the user can click the video recording control to trigger the terminal device to start shooting video, read the imu data of the terminal device at the same time, and obtain the Wi-Fi fingerprint.
  • the user when the user triggers the terminal device to start shooting video, the user moves within the target area, so that the terminal device moves within the target area.
  • the user can trigger the terminal device to move in the target area, and simultaneously collect video frames, Wi-Fi fingerprints of the target area, and imu data of the terminal device.
  • the terminal device has a trigger control, which can be triggered by the user selecting the trigger control.
  • the user can perform voice interaction with the terminal device to trigger, for example, the user can say "Go and collect the spatial distribution of Wi-Fi fingerprints" to trigger.
  • the Wi-Fi function switch of the terminal device may be turned on in advance.
  • the terminal device turns on the Wi-Fi function switch to realize the collection of the Wi-Fi fingerprint.
  • the terminal device can obtain the video frame of the target area (ie, a monocular RGB image sequence) at a certain frequency (such as 30 Hz), and simultaneously activate the imu reading function (such as 200 Hz), obtain imu data, and activate Wi- Fi scanning function (such as 0.3Hz) to obtain Wi-Fi fingerprints.
  • the imu data may include the angular velocity and acceleration of the terminal device.
  • Wi-Fi fingerprints can include Wi-Fi signal strength, frequency band, and Wi-Fi name (such as ESS ID), Wi-Fi channel (channel), Wi-Fi physical address (adress), and whether the Wi-Fi signal is for the encryption type, etc.
  • the prior art provides a method for locating a terminal device by using an image, wherein the RGB-D depth camera is used for calibration to obtain the internal parameter information of the RGB-D depth camera, and then the depth is obtained through the depth camera. image, and then locate the terminal device according to the depth image.
  • the cost of using an RGB-D depth camera in this method is high and the scope of application is small, and many terminal devices do not have an RGB-D depth camera.
  • the RGB camera set on the terminal device is used to obtain the RGB video of the target area, and the terminal device can be positioned in combination with the imu data of the terminal device, which saves the cost and expands the applicable scope of the method.
  • the terminal device can be positioned based on the video and imu data, that is, the pose of the terminal device during the movement of the target area can be obtained, and the terminal device can specifically perform the positioning on the terminal device according to the characteristics of the video frame and the imu data. position.
  • the features of the video frame may be point features and/or line features of the video, and the point features may also be referred to as feature points.
  • the terminal device can determine the relative movement information of the terminal device according to the characteristics of the video frame. Exemplarily, the terminal device may determine the relative movement information of the terminal device according to changes in the positions of the point features in the two video frames collected by the terminal device.
  • the real-time position of the terminal device when collecting each video frame can be determined according to the relative movement information of the terminal device.
  • the imu data can characterize the motion posture of the terminal device during the movement, such as angular velocity and acceleration.
  • the posture of the terminal device during the movement of the target area is determined according to the movement posture of the terminal device during the movement of the target area. To sum up, the terminal device can not only obtain the real-time position and motion posture of the terminal device during the movement of the target area, but also obtain the pose of the terminal device during the movement of the target area.
  • the terminal device can use the video of the target area in the current time window, the Wi-Fi fingerprint, and the data of the terminal device. imu data to obtain the pose of the terminal device in the current time window.
  • the set of poses of the terminal device in each time window may be used as the pose of the terminal device in the process of moving in the target area.
  • the terminal device may collect video frames, Wi-Fi fingerprints of the target area, and imu data of the terminal device at the same collection frequency.
  • the real-time pose of the terminal device is obtained from the video frame and imu data (that is, the pose of the terminal device is obtained at the same frequency).
  • the terminal device can store the real-time pose corresponding to the Wi-Fi fingerprint collected when collecting imu data of the target area, and can obtain the spatial distribution of Wi-Fi fingerprints in the target area.
  • the above collection frequency is 20 Hz, that is, an image (ie, one video frame), one Wi-Fi fingerprint, and one imu data are collected every 0.05s.
  • the terminal device can obtain the pose of the terminal device at 0.05s according to the collected video frames and imu data, and then store the pose of the terminal device at 0.05s and the Wi-Fi fingerprint collected at 0.05s correspondingly. In the same way, the spatial distribution of Wi-Fi fingerprints in the target area can be obtained.
  • the collection frequency of video frames and imu data can be increased, while the collection frequency of Wi-Fi fingerprints can be relatively low. That is to say, in this embodiment of the present application, the terminal device may collect video frames, Wi-Fi fingerprints of the target area, and imu data of the terminal device at different collection frequencies.
  • the pose of the terminal device during the movement process and the Wi-Fi fingerprint collected by the terminal device during the movement process can be aligned according to time, and the spatial distribution of the Wi-Fi fingerprint of the target area can be obtained.
  • a Wi-Fi fingerprint whose collection time is within a preset time range may be aligned with the pose of the terminal device.
  • the preset time range may be 0.02s.
  • the terminal device can obtain the pose of the terminal device at 0.05s according to the collected video frames and imu data, but at 0.05s, the terminal device does not collect Wi-Fi fingerprints, but collects Wi-Fi at 0.06s Therefore, the pose of the terminal device in 0.05s and the Wi-Fi fingerprint collected in 0.06s can be stored correspondingly.
  • cubic spline interpolation method may be used to align the pose of the terminal device during the movement process with the collected Wi-Fi fingerprints .
  • the cubic spline interpolation method is an interpolation function preset in the terminal device.
  • the terminal device may use the time and the Wi-Fi fingerprint collected at each time as parameters, use the interpolation function to calculate the Wi-Fi fingerprint at 0.05s, and then use the calculated Wi-Fi fingerprint at 0.05s. It is stored corresponding to the pose of the terminal device at 0.05s.
  • the method for obtaining the spatial distribution of Wi-Fi fingerprints includes: during the movement of the terminal device in the target area, collecting video frames, Wi-Fi fingerprints, and imu of the terminal device in the target area within the current time window. Data, based on the video frame and imu data, obtain the pose of the terminal device during the movement process; align the pose of the terminal device during the movement process and the Wi-Fi fingerprint collected by the terminal device during the movement process according to time to obtain the target area The spatial distribution of Wi-Fi fingerprints.
  • the Wi-Fi fingerprint is collected during the movement of the terminal device in the target area, and the terminal device accurately determines the pose during the movement process according to the imu data and video, so as to obtain the Wi-Fi fingerprint of the target area.
  • Fingerprint spatial distribution In this embodiment of the present application, users can obtain the spatial distribution of Wi-Fi fingerprints through their own terminal equipment, which has high applicability, avoids the need to use special equipment, and this method can avoid problems caused by pre-setting the location of external anchor points. In view of the problem of low collection efficiency, in view of the fact that the terminal device in the embodiment of the present application can not only accurately locate the position, but also obtain an accurate posture, so the accuracy of the spatial distribution of Wi-Fi fingerprints can also be improved.
  • FIG. 4 is a schematic flowchart of another embodiment of the method for obtaining the spatial distribution of Wi-Fi fingerprints provided by an embodiment of the present application.
  • the method for obtaining the spatial distribution of Wi-Fi fingerprints provided by this embodiment of the present application may include:
  • S401 collect video frames, Wi-Fi fingerprints, and imu data of a terminal device in a target area within a current time window.
  • S402 according to the imu data, obtain relative movement information of the terminal device when collecting video frames.
  • S403 Acquire features of the video frame, where the features of the video frame include point features and/or line features of the video frame.
  • S404 according to the relative movement information and the characteristics of the video frame, obtain the pose when the terminal device collects the video frame, so as to obtain the pose of the terminal device during the movement of the target area.
  • the pose of the terminal device changes in real time in the process of moving, that is to say, the pose of the terminal device changes in real time when the video is collected.
  • the imu data in the embodiment of the present application is used to represent the movement process of the terminal device, and because the video is composed of multiple consecutive video frames, in the embodiment of the present application, the imu data when two adjacent video frames are collected by the terminal device can be collected Data, to obtain the change of the pose of the terminal device when collecting video frames.
  • relative movement information can be obtained according to the imu data collected by the terminal device between the video frame and the previous video frame.
  • the movement of the video frame It should be understood that the relative movement information represents the relative movement information relative to the previous video frame when the terminal device collects the latter video frame of the two video frames.
  • the imu data in this embodiment of the present application includes the angular velocity and acceleration of the terminal device.
  • integration processing may be performed on imu data collected between two adjacent video frames.
  • W t is the true value of the angular velocity at time t
  • a t is the true value of the acceleration at time t
  • the accelerometer random walk error of the gyro sensor is the random walk error of the acceleration sensor
  • n w is the measurement noise of the angular velocity
  • n a is the measurement noise of the acceleration
  • n w and n a Gaussian distribution with mean 0 are distributed.
  • the relative movement information of the terminal device in this embodiment of the present application may include relative translation, relative rotation and relative speed.
  • the terminal device collects the two adjacent video frames, it collects several frames of imu data.
  • the terminal device integrates the imu data to obtain the relative translation from the video frame k to the video frame k+1 frame Relative velocity and relative rotation in quaternions
  • the relative translation, relative velocity and relative rotation can be expressed by the following formulae 3, 4 and 5 respectively:
  • the embodiments of the present application can acquire the relative translation, relative rotation and relative speed between two adjacent video frames caused by the movement of the terminal device, that is, the relative translation, relative rotation and relative speed can indicate that the terminal device is in The relative translation, relative rotation and relative velocity of the pose when shooting two adjacent video frames, respectively Express.
  • the relative translation, relative rotation and relative speed obtained by using the above formulas 3-5 in real time in the present application may be the relative movement information of the next video frame in the adjacent two video frames.
  • the point feature is used to represent the local feature in the video frame, and one video frame can be represented by multiple point features.
  • the detection of the line feature in the video frame may be the detection of pixels with a large gradient change in the video frame.
  • contours are certain special areas in a video frame where, for grayscale images, the grayscale changes drastically from black to white or from white to black.
  • a line feature may be a set of pixels in a video frame, which is used to characterize the feature of the video frame.
  • the point features and line features in the video frame can both be abstract representations of objects in the video frame, and the point features and line features in the video frame are both extracted from the objects in the video frame and used to represent the object in the video frame.
  • the object may be an object, a person, or the like in the video frame.
  • the point feature and line feature of the video frame are used, compared with the way of using the point feature or the line feature, because the features in the acquired video frame are increased, so the video frame can be more accurate. Then, the pose of the terminal device when the video frame is collected can be more accurately obtained.
  • the features of the video frame include point features and line features of the video frame as an example for description. When the features of the video frame include point features or line features of the video frame, the following embodiments may be used. Replace "point and line features" with "point or line features" and perform the steps that include a point or line feature.
  • the terminal device may use the Fast corner detection algorithm to acquire point features in the video frame.
  • the corner point is a local feature, which has the characteristics of rotation invariance and does not change with the change of lighting conditions.
  • the point in the video frame with high enough curvature or obvious curvature change is regarded as the corner point.
  • point feature detection parameters may be preset, and the point feature detection parameters may include: the maximum number of corner points (max corners), the minimum distance between corner points (min distance), the corner point quality level ( quality level), the minimum number of point features (min point features), etc.
  • the terminal device may use the Fast corner detection algorithm in the first video frame to obtain point features in the first video frame.
  • the first video frame satisfies the above-mentioned preset point feature detection parameters. That is, the first video frame satisfies the following conditions: the number of corner points in the first video frame is less than or equal to the second number threshold; and, the number of point features of the first video frame is greater than or equal to the first number and, the distance between the corners in the first video frame is greater than or equal to the first distance threshold; and, the quality level of the corners in the first video frame is greater than a preset level.
  • the first video frame is the first video frame collected by the terminal device.
  • a first quantity threshold may be preset, and the calculation amount of the terminal device may be reduced on the basis of ensuring the accuracy of the description of the characteristics of the video frame.
  • the terminal device can use an optical flow algorithm (such as the Lucas-Kanade optical flow algorithm) to track the point features of the first video frame. That is to say, the terminal device can use the LK optical flow algorithm to track the point features in the first video frame to the second video frame. In order to ensure that the number of point features in the second video frame is greater than the first number threshold, If all the point features in the first video frame can be tracked in the second video frame, the point features in the second video frame can continue to be tracked in the third video frame.
  • an optical flow algorithm such as the Lucas-Kanade optical flow algorithm
  • the point feature tracking method is adopted instead of the point feature matching method to obtain the point feature in each video frame, which can reduce the amount of calculation and improve the speed of obtaining the point feature.
  • the erroneously tracked point feature in the process of tracking the point feature, the erroneously tracked point feature may be acquired, and the erroneously tracked point feature may be deleted in the video frame, so as to improve the tracking accuracy of the point feature.
  • RANSAC random sample consensus
  • an essential matrix (essential matrix) test can be used to obtain the point features with tracking errors, and the point features with tracking errors can be deleted. .
  • the terminal device acquires the line features in the video frame by using the line feature matching method.
  • the terminal device can use the line segment detector (LSD) algorithm to detect and obtain the line features in the video frame, and use the line band descriptor (LBD) algorithm to describe the line features in the video frame. Then the LBD descriptors of the line features in the video frame are obtained.
  • LSD line segment detector
  • LBD line band descriptor
  • the initial line pair may be established according to matching the line feature of the video frame with the line feature of the previous video frame.
  • the initial line pair indicates two line features that are successfully matched in the video frame and the previous video frame.
  • the two line features that are successfully matched in the video frame and the previous video frame can be: the distance between the line features
  • Two line features less than a preset distance The distance between the line features may be the Euclidean distance or the cosine similarity between the line features, or the like.
  • the terminal device acquires the line feature in the first video frame and the line feature in the second video frame, and acquires the LBD descriptor in the first video frame and the LBD in the second video frame. descriptor, and then match the LBD descriptor in the second video frame with the LBD descriptor in the first video frame, and use the successfully matched line pair as the initial line pair.
  • the method for matching the LBD descriptor in the second video frame with the LBD descriptor in the first video frame may be the above-mentioned method of obtaining the distance between line features.
  • the terminal device may delete the wrong line pairs from the initial line pairs to obtain line features in the video frame.
  • the following manner may be used to determine an erroneous line pair in the initial line pair.
  • the terminal device can obtain the distance ⁇ r of the two line features in the line pair, the length difference ⁇ d of the line segment, and the included angle of the line segment.
  • the line features in the video frame may be represented in the form of line segments in the video frame, and in this embodiment of the present application, the distance between the line segments may be used as the distance between two line features.
  • the terminal device may regard a line pair whose distance between the two line features is greater than the second distance threshold as an erroneous line pair, or may regard a line pair whose line segment length difference between the two line features is greater than the length difference threshold as an erroneous line pair , or, a line pair whose line segment angle between the two line features is greater than the second angle threshold may be regarded as an incorrect line pair. That is to say, in this embodiment of the present application, the terminal device may delete a line pair whose distance between two line features is greater than the second distance threshold from the initial line pair; and delete the line segment length difference between the two line features that is greater than the length difference threshold.
  • the steps of acquiring the point feature and the line feature in the video frame by the terminal device may be performed simultaneously.
  • the terminal device after acquiring the relative movement information of the video frame and the special feature in the video frame, the terminal device can acquire the pose of the terminal device when the video frame is collected, and then obtain the terminal device according to this method. The pose of the device during the movement of the target area.
  • the pose of the terminal device during the movement of the target area may include the pose of the terminal device when collecting each video frame.
  • the terminal device can obtain the pose of the adjacent key frame before the video frame, and then according to the pose of the adjacent key frame before the video frame, and between the video frame and the adjacent key frame before the video frame
  • the relative displacement information of the terminal device determines the pose when the terminal device collects the video frame.
  • the adjacent key frame before the video frame may be a key frame before the video frame that is closest in time to the video frame.
  • the relative movement information in this embodiment of the present application includes: relative displacement and relative angle, and the terminal device can determine whether the video frame is Keyframe.
  • the key frame in the embodiment of the present application satisfies at least one of the following conditions: the relative displacement is greater than the first displacement threshold; or the relative angle is greater than the first angle threshold; or the time difference is greater than the time difference threshold.
  • the video frame is determined to be Keyframe. It should be understood that in this embodiment of the present application, after each video frame is collected, it may be determined whether the video frame is a key frame according to the relative displacement information of each video frame and the time of the previous key frame.
  • the terminal device can obtain the pose of the terminal device when collecting the video frame according to the relative displacement information between the video frame and the adjacent key frames before the video frame, and then collect the previous key frame on the terminal device.
  • the pose when the terminal device collects the video frame can be obtained.
  • the video frame and the adjacent key frame before the video frame may be regarded as two adjacent video frames, and then the relative displacement information is obtained according to the above formula 3-formula 5. It should be understood that, for the acquisition manner of the pose of the terminal device when collecting the previous key frame, reference may be made to the related description in another possible implementation manner below.
  • the terminal device collects 5 video frames
  • the third video frame is a key frame
  • the fourth video frame and the fifth video frame are non-key frames
  • the terminal device acquires the fourth video frame and
  • the pose of the fifth video frame is the same, and the following describes the pose of the terminal device when the fifth video frame is collected.
  • the terminal device uses the fifth video frame and the third video frame as two adjacent video frames, and then obtains the difference between the fifth video frame and the third video frame according to the above formula 3-formula 5 relative displacement information.
  • the pose when the terminal device collects the fifth video frame can be obtained. It should be understood that, for the acquisition method of the pose when the terminal device collects the third video frame (ie, the key frame), reference may be made to the related description in the following another possible implementation manner.
  • the pose of the terminal device during the movement process may include the pose of the terminal device when the key frame is collected. That is to say, compared with the previous possible implementation manner, in this manner, the terminal device only calculates and obtains the pose of the terminal device when the key frame is collected. According to the above, the terminal device can obtain whether the video frame is a key frame, and when determining that the video frame is a key frame, calculate the pose when capturing the key frame.
  • the key frame collected in the current time window can be obtained according to the relative displacement information of the key frame in the current time window, the point feature and the line feature of the key frame.
  • the pose of the end device at frame time can be obtained according to the relative displacement information of the key frame in the current time window, the point feature and the line feature of the key frame.
  • the terminal device may use the relative movement information of the key frame, the point feature of the key frame, and the line feature of the key frame as parameters to construct the state vector to be estimated, and then establish the maximum posterior probability objective function, and the The estimated state vector is substituted into the maximum a posteriori probability objective function, and the maximum a posteriori probability objective function is solved to obtain the initial pose when the terminal device collects key frames within the current time window.
  • the key frames in the current time window are n frames.
  • the state vector to be estimated can be expressed as the following formula 6:
  • x n is the motion data of the n-th key frame, including the relative displacement of the n-th key frame Relative velocity and relative angle and random walk error and [lambda] m for the n-th keyframe m-th feature point inverse depth
  • ⁇ k is the n th k-th keyframe characteristic line orthogonal representation.
  • the inverse depth is a parameterization method, that is, the reciprocal of the depth is considered to obey a Gaussian distribution
  • the orthogonal representation is a representation method of the line feature.
  • the terminal device can construct a maximum a posteriori probability objective function, as shown in the following formula 7:
  • r p is the prior error term, which is obtained by Gaussian elimination method during marginalization
  • r b is the error term of the motion data of the key frame, is the observation of the motion data of the key frame
  • r f is the point feature reprojection error term, is the predicted value of the j-th point feature at the i-th key frame
  • r l is the line feature reprojection error term, is the predicted value of the l-th line feature in the i-th frame
  • is the L2 norm
  • ⁇ () is the kernel function robust to outliers
  • min is the minimization function
  • B is the motion data of the key frame
  • sequence of motion data of key frames refers to the set of motion data of each key frame
  • sequence of key frames refers to the set of key frames
  • sequence of point features refers to the set of point features in each key frame
  • line feature sequence refers to the set of line features in each keyframe.
  • the state vector to be estimated constructed by the key frame in the current time window can be substituted into the maximum a posteriori probability objective function, and the LM iterative optimization algorithm in the Ceres library can be used to solve the maximum a posteriori probability objective function, Obtain X that satisfies the maximum posterior probability, that is, the initial pose of the terminal device when collecting each key frame.
  • the initial pose of the terminal device when the adjacent non-key frames after the key frame is collected obtained according to the above formula 7, may be regarded as the adjacent non-key frames of the terminal device after the key frame is collected. Pose at frame time.
  • the terminal device may not acquire the pose of the non-key frame before the first key frame, and when acquiring the pose of the first key frame, the terminal device may acquire the pose of the For the pose of the adjacent non-key frames after the key frame, reference may be made to the relevant descriptions in the above possible implementation manners for details.
  • FIG. 5 is a schematic diagram of a pose and a point-line feature map provided by an embodiment of the present application.
  • FIG. 5 shows the pose of the terminal device when the key frame is acquired in the current time window, and the set of point features and line features in the video frames that have been acquired at the current moment. It should be understood that the figure also displays the pose, point feature and line feature when the terminal device collects the key frame in the time window before the current time window. Wherein, the line segment shown in FIG.
  • the pose of the terminal device can be represented by 6 degrees of freedom, and the 6 degrees of freedom can include the spatial position and spatial angle (also called spatial posture) of the terminal device, and the spatial position is such as the three-dimensional coordinates of the terminal device (x, y, z), the spatial angle such as the yaw angle (yaw angle) ⁇ , the roll angle (raw angle) ⁇ and the pitch angle (pitch angle) ⁇ of the terminal device.
  • the current keyframe may refer to the last keyframe within the current time window.
  • FIG. 6 is a schematic interface diagram of a terminal device provided by an embodiment of the present application.
  • a selection control may also be displayed on the interface, and the selection control is used to instruct the user to select whether the terminal device obtains the pose when capturing each video frame or whether the terminal device is capturing Pose at keyframe time.
  • the interface as shown in Figure 6 displays "Key Frames" and "Video Frames".
  • Video Frames can be selected (that is, the terminal device only obtains the acquisition key If the user needs to obtain more detailed spatial distribution of Wi-Fi fingerprints in the target area, you can select "video frame” (that is, the terminal device only obtains the pose when each video frame is collected).
  • the user can customize and determine the pose of the terminal device when acquiring each video frame, or acquire the pose when capturing key frames, which can improve flexibility.
  • the tracking method is used to obtain the point features in the video frame, which can reduce the calculation amount of the terminal device compared with the point feature matching method.
  • the calculation of the pose for each video frame can also reduce the calculation amount of the terminal device, and the point and line features in the key frame and the relative displacement information are used in the embodiment of the present application, and the continuous iterative optimization algorithm is used, which can improve the obtained The accuracy of the pose of the end device.
  • the relevant descriptions in the foregoing embodiments for other beneficial effects in the embodiments of the present application, reference may be made to the relevant descriptions in the foregoing embodiments.
  • the terminal device obtains the initial pose of the terminal device when collecting the key frame according to formula 7, as the pose of the terminal device when collecting the key frame.
  • closed-loop detection and spatio-temporal consistency verification can be performed on key frames in the current time window. According to the test result, the initial pose is processed to obtain the pose when the terminal device collects key frames with higher accuracy.
  • the cumulative drift error caused by the continuous integration of the above formulas 3-5 can be significantly reduced, thereby improving the accuracy of the pose.
  • the terminal device moves rapidly, there will be a problem that insufficient feature points cannot be extracted due to the motion blur of the video frame. The motion of moving objects leads to inaccurate pose estimation, or even failure. Therefore, if the space-time consistency check is not performed after the closed-loop detection is successful, the above method can only be applied to static scenes with slow and steady motion, not fast Scenes with rotating and dynamic objects have poor robustness.
  • the key frame can be checked for space-time consistency, so that the method in the embodiment of the present application rotates rapidly in the terminal device, the target area contains dynamic objects, and the target area has weak textures , illumination changes and other complex environments, it has good robustness.
  • a closed loop may be performed according to the point feature and line feature of the current key frame in the current time window, and the point feature and line feature of the historical key frame. Detect, and perform spatiotemporal consistency check on the current key frame.
  • the terminal device obtains the pose when the terminal device collects the current key frame according to the initial pose of the terminal device, the closed-loop detection result, and the verification result of the current key frame, so as to obtain the pose of the terminal device in the moving process.
  • the historical key frames are: key frames in the time window before the current time window when the terminal device moves in the target area. Exemplarily, if the current time window is 5s-10s, the historical keyframes may be keyframes in the time window of 0s-5s.
  • the terminal device may acquire the descriptor of the point feature of the current key frame, and then construct the first current vector according to the descriptor of the point feature of the current key frame.
  • the terminal device also constructs the first history vector according to the descriptor of the point feature of the historical key frame. It should be understood that when constructing the first current vector, the terminal device constructs the first current vector by using the descriptors of all point features in the current key frame. When constructing the first history vector, the terminal device constructs the first history vector by using descriptors of all point features in a history key frame.
  • the descriptor of the point feature is used to describe the point feature, for example, the descriptor may represent the point feature in a matrix, binary or other manner.
  • the terminal device can also construct a second current vector according to the descriptor of the line feature of the current key frame, and correspondingly, the terminal device can also construct a second history vector according to the descriptor of the line feature of the historical key frame.
  • the terminal device constructs the second current vector by using the descriptors of all the line features in the current key frame.
  • the terminal device constructs the second history vector by using the descriptors of all the line features in a history key frame.
  • the descriptor of the line feature is used to describe the line feature, for example, the descriptor may represent the line feature in a matrix, binary or other manner.
  • a current vector (a first current vector and a second current vector) and a history vector (a first history vector and a second history vector) can be constructed based on a bag of words model (bag of words, BOW),
  • bag of words BOW
  • the "vector" in this embodiment of the present application may be replaced with a "bag of words vector”.
  • other models may also be used to extract the vectors of point feature line features.
  • the terminal device obtains the similarity between the first current vector and the first history vector, and uses the key frame corresponding to the first history vector whose similarity is greater than the similarity threshold as the first candidate frame, and the terminal device obtains the first candidate frame. 2.
  • the similarity between the current vector and the second history vector, and the key frame corresponding to the second history vector whose similarity is greater than the similarity threshold is used as the second candidate frame.
  • the terminal device may perform closed-loop detection according to the similarity of the first candidate frame and the similarity of the second candidate frame, and perform a spatiotemporal consistency check on the current key frame. It should be understood that the similarity threshold is preset.
  • the terminal device may perform weighting processing on the similarity of the first candidate frame and the similarity of the second candidate frame to obtain a weighted score.
  • the similarity of the first candidate frame and the similarity of the second candidate frame are pre-set with corresponding weights, and then the weight of the similarity of the first candidate frame is based on the similarity of the first candidate frame. and the product of the similarity of the second candidate frame and the weight of the similarity of the second candidate frame to obtain a weighted score.
  • the weighted score S p1 can be expressed as the following formula 8:
  • n p and n l are the number of point features and line features in the current key frame, respectively; ⁇ is a hyperparameter, ⁇ 1, and the larger ⁇ , the greater the weight of the line feature, indicating the structure of the target area. the higher the degree.
  • the closed-loop detection passes; if the weighted score is less than the score threshold, the closed-loop detection fails.
  • the terminal device can perform the spatiotemporal consistency check on the key frame.
  • the following describes the process of performing the spatiotemporal consistency check on the current key frame in the embodiment of the present application:
  • the spatiotemporal consistency check includes spatial consistency check: obtaining the relative displacement and relative angle between the first candidate frame and the current key frame. It should be understood that, in this embodiment of the present application, the relative displacement and relative angle between the first candidate frame and the current key frame may be obtained according to the above formulae 3-5, which will not be repeated here.
  • the spatiotemporal consistency check also includes a time consistency check: if the weighted scores of the first N key frames of the current key frame are all greater than the score threshold, it is determined that the current key frame passes the time consistency check, and N is greater than or equal to An integer of 1. It should be understood that, in the embodiment of the present application, the terminal device may obtain the weighted score of the key frame every time it determines that a video frame is a key frame, and then the weighted scores of the first N key frames of the key frame are all greater than the score of the key frame. value threshold, it is determined that the key frame passes the time consistency check.
  • the terminal device may acquire the pose of the terminal device during the movement process according to the initial pose of the terminal device, the closed-loop detection result, and the verification result of the current key frame.
  • the initial pose of the terminal device and the pose of the terminal device when collecting historical key frames are used as the terminal device during the movement process. 's pose. Because the pose of the terminal device in the current time window is obtained according to the above formula 7, and it should be understood that the pose of the terminal device in the time window before the current time window can also be obtained through the above formula 7.
  • the terminal device may use the initial pose obtained by the above formula 7 as the position of the terminal device in the current time window.
  • pose that is to say, the pose of the terminal device during the movement of the target area includes the initial pose and the pose when the terminal device collects historical key frames.
  • the pose when the terminal device collects the historical key frame may be obtained according to the method in the embodiment of the present application, that is, the time window before the current time window is taken as the current time window.
  • the terminal device corrects the initial pose of the terminal device and the pose of the terminal device when collecting historical key frames through the fused point features and the fused line features, and obtains the pose of the terminal device during the movement process.
  • the terminal device can correct the initial pose of the terminal device and the pose when the terminal device collects historical key frames through the fused point features and the fused line features as follows: the terminal device can Match the point and line feature set (ie the set of point features and line features) ⁇ P n , L n ⁇ in the current key frame with the point and line feature set ⁇ P h , L h ⁇ in the first candidate frame, P n is the point feature in the current key frame, L n is the line feature in the current key frame, P h is the point feature in the first candidate frame, and L h is the line feature in the historical key frame.
  • the terminal device can Match the point and line feature set (ie the set of point features and line features) ⁇ P n , L n ⁇ in the current key frame with the point and line feature set ⁇ P h , L h ⁇ in the first candidate frame, P n is the point feature in the current key frame, L n is the line feature in the current key frame, P h is
  • the matched line features and point features can be fused, and the re-projection error can be minimized through the cost function, so as to correct the initial pose of the terminal device and the pose when the terminal device collects historical key frames, Then, the corrected pose of the terminal device is obtained.
  • the corrected pose of the terminal device may be used as the pose of the terminal device during the movement process, so as to improve the accuracy of the pose of the terminal device.
  • the terminal device performs closed-loop detection, which can significantly reduce the accumulated drift error caused by integration, thereby improving the accuracy of the pose. It is also possible to perform a spatiotemporal consistency check on the key frames after the closed-loop detection is passed, so that the method in this embodiment of the present application rotates rapidly in the terminal device, the target area contains dynamic objects, and the target area has weak textures, illumination changes, etc. In the environment, it has good robustness.
  • FIG. 7 is a schematic flowchart of another embodiment of the method for obtaining the spatial distribution of Wi-Fi fingerprints provided by the embodiment of the present application. As shown in FIG. 7, after the above S404, S406 may also be included.
  • S406 Output the pose of the terminal device during the movement of the target area, the time corresponding to the pose, and the video frame collected by the terminal device at the corresponding time.
  • the terminal device can obtain the pose when each video frame is collected, or the terminal device can obtain the pose when collecting key frames.
  • the terminal device may output the pose of the terminal device during the movement process, the time corresponding to the pose, and the video frames collected by the terminal device at time.
  • the output method of the terminal device may be to display the pose of the terminal device during the movement process, the time corresponding to the pose, and the video frames collected by the terminal device in time on the interface, as shown in FIG. 5 above.
  • the terminal device may output the pose when each video frame is collected, the time corresponding to the pose, and the video frame collected by the terminal device at the corresponding time during the movement.
  • the terminal device may output the pose when each key frame is collected, the time corresponding to the pose, and the key frame collected by the terminal device at the corresponding time during the movement.
  • the pose of the terminal device during the movement process, the time corresponding to the pose, and the video frames collected by the terminal device in time can be output in real time, so that the user can intuitively see the real-time pose of the terminal device. time, and captured video frames.
  • FIG. 8 is a schematic flowchart of another embodiment of the method for obtaining the spatial distribution of Wi-Fi fingerprints provided by an embodiment of the present application. As shown in FIG. 8 , the method for obtaining the spatial distribution of Wi-Fi fingerprints provided by this embodiment of the present application may include:
  • S801 collect video frames, Wi-Fi fingerprints, and imu data of a terminal device of a target area within a current time window.
  • S803 Acquire features of the video frame, where the features of the video frame include point features and/or line features of the video frame.
  • S804 according to the relative movement information and the characteristics of the video frame, obtain the pose when the terminal device collects the video frame, so as to obtain the pose of the terminal device during the movement of the target area.
  • the terminal device filters the collected Wi-Fi fingerprints according to the application scenario of the spatial distribution of the Wi-Fi fingerprints.
  • the terminal device may filter data irrelevant to positioning in the Wi-Fi fingerprint. If a signal strength threshold is preset in the terminal device, the terminal device can delete the Wi-Fi fingerprint whose Wi-Fi signal strength is less than the signal strength threshold in the Wi-Fi fingerprint.
  • the signal strength threshold may be -75dBm, for example.
  • the terminal device may retain the Wi-Fi fingerprint with the preset Wi-Fi name, such as retaining the Wi-Fi name. For the Wi-Fi fingerprint of essid_home, delete other Wi-Fi fingerprints that are not the default Wi-Fi names.
  • the first application scenario the adjustment of the router position.
  • the user can hold the terminal device (the camera of the terminal device is facing forward) and walk around each position of the target area to obtain the spatial distribution map of Wi-Fi fingerprints, as shown in Figures 7 and 8 below.
  • Wi-Fi fingerprint spatial distribution map The user adjusts the position of the router according to the spatial distribution map of Wi-Fi fingerprints, so that each position in the target area can cover the Wi-Fi signal of a certain strength.
  • the second application scenario router antenna pointing optimization.
  • the user adjusts the antenna pointing of the router, so that each location in the target area can cover the Wi-Fi signal of a certain strength.
  • the embodiment of the present application is used to give an indication of the spatial distribution map of Wi-Fi fingerprints, and as for how to adjust the direction of the router antenna according to the indication map, the embodiment of the present application does not limit it.
  • the third application scenario improving the accuracy of terminal device positioning.
  • the user adjusts the position of the router in the target area according to the instructions of the Wi-Fi fingerprint space distribution map, so that the Wi-Fi signal in the target area and the Wi-Fi signal in the non-target area have a certain configuration, so as to improve the positioning accuracy of the terminal device.
  • the embodiment of the present application is used to give an indication of the spatial distribution of Wi-Fi fingerprints.
  • the embodiment of the present application does not limit it.
  • the fourth application scenario router layout location recommendation.
  • the user can determine the location with weak Wi-Fi signal strength according to the current Wi-Fi fingerprint spatial distribution map of the target area, and then deploy a new route at this location to form a router relay .
  • the related description in FIG. 11 below.
  • the terminal device may acquire the Wi-Fi fingerprint of the target area according to the Wi-Fi fingerprint spatial distribution of the target area.
  • the spatial distribution map, and the Wi-Fi fingerprint spatial distribution map of the target area is displayed.
  • the user can view the Wi-Fi fingerprint spatial distribution map of the target area on the terminal device.
  • the spatial distribution of Wi-Fi fingerprints represents the corresponding Wi-Fi fingerprints of the terminal device at each position of the target area.
  • the pose of the terminal device is taken as an example of 6 components for description, for example, the pose of the terminal device is ⁇ x t , y t , z t , roll t , pitch t , yaw t ⁇ .
  • x t , y t , and z t represent the spatial position of the terminal device
  • roll t , pitch t , and yaw t represent the spatial attitude of the terminal device.
  • FIG. 9 is a first spatial distribution diagram of Wi-Fi fingerprints provided by an embodiment of the present application.
  • Fig. 9 is a point cloud diagram, wherein the X axis and the Y axis in Fig. 9 represent the abscissa and the ordinate in the position of the terminal device respectively, and the Z axis in Fig. 9 represents the strength of the Wi-Fi signal corresponding to the position of the terminal device , the greater the value of the Z axis, the greater the strength of the Wi-Fi signal.
  • FIG. 9 is a point cloud diagram, wherein the X axis and the Y axis in Fig. 9 represent the abscissa and the ordinate in the position of the terminal device respectively, and the Z axis in Fig. 9 represents the strength of the Wi-Fi signal corresponding to the position of the terminal device , the greater the value of the Z axis, the greater the strength of the Wi-Fi signal.
  • FIG. 10 is a second spatial distribution diagram of Wi-Fi fingerprints provided by an embodiment of the present application.
  • Figure 10 is a heat map.
  • the intensity of the Wi-Fi signal at each location in the target area is represented by grayscale. The larger the grayscale value, the greater the intensity of the Wi-Fi signal.
  • the terminal device may determine the location to be installed of the router according to the spatial distribution of Wi-Fi fingerprints, so that each location in the target area can cover Wi-Fi signals of a certain strength.
  • FIG. 11 is a third spatial distribution diagram of Wi-Fi fingerprints provided by an embodiment of the present application.
  • the Wi-Fi fingerprint spatial distribution map displayed by the terminal device in this embodiment of the present application may display the location to be installed of the router, and the user can install the router using this illustration.
  • a blind area may also be displayed in the Wi-Fi fingerprint spatial distribution map, where the blind area represents a position where the strength of the Wi-Fi signal is less than a strength threshold.
  • the terminal device may filter the Wi-Fi fingerprints according to the application scenario of the spatial distribution of the Wi-Fi fingerprints, so as to obtain the Wi-Fi fingerprints that are more suitable for use and reduce the space occupied by the data.
  • a Wi-Fi fingerprint spatial distribution map may also be displayed, and the user can intuitively determine the Wi-Fi fingerprint spatial distribution of the target area.
  • FIG. 12 is a schematic structural diagram of an apparatus for obtaining the spatial distribution of Wi-Fi fingerprints according to an embodiment of the present application.
  • the apparatus for obtaining the spatial distribution of Wi-Fi fingerprints in this embodiment of the present application may be the aforementioned terminal device, or may be a chip applied to the terminal device.
  • the apparatus for obtaining the spatial distribution of Wi-Fi fingerprints may be used to perform the actions of the terminal device in the foregoing method embodiments.
  • the apparatus for obtaining the spatial distribution of Wi-Fi fingerprints may include: a collection module 1201 , a processing module 1202 , and a display module 1203 . in,
  • the collection module 1201 is configured to collect video frames, Wi-Fi fingerprints, and inertial measurement unit imu data of the target area within the current time window.
  • the processing module 1202 is used to obtain the pose of the terminal device during the movement of the target area based on the video frame and the imu data, and to compare the pose of the terminal device during the movement of the target area and the pose of the terminal device during the movement of the target area. Wi-Fi fingerprints are aligned according to time, and the spatial distribution of Wi-Fi fingerprints in the target area is obtained.
  • the processing module 1202 is specifically configured to obtain relative movement information of the terminal device when collecting video frames according to the imu data, obtain the characteristics of the video frames, and obtain the relative movement information and the characteristics of the video frames according to the relative movement information and the characteristics of the video frames. , to obtain the pose of the terminal device when the video frame is collected, so as to obtain the pose of the terminal device during the movement of the target area, wherein the features of the video frame include point features and/or line features.
  • the processing module 1202 is specifically configured to obtain relative movement information according to the imu data collected by the terminal device between the video frame and the previous video frame. Movement when the previous video frame was captured.
  • the feature of the video frame includes the point feature of the video frame
  • the processing module 1202 is specifically configured to track the point feature of the previous video frame in the video frame, if the point feature obtained by tracking in the video frame is less than the first number threshold, extract new point features in the video frame to obtain point features of the first number threshold, and use the point features and new point features tracked in the video frame as the point features of the video frame .
  • the first video frame satisfies the following conditions:
  • the number of corner points in the first video frame is less than or equal to the second number threshold
  • the number of point features of the first video frame is greater than or equal to the first number threshold
  • the distance between the corner points in the first video frame is greater than or equal to the first distance threshold
  • the quality level of the corners in the first video frame is greater than the preset level.
  • the processing module 1202 is further configured to acquire the point features with tracking errors, and delete the point features with tracking errors in the video frame.
  • the feature of the video frame includes the line feature of the video frame.
  • the processing module 1202 is specifically configured to match the line feature of the video frame with the line feature of the previous video frame, and establish an initial line pair. Line pairs indicate two line features that are successfully matched in the video frame and the previous video frame; in the initial line pair, the wrong line pair is removed to get the line features in the video frame.
  • the processing module 1202 is specifically configured to obtain the distance between the two line features in the line pair, the difference in the length of the line segment, and the included angle between the line segments; delete the line pair whose distance between the two line features is greater than the second distance threshold ; and, delete the line pair whose line segment length difference of the two line features is greater than the length difference threshold; and, delete the line pair whose line segment angle of the two line features is greater than the second angle threshold.
  • the relative movement information includes: relative displacement and relative angle.
  • the processing module 1202 is further configured to obtain the key frame according to the relative displacement, the relative angle or the time difference between the video frame and the previous key frame.
  • the key frame satisfies at least one of the following conditions: the relative displacement is greater than the first displacement threshold; the relative angle is greater than the first angle threshold; the time difference is greater than the time difference threshold.
  • the pose of the terminal device during the movement of the target area includes: the pose of the terminal device when collecting video frames within the current time window.
  • the processing module 1202 is specifically used to obtain the position of the adjacent key frame before the video frame, and according to the pose of the adjacent key frame before the video frame, and the relative displacement between the video frame and the adjacent key frame before the video frame information to obtain the pose of the terminal device when capturing video frames.
  • the processing module 1202 is specifically configured to use the relative movement information of the key frame, the point feature of the key frame and the line feature of the key frame as parameters to construct the state vector to be estimated; establish the maximum posterior probability The objective function, and the state vector to be estimated is substituted into the maximum a posteriori probability objective function, the maximum a posteriori probability objective function is solved, and the initial pose of the terminal device when the key frame is collected in the current time window is obtained; Pose, obtain the pose when the terminal device collects key frames, so as to obtain the pose of the terminal device during the movement of the target area.
  • the key frame is the current key frame in the current time window, and the features of the video frame include point features and line features of the video frame;
  • the processing module 1202 is specifically configured to The point feature and line feature of the frame, as well as the point feature and line feature of the historical key frame, perform closed-loop detection, and perform spatiotemporal consistency check on the current key frame.
  • the historical key frame is: the current time window when the terminal device moves in the target area The key frame in the previous time window; according to the initial pose of the terminal device, the closed-loop detection result, and the verification result of the current key frame, obtain the pose of the terminal device when the key frame was collected, so as to obtain the movement process of the terminal device in the target area pose in .
  • the processing module 1202 is specifically configured to construct a first current vector according to the descriptor of the point feature of the current key frame, where the current key frame is a key frame in the current time window; obtain the first current vector The similarity with the first historical vector, and the key frame corresponding to the first historical vector whose similarity is greater than the similarity threshold is used as the first candidate frame, and the first historical vector is constructed according to the descriptor of the point feature of the historical key frame; According to the descriptor of the line feature of the current key frame, a second current vector is constructed; the similarity between the second current vector and the second historical vector is obtained, and the key frame corresponding to the second historical vector whose similarity is greater than the similarity threshold is used as the first Two candidate frames, the second history vector is constructed according to the descriptor of the line feature of the historical key frame; according to the similarity of the first candidate frame and the similarity of the second candidate frame, the closed-loop detection is performed, and the current key frame is spatio-temporally
  • the processing module 1202 is specifically configured to perform weighted processing on the similarity of the first candidate frame and the similarity of the second candidate frame to obtain a weighted score; if the weighted score is greater than or equal to the score If the weighted score is less than the score threshold, the closed-loop detection fails.
  • the space-time consistency check includes a spatial consistency check.
  • the processing module 1202 is specifically configured to obtain the relative displacement and relative angle between the first candidate frame and the current key frame; if the relative displacement is less than the second displacement threshold and the relative angle is less than the third angle threshold, then determine that the current key frame passes through the space Consistency check.
  • the spatiotemporal consistency check further includes a time consistency check.
  • the processing module 1202 is specifically configured to determine that the current key frame has passed the time consistency check if the weighted scores of the first N key frames of the current key frame are all greater than the score threshold, and N is an integer greater than or equal to 1.
  • the processing module 1202 is specifically configured to fuse the point feature of the current key frame and the point feature of the first candidate frame if the closed-loop detection passes and the current key frame passes the spatiotemporal consistency check; fusion; The line feature of the current key frame and the line feature of the second candidate frame; through the fused point feature and the fused line feature, the initial pose of the terminal device and the pose of the terminal device when collecting historical key frames are corrected to obtain The pose of the terminal device during the movement of the target area.
  • the processing module 1202 is specifically configured to collect the initial pose of the terminal device and the terminal device if the closed-loop detection fails, or if the closed-loop detection passes and the current key frame fails the spatiotemporal consistency check
  • the pose of the historical key frame is used as the pose of the terminal device during the movement of the target area.
  • the processing module 1202 is further configured to store the first current vector and the second current vector corresponding to the current key frame if the current key frame passes the spatiotemporal consistency check.
  • the processing module 1202 is further configured to output the pose of the terminal device during the movement process, the time corresponding to the pose, and the video frames collected by the terminal device at time.
  • the processing module 1202 is further configured to filter the Wi-Fi fingerprints according to the application scenario of the spatial distribution of the Wi-Fi fingerprints in the target area.
  • the processing module 1202 is further configured to obtain a Wi-Fi fingerprint spatial distribution map of the target area according to the Wi-Fi fingerprint spatial distribution of the target area.
  • the display module 1203 is used to display the Wi-Fi fingerprint spatial distribution map of the target area.
  • the display module 1203 is further configured to display in the Wi-Fi fingerprint spatial distribution map of the target area if the application scenario of the spatial distribution of Wi-Fi fingerprints in the target area is to obtain the installation location of the router The location where the router is to be installed.
  • the apparatus for obtaining the spatial distribution of Wi-Fi fingerprints provided in the embodiments of the present application can perform the actions of the terminal device in the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and are not repeated here.
  • the above transceiver module may be a transceiver, or include a transmitter and a receiver when actually implemented.
  • the processing module can be implemented in the form of software calling through processing elements; it can also be implemented in the form of hardware.
  • the processing module may be a separately established processing element, or may be integrated into a certain chip of the above-mentioned device to be implemented, in addition, it may also be stored in the memory of the above-mentioned device in the form of program code, and a certain processing element of the above-mentioned device Call and execute the functions of the above processing modules.
  • all or part of these modules can be integrated together, and can also be implemented independently.
  • the processing element described here may be an integrated circuit with signal processing capability.
  • each step of the above-mentioned method or each of the above-mentioned modules can be completed by an integrated logic circuit of hardware in the processor element or an instruction in the form of software.
  • the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more application specific integrated circuits (ASIC), or one or more microprocessors (digital) signal processor, DSP), or, one or more field programmable gate arrays (field programmable gate array, FPGA), etc.
  • ASIC application specific integrated circuits
  • DSP digital signal processor
  • FPGA field programmable gate array
  • the processing element may be a general-purpose processor, such as a central processing unit (central processing unit, CPU) or other processors that can call program codes.
  • these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).
  • SOC system-on-a-chip
  • a computer program product includes one or more computer instructions.
  • the computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device.
  • Computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server, or data center over a wire (e.g.
  • a computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, or the like that contains one or more of the available mediums integrated.
  • Useful media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)), among others.
  • plural refers to two or more.
  • the term “and/or” in this article is only an association relationship to describe the associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, it can mean that A exists alone, A and B exist at the same time, and A and B exist independently B these three cases.
  • the character "/" in this article generally indicates that the related objects before and after are an “or” relationship; in the formula, the character "/" indicates that the related objects are a "division" relationship.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Image Analysis (AREA)

Abstract

本申请实施例提供一种获取Wi-Fi指纹空间分布的方法、装置和电子设备，该方法包括：采集当前时间窗口内的目标区域的视频帧、Wi-Fi指纹，以及终端设备的惯性测量单元imu数据；基于视频帧和imu数据，获取终端设备在目标区域移动过程中的位姿；将终端设备在目标区域移动过程中的位姿和终端设备在目标区域移动过程中采集的Wi-Fi指纹按照时间对齐，得到目标区域的Wi-Fi指纹空间分布。本申请实施例可以采用终端设备获取Wi-Fi指纹空间分布，适用性高，且还能够避免预先设置外部锚点的位置导致的采集效率低的问题，另外终端设备不仅能够精准定位位置，还能够获取准确的设备姿态，提高了Wi-Fi指纹空间分布的准确性。

Description

获取Wi-Fi指纹空间分布的方法、装置和电子设备
本申请要求于2020年07月21日提交国家知识产权局、申请号为202010705291.X、申请名称为“获取Wi-Fi指纹空间分布的方法、装置和电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及计算机视觉技术，尤其涉及一种获取Wi-Fi指纹空间分布的方法、装置和电子设备。
背景技术
随着互联网的普及，大部分室内环境都存在Wi-Fi网络。Wi-Fi指纹是Wi-Fi信号的属性信息，如Wi-Fi信号的强度、频段等。路由器作为电子设备接入Wi-Fi网络的设备，布局在室内空间中，若路由器的布局合理，则Wi-Fi信号能够稳定覆盖室内的每个位置。因此，获取室内Wi-Fi指纹空间分布，即室内各位置处的Wi-Fi指纹至关重要。
现有技术中，为了获取室内Wi-Fi指纹空间分布，可以预先将室内划分为多个已知位置的网格，用户可以手持专用设备在已知位置处采集Wi-Fi指纹，进而将已知位置和已知位置处的Wi-Fi指纹对应存储，得到Wi-Fi指纹空间分布。
现有技术的采集方式中，需要预先确定已知位置，进而采集该已知位置处的Wi-Fi指纹进行存储，采集效率低，且目前需要采用专用设备采集Wi-Fi指纹，适用性低。
发明内容
本申请实施例提供一种获取Wi-Fi指纹空间分布的方法、装置和电子设备，不仅可以采用终端设备获取Wi-Fi指纹空间分布，适用性高，且还能够提高Wi-Fi指纹空间分布的获取效率和准确性。
第一方面，本申请实施例提供一种获取Wi-Fi指纹空间分布的方法，该方法可以应用于终端设备、也可以应用于终端设备中的芯片。下面以应用于终端设备为例对该方法进行描述，该方法中，在终端设备在目标区域移动的过程中，终端设备采集当前时间窗口内的所述目标区域的视频帧、Wi-Fi指纹，以及所述终端设备的惯性测量单元imu数据。应理解，本申请实施例中，终端设备可以每隔预设时间窗口，获取目标区域的视频帧、Wi-Fi指纹和imu数据，且可以根据当前时间窗口内的目标区域的视频帧、Wi-Fi指纹和imu数据，获取终端设备在当前时间窗口内在目标区域移动过程中的位姿。本申请实施例中，可以根据基于所述视频帧和所述imu数据，对所述终端设备进行定位，获取所述终端设备在移动过程中的位姿。也就是说，本申请实施例中，可以根据终端设备在当前时间窗口内的位姿，对历史时间窗口内获取的终端设备的位姿进行校正或处理，以得到所述终端设备在目标区域移动过程中的位姿。
在获取终端设备在目标区域移动过程中的位姿后，可以将所述终端设备在目标区域移动过程中的位姿和所述终端设备在目标区域移动过程中采集的Wi-Fi指纹按照时间对齐，得到所述目标区域的Wi-Fi指纹空间分布。其中，Wi-Fi指纹空间分布可以理解为终端设备在目标区 域内移动时的位姿和Wi-Fi指纹的对应关系。本申请实施例中用户可以通过自己的终端设备就能够获取Wi-Fi指纹空间分布，适用性高，避免了需要采用专用设备的问题。另外本申请实施例中是通过终端设备自身获取终端设备的位姿，一方面能够避免预先设置外部锚点的位置导致的采集效率低的问题，另一方面终端设备不仅能够精准定位位置，还能够获取准确的设备姿态，提高了Wi-Fi指纹空间分布的准确性。
应注意，本申请实施例中在获取终端设备在目标区域移动过程中的位姿时，可以根据所述imu数据，获取所述终端设备在采集所述视频帧时的相对移动信息。应理解，终端设备在目标区域移动的过程中可以采集多个视频帧，终端设备在采集每个视频帧时均具有对应的位姿，因为imu数据可以表征终端设备在移动过程中的运动姿态，则终端设备可以通过imu数据，获取终端设备在采集所述视频帧时的相对移动信息。本申请实施例中可以根据所述视频帧和前一个视频帧之间所述终端设备采集的imu数据，得到所述相对移动信息，所述相对移动信息表征终端设备采集所述视频帧时相较于采集前一个视频帧时的移动情况，该移动情况可以表征终端设备位置的移动。也就是说，终端设备在采集相邻两个视频帧时，可以采集imu数据，进而根据该相邻两个视频帧之间所述终端设备采集的imu数据，得到终端设备在采集后一个视频帧时，相对于前一个视频帧的相对移动信息。本申请实施例中，终端设备可以获取所述视频帧的特征，该视频帧的特征包括视频帧的点特征和/或线特征。其中，终端设备可以获取视频帧的点特征和/或线特征，进而根据所述相对移动信息、所述点特征和/或所述线特征(视频帧的特征)，获取所述终端设备采集所述视频帧时的位姿，以得到所述终端设备在所述目标区域移动过程中的位姿。应理解，本申请实施例中采用视频帧的点特征和线特征，相较于采用点特征或线特征的方式，因为视频帧中的特征增多，因此可以提高终端设备定位的准确性(即获取所述终端设备采集所述视频帧时的位姿的准确性)。
下述首先介绍本申请实施例中获取视频帧的点特征的过程：
本申请实施例中，终端设备可以获取视频帧中的点特征，其中，若在一视频帧中采集的点特征过少，则点特征对该视频帧的特征描述的准确度可能较低，但若在一视频帧中采集的点特征过多，则加大了导致终端设备的计算量，因此，本申请实施例中可以预先设置第一数量阈值，可以在保证视频帧的特征的描述的准确性的基础上，减少终端设备的计算量。其中，第一个视频帧满足上述预先设置点特征检测参数。也就是说，第一个视频帧满足如下条件：第一个视频帧中的角点的数量小于或等于第二数量阈值，以及第一个视频帧的点特征的数量大于或等于第一数量阈值，以及第一个视频帧中的角点之间的距离大于或等于第一距离阈值，以及第一个视频帧中的角点的质量等级大于预设等级。本申请实施例中，可以采用点特征追踪的方法，而并非点特征匹配的方式在每个视频帧中获取点特征，可以减少计算量，提高获取点特征的速度。
也就是说，本申请实施例可以在所述视频帧中，追踪前一个视频帧的点特征，其中，若所述视频帧中追踪得到的点特征的数量小于第一数量阈值，则在所述视频帧中提取新的点特征，以得到所述第一数量阈值的点特征，进而将所述视频帧中追踪得到的点特征和所述新的点特征作为所述视频帧的点特征。其中，需要注意的是，若所述视频帧中追踪得到的点特征的数量等于第一数量阈值，则在下一个所述视频帧中追踪该视频帧中的点特征。其中，为了提高在视频帧中追踪的点特征的准确性，还可以获取追踪错误的点特征，且在所述视频帧中删除所述追踪错误的点特征。
其次介绍本申请实施例中获取视频帧的线特征的过程：
本申请实施例中，采用线特征匹配的方式，获取将所述视频帧的线特征与前一个视频帧 的线特征进行匹配，进而建立初始线对，其中，所述初始线对指示在所述视频帧和所述前一个视频帧中匹配成功的两个线特征。为了保证本申请实施例中线特征的匹配的准确性，进而提高终端设备的位姿的准确性，本申请实施例中可以在所述初始线对中，删除错误的线对，以得到所述视频帧中的线特征。
本申请实施例中可以获取线对中获取两个线特征的距离、线段长度差以及线段夹角，进而根据两个线特征的距离、线段长度差以及线段夹角确定错误的线对。其中，错误的线对具备：两个线特征的距离大于第二距离阈值的线对，或者两个线特征的线段长度差大于长度差阈值的线对，或者两个线特征的线段夹角大于第二角度阈值的线对。另外，值得注意的是，本申请实施例中在根据所述相对移动信息、所述点特征和所述线特征，获取所述终端设备在移动过程中的位姿的方式可以包括下述两种方式。其中，相对移动信息包括：相对位移和相对角度，且此处对本申请实施例中的关键帧进行介绍：
本申请实施例中，可以根据所述相对位移、所述相对角度或所述视频帧与前一个关键帧之间的时间差，得到关键帧。其中，所述关键帧满足如下至少一项条件：所述相对位移大于第一位移阈值，或者所述相对角度大于第一角度阈值，或者所述时间差大于时间差阈值。应注意的是，终端设备在采集一视频帧后，可以依据上述条件，判断该视频帧是否为关键帧，进而选择采用下述对应的方式获取终端设备在移动过程中的位姿。应理解，本申请实施例中，终端设备在移动过程中的位姿是采用下述两种方式中的哪种方式，可以有用户自定义设置。
第一种方式：获取终端设备在移动过程中采集每个视频帧时的位姿。
在该种方式中，所述终端设备在移动过程中的位姿包括：所述当前时间窗口内所述终端设备在采集所述视频帧时的位姿。其中，可以根据所述视频帧和所述视频帧之前相邻的关键帧之间的相对位移信息，获取所述终端设备在采集所述视频帧时的位姿。应理解，在该种方式中，终端设备采集视频帧的前一个关键帧时的位姿可以采用下述第二种方式获取。其中，本申请实施例中可以在终端设备采集视频帧之前相邻的关键帧时的位姿的基础上，加上相对位移和相对角度，以得到终端设备在采集所述视频帧时的位姿
第二种方式：获取终端设备在移动过程中采集关键帧时的位姿。本申请实施例中，以关键帧为当前时间窗口内的当前关键帧为例说明。终端设备可以以当前时间窗口内的关键帧的相对移动信息、所述关键帧的点特征和所述关键帧的线特征作为参数，构建待估计的状态向量，建立最大后验概率目标函数，且将所述待估计的状态向量代入所述最大后验概率目标函数中，对所述最大后验概率目标函数进行求解，得到所述终端设备在所述当前时间窗口内采集所述关键帧时的初始位姿；进而根据所述终端设备初始位姿，获取所述终端设备采集所述关键帧时的位姿，以得到获取所述终端设备在目标区域移动过程中的位姿。
在该种方式中，下面以视频帧的特征包括视频帧的点特征和线特征为例进行说明。应理解，当视频帧的特征包括视频帧的点特征或线特征时，执行下述点特征或线特征对应的步骤即可。其中，终端设备可以根据所述当前时间窗口内的当前关键帧的点特征和线特征，以及历史关键帧的点特征和线特征，进行闭环检测，以及对所述当前关键帧进行时空一致性校验。其中，所述历史关键帧为：所述终端设备在所述目标区域移动时所述当前时间窗口之前的时间窗口内的关键帧。进而，根据所述终端设备的初始位姿、闭环检测结果、以及所述当前关键帧的校验结果，获取所述终端设备在移动过程中的位姿。
应注意，本申请实施例中进行闭环检测，可以显著地减小因为不断积分导致的累积漂移误差，进而提高位姿的准确性。当终端设备快速运动时，会存在由于视频帧运动模糊而提取不到足够多的特征点而失效的问题，或者当目标区域中存在运动的人或物时，由于不能区分 是终端设备的运动还是运动物体的运动而导致位姿估计不准，甚至失效，因此若闭环检测成功后，不进行时空一致性校验，则上述方法仅能应用于慢速平稳运动的静态场景，而不适用于快速旋转、动态物体的场景，鲁棒性差。而本申请实施例中能够在闭环检测成功后，对关键帧进行时空一致性校验，使得本申请实施例中的方法在终端设备快速旋转、目标区域包含有动态物体、且目标区域存在弱纹理、光照变化等复杂环境时，具有很好的鲁棒性。
其中，若闭环检测通过，且所述当前关键帧通过时空一致性校验，则融合所述当前关键帧的点特征与所述第一候选帧的点特征，融合所述当前关键帧的线特征与所述第二候选帧的线特征，进而通过融合后的点特征和融合后的线特征，对所述终端设备的初始位姿以及所述终端设备采集所述历史关键帧时的位姿进行校正，获取所述终端设备在目标区域移动过程中的位姿。本申请实施例中，在当前关键帧通过时空一致性校验，则存储所述当前关键帧对应的第一当前向量和第二当前向量，进而可以作为下一个时间窗口的历史关键帧。
其中，若闭环检测未通过，或者闭环检测通过且所述当前关键帧未通过时空一致性校验，则将所述终端设备的初始位姿和所述终端设备采集所述历史关键帧时的位姿作为所述终端设备在目标区域移动过程中的位姿。
下述对本申请实施例中进行闭环检测，以及对所述当前关键帧进行时空一致性校验的过程进行说明：
本申请实施例中，可以根据所述当前关键帧的点特征的描述子，构建第一当前向量，所述当前关键帧为所述当前时间窗口内的关键帧，获取第一当前向量与第一历史向量的相似度，且将相似度大于相似度阈值的第一历史向量对应的关键帧作为第一候选帧，所述第一历史向量是根据所述历史关键帧的点特征的描述子构建的。根据所述当前关键帧的线特征的描述子，构建第二当前向量；获取第二当前向量与第二历史向量的相似度，且将相似度大于所述相似度阈值的第二历史向量对应的关键帧作为第二候选帧，所述第二历史向量是根据所述历史关键帧的线特征的描述子构建的。进而，根据所述第一候选帧的相似度和所述第二候选帧的相似度，进行闭环检测，以及对所述当前关键帧进行时空一致性校验。其中，终端设备可以对所述第一候选帧的相似度和所述第二候选帧的相似度进行加权处理，得到加权分值；若所述加权分值大于或等于分值阈值，则闭环检测通过；若所述加权分值小于分值阈值，则闭环检测未通过。
其中，时空一致性校验还包括空间一致性校验和时间一致性校验。终端设备可以获取所述第一候选帧和所述当前关键帧之间的相对位移和相对角度；若所述相对位移小于第二位移阈值，以及所述相对角度小于第三角度阈值，则确定所述当前关键帧通过空间一致性校验。其中，若所述当前关键帧的前N个关键帧的加权分值均大于所述分值阈值，则确定所述当前关键帧通过时间一致性校验，所述N为大于或等于1的整数。
应注意，在获取终端设备在目标区域移动过程中的位姿之后，终端设备还可以输出所述终端设备在目标区域移动过程中的位姿、位姿对应的时间，以及所述终端设备在所述时间采集的视频帧。其中，本申请实施例中，终端设备可以输出终端设备在目标区域移动过程中的位姿、位姿对应的时间，以及所述终端设备在所述时间采集的视频帧相对应的示意图，使得用户可以实时直观地观察到终端设备的位姿。
在一种可能的实现方式中，本申请实施例获取上述Wi-Fi指纹空间分布，是为了应用在具体的场景中。例如，应用场景可以为路由器位置的调整、路由器天线指向优化、提高终端设备定位的精度或路由器布局位置推荐等。本申请实施例中，还可以根据所述目标区域的Wi-Fi指纹空间分布的应用场景，过滤所述Wi-Fi指纹，进而减少终端设备的存储空间。
其中，本申请实施例中还可以根据所述目标区域的Wi-Fi指纹空间分布，获取所述目标区域的Wi-Fi指纹空间分布图，进而显示所述目标区域的Wi-Fi指纹空间分布图，使得用户能够更为直观地获取目标区域的Wi-Fi指纹空间分布。应注意的是，若所述目标区域的Wi-Fi指纹空间分布的应用场景为获取路由器的安装位置，则本申请实施例中可以在所述目标区域的Wi-Fi指纹空间分布图中显示所述路由器的待安装位置。应理解，上述路由器位置的调整、路由器天线指向优化、提高终端设备定位的精度等应用场景中，终端设备也可以显示调整的路由器的位置、或者推荐的路由器天线指向等，本申请实施例中对终端设备的应用场景和应用后终端设备的显示以及处理过程不做限制。
第二方面，本申请实施例提供一种获取Wi-Fi指纹空间分布的装置，该装置包括：
采集模块，用于采集当前时间窗口内的目标区域的视频帧、Wi-Fi指纹，以及所述终端设备的惯性测量单元imu数据。
处理模块，用于基于所述视频帧和所述imu数据，对所述终端设备进行定位，获取所述终端设备在目标区域移动过程中的位姿，且将所述终端设备在目标区域移动过程中的位姿和所述终端设备在目标区域移动过程中采集的Wi-Fi指纹按照时间对齐，得到所述目标区域的Wi-Fi指纹空间分布。
在一种可能的实现方式中，所述处理模块，具体用于根据所述imu数据，获取所述终端设备在采集所述视频帧时的相对移动信息，且获取所述视频帧的特征，以及根据所述相对移动信息、所述视频帧的特征特征，获取所述终端设备采集所述视频帧时的位姿，以得到所述终端设备在所述目标区域移动过程中的位姿。其中，视频帧的特征可以包括点特征和/或线特征。
在一种可能的实现方式中，所述处理模块，具体用于根据视频帧和前一个视频帧之间所述终端设备采集的imu数据，得到所述相对移动信息，所述相对移动信息表征终端设备采集所述视频帧时相较于采集前一个视频帧时的移动情况。
在一种可能的实现方式中，所述视频帧的特征包括所述视频帧的点特征，所述处理模块，具体用于在所述视频帧中，追踪前一个视频帧的点特征，若所述视频帧中追踪得到的点特征的数量小于第一数量阈值，则在所述视频帧中提取新的点特征，以得到所述第一数量阈值的点特征，以及将所述视频帧中追踪得到的点特征和所述新的点特征作为所述视频帧的点特征。
在一种可能的实现方式中，若所述视频帧为第一个视频帧，则所述第一个视频帧满足如下条件：
所述第一个视频帧中的角点的数量小于或等于第二数量阈值；和，
所述第一个视频帧的点特征的数量大于或等于所述第一数量阈值；和，
所述第一个视频帧中的角点之间的距离大于或等于第一距离阈值；和，
所述第一个视频帧中的角点的质量等级大于预设等级。
在一种可能的实现方式中，所述处理模块，还用于获取追踪错误的点特征，且在所述视频帧中删除所述追踪错误的点特征。
在一种可能的实现方式中，所述视频帧的特征包括所述视频帧的线特征，所述处理模块，具体用于将所述视频帧的线特征与前一个视频帧的线特征进行匹配，建立初始线对，所述初始线对指示在所述视频帧和所述前一个视频帧中匹配成功的两个线特征；在所述初始线对中，删除错误的线对，以得到所述视频帧中的线特征。
在一种可能的实现方式中，所述处理模块，具体用于获取所述线对中两个线特征的距离、线段长度差以及线段夹角；删除所述两个线特征的距离大于第二距离阈值的线对；和，删除所述两个线特征的线段长度差大于长度差阈值的线对；和，删除所述两个线特征的线段夹角 大于第二角度阈值的线对。
在一种可能的实现方式中，所述相对移动信息包括：相对位移和相对角度。
所述处理模块，还用于根据所述相对位移、所述相对角度或所述视频帧与前一个关键帧之间的时间差，得到关键帧。
所述关键帧满足如下至少一项条件：所述相对位移大于第一位移阈值；所述相对角度大于第一角度阈值；所述时间差大于时间差阈值。
在一种可能的实现方式中，所述终端设备在目标区域移动过程中的位姿包括：所述当前时间窗口内所述终端设备在采集所述视频帧时的位姿。所述处理模块，具体用于获取所述视频帧之前相邻的关键帧的位姿；根据所述视频帧之前相邻的关键帧的位姿，以及所述视频帧和所述视频帧之前相邻的关键帧之间的相对位移信息，获取所述终端设备在采集所述视频帧时的位姿。
在一种可能的实现方式中，所述处理模块，具体用于以关键帧的相对移动信息、所述关键帧的点特征和所述关键帧的线特征作为参数，构建待估计的状态向量；建立最大后验概率目标函数，且将所述待估计的状态向量代入所述最大后验概率目标函数中，对所述最大后验概率目标函数进行求解，得到所述终端设备在所述当前时间窗口内采集所述关键帧时的初始位姿；根据所述终端设备初始位姿，获取所述终端设备采集所述关键帧时的位姿，以得到所述终端设备在目标区域移动过程中的位姿。
在一种可能的实现方式中，所述关键帧为所述当前时间窗口内的当前关键帧，所述视频帧的特征包括所述视频帧的点特征和线特征；所述处理模块，具体用于根据所述当前时间窗口内的当前关键帧的点特征和线特征，以及历史关键帧的点特征和线特征，进行闭环检测，以及对所述当前关键帧进行时空一致性校验，所述历史关键帧为：所述终端设备在所述目标区域移动时所述当前时间窗口之前的时间窗口内的关键帧；根据所述终端设备的初始位姿、闭环检测结果、以及所述当前关键帧的校验结果，获取所述终端设备采集所述当前关键帧时的位姿，以得到所述终端设备在目标区域移动过程中的位姿。
在一种可能的实现方式中，所述处理模块，具体用于根据所述当前关键帧的点特征的描述子，构建第一当前向量，所述当前关键帧为所述当前时间窗口内的关键帧；获取第一当前向量与第一历史向量的相似度，且将相似度大于相似度阈值的第一历史向量对应的关键帧作为第一候选帧，所述第一历史向量是根据所述历史关键帧的点特征的描述子构建的；根据所述当前关键帧的线特征的描述子，构建第二当前向量；获取第二当前向量与第二历史向量的相似度，且将相似度大于所述相似度阈值的第二历史向量对应的关键帧作为第二候选帧，所述第二历史向量是根据所述历史关键帧的线特征的描述子构建的；根据所述第一候选帧的相似度和所述第二候选帧的相似度，进行闭环检测，以及对所述当前关键帧进行时空一致性校验。
在一种可能的实现方式中，所述处理模块，具体用于对所述第一候选帧的相似度和所述第二候选帧的相似度进行加权处理，得到加权分值；若所述加权分值大于或等于分值阈值，则闭环检测通过；若所述加权分值小于分值阈值，则闭环检测未通过。
在一种可能的实现方式中，所述时空一致性校验包括空间一致性校验。所述处理模块，具体用于获取所述第一候选帧和所述当前关键帧之间的相对位移和相对角度；若所述相对位移小于第二位移阈值，以及所述相对角度小于第三角度阈值，则确定所述当前关键帧通过空间一致性校验。
在一种可能的实现方式中，所述时空一致性校验还包括时间一致性校验。所述处理模块， 具体用于若所述当前关键帧的前N个关键帧的加权分值均大于所述分值阈值，则确定所述当前关键帧通过时间一致性校验，所述N为大于或等于1的整数。
在一种可能的实现方式中，所述处理模块，具体用于若闭环检测通过，且所述当前关键帧通过时空一致性校验，则融合所述当前关键帧的点特征与所述第一候选帧的点特征；融合所述当前关键帧的线特征与所述第二候选帧的线特征；通过融合后的点特征和融合后的线特征，对所述终端设备的初始位姿以及所述终端设备采集所述历史关键帧时的位姿进行校正，获取所述终端设备在目标区域移动过程中的位姿。
在一种可能的实现方式中，所述处理模块，具体用于若闭环检测未通过，或者闭环检测通过且所述当前关键帧未通过时空一致性校验，则将所述终端设备的初始位姿和所述终端设备采集所述历史关键帧时的位姿作为所述终端设备在目标区域移动过程中的位姿。
在一种可能的实现方式中，所述处理模块，还用于若所述当前关键帧通过时空一致性校验，则存储所述当前关键帧对应的第一当前向量和第二当前向量。
在一种可能的实现方式中，所述处理模块，还用于输出所述终端设备在移动过程中的位姿、位姿对应的时间，以及所述终端设备在所述时间采集的视频帧。
在一种可能的实现方式中，所述处理模块，还用于根据所述目标区域的Wi-Fi指纹空间分布的应用场景，过滤所述Wi-Fi指纹。
在一种可能的实现方式中，所述处理模块，还用于根据所述目标区域的Wi-Fi指纹空间分布，获取所述目标区域的Wi-Fi指纹空间分布图。
显示模块，用于显示所述目标区域的Wi-Fi指纹空间分布图。
在一种可能的实现方式中，所述显示模块，还用于若所述目标区域的Wi-Fi指纹空间分布的应用场景为获取路由器的安装位置，则在所述目标区域的Wi-Fi指纹空间分布图中显示所述路由器的待安装位置。
本申请实施例提供的获取Wi-Fi指纹空间分布的装置，可以执行上述方法实施例中终端设备的动作，其实现原理和技术效果类似，在此不再赘述。
第三方面，本申请实施例提供一种电子设备，该电子设备可以为上述第一方面中的终端设备。所述电子设备包括：处理器、存储器、收发器；所述收发器耦合至所述处理器，所述处理器控制所述收发器的收发动作；其中，存储器用于存储计算机可执行程序代码，程序代码包括指令；当处理器执行指令时，指令使所述电子设备执行如第一方面所提供的方法。
第四方面，本申请实施例提供一种电子设备，包括用于执行以上第一方面可能的设计所提供的方法的单元、模块或电路。该电子设备可以为终端设备，也可以为应用于终端设备的一个模块，例如，可以为应用于终端设备的芯片。
第五方面，本申请实施例提供一种芯片，所述芯片上存储有计算机程序，在所述计算机程序被所述芯片执行时，实现如第一方面所提供的方法。
第六方面，本申请实施例提供一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述第一方面中的方法。
第七方面，本申请实施例提供一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行上述第一方面中的方法。
本申请实施例提供一种获取Wi-Fi指纹空间分布的方法、装置和电子设备，该方法包括：采集当前时间窗口内的目标区域的视频帧、Wi-Fi指纹，以及终端设备的惯性测量单元imu数据；基于视频帧和imu数据，获取终端设备在目标区域移动过程中的位姿；将终端设备在目标区域移动过程中的位姿和终端设备在目标区域移动过程中采集的Wi-Fi指纹按照时间对 齐，得到目标区域的Wi-Fi指纹空间分布。本申请实施例一方面能够避免预先设置外部锚点的位置导致的采集效率低的问题，另一方面终端设备不仅能够精准定位位置，还能够获取准确的设备姿态，提高了Wi-Fi指纹空间分布的准确性。
附图说明
图1为本申请实施例提供的终端设备的结构示意图；
图2为本申请实施例提供的获取Wi-Fi指纹空间分布的方法的一实施例的流程示意图；
图3为本申请实施例提供的终端设备的界面变化示意图；
图4为本申请实施例提供的获取Wi-Fi指纹空间分布的方法的另一实施例的流程示意图；
图5为本申请实施例提供的位姿和点线特征图的示意图；
图6为本申请实施例提供的终端设备的界面示意图；
图7为本申请实施例提供的获取Wi-Fi指纹空间分布的方法的另一实施例的流程示意图；
图8为本申请实施例提供的获取Wi-Fi指纹空间分布的方法的另一实施例的流程示意图；
图9为本申请实施例提供的Wi-Fi指纹空间分布图一；
图10为本申请实施例提供的Wi-Fi指纹空间分布图二；
图11为本申请实施例提供的Wi-Fi指纹空间分布图三；
图12为本申请实施例提供的一种获取Wi-Fi指纹空间分布的装置的结构示意图。
具体实施方式
为了解决目前获取Wi-Fi指纹空间分布的适用性低和采集效率低的问题，本申请提供了一种获取Wi-Fi指纹空间分布的方法，在终端设备在目标区域移动的过程中，采用终端设备采集Wi-Fi指纹，且由终端设备自身在移动过程中进行位姿的精准确定，以得到目标区域的Wi-Fi指纹空间分布。本申请实施例，用户可以通过自己的终端设备就能够获取Wi-Fi指纹空间分布，适用性高，避免了需要采用专用设备的问题。且终端设备不仅精准定位位置，还能够获取准确的姿态，进而提高了Wi-Fi指纹空间分布的准确性。
应理解，本申请实施例中的终端设备可以指用户设备、接入终端、用户单元、用户站、远程终端、移动设备、用户终端、终端、无线通信设备、用户代理或用户装置。终端设备可以是手机(mobile phone)、平板电脑(pad)、带无线收发功能的电脑、个人数字助理(personal digital assistant，PDA)、具有无线通信功能的手持设备、计算机或其它处理设备、可穿戴设备、虚拟现实(virtual reality，VR)终端设备、增强现实(augmented reality，AR)终端设备、智慧家庭(smart home)中的无线终端、智能机器人、5G网络中的终端设备或者未来演进的公用陆地移动通信网络(public land mobile network，PLMN)中的终端设备等。
下述首先对本申请实施例中提供的终端设备的结构进行说明。图1为本申请实施例提供的终端设备的结构示意图。如图1所示，本申请实施例中的终端设备100可以包括：处理器101，存储器102，无线通信模块103，传感器104，摄像头105和显示屏106等。可以理解的是，本实施例示意的结构并不构成对终端设备100的具体限定。在本申请另一些实施例中，终端设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件，或软件和硬件的组合实现。
处理器101可以包括一个或多个处理单元，例如：处理器101可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)， 图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，显示处理单元(display process unit，DPU)，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。在一些实施例中，终端设备100也可以包括一个或多个处理器101。其中，处理器可以是终端设备100的神经中枢和指挥中心。控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。处理器101中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器101中的存储器为高速缓冲存储器。该存储器可以保存处理器101刚用过或循环使用的指令或数据。如果处理器101需要再次使用该指令或数据，可从所述存储器中直接调用。这就避免了重复存取，减少了处理器101的等待时间，因而提高了终端设备100的效率。
在一些实施例中，处理器101可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，用户标识模块(subscriber identity module，SIM)接口，和/或通用串行总线(universal serial bus，USB)接口等。可以理解的是，本发明实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对终端设备100的结构限定。在本申请另一些实施例中，终端设备100也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。
存储器102可以用于存储一个或多个计算机程序，该一个或多个计算机程序包括指令。处理器101可以通过运行存储在存储器102的指令，从而使得终端设备100执行下述实施例中的相关动作。存储器102可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统；该存储程序区还可以存储一个或多个应用程序(比如图库、联系人等)等。存储数据区可存储终端设备100使用过程中所创建的数据(比如照片，联系人等)等。此外，存储器102可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。在一些实施例中，处理器101可以通过运行存储在存储器102的指令，和/或存储在设置于处理器101中的存储器的指令，来使得终端设备100执行各种功能应用及数据处理。
终端设备100的无线通信功能可以通过无线通信模块103实现。无线通信模块103可以提供应用在终端设备100上的包括无线局域网(wireless local area networks，WLAN)，蓝牙，全球导航卫星系统(global navigation satellite system，GNSS)，调频(frequency modulation，FM)，NFC，红外技术(infrared，IR)等无线通信的解决方案。本申请实施例中的无线通信模块103可以为终端设备中的无线网卡。无线通信模块103可以是集成至少一个通信处理模块的一个或多个器件。本申请实施例中的无线通信模块103用于实现电子设备的收发功能，如实现与上述图1中的服务器的通信。
传感器104可以包括压力传感器104A、陀螺仪传感器104B和加速度传感器104C等。其中，压力传感器104A用于感受压力信号，可以将压力信号转换成电信号。在一些实施例中，压力传感器104A可以设置于显示屏106，终端设备100根据压力传感器104A检测所述触摸操作强度。陀螺仪传感器104B可以用于确定终端设备100的运动姿态。在一些实施例中，可以通过陀螺仪传感器104B确定终端设备100围绕三个轴(即，x，y和z轴)的角速度。陀螺仪传感器104B可以用于拍摄防抖。示例性的，当按下快门，陀螺仪传感器104B检测终 端设备100抖动的角度，根据角度计算出镜头模组需要补偿的距离，让镜头通过反向运动抵消终端设备100的抖动，实现防抖。加速度传感器104C可检测终端设备100在各个方向上(一般为三轴)加速度的大小。当终端设备100静止时可检测出重力的大小及方向。还可以用于识别电子设备姿态，应用于横竖屏切换，计步器等应用。
终端设备100可以通过一个或多个摄像头105实现拍摄功能。另外，终端设备100通过显示屏106可以实现显示功能。显示屏106用于显示图像，视频等。显示屏106包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。
其中，本申请实施例中的终端设备100还可以包括：音频模块407，麦克风408。终端设备100可以通过音频模块407、麦克风408实现音频功能，如音乐播放，录音，与用户的语音交互等。其中，音频模块407用于将数字音频信息转换成模拟音频信号输出，也用于将模拟音频输入转换为数字音频信号。音频模块407还可以用于对音频信号编码和解码。在一些实施例中，音频模块407可以设置于处理器101中，或将音频模块407的部分功能模块设置于处理器101中。终端设备100可以设置至少一个麦克风408。在另一些实施例中，终端设备100可以设置两个麦克风408，除了采集声音信号，还可以实现降噪功能。在另一些实施例中，终端设备100还可以设置三个，四个或更多麦克风408，实现采集声音信号，降噪，还可以识别声音来源，实现定向录音功能等。
本申请实施例中，摄像头105可以采集目标区域的视频，陀螺仪传感器104B和加速度传感器104C可以采集终端设备的惯性测量(inertial measurement unit，imu)数据。其中，imu数据中可以包括终端设备的加速度和角速度，陀螺仪传感器104B可以采集终端设备的角速度，加速度传感器104C可以采集终端设备的加速度。处理器401用于执行下述实施例中的图2、图4、图7或图8中所示的动作。基于图1所示的终端设备可以实现下述实施例中的获取Wi-Fi指纹空间分布的方法，具体的技术方案和技术效果参照下述实施例的相关描述。应理解，图1中的结构说明为终端设备的示例，下述实施例中以执行获取Wi-Fi指纹空间分布的方法的执行主体为终端设备进行说明，终端设备中集成有上述图1所示的各模块，能够实现上述各模块的功能。
下面结合具体的实施例对本申请实施例提供的获取Wi-Fi指纹空间分布的方法进行说明。下面这几个实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例不再赘述。图2为本申请实施例提供的获取Wi-Fi指纹空间分布的方法的一实施例的流程示意图。如图2所示，本申请实施例提供的获取Wi-Fi指纹空间分布的方法可以包括：
S201，采集当前时间窗口内的目标区域的视频帧、Wi-Fi指纹，以及终端设备的惯性测量单元imu数据。
S202，基于视频帧和imu数据，获取终端设备在目标区域移动过程中的位姿。
S203，将终端设备在目标区域移动过程中的位姿，以及终端设备在目标区域移动过程中采集的Wi-Fi指纹按照时间对齐，得到目标区域的Wi-Fi指纹空间分布。
上述S201中，应理解，本申请实施例中，终端设备在目标区域移动的过程中可以采集当前时间窗口内的目标区域的视频帧、Wi-Fi指纹，以及终端设备的惯性测量单元imu数据。其 中，目标区域可以但不限于为室内空间。其中，终端设备在目标区域中的移动方式可以为：用户手持终端设备在目标区域中移动，或者终端设备自身在目标区域中移动。示例性的，如终端设备为智能手机时，用户可以手持智能手机在目标区域中移动，使得终端设备在目标区域中移动；或者终端设备为机器人时，机器人可以在目标区域中移动。
应理解，终端设备在目标区域移动的过程中会持续采集目标区域的视频帧、Wi-Fi指纹，以及终端设备的imu数据。而本申请实施例中为了减少终端设备的计算量，可以根据当前时间窗口内的目标区域的视频帧、Wi-Fi指纹，以及终端设备的imu数据，获取终端设备的位姿。应注意，本申请实施例中的时间窗口可以理解为预设时长。示例性的，如以终端设备开始采集目标区域的视频帧为例，每隔预设时长为一个时间窗口。如时间窗口为5s，则从终端设备开始采集目标区域的视频帧，0-5s为一个时间窗口，5s-10s为一个时间窗口，依次类推。
示例性的，时间窗口为5s，终端设备开始移动后，会采集目标区域的视频帧、Wi-Fi指纹，以及终端设备的imu数据。在5s时，0-5s内采集的目标区域的视频、Wi-Fi指纹，以及终端设备的imu数据可以作为当前时间窗口的目标区域的视频帧、Wi-Fi指纹，以及终端设备的imu数据。本申请实施例中的目标区域的视频帧可以为终端设备采集的目标区域的视频中包括的视频帧，或者是终端设备每隔预设时长采集的视频帧，视频帧也可以称为图像帧。其中，Wi-Fi指纹指的是Wi-Fi信号的属性信息，如Wi-Fi信号的强度、频段、Wi-Fi名称等，室内各位置处的Wi-Fi指纹的集合即为室内Wi-Fi指纹空间分布。终端设备的imu数据可以为终端设备中设置的惯性测量单元(inertial measurement unit，IMU)采集的数据。
其中，用户可以触发终端设备开始采集目标区域的视频帧、Wi-Fi指纹，以及终端设备的imu数据。在一种可能的场景中，终端设备上设置有获取Wi-Fi指纹空间分布的控件(如应用程序中的功能)，或者终端设备上设置有获取Wi-Fi指纹空间分布的应用程序，用户点击该控件或者打开该应用程序可以触发终端设备采集上述目标区域的视频帧、Wi-Fi指纹，以及终端设备的imu数据。
图3为本申请实施例提供的终端设备的界面变化示意图。如图3中的界面301所示，该界面301上显示有“Wi-Fi指纹空间分布”的应用程序的图标，用户点击该图标，界面301跳转至界面302，该界面302可以类似于智能手机录像界面。值得注意的是，界面301跳转至界面302后，即触发终端设备开始拍摄视频(即采集视频帧)、同时读取终端设备的imu数据，以及获取Wi-Fi指纹。或者，界面301跳转至界面302后，界面302上显示有录像控件，用户可以点击该录像控件，触发终端设备开始拍摄视频、同时读取终端设备的imu数据，以及获取Wi-Fi指纹。其中，用户可以在触发终端设备开始拍摄视频时，用户在目标区域内移动，以使得终端设备在目标区域内移动。
在另一种可能的场景中，用户可以触发终端设备在目标区域移动，且同时采集目标区域的视频帧、Wi-Fi指纹，以及终端设备的imu数据。示例性的，终端设备上具有触发控件，用户选择该触发控件即可触发。或者，用户可以与终端设备进行语音交互进行触发，如用户可以说“去采集下Wi-Fi指纹空间分布吧”进行触发。应理解，在上述两种场景中，终端设备的Wi-Fi功能开关可以预先处于打开状态。或者，在用户触发终端设备采集目标区域的视频、Wi-Fi指纹，以及终端设备的imu数据时，终端设备打开该Wi-Fi功能开关，以实现对Wi-Fi指纹的采集。
本申请实施例中，终端设备可以按一定频率(如30Hz)获得目标区域的视频帧(即单目RGB图像序列)，同时启动imu读取功能(如200Hz)，获取imu数据，且启动Wi-Fi扫描功能(如0.3Hz)，获取Wi-Fi指纹。其中，imu数据可以包括终端设备的角速度和加速度。Wi-Fi 指纹可以包括Wi-Fi信号的强度、频段、以及Wi-Fi名称(如ESS ID)、Wi-Fi通道(channel)、Wi-Fi的物理地址(adress)，以及Wi-Fi信号是否为加密类型等。
上述S202中，目前现有技术中提供一种采用图像对终端设备进行定位的方法，其中，通过RGB-D深度相机进行标定，获得RGB-D深度相机的内参信息，之后通过该深度相机获取深度图像，进而根据该深度图像对终端设备进行定位。该方法中采用RGB-D深度相机的成本高、适用范围小，很多终端设备中并未设置RGB-D深度相机。而本申请实施例中采用终端设备上设置的RGB相机，获取目标区域的RGB视频，再结合终端设备imu数据可以实现终端设备的定位，节省了成本，扩展了方法的适用范围。
本申请实施例中，基于视频和imu数据可以对终端设备进行定位，即获取终端设备在目标区域移动过程中的位姿，其中终端设备具体是根据视频帧的特征和imu数据可以对终端设备进行定位。应理解，视频帧的特征可以为视频的点特征和/或线特征，点特征也可以称为特征点。终端设备可以根据视频帧的特征，确定终端设备的相对移动信息。示例性的，终端设备可以根据终端设备采集的前后两个视频帧中的点特征的位置的变化，确定终端设备的相对移动信息。如终端设备在开始采集视频帧时的位置为A，则可以根据该终端设备的相对移动信息，确定终端设备在采集每个视频帧时的实时位置。另外，鉴于imu数据可以表征终端设备在移动过程中运动姿态，如角速度和加速度。本申请实施例中根据终端设备在目标区域移动过程中运动姿态，确定终端设备在目标区域移动过程中的姿态。综上，终端设备既可以获取终端设备在目标区域移动过程中的实时位置以及运动姿态，则可以获取终端设备在目标区域移动过程中的位姿。
应理解，因为本申请实施例中终端设备采集的数据为当前时间窗口内的数据，则相对应的，终端设备可以根据当前时间窗口内的目标区域的视频、Wi-Fi指纹，以及终端设备的imu数据，获取终端设备在当前时间窗口内的位姿。进而，本申请实施例中可以将每个时间窗口内的终端设备的位姿的集合作为终端设备在目标区域移动的过程中的位姿。
上述S203中，本申请实施例中，终端设备可以以相同的采集频率采集目标区域的视频帧、Wi-Fi指纹，以及终端设备的imu数据，在该种场景下，终端设备可以根据目标区域的视频帧和imu数据获取终端设备的实时位姿(即按照相同的频率获取终端设备的位姿)。终端设备可以将实时位姿与采集目标区域的imu数据时采集的Wi-Fi指纹对应存储，可以得到目标区域的Wi-Fi指纹空间分布。
示例性的，上述的采集频率为20Hz，即每隔0.05s采集一次图像(即一个视频帧)、一次Wi-Fi指纹、一次imu数据。在0.05s时，终端设备可以根据采集的视频帧和imu数据获取终端设备在0.05s的位姿，进而将0.05s时的终端设备的位姿和0.05s时采集的Wi-Fi指纹对应存储。按照相同的方式，可以得到目标区域的Wi-Fi指纹空间分布。
为了更为准确的获取终端设备的位姿，可以将视频帧和imu数据的采集频率提高，而Wi-Fi指纹的采集频率可以相对较低。也就是说，本申请实施例中终端设备可以以不同的采集频率采集目标区域的视频帧、Wi-Fi指纹，以及终端设备的imu数据。在该种场景下，可以将终端设备在移动过程中的位姿和终端设备在移动过程中采集的Wi-Fi指纹按照时间对齐，得到目标区域的Wi-Fi指纹空间分布。例如，本申请实施例中可以将采集时间在预设时间范围的Wi-Fi指纹与终端设备的位姿对齐。
示例性的，预设时间范围可以为0.02s。在0.05s时，终端设备可以根据采集的视频帧和imu数据获取终端设备在0.05s的位姿，而在0.05s时终端设备并未采集Wi-Fi指纹，而是在0.06s采集Wi-Fi指纹，因此可以将0.05s时终端设备的位姿和0.06s采集的Wi-Fi指纹对应存储。
本申请实施例中，为了提高目标区域的Wi-Fi指纹空间分布的准确性，可以采用三次样条插值法(spline)，将终端设备在移动过程中的位姿和采集的Wi-Fi指纹对齐。其中，三次样条插值法为终端设备中预先设置的插值函数。示例性的，终端设备可以以时间和各时间处采集的Wi-Fi指纹为参数，采用该插值函数计算0.05s时的Wi-Fi指纹，进而将该计算得到的0.05s时的Wi-Fi指纹与0.05s时终端设备的位姿对应存储。
本申请实施例提供的获取Wi-Fi指纹空间分布的方法包括：在终端设备在目标区域移动的过程中，采集当前时间窗口内的目标区域的视频帧、Wi-Fi指纹，以及终端设备的imu数据，基于视频帧和imu数据，获取终端设备在移动过程中的位姿；将终端设备在移动过程中的位姿和终端设备在移动过程中采集的Wi-Fi指纹按照时间对齐，得到目标区域的Wi-Fi指纹空间分布。本申请实施例中，在终端设备在目标区域移动的过程中采集Wi-Fi指纹，且由终端设备根据imu数据和视频在移动过程中进行位姿的精准确定，以得到目标区域的Wi-Fi指纹空间分布。本申请实施例中，用户可以通过自己的终端设备就能够获取Wi-Fi指纹空间分布，适用性高，避免了需要采用专用设备的问题，且该方法能够避免预先设置外部锚点的位置导致的采集效率低的问题，鉴于本申请实施例中的终端设备不仅精准定位位置，还能够获取准确的姿态，因此还能够提高了Wi-Fi指纹空间分布的准确性。
在上述实施例的基础上，下述结合实施例对本申请实施例中终端设备进行定位的过程进行详细说明。图4为本申请实施例提供的获取Wi-Fi指纹空间分布的方法的另一实施例的流程示意图。如图4所示，本申请实施例提供的获取Wi-Fi指纹空间分布的方法可以包括：
S401，采集当前时间窗口内的目标区域的视频帧、Wi-Fi指纹，以及终端设备的imu数据。
S402，根据imu数据，获取终端设备在采集视频帧时的相对移动信息。
S403，获取视频帧的特征，视频帧的特征包括视频帧的点特征和/或线特征。
S404，根据相对移动信息、视频帧的特征，获取终端设备采集视频帧时的位姿，以得到终端设备在目标区域移动过程中的位姿。
S405，将终端设备在目标区域移动过程中的位姿和终端设备在目标区域移动过程中采集的Wi-Fi指纹按照时间对齐，得到目标区域的Wi-Fi指纹空间分布。
应理解，本申请实施例中的S401、S405中的实施方式可以参照上述实施例中S201和S203中的相关描述，在此不做赘述。
上述S402中，终端设备在移动的过程中的位姿是实时变化的，也就是说终端设备在采集视频时的位姿是实时变化的。本申请实施例中的imu数据用于表征终端设备的移动过程，且因为视频是由连续的多个视频帧组成，因此本申请实施例中可以根据终端设备采集相邻两个视频帧时的imu数据，获取终端设备在采集视频帧时的位姿的变化。
也就是说，本申请实施例中可以根据该视频帧和前一个视频帧之间的终端设备采集的imu数据，得到相对移动信息，相对移动信息表征终端设备采集视频帧时相较于采集前一个视频帧时的移动情况。应理解，相对移动信息表征终端设备在采集两个视频帧中后一个视频帧时，相对于前一个视频帧的相对移动信息。
应理解，本申请实施例中的imu数据包括终端设备的角速度和加速度。为了提高获取终端设备的相对移动信息的效率和精度，本申请实施例中可以对相邻两个视频帧之间采集的imu数据进行积分处理。
本申请实施例中t时刻采集的imu数据中的角速度
Figure PCTCN2021085972-appb-000001
和加速度
Figure PCTCN2021085972-appb-000002
分别可如下公式一和公式二所示：
Figure PCTCN2021085972-appb-000003
Figure PCTCN2021085972-appb-000004
其中，w t为t时刻的真实角速度值，a t为t时刻的真实加速度值，
Figure PCTCN2021085972-appb-000005
为陀螺仪传感器的加速度计随机游走误差，
Figure PCTCN2021085972-appb-000006
为加速度传感器的随机游走误差，n w为角速度的测量噪声，n a为加速度的测量噪声，
Figure PCTCN2021085972-appb-000007
n w和n a均服从均值为0的高斯分布。应理解，本申请实施例中t时刻采集的角速度和加速度分别为
Figure PCTCN2021085972-appb-000008
和
Figure PCTCN2021085972-appb-000009
上述公式是为了表征传感器中计算获取角速度和加速度的过程。
应理解，本申请实施例中终端设备的相对移动信息可以包括相对平移、相对旋转和相对速度。对于两个视频帧，如视频帧k和视频帧k+1，终端设备在采集该相邻两个视频帧时，采集了若干帧imu数据。本申请实施例中，终端设备将imu数据进行积分可得到视频帧k到视频帧k+1帧的相对平移
Figure PCTCN2021085972-appb-000010
相对速度
Figure PCTCN2021085972-appb-000011
和用四元数表示的相对旋转
Figure PCTCN2021085972-appb-000012
相对平移、相对速度和相对旋转可分别如下公式三、公式四和公式五表示：
Figure PCTCN2021085972-appb-000013
Figure PCTCN2021085972-appb-000014
Figure PCTCN2021085972-appb-000015
其中，
Figure PCTCN2021085972-appb-000016
为四元数
Figure PCTCN2021085972-appb-000017
对应的旋转矩阵，
Figure PCTCN2021085972-appb-000018
由
Figure PCTCN2021085972-appb-000019
及其反对称矩阵构成，
Figure PCTCN2021085972-appb-000020
均采用一阶偏导数组成的雅克比矩阵近似后进行数值积分计算。
通过以上步骤，本申请实施例可以获取由于终端设备的运动导致的相邻两个视频帧之间的相对平移、相对旋转和相对速度，即该相对平移、相对旋转和相对速度可以表征终端设备在拍摄两个相邻的视频帧时的位姿的相对平移、相对旋转和相对速度，分别用
Figure PCTCN2021085972-appb-000021
表示。应理解，本申请实时中采用上述公式三-公式五获取的相对平移、相对旋转和相对速度可以为相邻两个视频帧中后一个视频帧的相对移动信息。
上述S403中，点特征用于表征视频帧中局部性的特征，一个视频帧可以通过多个点特征进行表示。应理解，检测视频帧中的线特征，可以为检测视频帧中梯度变化较大的像素。示例性的，轮廓是视频帧中的某些特殊区域，对于灰度图来说，在这些特殊区域，灰度从黑到白或者从白到黑的剧烈变化。简单理解来说，线特征可以为视频帧中的一组像素的集合，用于表征视频帧的特征。其中，视频帧中的点特征和线特征均可以为视频帧中的对象的抽象表征，视频帧中的点特征和线特征均为在视频帧中的对象中抽离获取的，用于表征该视频帧中的对象。其中，对象可以为视频帧中的物体、人物等。
应理解，本申请实施例中采用视频帧的点特征和线特征，相较于采用点特征或线特征的方式，因为获取的视频帧中的特征增多，因此可以更为准确地对视频帧中的对象进行定位，进而能够更为准确的获取终端设备采集视频帧时的位姿。应理解，下述实施例中以视频帧的特征包括视频帧的点特征和线特征为例进行说明，当视频帧的特征包括视频帧的点特征或线特征时，可以将下述实施例中的“点特征和线特征”替换为“点特征或线特征”，且执行包含有点特征或线特征的步骤。
下面首先对终端设备获取视频帧中的点特征的过程进行说明：
本申请实施例中，终端设备可以采用Fast角点检测算法，获取视频帧中的点特征。其中，角点是一种局部特征，具有旋转不变性和不随光照条件变化而变化的特点，一般将视频帧中曲率足够高或者曲率变化明显的点作为角点。应理解，本申请实施例中可以预先设置点特征 检测参数，点特征检测参数可以包括：最大角点的数量(max corners)、角点之间的最小距离(min distance)，角点质量等级(quality level)，最少的点特征的个数(min point features)等。
终端设备可以在第一个视频帧中，采用Fast角点检测算法，获取第一个视频帧中的点特征。其中，第一个视频帧满足上述预先设置点特征检测参数。也就是说，第一个视频帧满足如下条件：第一个视频帧中的角点的数量小于或等于第二数量阈值；以及，第一个视频帧的点特征的数量大于或等于第一数量阈值；以及，第一个视频帧中的角点之间的距离大于或等于第一距离阈值；以及，第一个视频帧中的角点的质量等级大于预设等级。应理解，第一个视频帧为终端设备采集的第一个视频帧。应理解，若在一视频帧中采集的点特征过少，则点特征对该视频帧的特征描述的准确度可能较低，但若在一视频帧中采集的点特征过多，则加大了导致终端设备的计算量，因此，本申请实施例中可以预先设置第一数量阈值，可以在保证视频帧的特征的描述的准确性的基础上，减少终端设备的计算量。
在第二个视频帧中，终端设备可以使用光流算法(如Lucas-Kanade光流算法)，追踪第一个视频帧的点特征。也就是说，终端设备可以采用LK光流算法将第一个视频帧中的点特征跟踪到第二个视频帧中，为了保证第二个视频帧中的点特征的数量大于第一数量阈值，若在第二个视频帧中可以追踪第一个视频帧中的全部点特征，则可以继续在第三个视频帧中追踪第二个视频帧中的点特征。若在第二个视频帧中追踪的第一个视频帧中的点特征的数量小于第一数量阈值，则可以在第二个视频帧中提取新的点特征，并将在第二个视频帧中追踪得到的点特征和提取的新的点特征作为第二个视频帧的点特征，并在第三个视频帧中继续追踪第二个视频帧中的点特征。应注意，本申请实施例中，采用点特征追踪的方法，而并非点特征匹配的方式在每个视频帧中获取点特征，可以减少计算量，提高获取点特征的速度。
其中，本申请实施例中可以在追踪点特征的过程中，可以获取追踪错误的点特征，且在视频帧中删除追踪错误的点特征，以提高点特征的追踪准确性。在一种可能的实现方式中，本申请实施例中可以基于随机抽样一致(random sample consensus，RANSAC)算法，通过本质矩阵(essential matrix)测试获取追踪错误的点特征，且删除追踪错误的点特征。
其次对终端设备获取视频帧中的线特征的过程进行说明：
本申请实施例中，终端设备采用线特征匹配的方法获取视频帧中的线特征。其中，终端设备可以采用直线段检测(line segment detector，LSD)算法检测获取视频帧中的线特征，并使用线条带描述子(line band discriptor，LBD)算法对视频帧中的线特征进行描述，进而得到视频帧中的线特征的LBD描述子。
本申请实施例中可以根据将视频帧的线特征与前一个视频帧的线特征进行匹配，建立初始线对。其中，初始线对指示在视频帧和前一个视频帧中匹配成功的两个线特征，应理解，视频帧和前一个视频帧中匹配成功的两个线特征可以为：线特征之间的距离小于预设距离的两个线特征。其中，线特征之间的距离可以为线特征之间的欧式距离或余弦相似度等。
示例性的，终端设备获取第一个视频帧中的线特征，以及第二个视频帧中的线特征，且获取第一个视频帧中的LBD描述子，以及第二个视频帧中的LBD描述子，进而将第二个视频帧中的LBD描述子与第一个视频帧中的LBD描述子进行匹配，且将匹配成功的线对作为初始线对。应理解，此处对第二个视频帧中的LBD描述子与第一个视频帧中的LBD描述子进行匹配的方式可以为采用上述获取线特征之间的距离的方式。
本申请实施例中，在匹配线特征的过程中，难免存在匹配错误的情况。为了保证匹配的初始线对的准确性，终端设备可以在初始线对中，删除错误的线对，以得到视频帧中的线特 征。其中，本申请实施例中可以采用下述方式确定初始线对中错误的线对。
终端设备可以获取线对中两个线特征的距离Δr、线段长度差Δd以及线段夹角
Figure PCTCN2021085972-appb-000022
应理解，视频帧中的线特征在视频帧中可以以线段的方式表示，本申请实施例中可以将线段之间的距离作为两个线特征的距离。其中，终端设备可以将两个线特征的距离大于第二距离阈值的线对作为错误的线对，或者，可以将两个线特征的线段长度差大于长度差阈值的线对作为错误的线对，或者，可以将两个线特征的线段夹角大于第二角度阈值的线对作为错误的线对。也就是说，本申请实施例中，终端设备可以在初始线对中，删除两个线特征的距离大于第二距离阈值的线对；以及，删除两个线特征的线段长度差大于长度差阈值的线对；以及，删除两个线特征的线段夹角大于第二角度阈值的线对，进而得到第二个视频帧中的线特征。应理解，本申请实施例中可以采用相同的方式，获取第三个视频帧、第四个视频帧…...最后一个视频帧的线特征。
应理解，本申请实施例中终端设备获取视频帧中的点特征和线特征的步骤可以同时执行。
上述S404中，本申请实施例中，终端设备在获取视频帧的相对移动信息、视频帧中的特后，可以获取终端设备在采集该视频帧时的位姿，进而根据该种方式，得到终端设备在目标区域移动过程中的位姿。
应理解，本申请实施例中的第一种方式，终端设备在目标区域移动过程中的位姿可以包括终端设备在采集每个视频帧时的位姿。在该种方式中，终端设备可以获取视频帧之前相邻的关键帧的位姿，进而根据视频帧之前相邻的关键帧的位姿，以及视频帧和视频帧之前相邻的关键帧之间的相对位移信息，确定终端设备采集该视频帧时的位姿。其中，该视频帧之前的相邻关键帧可以为该视频帧之前的、距离该视频帧时间最近的一个关键帧。其中，本申请实施例中获取视频帧之前相邻的关键帧的位姿的方式可以参照下述第二种方式中的相关描述。
为了更为清楚地说明本申请实施例中获取终端设备的位姿的方法，下面先对本申请实施例中的关键帧进行说明：
依据上述，本申请实施例中的相对移动信息包括：相对位移和相对角度，终端设备可以根据视频帧的相对位移、相对角度以及视频帧与前一个关键帧之间的时间差，确定视频帧是否为关键帧。其中，本申请实施例中的关键帧满足如下至少一项条件：相对位移大于第一位移阈值；或者，相对角度大于第一角度阈值；或者，时间差大于时间差阈值。
也就是说，若视频帧的相对位移大于第一位移阈值，或者视频帧的相对角度大于第一角度阈值，或者视频帧与前一个关键帧之间的时间差大于时间差阈值，则确定该视频帧为关键帧。应理解，本申请实施例中可以在采集到每个视频帧后，可以根据每个视频帧的相对位移信息，以及上一个关键帧的时间，确定该视频帧是否为关键帧。
在第一种方式中，终端设备可以根据视频帧和视频帧之前相邻的关键帧之间的相对位移信息，获取终端设备在采集视频帧时的位姿，进而在终端设备采集前一个关键帧时的位姿的基础上，根据该相对位移信息中的相对位移和相对角度，可以得到终端设备采集该视频帧时的位姿。应注意，本申请实施例中获取视频帧和视频帧之前相邻的关键帧之间的相对位移信息的方式，可以参照上述获取相邻两个视频帧之间的相对位移信息的相关描述。也就是说，本申请实施例中可以将视频帧和视频帧之前相邻的关键帧作为相邻的两个视频帧，进而根据上述公式三-公式五获取相对位移信息。应理解，终端设备在采集该前一个关键帧时的位姿的获取方式可以参照下述另一种可能的实现方式中的相关描述。
示例性的，终端设备采集了5个视频帧，第3个视频帧为关键帧，第4个视频帧和第5个视频帧均为非关键帧，终端设备在获取采集第4个视频帧和第5个视频帧时的位姿的方式 相同，下述以终端设备采集第5个视频帧时的位姿进行说明。本申请实施例中，终端设备根据第5个视频帧和第3个视频帧作为相邻的两个视频帧，进而根据上述公式三-公式五获取第5个视频帧和第3个视频帧之间的相对位移信息。终端设备在采集第3个视频帧时的位姿的基础上，加上相对位移信息中的相对位移和相对角度，可以得到终端设备采集第5个视频帧时的位姿。应理解，终端设备在采集该第3个视频帧(即关键帧)时的位姿的获取方式可以参照下述另一种可能的实现方式中的相关描述。
本申请实施例的第二种方式中，为了减少终端设备的计算量，终端设备在移动过程中的位姿可以包括终端设备在采集关键帧时的位姿。也就是说，相较于上一种可能的实现方式，该方式中终端设备仅计算获取终端设备在采集关键帧时的位姿。依据上述，终端设备可以获取该视频帧是否为关键帧，在确定视频帧为关键帧时，计算采集该关键帧时的位姿。应理解，本申请实施例中可以在确定当前时间窗口内的关键帧后，根据该当前时间窗口内的关键帧的相对位移信息、关键帧的点特征和线特征，获取当前时间窗口内采集关键帧时的终端设备的位姿。
本申请实施例中，终端设备可以以关键帧的相对移动信息、关键帧的点特征和关键帧的线特征作为参数，构建待估计的状态向量，进而建立最大后验概率目标函数，且将待估计的状态向量代入最大后验概率目标函数中，对最大后验概率目标函数进行求解，得到终端设备在当前时间窗口内采集关键帧时的初始位姿。
其中，示例性的，如当前时间窗口内的关键帧为n帧。则待估计的状态向量可以如下述公式六表示：
χ＝[x 0,x 1,…,x N,λ 0,λ 1,…,λ M,μ 0,μ 1,…,μ K]     公式六
其中，x n为第n个关键帧的运动数据，包含第n个关键帧的相对位移
Figure PCTCN2021085972-appb-000023
相对速度
Figure PCTCN2021085972-appb-000024
和相对角度
Figure PCTCN2021085972-appb-000025
及随机游走误差
Figure PCTCN2021085972-appb-000026
和
Figure PCTCN2021085972-appb-000027
λ m为第n个关键帧中第m个点特征的逆深度，μ k为第n个关键帧中第k个线特征的正交表示。应理解，逆深度为一种参数化方法，即认为深度的倒数服从高斯分布，正交表示是线特征的一种表示方法。
本申请实施例中终端设备可以构建最大后验概率目标函数，如下公式七所示：
Figure PCTCN2021085972-appb-000028
其中，r p为先验误差项，在边缘化时由高斯消元法获得；r b为关键帧的运动数据的误差项，
Figure PCTCN2021085972-appb-000029
为关键帧的运动数据的观测；r f为点特征重投影误差项，
Figure PCTCN2021085972-appb-000030
为第j个点特征在第i个关键帧的预测值，r l为线特征重投影误差项，
Figure PCTCN2021085972-appb-000031
为第l个线特征在第i帧的预测值；||||为L2范数，ρ()为对离群点鲁棒的核函数，min为最小化函数；B为关键帧的运动数据的序列，F为点特征序列，L为线特征序列。应理解，关键帧的运动数据的序列指的是，每个关键帧的运动数据的集合，关键帧序列指的是关键帧的集合，点特征序列指的是每个关键帧中的点特征的集合，线特征序列指的是每个关键帧中的线特征的集合。
本申请实施例中，可以将当前时间窗口内的关键帧构建的待估计的状态向量代入最大后验概率目标函数中，利用Ceres库中的LM迭代优化算法对最大后验概率目标函数进行求解，得到满足最大后验概率的X，即终端设备在采集每个关键帧时的初始位姿。应理解，本申请实施例中可以将根据上述公式七获取的终端设备在采集该关键帧之后相邻的非关键帧时的初始位姿，作为终端设备在采集该关键帧之后相邻的非关键帧时的位姿。
应理解，本申请实施例中对如何求解最大后验概率目标函数的过程不做赘述，具体可以参照现有技术中的相关描述。应注意的是，本申请实施例中，终端设备可以不获取第一个关键帧之前的非关键帧的位姿，且在获取第一个关键帧的位姿时，可以获取终端设备在采集该关键帧之后相邻的非关键帧时的位姿，具体可以参照上述可能的实现方式中的相关描述。
可选的，本申请实施例中，终端设备在根据上述公式七获取终端设备采集关键帧时的位姿后，可以存储终端设备采集关键帧时的位姿和点线特征图。图5为本申请实施例提供的位姿和点线特征图的示意图。图5所示的为终端设备在当前时间窗口获取采集关键帧时的位姿，以及当前时刻已经获取的视频帧中的点特征和线特征的集合。应理解，该图示中还显示有当前时间窗口之前的时间窗口内终端设备采集关键帧时的位姿、点特征和线特征。其中，图5中所示的线段即为线特征，点即为点特征，虚线框所示的为终端设备在当前时间窗口获取采集当前关键帧时的位姿。其中，终端设备的位姿可以由6个自由度表示，6个自由度可以包括终端设备的空间位置和空间角度(也称为空间姿态)，空间位置如终端设备的三维坐标(x，y，z)，空间角度如终端设备的偏航角(yaw角)α、滚动角(raw角)β以及俯仰角(pitch角)θ。当前关键帧可以指的是当前时间窗口内最后一个关键帧。
应理解，本申请实施例中既可以获取终端设备在采集每个视频帧时的位姿，也可以获取终端设备在采集关键帧时的位姿，至于终端设备是获取采集每个视频帧时的位姿还是以获取终端设备在采集关键帧时的位姿，用户可以自定义设置。图6为本申请实施例提供的终端设备的界面示意图。如图6所示，在上述界面302的基础上，该界面上还可以显示有选择控件，选择控件用于指示用户选择终端设备获取采集每个视频帧时的位姿还是以获取终端设备在采集关键帧时的位姿。如图6所示的界面上显示有“关键帧”和“视频帧”，若用户需求快速获取目标区域的Wi-Fi指纹空间分布，则可以选择“视频帧”(即终端设备只获取采集关键帧时的位姿)，若用户需求获取更为详细的目标区域的Wi-Fi指纹空间分布，则可以选择“视频帧”(即终端设备只获取采集每个视频帧时的位姿)。
本申请实施例中，用户可以自定义确定终端设备获取采集每个视频帧时的位姿，或者获取采集关键帧时的位姿，可以提高灵活性。且本申请实施例中采用追踪的方式获取视频帧中的点特征，相对于采用点特征匹配的方式可以减少终端设备的计算量，另外若终端设备获取采集关键帧时的位姿，不必要对每个视频帧进行位姿的计算，也能够减少终端设备的计算量，且本申请实施例中采用关键帧中的点线特征，以及相对位移信息，采用不断迭代的优化算法，能够提高获取的终端设备的位姿的准确性。另，本申请实施例中的其他有益效果可以参照上述实施例中的相关描述。
上述实施例中，终端设备在根据公式七获取终端设备在采集关键帧时的初始位姿，作为终端设备在采集关键帧时的位姿。本申请实施例中，为了进一步提高终端设备在采集关键帧时的位姿，可以对当前时间窗口内的关键帧进行闭环检测，以及时空一致性校验，根据闭环检测结果，以及时空一致性校验结果，对初始位姿进行处理，以得到准确性更高的终端设备采集关键帧时的位姿。
其中，如果闭环检测成功，可以显著地减小因为上述公式三-公式五因为不断积分导致的累积漂移误差，进而提高位姿的准确性。当终端设备快速运动时，会存在由于视频帧运动模糊而提取不到足够多的特征点而失效的问题，或者当目标区域中存在运动的人或物时，由于不能区分是终端设备的运动还是运动物体的运动而导致位姿估计不准，甚至失效，因此若闭环检测成功后，不进行时空一致性校验，则上述方法仅能应用于慢速平稳运动的静态场景， 而不适用于快速旋转、动态物体的场景，鲁棒性差。而本申请实施例中能够在闭环检测成功后，对关键帧进行时空一致性校验，使得本申请实施例中的方法在终端设备快速旋转、目标区域包含有动态物体、且目标区域存在弱纹理、光照变化等复杂环境时，具有很好的鲁棒性。
相对应的，在上述另一种可能实现的方式中，本申请实施例中可以根据当前时间窗口内的当前关键帧的点特征和线特征，以及历史关键帧的点特征和线特征，进行闭环检测，以及对当前关键帧进行时空一致性校验。终端设备根据终端设备的初始位姿、闭环检测结果、以及当前关键帧的校验结果，获取终端设备采集当前关键帧时的位姿，以得到终端设备在移动过程中的位姿。其中，历史关键帧为：终端设备在目标区域移动时当前时间窗口之前的时间窗口内的关键帧。示例性的，当前时间窗口为5s-10s，则历史关键帧可以为时间窗口为0s-5s中的关键帧。
下述首先对本申请实施例中的闭环检测过程进行说明：
以当前时间窗口内的当前关键帧为例，本申请实施例中，终端设备可以获取当前关键帧的点特征的描述子，进而根据当前关键帧的点特征的描述子，构建第一当前向量。另外，终端设备还根据历史关键帧的点特征的描述子构建第一历史向量。应理解，在构建第一当前向量时，终端设备采用当前关键帧中的所有的点特征的描述子构建该第一当前向量。在构建第一历史向量时，终端设备采用一个历史关键帧中的所有的点特征的描述子构建该第一历史向量。应理解，点特征的描述子用于描述点特征，如描述子可以将点特征以矩阵、二进制等方式进行表示。
进一步的，终端设备还可以根据当前关键帧的线特征的描述子，构建第二当前向量，对应的，终端设备还根据历史关键帧的线特征的描述子构建第二历史向量。相应的，在构建第二当前向量时，终端设备采用当前关键帧中的所有的线特征的描述子构建该第二当前向量。在构建第二历史向量时，终端设备采用一个历史关键帧中的所有的线特征的描述子构建该第二历史向量。应理解，线特征的描述子用于描述线特征，如描述子可以将线特征以矩阵、二进制等方式进行表示。
应理解，本申请实施例中可以基于词袋模型(bag of words，BOW)，构建当前向量(第一当前向量和第二当前向量)和历史向量(第一历史向量和第二历史向量)，相应的，本申请实施例中的“向量”可以替换为“词袋向量”。应注意，本申请实施例中还可以采用其他模型提取点特征线特征的向量。应注意的是，当上述实施例中采用视频帧中的点特征或线特征获取终端设备的位姿时，本申请实施例在进行闭环检测时，仅执行点特征或线特征对应的步骤即可。
本申请实施例中，终端设备获取第一当前向量与第一历史向量的相似度，且将相似度大于相似度阈值的第一历史向量对应的关键帧作为第一候选帧，且终端设备获取第二当前向量与第二历史向量的相似度，且将相似度大于相似度阈值的第二历史向量对应的关键帧作为第二候选帧。其中，终端设备可以根据第一候选帧的相似度和第二候选帧的相似度，进行闭环检测，以及对当前关键帧进行时空一致性校验。应理解，相似度阈值为预先设置的。
其中，终端设备可以对第一候选帧的相似度和第二候选帧的相似度进行加权处理，得到加权分值。应理解，本申请实施例中预先对第一候选帧的相似度和第二候选帧的相似度设置有对应的权重，进而根据第一候选帧的相似度的第一候选帧的相似度的权重的乘积，以及第二候选帧的相似度的第二候选帧的相似度的权重的乘积，得到加权分值。
示例性的，如第一候选帧的相似度为S p，第二候选帧的相似度为S l，则加权分值S pl可如下述公式八表示：
Figure PCTCN2021085972-appb-000032
其中，n p和n l分别为当前关键帧中的点特征和线特征的个数；λ为超参数，λ≥1，且λ越大，线特征的权重越大，表示目标区域的结构化程度越高。
本申请实施例中，若加权分值大于或等于分值阈值，则闭环检测通过；若加权分值小于分值阈值，则闭环检测未通过。
在闭环检测通过后，终端设备可以对关键帧进行时空一致性校验，下述对本申请实施例中对当前关键帧进行时空一致性校验的过程进行说明：
时空一致性校验包括空间一致性校验：获取第一候选帧和当前关键帧之间的相对位移和相对角度。应理解，本申请实施例中可以根据如上公式三-公式五获取第一候选帧和当前关键帧之间的相对位移和相对角度，在此不做赘述。
其中，若第一候选帧和当前关键帧之间的相对位移小于第二位移阈值，以及相对角度小于第三角度阈值，则确定当前关键帧通过空间一致性校验。
时空一致性校验还包括时间一致性校验：若当前关键帧的前N个关键帧的加权分值均大于分值阈值，则确定当前关键帧通过时间一致性校验，N为大于或等于1的整数。应理解，本申请实施例中，终端设备可以在每确定一视频帧为关键帧时，获取该关键帧的加权分值，进而在该关键帧的前N个关键帧的加权分值均大于分值阈值，则确定该关键帧通过时间一致性校验。
相对应的，本申请实施例中终端设备可以根据终端设备的初始位姿、闭环检测结果、以及当前关键帧的校验结果，获取终端设备在移动过程中的位姿。
其中，若闭环检测未通过，或者闭环检测通过且当前关键帧未通过时空一致性校验，则将终端设备的初始位姿和终端设备采集历史关键帧时的位姿作为终端设备在移动过程中的位姿。因为根据上述公式七获取的为当前时间窗口的终端设备的位姿，且应理解，在当前时间窗口之前的时间窗口内的终端设备的位姿也可以通过上述公式七获取。本申请实施例中，若闭环检测未通过，或者闭环检测通过且当前关键帧未通过时空一致性校验，终端设备可以将由上述公式七获取的初始位姿作为当前时间窗口内的终端设备的位姿，也就是说，终端设备在目标区域移动的过程中的位姿包括该初始位姿和终端设备采集历史关键帧时的位姿。应理解，终端设备采集历史关键帧时的位姿可以根据本申请实施例中的方法获取，即将当前时间窗口之前的时间窗口作为当前时间窗口。
其中，若闭环检测通过，且当前关键帧通过时空一致性校验，则融合当前关键帧的点特征与第一候选帧的点特征，以及融合当前关键帧的线特征与第二候选帧的线特征。终端设备通过融合后的点特征和融合后的线特征，对终端设备的初始位姿以及终端设备采集历史关键帧时的位姿进行校正，获取终端设备在移动过程中的位姿。
本申请实施例中，终端设备对通过融合后的点特征和融合后的线特征，对终端设备的初始位姿以及终端设备采集历史关键帧时的位姿进行校正的方式可以为：终端设备可以将当前关键帧中的点线特征集合(即点特征和线特征的集合){P n，L n}和第一候选帧中的点线特征集合{P h，L h}进行匹配，P n为当前关键帧中的点特征，L n为当前关键帧中的线特征，P h为第一候选帧中的点特征，L h为历史关键帧中的线特征。应注意，本申请实施例中可以融合匹配的线特征和点特征，通过代价函数进行最小化重投影误差，以对终端设备的初始位姿以及终端设备采集历史关键帧时的位姿进行校正，进而得到矫正后的终端设备位姿。本申请实施例中可以将终端设备的矫正后位姿，作为述终端设备在移动过程中的位姿，以提高终端设备的位 姿的准确性。
本申请实施例中，终端设备进行闭环检测，可以显著地减小因为积分导致的累积漂移误差，进而提高位姿的准确性。还可以在闭环检测通过后，对关键帧进行时空一致性校验，使得本申请实施例中的方法在终端设备快速旋转、目标区域包含有动态物体、且目标区域存在弱纹理、光照变化等复杂环境时，具有很好的鲁棒性。
在一种可能的实现方式中，图7为本申请实施例提供的获取Wi-Fi指纹空间分布的方法的另一实施例的流程示意图。如图7所示，在上述S404之后，还可以包括S406。
S406，输出终端设备在目标区域移动过程中的位姿、位姿对应的时间，以及终端设备在对应的时间采集的视频帧。
应理解，上述S405和S406可以同时执行。
上述实施例中描述了终端设备可以获取采集每个视频帧时的位姿，或者终端设备可以获取采集关键帧时的位姿。相对应的，本申请实施例中终端设备可以输出终端设备在移动过程中的位姿、位姿对应的时间，以及终端设备在时间采集的视频帧。其中，终端设备输出的方式可以为在界面上显示终端设备在移动过程中的位姿、位姿对应的时间，以及终端设备在时间采集的视频帧，如上述图5所示。
也就是说，一种可能的实现方式中，终端设备可以在移动过程中，可以输出采集每个视频帧时的位姿、位姿对应的时间，以及终端设备在对应的时间采集的视频帧。另一种可能的实现方式中，终端设备在移动过程中，可以输出采集每个关键帧时的位姿、位姿对应的时间，以及终端设备在对应的时间采集的关键帧。
本申请实施例中，可以实时输出终端设备在移动过程中的位姿、位姿对应的时间，以及终端设备在时间采集的视频帧，可以使得用户可以直观地看到终端设备的实时位姿，时间，以及采集的视频帧。
在上述实施例的基础上，本申请实施例中还可以根据Wi-Fi指纹空间分布的应用场景，获取Wi-Fi指纹空间分布。图8为本申请实施例提供的获取Wi-Fi指纹空间分布的方法的另一实施例的流程示意图。如图8所示，本申请实施例提供的获取Wi-Fi指纹空间分布的方法可以包括：
S801，采集当前时间窗口内的目标区域的视频帧、Wi-Fi指纹，以及终端设备的imu数据。
S802，根据imu数据，获取终端设备在采集视频帧时的相对移动信息。
S803，获取视频帧的特征，视频帧的特征包括视频帧的点特征和/或线特征。
S804，根据相对移动信息、视频帧的特征，获取终端设备采集视频帧时的位姿，以得到终端设备在目标区域移动过程中的位姿。
S805，根据目标区域的Wi-Fi指纹空间分布的应用场景，过滤Wi-Fi指纹。
S806，将终端设备在目标区域移动过程中的位姿和终端设备在目标区域移动过程中采集的Wi-Fi指纹按照时间对齐，得到目标区域的Wi-Fi指纹空间分布。
S807，根据目标区域的Wi-Fi指纹空间分布，获取目标区域的Wi-Fi指纹空间分布图，且显示目标区域的Wi-Fi指纹空间分布图。
应理解，本申请实施例中的S801-S804、S806中的实施方式可以参照上述实施例中S401-S404、S405中的相关描述，在此不做赘述。
上述S805中，本申请实施例中终端设备根据Wi-Fi指纹空间分布的应用场景，过滤采集的Wi-Fi指纹。
示例性的，若终端设备获取Wi-Fi指纹空间分布是为了对终端设备进行定位，则终端设备可以将Wi-Fi指纹中与定位不相关的数据进行过滤。如终端设备中预先设置有信号强度阈值，终端设备可以将Wi-Fi指纹中Wi-Fi信号的强度小于该信号强度阈值的Wi-Fi指纹删除。信号强度阈值如可以为-75dBm。
示例性的，若终端设备获取Wi-Fi指纹空间分布是为了调整路由器位置或对路由器的天线指向优化，则终端设备可以保留预设Wi-Fi名称的Wi-Fi指纹，如保留Wi-Fi名称为essid_home的Wi-Fi指纹，删除其他非该保留预设Wi-Fi名称的Wi-Fi指纹。
下面针对本申请实施例中Wi-Fi指纹空间分布可能的应用场景进行说明：
第一种应用场景：路由器位置的调整。
将路由器静置通电后，用户可以手持终端设备(终端设备的摄像头朝前)绕目标区域的各位置走一圈，以获取Wi-Fi指纹空间分布图，见下述图7和图8中的Wi-Fi指纹空间分布图。用户根据Wi-Fi指纹空间分布图，调整路由器的位置，使得目标区域的各位置都能覆盖到一定强度的Wi-Fi信号。
第二种应用场景：路由器天线指向优化。
用户根据Wi-Fi指纹空间分布图，调整路由器天线指向，使得目标区域的各位置都能覆盖到一定强度的Wi-Fi信号。应理解，本申请实施例中用于给出Wi-Fi指纹空间分布图指示，至于具体如何根据该指示图调整路由器天线指向，本申请实施例不做限制。
第三种应用场景：提高终端设备定位的精度。
用户根据Wi-Fi指纹空间分布图指示，调整目标区域的路由器的位置，使得目标区域Wi-Fi信号与非目标区域的Wi-Fi信号呈现一定的构型，以便提高终端设备定位的精度。应理解，本申请实施例中用于给出Wi-Fi指纹空间分布图指示，至于具体如何根据该Wi-Fi指纹空间分布图调整路由器构型，本申请实施例不做限制。
第四种应用场景：路由器布局位置推荐。
在目标区域需要新部署一个路由器时，用户根据目前目标区域的Wi-Fi指纹空间分布图，可以确定Wi-Fi信号强度弱的位置，进而在该位置处部署一个新的路由，以形成路由器接力。该应用场景可以参照下述图11中的相关描述。
上述S807中，为了使得用户能够更为直观地获取目标区域的Wi-Fi指纹空间分布，本申请实施例中终端设备可以根据目标区域的Wi-Fi指纹空间分布，获取目标区域的Wi-Fi指纹空间分布图，且显示目标区域的Wi-Fi指纹空间分布图。用户可以在终端设备上观看到目标区域的Wi-Fi指纹空间分布图。
其中，Wi-Fi指纹空间分布表征终端设备在目标区域的各位置处对应的Wi-Fi指纹。本申请实施例中以终端设备的位姿为6分量为例进行说明，如终端设备的位姿为{x t,y t,z t,roll t,pitch t,yaw t}。其中，x t,y t,z t表示终端设备的空间位置，roll t,pitch t,yaw t表示终端设备的空间姿态。
本申请实施例中在终端设备上显示的Wi-Fi指纹空间分布图可以为点云图，也可以为热度图，或者其他形式的图。图9为本申请实施例提供的Wi-Fi指纹空间分布图一。图9为点云图，其中，图9中的X轴和Y轴分别表征终端设备的位置中的横坐标和纵坐标，图9中的Z轴表征终端设备的位置对应的Wi-Fi信号的强度，Z轴的数值越大，代表Wi-Fi信号的强度越大。图10为本申请实施例提供的Wi-Fi指纹空间分布图二。图10为热度图，图10中以灰度表征目标区域各位置处的Wi-Fi信号的强度，灰度值越大，代表Wi-Fi信号的强度越大。
在一种可能的实现方式中，若目标区域的Wi-Fi指纹空间分布的应用场景为获取路由器 的安装位置，则本申请实施例中可以根据目标区域的Wi-Fi指纹空间分布推荐路由器的待安装位置，进而在目标区域的Wi-Fi指纹空间分布图中显示路由器的待安装位置。应注意，本申请实施例中终端设备可以根据Wi-Fi指纹空间分布，确定路由器的待安装位置，使得目标区域的各位置都能覆盖到一定强度的Wi-Fi信号。
图11为本申请实施例提供的Wi-Fi指纹空间分布图三。与上述图10不同的是，本申请实施例终端设备显示的Wi-Fi指纹空间分布图中可以显示有路由器的待安装位置，用户可以该图示，安装路由器。如图11所示，本申请实施例中还可以在Wi-Fi指纹空间分布图中显示盲区，其中，盲区表示Wi-Fi信号的强度小于强度阈值的位置。
本申请实施例中，终端设备可以根据Wi-Fi指纹空间分布的应用场景，过滤Wi-Fi指纹，以得到更为贴合使用的Wi-Fi指纹，减少数据占用空间。另外，本申请实施例中还可以显示Wi-Fi指纹空间分布图，用户可以直观地确定目标区域的Wi-Fi指纹空间分布。
图12为本申请实施例提供的一种获取Wi-Fi指纹空间分布的装置的结构示意图。本申请实施例的获取Wi-Fi指纹空间分布的装置可以为前述所说的终端设备，也可以为应用于终端设备的芯片。该获取Wi-Fi指纹空间分布的装置可以用于执行上述方法实施例中终端设备的动作。如图12所示，该获取Wi-Fi指纹空间分布的装置可以包括：采集模块1201、处理模块1202、显示模块1203。其中，
采集模块1201，用于采集当前时间窗口内的目标区域的视频帧、Wi-Fi指纹，以及终端设备的惯性测量单元imu数据。
处理模块1202，用于基于视频帧和imu数据，获取终端设备在目标区域移动过程中的位姿，且将终端设备在目标区域移动过程中的位姿和终端设备在目标区域移动过程中采集的Wi-Fi指纹按照时间对齐，得到目标区域的Wi-Fi指纹空间分布。
在一种可能的实现方式中，处理模块1202，具体用于根据imu数据，获取终端设备在采集视频帧时的相对移动信息，且获取视频帧的特征，以及根据相对移动信息、视频帧的特征，获取终端设备采集视频帧时的位姿，以得到终端设备在目标区域移动过程中的位姿，其中，视频帧的特征包括点特征和/或线特征。
在一种可能的实现方式中，处理模块1202，具体用于根据视频帧和前一个视频帧之间终端设备采集的imu数据，得到相对移动信息，相对移动信息表征终端设备采集视频帧时相较于采集前一个视频帧时的移动情况。
在一种可能的实现方式中，视频帧的特征包括视频帧的点特征，处理模块1202，具体用于在视频帧中，追踪前一个视频帧的点特征，若视频帧中追踪得到的点特征的数量小于第一数量阈值，则在视频帧中提取新的点特征，以得到第一数量阈值的点特征，以及将视频帧中追踪得到的点特征和新的点特征作为视频帧的点特征。
在一种可能的实现方式中，若视频帧为第一个视频帧，则第一个视频帧满足如下条件：
第一个视频帧中的角点的数量小于或等于第二数量阈值；和，
第一个视频帧的点特征的数量大于或等于第一数量阈值；和，
第一个视频帧中的角点之间的距离大于或等于第一距离阈值；和，
第一个视频帧中的角点的质量等级大于预设等级。
在一种可能的实现方式中，处理模块1202，还用于获取追踪错误的点特征，且在视频帧中删除追踪错误的点特征。
在一种可能的实现方式中，视频帧的特征包括视频帧的线特征，处理模块1202，具体用 于将视频帧的线特征与前一个视频帧的线特征进行匹配，建立初始线对，初始线对指示在视频帧和前一个视频帧中匹配成功的两个线特征；在初始线对中，删除错误的线对，以得到视频帧中的线特征。
在一种可能的实现方式中，处理模块1202，具体用于获取线对中两个线特征的距离、线段长度差以及线段夹角；删除两个线特征的距离大于第二距离阈值的线对；和，删除两个线特征的线段长度差大于长度差阈值的线对；和，删除两个线特征的线段夹角大于第二角度阈值的线对。
在一种可能的实现方式中，相对移动信息包括：相对位移和相对角度。
处理模块1202，还用于根据相对位移、相对角度或视频帧与前一个关键帧之间的时间差，得到关键帧。关键帧满足如下至少一项条件：相对位移大于第一位移阈值；相对角度大于第一角度阈值；时间差大于时间差阈值。
在一种可能的实现方式中，终端设备在目标区域移动过程中的位姿包括：当前时间窗口内终端设备在采集视频帧时的位姿。处理模块1202，具体用于获取视频帧之前相邻的关键帧的位，且根据视频帧之前相邻的关键帧的位姿，以及视频帧和视频帧之前相邻的关键帧之间的相对位移信息，获取终端设备在采集视频帧时的位姿。
在一种可能的实现方式中，处理模块1202，具体用于以关键帧的相对移动信息、关键帧的点特征和关键帧的线特征作为参数，构建待估计的状态向量；建立最大后验概率目标函数，且将待估计的状态向量代入最大后验概率目标函数中，对最大后验概率目标函数进行求解，得到终端设备在当前时间窗口内采集关键帧时的初始位姿；根据终端设备初始位姿，获取终端设备采集关键帧时的位姿，以得到获取终端设备在目标区域移动过程中的位姿。
在一种可能的实现方式中，关键帧为当前时间窗口内的当前关键帧，视频帧的特征包括视频帧的点特征和线特征；处理模块1202，具体用于根据当前时间窗口内的当前关键帧的点特征和线特征，以及历史关键帧的点特征和线特征，进行闭环检测，以及对当前关键帧进行时空一致性校验，历史关键帧为：终端设备在目标区域移动时当前时间窗口之前的时间窗口内的关键帧；根据终端设备的初始位姿、闭环检测结果、以及当前关键帧的校验结果，获取终端设备采集关键帧时的位姿，以得到终端设备在目标区域移动过程中的位姿。
在一种可能的实现方式中，处理模块1202，具体用于根据当前关键帧的点特征的描述子，构建第一当前向量，当前关键帧为当前时间窗口内的关键帧；获取第一当前向量与第一历史向量的相似度，且将相似度大于相似度阈值的第一历史向量对应的关键帧作为第一候选帧，第一历史向量是根据历史关键帧的点特征的描述子构建的；根据当前关键帧的线特征的描述子，构建第二当前向量；获取第二当前向量与第二历史向量的相似度，且将相似度大于相似度阈值的第二历史向量对应的关键帧作为第二候选帧，第二历史向量是根据历史关键帧的线特征的描述子构建的；根据第一候选帧的相似度和第二候选帧的相似度，进行闭环检测，以及对当前关键帧进行时空一致性校验。
在一种可能的实现方式中，处理模块1202，具体用于对第一候选帧的相似度和第二候选帧的相似度进行加权处理，得到加权分值；若加权分值大于或等于分值阈值，则闭环检测通过；若加权分值小于分值阈值，则闭环检测未通过。
在一种可能的实现方式中，时空一致性校验包括空间一致性校验。处理模块1202，具体用于获取第一候选帧和当前关键帧之间的相对位移和相对角度；若相对位移小于第二位移阈值，以及相对角度小于第三角度阈值，则确定当前关键帧通过空间一致性校验。
在一种可能的实现方式中，时空一致性校验还包括时间一致性校验。处理模块1202，具 体用于若当前关键帧的前N个关键帧的加权分值均大于分值阈值，则确定当前关键帧通过时间一致性校验，N为大于或等于1的整数。
在一种可能的实现方式中，处理模块1202，具体用于若闭环检测通过，且当前关键帧通过时空一致性校验，则融合当前关键帧的点特征与第一候选帧的点特征；融合当前关键帧的线特征与第二候选帧的线特征；通过融合后的点特征和融合后的线特征，对终端设备的初始位姿以及终端设备采集历史关键帧时的位姿进行校正，获取终端设备在目标区域移动过程中的位姿。
在一种可能的实现方式中，处理模块1202，具体用于若闭环检测未通过，或者闭环检测通过且当前关键帧未通过时空一致性校验，则将终端设备的初始位姿和终端设备采集历史关键帧时的位姿作为终端设备在目标区域移动过程中的位姿。
在一种可能的实现方式中，处理模块1202，还用于若当前关键帧通过时空一致性校验，则存储当前关键帧对应的第一当前向量和第二当前向量。
在一种可能的实现方式中，处理模块1202，还用于输出终端设备在移动过程中的位姿、位姿对应的时间，以及终端设备在时间采集的视频帧。
在一种可能的实现方式中，处理模块1202，还用于根据目标区域的Wi-Fi指纹空间分布的应用场景，过滤Wi-Fi指纹。
在一种可能的实现方式中，处理模块1202，还用于根据目标区域的Wi-Fi指纹空间分布，获取目标区域的Wi-Fi指纹空间分布图。
显示模块1203，用于显示目标区域的Wi-Fi指纹空间分布图。
在一种可能的实现方式中，显示模块1203，还用于若目标区域的Wi-Fi指纹空间分布的应用场景为获取路由器的安装位置，则在目标区域的Wi-Fi指纹空间分布图中显示路由器的待安装位置。
本申请实施例提供的获取Wi-Fi指纹空间分布的装置，可以执行上述方法实施例中终端设备的动作，其实现原理和技术效果类似，在此不再赘述。
需要说明的是，应理解以上收发模块实际实现时可以为收发器、或者包括发送器和接收器。而处理模块可以以软件通过处理元件调用的形式实现；也可以以硬件的形式实现。例如，处理模块可以为单独设立的处理元件，也可以集成在上述装置的某一个芯片中实现，此外，也可以以程序代码的形式存储于上述装置的存储器中，由上述装置的某一个处理元件调用并执行以上处理模块的功能。此外这些模块全部或部分可以集成在一起，也可以独立实现。这里所述的处理元件可以是一种集成电路，具有信号的处理能力。在实现过程中，上述方法的各步骤或以上各个模块可以通过处理器元件中的硬件的集成逻辑电路或者软件形式的指令完成。
例如，以上这些模块可以是被配置成实施以上方法的一个或多个集成电路，例如：一个或多个专用集成电路(application specific integrated circuit，ASIC)，或，一个或多个微处理器(digital signal processor，DSP)，或，一个或者多个现场可编程门阵列(field programmable gate array，FPGA)等。再如，当以上某个模块通过处理元件调度程序代码的形式实现时，该处理元件可以是通用处理器，例如中央处理器(central processing unit，CPU)或其它可以调用程序代码的处理器。再如，这些模块可以集成在一起，以片上系统(system-on-a-chip，SOC)的形式实现。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。计算机程序产品包括一 个或多个计算机指令。在计算机上加载和执行计算机程序指令时，全部或部分地产生按照本申请实施例的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘Solid State Disk(SSD))等。
本文中的术语“多个”是指两个或两个以上。本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系；在公式中，字符“/”，表示前后关联对象是一种“相除”的关系。
可以理解的是，在本申请的实施例中涉及的各种数字编号仅为描述方便进行的区分，并不用来限制本申请的实施例的范围。
可以理解的是，在本申请的实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请的实施例的实施过程构成任何限定。

Claims (26)

  1. 一种获取Wi-Fi指纹空间分布的方法，应用于终端设备，其特征在于，包括：
    采集当前时间窗口内的目标区域的视频帧、Wi-Fi指纹，以及终端设备的惯性测量单元imu数据；
    基于所述视频帧和所述imu数据，获取所述终端设备在所述目标区域移动过程中的位姿；
    将所述终端设备在所述目标区域移动过程中的位姿，以及所述终端设备在所述目标区域移动过程中采集的Wi-Fi指纹按照时间对齐，得到所述目标区域的Wi-Fi指纹空间分布。
  2. 根据权利要求1所述的方法，其特征在于，所述基于所述视频帧和所述imu数据，获取所述终端设备在所述目标区域移动过程中的位姿，包括：
    根据所述imu数据，获取所述终端设备在采集所述视频帧时的相对移动信息；
    获取所述视频帧的特征，所述视频帧的特征包括所述视频帧的点特征和/或线特征；
    根据所述相对移动信息、所述视频帧的特征，获取所述终端设备采集所述视频帧时的位姿，以得到所述终端设备在所述目标区域移动过程中的位姿。
  3. 根据权利要求2所述的方法，其特征在于，所述根据所述imu数据，获取所述终端设备在采集所述视频帧时的相对移动信息，包括：
    根据所述视频帧和前一个视频帧之间所述终端设备采集的imu数据，得到所述相对移动信息，所述相对移动信息表征所述终端设备采集所述视频帧时相较于采集所述前一个视频帧时的移动情况。
  4. 根据权利要求2或3所述的方法，其特征在于，所述视频帧的特征包括所述视频帧的点特征，所述获取所述视频帧的特征，包括：
    在所述视频帧中，追踪前一个视频帧的点特征；
    若所述视频帧中追踪得到的点特征的数量小于第一数量阈值，则在所述视频帧中提取新的点特征，以得到所述第一数量阈值的点特征；
    将所述视频帧中追踪得到的点特征和所述新的点特征作为所述视频帧的点特征。
  5. 根据权利要求4所述的方法，其特征在于，所述方法还包括：若所述视频帧为所述当前时间窗口内的第一个视频帧，则所述第一个视频帧满足如下条件：
    所述第一个视频帧中的角点的数量小于或等于第二数量阈值；和，
    所述第一个视频帧的点特征的数量大于或等于所述第一数量阈值；和，
    所述第一个视频帧中的角点之间的距离大于或等于第一距离阈值；和，
    所述第一个视频帧中的角点的质量等级大于预设等级。
  6. 根据权利要求4或5所述的方法，其特征在于，所述追踪前一个视频帧的点特征之后，还包括：
    获取追踪错误的点特征，且在所述视频帧中删除所述追踪错误的点特征。
  7. 根据权利要求2-6中任一项所述的方法，其特征在于，所述视频帧的特征包括所述视频帧的线特征，所述获取所述视频帧的特征，包括：
    将所述视频帧的线特征与前一个视频帧的线特征进行匹配，建立初始线对，所述初始线对指示在所述视频帧和所述前一个视频帧中匹配成功的两个线特征；
    在所述初始线对中，删除错误的线对，以得到所述视频帧中的线特征。
  8. 根据权利要求7所述的方法，其特征在于，所述删除错误的线对，包括：
    获取所述线对中两个线特征的距离、线段长度差以及线段夹角；
    删除所述两个线特征的距离大于第二距离阈值的线对；和，
    删除所述两个线特征的线段长度差大于长度差阈值的线对；和，
    删除所述两个线特征的线段夹角大于第二角度阈值的线对。
  9. 根据权利要求3所述的方法，其特征在于，所述相对移动信息包括：相对位移和相对角度；所述得到所述相对移动信息之后，还包括：
    根据所述相对位移、所述相对角度或所述视频帧与前一个关键帧之间的时间差，得到关键帧；
    所述关键帧满足如下至少一项条件：
    所述相对位移大于第一位移阈值；
    所述相对角度大于第一角度阈值；
    所述时间差大于时间差阈值。
  10. 根据权利要求9所述的方法，其特征在于，所述终端设备在所述目标区域移动过程中的位姿包括：所述当前时间窗口内所述终端设备在采集所述视频帧时的位姿；所述根据所述相对移动信息、所述视频帧的特征，获取所述终端设备采集所述视频帧时的位姿，包括：
    获取所述视频帧之前相邻的关键帧的位姿；
    根据所述视频帧之前相邻的关键帧的位姿，以及所述视频帧和所述视频帧之前相邻的关键帧之间的相对位移信息，获取所述终端设备在采集所述视频帧时的位姿。
  11. 根据权利要求9所述的方法，其特征在于，所述根据所述相对移动信息、所述视频帧的特征，获取所述终端设备采集所述视频帧时的位姿，以得到所述终端设备在所述目标区域移动过程中的位姿，包括：
    以关键帧的相对移动信息、所述关键帧的点特征和所述关键帧的线特征作为参数，构建待估计的状态向量；
    建立最大后验概率目标函数，且将所述待估计的状态向量代入所述最大后验概率目标函数中，对所述最大后验概率目标函数进行求解，得到所述终端设备在所述当前时间窗口内采集所述关键帧时的初始位姿；
    根据所述终端设备初始位姿，获取所述终端设备采集所述关键帧时的位姿，以得到所述终端设备在所述目标区域移动过程中的位姿。
  12. 根据权利要求11所述的方法，其特征在于，所述关键帧为所述当前时间窗口内的当前关键帧，所述视频帧的特征包括所述视频帧的点特征和线特征；所述根据所述终端设备初始位姿，获取所述终端设备采集所述关键帧时的位姿，以得到所述终端设备在所述目标区域移动过程中的位姿，包括：
    根据所述当前时间窗口内的当前关键帧的点特征和线特征，以及历史关键帧的点特征和线特征，进行闭环检测，以及对所述当前关键帧进行时空一致性校验，所述历史关键帧为：所述终端设备在所述目标区域移动时所述当前时间窗口之前的时间窗口内的关键帧；
    根据所述终端设备的初始位姿、闭环检测结果、以及所述当前关键帧的校验结果，获取所述终端设备采集所述当前关键帧时的位姿，以得到所述终端设备在所述目标区域移动过程中的位姿。
  13. 根据权利要求12所述的方法，其特征在于，所述根据所述当前时间窗口内的当前关键帧的点特征和线特征，以及历史关键帧的点特征和线特征，进行闭环检测，以及对所述当前关键帧进行时空一致性校验，包括：
    根据所述当前关键帧的点特征的描述子，构建第一当前向量，所述当前关键帧为所述当 前时间窗口内的关键帧；
    获取第一当前向量与第一历史向量的相似度，且将相似度大于相似度阈值的第一历史向量对应的关键帧作为第一候选帧，所述第一历史向量是根据所述历史关键帧的点特征的描述子构建的；
    根据所述当前关键帧的线特征的描述子，构建第二当前向量；
    获取第二当前向量与第二历史向量的相似度，且将相似度大于所述相似度阈值的第二历史向量对应的关键帧作为第二候选帧，所述第二历史向量是根据所述历史关键帧的线特征的描述子构建的；
    根据所述第一候选帧的相似度和所述第二候选帧的相似度，进行闭环检测，以及对所述当前关键帧进行时空一致性校验。
  14. 根据权利要求13所述的方法，其特征在于，所述根据所述第一候选帧的相似度和所述第二候选帧的相似度，进行闭环检测，包括：
    对所述第一候选帧的相似度和所述第二候选帧的相似度进行加权处理，得到加权分值；
    若所述加权分值大于或等于分值阈值，则闭环检测通过；
    若所述加权分值小于分值阈值，则闭环检测未通过。
  15. 根据权利要求14所述的方法，其特征在于，所述时空一致性校验包括空间一致性校验；根据所述第一候选帧的相似度和所述第二候选帧的相似度，对所述当前关键帧进行空间一致性校验，包括：
    获取所述第一候选帧和所述当前关键帧之间的相对位移和相对角度；
    若所述相对位移小于第二位移阈值，以及所述相对角度小于第三角度阈值，则确定所述当前关键帧通过空间一致性校验。
  16. 根据权利要求14或15所述的方法，其特征在于，所述时空一致性校验还包括时间一致性校验；根据所述第一候选帧的相似度和所述第二候选帧的相似度，对所述当前关键帧进行时间一致性校验，包括：
    若所述当前关键帧的前N个关键帧的加权分值均大于所述分值阈值，则确定所述当前关键帧通过时间一致性校验，所述N为大于或等于1的整数。
  17. 根据权利要求13-16中任一项所述的方法，其特征在于，所述根据所述终端设备的初始位姿、闭环检测结果、以及所述当前关键帧的校验结果，获取所述终端设备采集所述当前关键帧时的位姿，以得到所述终端设备在所述目标区域移动过程中的位姿，包括：
    若闭环检测通过，且所述当前关键帧通过时空一致性校验，则融合所述当前关键帧的点特征与所述第一候选帧的点特征；
    融合所述当前关键帧的线特征与所述第二候选帧的线特征；
    通过融合后的点特征和融合后的线特征，对所述终端设备的初始位姿以及所述终端设备采集所述历史关键帧时的位姿进行校正，获取所述终端设备在所述目标区域移动过程中的位姿。
  18. 根据权利要求13-16中任一项所述的方法，其特征在于，所述根据所述终端设备的初始位姿、闭环检测结果、以及所述当前关键帧的校验结果，获取所述终端设备采集所述当前关键帧时的位姿，以得到所述终端设备在所述目标区域移动过程中的位姿，包括：
    若闭环检测未通过，或者闭环检测通过且所述当前关键帧未通过时空一致性校验，则将所述终端设备的初始位姿和所述终端设备采集所述历史关键帧时的位姿作为所述终端设备在所述目标区域移动过程中的位姿。
  19. 根据权利要求13-18中任一项所述的方法，其特征在于，所述方法还包括：
    若所述当前关键帧通过时空一致性校验，则存储所述当前关键帧的第一当前向量和第二当前向量。
  20. 根据权利要求18或19所述的方法，其特征在于，所述获取所述终端设备在所述目标区域移动过程中的位姿之后，还包括：
    输出所述终端设备在所述目标区域移动过程中的位姿、位姿对应的时间，以及所述终端设备在所述时间采集的视频帧。
  21. 根据权利要求1-20中任一项所述的方法，其特征在于，所述将所述终端设备在所述目标区域移动过程中的位姿，以及所述终端设备在所述目标区域移动过程中采集的Wi-Fi指纹按照时间对齐之前，还包括：
    根据所述目标区域的Wi-Fi指纹空间分布的应用场景，过滤所述Wi-Fi指纹。
  22. 根据权利要求1-21中任一项所述的方法，其特征在于，所述得到所述目标区域的Wi-Fi指纹空间分布之后，还包括：
    根据所述目标区域的Wi-Fi指纹空间分布，获取所述目标区域的Wi-Fi指纹空间分布图；
    显示所述目标区域的Wi-Fi指纹空间分布图。
  23. 根据权利要求22所述的方法，其特征在于，所述显示所述目标区域的Wi-Fi指纹空间分布图，包括：
    若所述目标区域的Wi-Fi指纹空间分布的应用场景为获取路由器的安装位置，则在所述目标区域的Wi-Fi指纹空间分布图中显示所述路由器的待安装位置。
  24. 一种获取Wi-Fi指纹空间分布的装置，其特征在于，包括：
    采集模块，用于采集当前时间窗口内的目标区域的视频帧、Wi-Fi指纹，以及终端设备的惯性测量单元imu数据；
    处理模块，用于基于所述视频帧和所述imu数据，获取所述终端设备在所述目标区域移动过程中的位姿，且将所述终端设备在所述目标区域移动过程中的位姿和所述终端设备在所述目标区域移动过程中采集的Wi-Fi指纹按照时间对齐，得到所述目标区域的Wi-Fi指纹空间分布。
  25. 一种电子设备，其特征在于，包括：存储器、处理器和收发器；
    所述处理器用于与所述存储器耦合，读取并执行所述存储器中的指令，以实现权利要求1-23中任一项所述的方法；
    所述收发器与所述处理器耦合，由所述处理器控制所述收发器进行消息收发。
  26. 一种计算机可读存储介质，其特征在于，所述计算机存储介质存储有计算机指令，当所述计算机指令被计算机执行时，使得所述计算机执行权利要求1-23中任一项所述的方法。
PCT/CN2021/085972 2020-07-21 2021-04-08 获取Wi-Fi指纹空间分布的方法、装置和电子设备 WO2022016909A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010705291.X 2020-07-21
CN202010705291.XA CN112087728B (zh) 2020-07-21 2020-07-21 获取Wi-Fi指纹空间分布的方法、装置和电子设备

Publications (1)

Publication Number Publication Date
WO2022016909A1 true WO2022016909A1 (zh) 2022-01-27

Family

ID=73735138

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/085972 WO2022016909A1 (zh) 2020-07-21 2021-04-08 获取Wi-Fi指纹空间分布的方法、装置和电子设备

Country Status (2)

Country Link
CN (1) CN112087728B (zh)
WO (1) WO2022016909A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112087728B (zh) * 2020-07-21 2022-01-14 华为技术有限公司 获取Wi-Fi指纹空间分布的方法、装置和电子设备
CN113465602A (zh) * 2021-05-26 2021-10-01 北京三快在线科技有限公司 导航方法、装置、电子设备及可读存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160094951A1 (en) * 2014-09-26 2016-03-31 Shao-Wen Yang Wi-fi fingerprint location database creation and update from crowd sourced data
CN108882169A (zh) * 2018-04-10 2018-11-23 北京三快在线科技有限公司 一种WiFi位置指纹数据的获取方法及装置和机器人
CN110447277A (zh) * 2017-03-28 2019-11-12 华为技术有限公司 一种指纹定位方法及相关设备
CN110763251A (zh) * 2019-10-18 2020-02-07 华东交通大学 视觉惯性里程计优化的方法及系统
CN112087728A (zh) * 2020-07-21 2020-12-15 华为技术有限公司 获取Wi-Fi指纹空间分布的方法、装置和电子设备

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6318552B2 (ja) * 2013-10-31 2018-05-09 セイコーエプソン株式会社 角速度センサー、電子機器および移動体
CN104181500A (zh) * 2014-08-19 2014-12-03 北京邮电大学 一种基于惯性信息和机会无线信号特征的实时定位方法
CN107194984A (zh) * 2016-03-14 2017-09-22 武汉小狮科技有限公司 移动端实时高精度三维建模方法
CN108242079B (zh) * 2017-12-30 2021-06-25 北京工业大学 一种基于多特征视觉里程计和图优化模型的vslam方法
CN109001674B (zh) * 2018-05-31 2022-09-02 中国矿业大学 一种基于连续视频序列的WiFi指纹信息快速采集与定位方法
CN110766716B (zh) * 2019-09-10 2022-03-29 中国科学院深圳先进技术研究院 一种空间未知运动目标的信息获取方法及系统
CN110853100B (zh) * 2019-10-24 2022-11-18 东南大学 一种基于改进点线特征的结构化场景视觉slam方法
CN111160362B (zh) * 2019-11-27 2023-07-11 东南大学 一种fast特征均匀化提取与帧间特征误匹配去除方法
CN111238450B (zh) * 2020-02-27 2021-11-30 北京三快在线科技有限公司 视觉定位方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160094951A1 (en) * 2014-09-26 2016-03-31 Shao-Wen Yang Wi-fi fingerprint location database creation and update from crowd sourced data
CN110447277A (zh) * 2017-03-28 2019-11-12 华为技术有限公司 一种指纹定位方法及相关设备
CN108882169A (zh) * 2018-04-10 2018-11-23 北京三快在线科技有限公司 一种WiFi位置指纹数据的获取方法及装置和机器人
CN110763251A (zh) * 2019-10-18 2020-02-07 华东交通大学 视觉惯性里程计优化的方法及系统
CN112087728A (zh) * 2020-07-21 2020-12-15 华为技术有限公司 获取Wi-Fi指纹空间分布的方法、装置和电子设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LIU TAO: "Visual-Aided Indoor Mapping and Pedestrian Navigation Research", SCIENCE-ENGINEERING (A), CHINA DOCTORAL DISSERTATIONS FULL-TEXT DATABASE, no. 6, 15 June 2018 (2018-06-15), XP055889090 *

Also Published As

Publication number Publication date
CN112087728A (zh) 2020-12-15
CN112087728B (zh) 2022-01-14

Similar Documents

Publication Publication Date Title
US11657609B2 (en) Terminal device, information processing device, object identifying method, program, and object identifying system
CN110555883B (zh) 相机姿态追踪过程的重定位方法、装置及存储介质
CN108596976B (zh) 相机姿态追踪过程的重定位方法、装置、设备及存储介质
WO2019205853A1 (zh) 相机姿态追踪过程的重定位方法、装置、设备及存储介质
US10250800B2 (en) Computing device having an interactive method for sharing events
CN112020855B (zh) 用于稳定视频以减少相机和人脸移动的方法、系统和设备
WO2018107679A1 (zh) 一种动态三维图像获取的方法和设备
US9417689B1 (en) Robust device motion detection
US20200334836A1 (en) Relocalization method and apparatus in camera pose tracking process, device, and storage medium
CN111935393A (zh) 拍摄方法、装置、电子设备和存储介质
CN106164982A (zh) 基于影像的电子设备定位
WO2022016909A1 (zh) 获取Wi-Fi指纹空间分布的方法、装置和电子设备
CN103907340A (zh) 图像生成装置和图像生成方法
WO2020259544A1 (zh) 一种确定校准参数的方法及电子设备
CN112348886A (zh) 视觉定位方法、终端和服务器
JP2015082823A (ja) 撮影制御装置、撮影制御方法およびプログラム
CN116739908A (zh) 图像处理方法、装置及设备
US20220345621A1 (en) Scene lock mode for capturing camera images
CN116576866B (zh) 导航方法和设备
Chang et al. Augmented reality services of photos and videos from filming sites using their shooting locations and attitudes
CN115830110B (zh) 即时定位与地图构建方法、装置、终端设备及存储介质
US9781292B2 (en) Lifelog camera and method of obtaining a digital image tagged with corrected image direction information
CN115760584A (zh) 一种图像处理方法及相关设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21846125

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21846125

Country of ref document: EP

Kind code of ref document: A1