WO2021263248A1

WO2021263248A1 - Multi-person simultaneous location and mapping (slam) linkage positioning and navigation

Info

Publication number: WO2021263248A1
Application number: PCT/US2021/044182
Authority: WO
Inventors: Fuyao Zhang; Dingyi CHEN; Youjie XIA; Fan DENG
Original assignee: Innopeak Technology, Inc.
Priority date: 2021-08-02
Filing date: 2021-08-02
Publication date: 2021-12-30
Also published as: CN117795292A

Abstract

Novel tools and techniques are provided for implementing multi-person SLAM linkage positioning and navigation for super large-scale scenes based on wireless signal characteristics and visual feature point information. In various embodiments, for creating a map model, a software application on a mobile device located in an area may generate or update a wireless signal fingerprint map model of the area based on signal data received from wireless signal sources within or near the area, may generate or update a visual feature point map model of a portion of the area based on image data received from visual sensors, and may send the wireless signal fingerprint map model and visual feature point map model to a remote computing system over a network. The remote computing system may analyze and integrate the wireless signal fingerprint map model and visual feature point map model to generate a global map of the area.

Description

MULTI-PERSON SIMULTANEOUS LOCATION AND MAPPING (SLAM) LINKAGE POSITIONING AND NAVIGATION

COPYRIGHT STATEMENT

[0001] A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.

FIELD

[0002] The present disclosure relates, in general, to methods, systems, and apparatuses for implementing simultaneous location and mapping ("SLAM") functionalities, and, more particularly, to methods, systems, and apparatuses for implementing multi-person SLAM linkage positioning and navigation, in some cases, for super large-scale scenes based on wireless signal characteristics and visual feature point information.

BACKGROUND

[0003] Conventional positioning and navigation systems each utilizes one of the following: (1) wireless signal positioning and navigation techniques; (2) visual sensor positioning and navigation techniques; (3) lighthouse positioning and navigation techniques; (4) laser positioning and navigation techniques; (5) infrared positioning and navigation techniques; or (6) ultrasound positioning and navigation techniques. However, these techniques possess the following drawbacks or disadvantages. Regarding (1), indoor positioning accuracy based on wireless signal characteristics is poor and can easily to be interfered with by the surrounding environment. Regarding (2), indoor positioning and navigation based on visual sensors is greatly affected by light and cannot work in a dark environment. It also needs to process a large amount of image data. Further, mobile devices cannot carry super large-scale visual feature point map models (due to memory limitations on mobile devices), and thus the positioning area is necessarily limited. Regarding (3), indoor positioning of the lighthouse, without map data, leads to easy loss of navigation, such techniques require additional equipment to assist in its processes. Regarding (4), laser indoor positioning and navigation is costly. Regarding (5), for infrared indoor positioning and navigation, the minimum detection distance is too large, and it can be greatly interfered with by the surrounding environment. It cannot detect the distance for almost black and transparent objects, is only suitable for short-distance transmission, and cannot work normally when other objects are blocked. Further, it requires additional equipment support, resulting in higher costs. Regarding (6), ultrasonic positioning and navigation can be easily disturbed by the surrounding environment, obstacles, mirror reflections, rough surfaces, and/or other conditions. Ultrasound travels in the air for a short distance, so the positioning range is limited. At the same time, there are problems such as slow signal acquisition speed and poor navigation accuracy.

[0004] Hence, there is a need for more robust and scalable solutions for implementing simultaneous location and mapping ("SLAM") functionalities.

SUMMARY

[0005] The techniques of this disclosure generally relate to tools and techniques for implementing simultaneous location and mapping ("SLAM") functionalities, and, more particularly, to methods, systems, and apparatuses for implementing multi person SLAM linkage positioning and navigation, in some cases, for super large-scale scenes based on wireless signal characteristics and visual feature point information. [0006] In an aspect, a method may be provided for creating a map model. The method may comprise: generating or updating, using a software application running on a first mobile device that is located in a first area, a first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area; generating or updating, using the software application, a first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor, the first visual feature point map model being correlated with the first wireless signal fingerprint map model; sending, using the software application and over a network, the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system that is configured to generate a global map of the first area based on analysis and integration of the correlated first wireless signal fingerprint map model and first visual feature point map model.

[0007] In another aspect, a method may be provided for creating a map model.

The method may comprise: receiving, using a remote computing system and from a software application running on a first mobile device that is located in a first area, correlated first wireless signal fingerprint map model and first visual feature point map model, the software application being configured to generate or update the first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area and further configured to generate or update the first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor; analyzing, using the remote computing system, the correlated first wireless signal fingerprint map model and first visual feature point map model; and based on the analysis, integrating, using the remote computing system, the correlated received first wireless signal fingerprint map model and first visual feature point map model to generate a global map of the first area.

[0008] In yet another aspect, a system may be provided for creating a map model. The system may comprise a first mobile device that may be located in a first area.

The first mobile device might comprise at least one first processor and a first non- transitory computer readable medium communicatively coupled to the at least one first processor. The first non-transitory computer readable medium might have stored thereon computer software comprising a first set of instructions that, when executed by the at least one first processor, causes the first mobile device to: generate or update a first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area; generate or update a first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor, the first visual feature point map model being correlated with the first wireless signal fingerprint map model; and send the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system over a network, the remote computing system being configured to generate a global map of the first area based on analysis and integration of the correlated first wireless signal fingerprint map model and first visual feature point map model.

[0009] In still another aspect, a method may be provided for loading a map model. The method may comprise: generating, using a software application running on a first mobile device that is located in a first area, a first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area; generating, using the software application, a first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor, the first visual feature point map model being correlated with the first wireless signal fingerprint map model; sending, using the software application and over a network, the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system that is configured to extract a regional visual feature point map model based on analysis of the correlated first wireless signal fingerprint map model and first visual feature point map model; and receiving, using the software application over the network, the regional visual feature point map model from the remote computing system.

[0010] In another aspect, a method may be provided for loading a map model.

The method may comprise: receiving, using a remote computing system and from a software application running on a first mobile device that is located in a first area, correlated first wireless signal fingerprint map model and first visual feature point map model, the software application being configured to generate the first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area and further configured to generate the first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor; and analyzing, using the remote computing system, the correlated first wireless signal fingerprint map model and first visual feature point map model, by: comparing the first wireless signal fingerprint map model with one or more portions of a previously generated global wireless signal fingerprint map using wireless signal fingerprint matching; in response to identifying a matching portion of the previously generated global wireless signal fingerprint map based on the comparison, comparing the first visual feature point map model with one or more portions of a previously generated visual feature point map corresponding to the identified matching portion of the previously generated global wireless signal fingerprint map using visual feature point matching; and in response to identifying a matching portion of the previously generated visual feature point map, extracting the regional visual feature point map model based on the identified matching portion of the previously generated visual feature point map, and sending the regional visual feature point map model as positioning and navigation data to the software application running on the first mobile device.

[0011] In yet another aspect, a system may be provided for loading a map model. The system may comprise a first mobile device that is located in a first area. The first mobile device comprising: at least one first processor; and a first non-transitory computer readable medium communicatively coupled to the at least one first processor, the first non-transitory computer readable medium having stored thereon a software application comprising a first set of instructions that, when executed by the at least one first processor, causes the first mobile device to: generate a first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area; generate a first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor, the first visual feature point map model being correlated with the first wireless signal fingerprint map model; send the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system over a network, the remote computing system being configured to extract a regional visual feature point map model based on analysis of the correlated first wireless signal fingerprint map model and first visual feature point map model; and receive the regional visual feature point map model from the remote computing system [0012] Various modifications and additions can be made to the embodiments discussed without departing from the scope of the invention. For example, while the embodiments described above refer to particular features, the scope of this invention also includes embodiments having different combination of features and embodiments that do not include all of the above-described features.

[0013] The details of one or more aspects of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description and drawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

[0014] A further understanding of the nature and advantages of particular embodiments may be realized by reference to the remaining portions of the specification and the drawings, in which like reference numerals are used to refer to similar components. In some instances, a sub-label is associated with a reference numeral to denote one of multiple similar components. When reference is made to a reference numeral without specification to an existing sub-label, it is intended to refer to all such multiple similar components.

[0015] Fig. 1 is a schematic diagram illustrating a system for implementing multi person simultaneous location and mapping ("SLAM") linkage positioning and navigation, in accordance with various embodiments.

[0016] Fig. 2 is a schematic block flow diagram illustrating a non-limiting example of a system for implementing multi-person SLAM linkage positioning and navigation, in accordance with various embodiments.

[0017] Figs. 3A and 3B are schematic diagrams illustrating various non-limiting examples of wireless signal fingerprint maps and visual feature point maps that may be used when implementing multi-person SLAM linkage positioning and navigation, in accordance with various embodiments.

[0018] Figs. 4A-4I are flow diagrams illustrating a method for implementing multi-person SLAM linkage positioning and navigation, in accordance with various embodiments. [0019] Fig. 5 is a block diagram illustrating an example of computer or system hardware architecture, in accordance with various embodiments.

[0020] Fig. 6 is a block diagram illustrating a networked system of computers, computing systems, or system hardware architecture, which can be used in accordance with various embodiments.

DETAILED DESCRIPTION

[0021] Overview

[0022] Various embodiments provide tools and techniques for implementing simultaneous location and mapping ("SLAM") functionalities, and, more particularly, to methods, systems, and apparatuses for implementing multi-person SLAM linkage positioning and navigation, in some cases, for super large-scale scenes based on wireless signal characteristics and visual feature point information.

[0023] In various embodiments, for creating a map model, a software application, which may be running on a first mobile device that is located in a first area, may generate or update a first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within, or in proximity to, the first area. The software application may generate or update a first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor, the first visual feature point map model being correlated with the first wireless signal fingerprint map model. The software application may send the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system over a network, the remote computing system being configured to generate a global map of the first area based on analysis and integration of the correlated first wireless signal fingerprint map model and first visual feature point map model.

[0024] In some embodiments, the mobile device may comprise at least one of a smartphone, a tablet computer, an augmented reality (" AR") device, a virtual reality ("VR") device, a mixed reality ("MR") device, a portable gaming device, or a robot, and/or the like. [0025] Merely by way of example, in some cases, the first area may comprise at least one of a room, a corridor, a courtyard, a floor level of a building, a building, a rooftop of a building, a balcony of a building, a patio of a building, a porch of a building, a residential facility, a business facility, a commercial facility, a parking facility, a storage facility, a facility operated by a public utility, a government facility, a military facility, an industrial facility, a transportation facility, a school facility, a school campus, a medical facility, a stadium, an athletics facility, an auditorium, a garden, a park, a zoo, a neighborhood, a city block of a municipality, a city center of a municipality, a municipality, a forest, a jungle, a field, agricultural land, a regional open space, a body of water, a waterway, an underwater area, an island region, a mountain region, a desert region, a rocky region, a sandy region, a terrestrial vehicle, an aquatic vehicle, an amphibious vehicle, an aircraft, a space vehicle, an aquatic facility, a space-based facility, or a portion of one of these areas, and/or the like.

[0026] According to some embodiments, the first signal data that is received from each of the at least one first wireless signal source may comprise at least one of Wi-Fi signal data, ultra-wideband ("UWB") signal data, ultra-wideband Doherty ("UWD") signal data, Bluetooth signal data, or Zig-Bee signal data, and/or the like. In some instances, the at least one first visual sensor may comprise at least one of a red-green- blue ("RGB") color camera, a wide-angle camera, an ultra-wide-angle camera, a fish- eye camera, or a telephoto camera, and/or the like.

[0027] In some embodiments, sending the correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network may comprise sending the correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network using a wireless network data transmission protocol, wherein the wireless network data transmission protocol comprises at least one of Wi-Fi protocol, Wi-Fi Direct protocol, other protocols under IEEE 802.11 standard, Bluetooth protocol, Zig-Bee protocol, other protocols under IEEE 802.15 standard, 4G broadband protocol, 4G LTE broadband protocol, 5G broadband protocol, or other cellular broadband protocol, and/or the like.

[0028] According to some embodiments, the first wireless signal fingerprint map model of the first area may be generated or updated based at least in part on characteristic information of the first signal data received from each first wireless signal source at a location at which the mobile device receives the first signal data. In some cases, the characteristic information of the first signal data may comprise at least one of name of a wireless network corresponding to said wireless signal source, a media access control ("MAC") address of said wireless signal source, or signal strength of said wireless signal source, and/or the like.

[0029] In some embodiments, the first visual feature point map model of at least a portion of the first area may be generated or updated by: extracting, using the software application, feature point data from the received image data of the at least a portion of the first area, the feature point data comprising at least one of object feature point data or environmental feature point data; and implementing, using the software application, at least one of simultaneous localization and mapping ("SLAM") techniques or image recognition techniques on the extracted feature point data to generate the first visual feature point map model of the at least a portion of the first area.

[0030] According to some embodiments, generating or updating the first visual feature point map model of the at least a portion of the first area and sending the correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network may comprise repeating the following processes until the first visual feature point map model covers all portions of the first area covered by the first wireless signal fingerprint map model: generating or updating, using the software application, a first visual feature point map model of one portion of the first area based on image data received from the at least one first visual sensor for said portion of the first area; and sending, using the software application, the correlated first wireless signal fingerprint map model and first visual feature point map model of said portion of the first area to the remote computing system over the network.

[0031] In some embodiments, a transceiver of the mobile device may continually or periodically receive updated first signal data from each of the at least one first wireless signal source within or in proximity to the first area. Based on a determination that the updated first signal data has changed compared with previously received updated first signal data, the software application may update the first wireless signal fingerprint map model of the first area based on the updated first signal data. At least one first visual sensor of the first mobile device may capture image data of objects within at least a portion of the first area. Based on a determination that the updated image data has changed compared with previously received updated image data, the software application may update the first visual feature point map model of the at least a portion of the first area based on the updated image data. The software application may send updated correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network, the remote computing system being further configured to generate an updated global map of the first area based on analysis and integration of the updated correlated first wireless signal fingerprint map model and first visual feature point map model.

[0032] According to some embodiments, as the mobile device has moved to a second area, the software application may generate or update a second wireless signal fingerprint map model of the second area based on second signal data that is received from each of at least one second wireless signal source within or in proximity to the second area, wherein the second area is one of a different portion of the first area, an area in proximity and overlapping with the first area, or an area in proximity with yet separate from the first area, or the like. The software application may generate or update a second visual feature point map model of at least a portion of the second area based on second image data received from the at least one first visual sensor, the second visual feature point map model being correlated with the second wireless signal fingerprint map model. The software application may send the correlated second wireless signal fingerprint map model and second visual feature point map model to the remote computing system over the network, the remote computing system being further configured to generate a global map of a combination of the first area and the second area based on analysis and integration of the correlated second wireless signal fingerprint map model and second visual feature point map model. [0033] Alternatively, or additionally, for creating a map model, a remote computing system may receive, from a software application running on a first mobile device that is located in a first area, correlated first wireless signal fingerprint map model and first visual feature point map model, the software application being configured to generate or update the first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area and further configured to generate or update the first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor. The remote computing system may analyze the correlated first wireless signal fingerprint map model and first visual feature point map model. Based on the analysis, the remote computing system may integrate the correlated received first wireless signal fingerprint map model and first visual feature point map model to generate a global map of the first area.

[0034] In some cases, the remote computing system may comprise at least one of a server computer over the network, an image processing server, a graphics processing unit ("GPU") -based server, a positioning and mapping server, a machine learning system, an artificial intelligence ("AI") system, a deep learning system, a neural network, a convolutional neural network ("CNN"), a fully convolutional network ("FCN"), a cloud computing system, or a distributed computing system, and/or the like.

[0035] In some embodiments, the first mobile device may be associated with a first user, wherein the first mobile device may be among a plurality of mobile devices located within the first area, the plurality of mobile devices being associated with a plurality of users. In such a case, the remote computing system may receive correlated wireless signal fingerprint map model and visual feature point map model from each of the plurality of mobile devices, may analyze the correlated wireless signal fingerprint map model and visual feature point map model from each of the plurality of mobile devices, and, based on the analysis, may integrate the correlated received wireless signal fingerprint map model and visual feature point map model from each of the plurality of mobile devices to generate the global map of the first area.

[0036] According to some embodiments, the remote computing system may analyze the correlated first wireless signal fingerprint map model and first visual feature point map model, by: comparing the first wireless signal fingerprint map model with one or more portions of a previously generated global wireless signal fingerprint map using wireless signal fingerprint matching; in response to identifying a matching portion of the previously generated global wireless signal fingerprint map based on the comparison, comparing the first visual feature point map model with one or more portions of a previously generated visual feature point map corresponding to the identified matching portion of the previously generated global wireless signal fingerprint map using visual feature point matching; and in response to identifying a matching portion of the previously generated visual feature point map, extracting a regional visual feature point map model based on the identified matching portion of the previously generated visual feature point map, and sending the regional visual feature point map model to a positioning and navigation server.

[0037] In some embodiments, in response to receiving the regional visual feature point map model, the positioning and navigation server may mount the received regional visual feature point map model to a simultaneous localization and mapping ("SLAM") system to locate and navigate the first area, and may concurrently send real-time local area pose information to the remote computing system. In response to receiving the real-time local area pose information, the remote computing system may convert the real-time local area pose information into global pose coordinates using a global coordinate conversion system, and may distribute the global pose coordinates to one or more second mobile devices either within the first area or in proximity of the first area, to establish pose information sharing and linkage between the first mobile device and each of the one or more second devices, wherein the one or more second mobile devices may each be associated with one or more other users.

[0038] In another aspect, for loading a map model, a software application, which may be running on a first mobile device that is located in a first area, may generate a first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area. The software application may generate a first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor, the first visual feature point map model being correlated with the first wireless signal fingerprint map model. The software application may send the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system over a network, the remote computing system being configured to extract a regional visual feature point map model based on analysis of the correlated first wireless signal fingerprint map model and first visual feature point map model. The software application may receive, over the network, the regional visual feature point map model from the remote computing system. [0039] Alternatively, or additionally, for loading a map model, a remote computing system may receive, from a software application running on a first mobile device that is located in a first area, correlated first wireless signal fingerprint map model and first visual feature point map model, the software application being configured to generate the first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area and further configured to generate the first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor. The remote computing system may analyze the correlated first wireless signal fingerprint map model and first visual feature point map model, by: comparing the first wireless signal fingerprint map model with one or more portions of a previously generated global wireless signal fingerprint map using wireless signal fingerprint matching; in response to identifying a matching portion of the previously generated global wireless signal fingerprint map based on the comparison, comparing the first visual feature point map model with one or more portions of a previously generated visual feature point map corresponding to the identified matching portion of the previously generated global wireless signal fingerprint map using visual feature point matching; and in response to identifying a matching portion of the previously generated visual feature point map, extracting the regional visual feature point map model based on the identified matching portion of the previously generated visual feature point map, and sending the regional visual feature point map model as positioning and navigation data to the software application running on the first mobile device.

[0040] In some embodiments, sending the regional visual feature point map model as positioning and navigation data to the software application running on the first mobile device may comprise: sending, using the remote computing system, the regional visual feature point map model to a positioning and navigation server; and in response to receiving the regional visual feature point map model, mounting, using the positioning and navigation server, the received regional visual feature point map model to a simultaneous localization and mapping ("SLAM") system to locate and navigate the first area.

[0041] According to some embodiments, concurrent with mounting the received regional visual feature point map model to the SLAM system, the positioning and navigation server may send real-time local area pose information to the remote computing system. In response to receiving the real-time local area pose information, the remote computing system may convert the real-time local area pose information into global pose coordinates using a global coordinate conversion system. The remote computing system may distribute the global pose coordinates to one or more second mobile devices either within the first area or in proximity of the first area, to establish pose information sharing and linkage between the first mobile device and each of the one or more second devices, wherein the one or more second mobile devices may each be associated with one or more other users.

[0042] The various aspects described herein provide a software system architecture based on mobile equipment, wireless networks, wireless signals, cloud services, and SLAM technology, and realizes accurate positioning and efficient local area map search in large-scale indoor (as well as outdoor) areas, while simultaneously enabling the real-time linkage of multiple people (via their mobile devices) to share their location relationships. The various embodiments include two parts: a mobile client and a back-end cloud service. The mobile client collects the wireless signal environment characteristics of the surrounding area to generate a corresponding fingerprint map model and draws a feature point map model of the surrounding environment through a computer vision algorithm. The mobile client sends the collected information to the back-end cloud service through the wireless network.

The back-end cloud service is responsible for integrating regional information (i.e., wireless signal fingerprint map model, feature point map model, etc.) to generate larger global map information. At the same time, the back-end cloud service compares and matches the area map, and returns it to the mobile device. The mobile device realizes high-precision positioning in the corresponding area according to the obtained map model data, and shares real-time pose information among multiple mobile devices through the back-end cloud service.

[0043] These and other aspects of the system and method for multi-person SLAM linkage positioning and navigation (in some cases, for super large-scale scenes based on wireless signal characteristics and visual feature point information) are described in greater detail with respect to the figures.

[0044] The following detailed description illustrates a few embodiments in further detail to enable one of skill in the art to practice such embodiments. The described examples are provided for illustrative purposes and are not intended to limit the scope of the invention.

[0045] In the following description, for the purposes of explanation, numerous details are set forth in order to provide a thorough understanding of the described embodiments. It will be apparent to one skilled in the art, however, that other embodiments of the present invention may be practiced without some of these details. In other instances, some structures and devices are shown in block diagram form. Several embodiments are described herein, and while various features are ascribed to different embodiments, it should be appreciated that the features described with respect to one embodiment may be incorporated with other embodiments as well. By the same token, however, no single feature or features of any described embodiment should be considered essential to every embodiment of the invention, as other embodiments of the invention may omit such features.

[0046] Unless otherwise indicated, all numbers used herein to express quantities, dimensions, and so forth used should be understood as being modified in all instances by the term "about." In this application, the use of the singular includes the plural unless specifically stated otherwise, and use of the terms "and" and "or" means "and/or" unless otherwise indicated. Moreover, the use of the term "including," as well as other forms, such as "includes" and "included," should be considered non exclusive. Also, terms such as "element" or "component" encompass both elements and components comprising one unit and elements and components that comprise more than one unit, unless specifically stated otherwise.

[0047] Various embodiments as described herein - while embodying (in some cases) software products, computer-performed methods, and/or computer systems - represent tangible, concrete improvements to existing technological areas, including, without limitation, positioning technology, navigation technology, positioning and navigation technology, simultaneous location and mapping ("SLAM") technology, mobile device linkage technology, and/or the like. In other aspects, some embodiments can improve the functioning of user equipment or systems themselves (e.g., positioning systems, navigation systems, positioning and navigation systems, simultaneous location and mapping ("SLAM") systems, mobile device linkage systems, etc.), for example, by generating or updating, using a software application running on a first mobile device that is located in a first area, a first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area; generating or updating, using the software application, a first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor, the first visual feature point map model being correlated with the first wireless signal fingerprint map model; sending, using the software application, the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system over a network; analyzing, using the remote computing system, the correlated first wireless signal fingerprint map model and first visual feature point map model; and based on the analysis, integrating, using the remote computing system, the correlated received first wireless signal fingerprint map model and first visual feature point map model to generate a global map of the first area; and/or the like.

[0048] In particular, to the extent any abstract concepts are present in the various embodiments, those concepts can be implemented as described herein by devices, software, systems, and methods that involve novel functionality (e.g., steps or operations), such as, generating or updating (a) a wireless signal fingerprint map model and (b) a visual feature point map model, analyzing these two models, and based on the analysis integrating them to generate a global map of an area of interest, which may subsequently be divided into regional sub-areas whose maps may be sent to future mobile devices based on feature matching of the wireless signal fingerprint map model with previously generated global wireless signal fingerprint map and subsequently feature matching of visual feature point map model from those mobile devices with previously generated global visual feature point map, and/or the like, to name a few examples, that extend beyond mere conventional computer processing operations. These functionalities can produce tangible results outside of the implementing computer system, including, merely by way of example, fast accurate, high-precision, positioning and navigation, as well as efficient local area map search in a large-scale environment, while simultaneously enabling real-time linkage of multiple mobile devices (and thus their users) to share location relationships and visual/virtual data (e.g., real-time pose information, map information, simulated or virtual constructs, etc.), at least some of which may be observed or measured by users, service providers, AR/VR/MR content and/or equipment developers, game/content developers, and/or user device manufacturers.

[0049] Some Embodiments

[0050] We now turn to the embodiments as illustrated by the drawings. Figs. 1-6 illustrate some of the features of the method, system, and apparatus for implementing simultaneous location and mapping ("SLAM") functionalities, and, more particularly, to methods, systems, and apparatuses for implementing multi-person SLAM linkage positioning and navigation, in some cases, for super large-scale scenes based on wireless signal characteristics and visual feature point information, as referred to above. The methods, systems, and apparatuses illustrated by Figs. 1-6 refer to examples of different embodiments that include various components and steps, which can be considered alternatives or which can be used in conjunction with one another in the various embodiments. The description of the illustrated methods, systems, and apparatuses shown in Figs. 1-6 is provided for purposes of illustration and should not be considered to limit the scope of the different embodiments.

[0051] With reference to the figures, Fig. 1 is a schematic diagram illustrating a system 100 for implementing multi-person simultaneous location and mapping ("SLAM") linkage positioning and navigation, in accordance with various embodiments.

[0052] In the non-limiting embodiment of Fig. 1, system 100 may comprise one or more mobile devices 105a-105n (collectively, "mobile device 105," "user devices 105," or the like) that may be located with a first area 110. In some embodiments, each mobile device 105 may include, without limitation, at least one of a smartphone, a tablet computer, an augmented reality ("AR") device, a virtual reality ("VR") device, a mixed reality ("MR") device, a portable gaming device, or a robot, and/or the like. Merely by way of example, in some cases, the first area 110 may include, but is not limited to, at least one of a room, a corridor, a courtyard, a floor level of a building, a building, a rooftop of a building, a balcony of a building, a patio of a building, a porch of a building, a residential facility, a business facility, a commercial facility, a parking facility, a storage facility, a facility operated by a public utility, a government facility, a military facility, an industrial facility, a transportation facility, a school facility, a school campus, a medical facility, a stadium, an athletics facility, an auditorium, a garden, a park, a zoo, a neighborhood, a city block of a municipality, a city center of a municipality, a municipality, a forest, a jungle, a field, agricultural land, a regional open space, a body of water, a waterway, an underwater area, an island region, a mountain region, a desert region, a rocky region, a sandy region, a terrestrial vehicle, an aquatic vehicle, an amphibious vehicle, an aircraft, a space vehicle, an aquatic facility, a space-based facility, or a portion of one of these areas, and/or the like.

[0053] System 100 may further comprise one or more wireless signal sources 115a-115n (collectively, "wireless signal sources 115," "signal sources 115," or the like) and one or more objects 120, both of which may be either located within first area 110 or in close proximity to (or, in the case of the object(s) 120, in line of sight ol) the first area 110. The wireless signal sources 115 may be any wireless communications device - including, but not limited to, WiFi hotspots, WiFi modems, Bluetooth™ devices, ultra-wideband ("UWB") communications devices, ultra- wideband Doherty ("UWD") communications devices, and/or the like - that broadcasts or transmits information, including, but not limited to, at least one of name of a wireless network corresponding to said wireless signal source, a media access control ("MAC") address of said wireless signal source, or signal strength of said wireless signal source, and/or the like.

[0054] According to some embodiments, the objects 120 may be any free standing, mounted, integrated, and/or suspended object that is visible within, or within line of sight of, the first area 110, and may include, without limitation, at least one of furniture, lighting fixtures, art pieces (e.g., paintings, photographs, sculptures, etc.), decorations, doors, windows, skylights, moonlights, panels, frames, clothing, curtains, utility ports (e.g., electrical sockets, USB ports, Ethernet cable ports, etc.), electronics devices (e.g., smart phones, mobile phones, gaming devices (consoles, portable devices, controllers), televisions, media playback (and/or recording) devices, remote controllers, tablet computers, laptop computers, desktop computers, servers, etc.), office equipment (e.g., desks, chairs, lamps, printers, scanners, photocopiers, fans, heaters, etc.), kitchen appliances (e.g., microwave oven, toaster, kettle, coffee machine, tea brewing machine, refrigerator, dish washer, oven, stove and/or range, etc.), home appliances (e.g., washer, dryer, air conditioner, furnace, thermostat, etc.), vehicles, and/or natural settings, nature-scape, or manmade landscape objects, or the like. [0055] In some embodiments, each mobile device 105 may include, without limitation, a wireless transceiver(s) 125, a visual sensor(s) 130, and a processor(s)

135. In some instances, the wireless transceiver 125 and the wireless signal sources 115 may each include wireless communications devices capable of communicating using protocols including, but not limited to, at least one of Bluetooth™ communications protocol, WiFi communications protocol, ultra-wideband ("UWB") communications protocol, ultra-wideband Doherty ("UWD") communications protocol, or other 802.11 suite of communications protocols, ZigBee or Zig-Bee communications protocol, Z-wave communications protocol, or other 802.15.4 suite of communications protocols, cellular communications protocol (e.g., 3G, 4G, 4G LTE, 5G, etc.), or other suitable communications protocols, and/or the like. In some instances, the visual sensor(s) 130 may include, without limitation, at least one of a red-green-blue ("RGB") color camera, a wide-angle camera, an ultra- wide-angle camera, a fish-eye camera, or a telephoto camera, and/or the like. The processor(s) 135, in some cases, may include, but is not limited to, at least one of a central processing unit ("CPU"), one or more graphics processing units ("GPUs"), or the like. [0056] According to some embodiments, a software application (or "app") 140 that is executed by the processor(s) 135 on the mobile device 105 may include, but is not limited to, a wireless signal fingerprint map model creator or creation module 145a, a visual feature point map model creator or creation module 145b, and a SLAM module 145c, and/or the like. The mobile device 105 may further include, without limitation, data storage 150 and display device(s) 155. The data storage 150 may be used to store at least wireless signal fingerprint maps, visual feature point maps, and/or local/regional maps, or the like, among other mobile device data. The display device(s) 155 - which may include, without limitation, at least one of a non touchscreen display device(s), a touchscreen display device(s), an AR/VR/MR headset display device(s), a projected display device(s), or a virtual display device(s), and/or the like - may be used to display the maps and/or any linked visual feature data (as described in detail below with respect to features of the system after loading map models).

[0057] System 100 may further comprise a remote computing system 160 that is accessible via network(s) 165. The lightning bolt symbols are used to denote wireless communications between (wireless transceiver(s) 125 of each) mobile device 105 or 105a-105n and network(s) 165 (in some cases, via network access points, telecommunications relay systems, or the like (not shown)). The remote computing system 160 may include, without limitation, a global map integrated generator or generation module 170, a feature matching system or module 175, a global coordinate converter or conversion module 180, and a positioning and navigation system 185. [0058] In some cases, the remote computing system 160 may include, but is not limited to, at least one of a server computer over the network, an image processing server, a graphics processing unit ("GPU") -based server, a positioning and mapping server, a machine learning system, an artificial intelligence ("AI") system, a deep learning system, a neural network, a convolutional neural network ("CNN"), a fully convolutional network ("FCN"), a cloud computing system, or a distributed computing system, and/or the like. In some embodiments, networks 165 may each include, without limitation, one of a local area network ("LAN"), including, without limitation, a fiber network, an Ethernet network, a Token-Ring™ network, and/or the like; a wide-area network ("WAN"); a wireless wide area network ("WWAN"); a virtual network, such as a virtual private network ("VPN"); the Internet; an intranet; an extranet; a public switched telephone network ("PSTN"); an infra-red network; a wireless network, including, without limitation, a network operating under any of the IEEE 802.11 suite of protocols, the Bluetooth™ protocol known in the art, and/or any other wireless protocol; and/or any combination of these and/or other networks. In a particular embodiment, the network(s) 165 may include an access network of the service provider (e.g., an Internet service provider ("ISP")). In another embodiment, the network(s) 165 may include a core network of the service provider and/or the Internet.

[0059] In operation, the system may provide two models for operation (as shown and described below with respect to Fig. 2) - namely, a "Create Map Model" and a "Load Map Model."

[0060] In an aspect, for creating a map model, a software application (e.g., software application 140, or the like) running on a first mobile device (e.g., mobile device 105 among mobile devices 105a-105n, or the like) that is located in a first area (e.g., first area 110, or the like) may generate or update a first wireless signal fingerprint map model of the first area (e.g., using wireless signal fingerprint map model creator 145a, or the like) based on first signal data that is received from each of at least one first wireless signal source (e.g., wireless signal sources 115a-115n, or the like, via wireless transceiver(s) 125, or the like) within, or in proximity to, the first area. According to some embodiments, the first signal data that is received from each of the at least one first wireless signal source may include, but is not limited to, at least one of Wi-Fi signal data, ultra-wideband ("UWB") signal data, ultra-wideband Doherty ("UWD") signal data, Bluetooth signal data, or Zig-Bee signal data, and/or the like.

[0061] The software application may generate or update a first visual feature point map model of at least a portion of the first area (e.g., using visual feature point map model creator 145b and/or SLAM 145c, or the like) based on first image data received from the at least one first visual sensor (e.g., visual sensor(s) 130 capturing images (and/or videos) of object(s) 120 within, or in line of sight of, the first area, or the like), the first visual feature point map model being correlated with the first wireless signal fingerprint map model. In some cases, the software application may send the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system over a network (e.g., remote computing system 160 via network(s) 165 via wireless transceiver(s) and any intermediate telecommunications relays systems (not shown), or the like).

[0062] The remote computing system may analyze the correlated first wireless signal fingerprint map model and first visual feature point map model. Based on the analysis, the remote computing system may integrate the correlated received first wireless signal fingerprint map model and first visual feature point map model to generate a global map of the first area (e.g., using global map integrated generator 170, or the like).

[0063] In some embodiments, sending the correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network may comprise sending the correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network using a wireless network data transmission protocol. In some cases, the wireless network data transmission protocol may include, without limitation, at least one of Wi-Fi protocol, Wi-Fi Direct protocol, other protocols under IEEE 802.11 standard, Bluetooth protocol, Zig-Bee protocol, other protocols under IEEE 802.15 standard, 4G broadband protocol, 4G LTE broadband protocol, 5G broadband protocol, or other cellular broadband protocol, and/or the like.

[0064] According to some embodiments, the first wireless signal fingerprint map model of the first area may be generated or updated based at least in part on characteristic information of the first signal data received from each first wireless signal source at a location at which the mobile device receives the first signal data. In some cases, the characteristic information of the first signal data may include, but is not limited to, at least one of name of a wireless network corresponding to said wireless signal source, a media access control ("MAC") address of said wireless signal source, or signal strength of said wireless signal source, and/or the like.

[0065] In some embodiments, the first visual feature point map model of at least a portion of the first area may be generated or updated by: the software application extracting feature point data from the received image data of the at least a portion of the first area, the feature point data including, but not limited to, at least one of object feature point data (i.e., feature point data of objects, or the like) or environmental feature point data (i.e., feature point data of the environment, or the like), or the like; and the software application implementing at least one of simultaneous localization and mapping ("SLAM") techniques or image recognition techniques, and/or the like, on the extracted feature point data to generate the first visual feature point map model of the at least a portion of the first area.

[0066] According to some embodiments, due to data storage limitations with respect to mobile devices and the fact that visual feature point map data requires greater memory requirements compared, e.g., with wireless signal fingerprint map data, it may be necessary to obtain visual feature point map data for a portion or fragment of the first area, send that data to the remote computing system, and continue obtaining visual feature point map data for the rest of the first area. Meanwhile, the system can obtain wireless signal fingerprint map data for the entire first area without needing to fragment such data collection. Accordingly, generating or updating the first visual feature point map model of the at least a portion of the first area and sending the correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network may comprise repeating the following processes until the first visual feature point map model covers all portions of the first area covered by the first wireless signal fingerprint map model: the software application generating or updating a first visual feature point map model of one portion of the first area based on image data received from the at least one first visual sensor for said portion of the first area; and the software application sending the correlated first wireless signal fingerprint map model and first visual feature point map model of said portion of the first area to the remote computing system over the network.

[0067] In some embodiments, a transceiver of the mobile device (e.g., wireless transceiver(s) 125 of mobile device 105, or the like) may continually or periodically receive updated first signal data from each of the at least one first wireless signal source within, or in proximity to, the first area. Based on a determination that the updated first signal data has changed compared with previously received updated first signal data, the software application may update the first wireless signal fingerprint map model of the first area (e.g., using wireless signal fingerprint map model creator 145 a, or the like) based on the updated first signal data. At least one first visual sensor of the first mobile device (e.g., visual sensor(s) 130 of mobile device 105, or the like) may capture image data of objects within at least a portion of the first area (i.e., while the object(s) 120 is within the field of view ("FOV") 130a of visual sensor(s) 130, or the like). Based on a determination that the updated image data has changed compared with previously received updated image data, the software application may update the first visual feature point map model of the at least a portion of the first area (e.g., using visual feature point map model creator 145b and/or SLAM 145c, or the like) based on the updated image data. The software application may then send updated correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network. The remote computing system may analyze the updated correlated first wireless signal fingerprint map model and first visual feature point map model. Based on the analysis, the remote computing system may integrate the correlated received first wireless signal fingerprint map model and first visual feature point map model to generate an updated global map of the first area (e.g., using global map integrated generator 170, or the like).

[0068] As the system is designed to enhance mapping and navigation within an area and not for static location determination or mapping, the system is capable of dynamic mapping and navigation as the user moves through the area. As such, according to some embodiments, as the mobile device moves to a second area, the software application may generate or update a second wireless signal fingerprint map model of the second area (e.g., using wireless signal fingerprint map model creator 145a, or the like) based on second signal data that is received from each of at least one second wireless signal source within, or in proximity to, the second area. Here, the second area may be one of a different portion of the first area, an area in proximity and overlapping with the first area, or an area in proximity with yet separate from the first area, or the like. The software application may generate or update a second visual feature point map model of at least a portion of the second area (e.g., using visual feature point map model creator 145b and/or SLAM 145c, or the like) based on second image data received from the at least one first visual sensor, the second visual feature point map model being correlated with the second wireless signal fingerprint map model. The software application may send the correlated second wireless signal fingerprint map model and second visual feature point map model to the remote computing system over the network. The remote computing system may analyze the correlated second wireless signal fingerprint map model and second visual feature point map model. Based on the analysis, the remote computing system may integrate the correlated received second wireless signal fingerprint map model and second visual feature point map model to generate a global map of a combination of the first area and the second area (e.g., using global map integrated generator 170, or the like). [0069] In some embodiments, the first mobile device may be associated with a first user, where the first mobile device may be among a plurality of mobile devices (e.g., mobile devices 105a-105n, or the like) located within the first area, the plurality of mobile devices being associated with a plurality of users. In such a case, the remote computing system may receive correlated wireless signal fingerprint map model and visual feature point map model from each of the plurality of mobile devices. The remote computing system may analyze the correlated wireless signal fingerprint map model and visual feature point map model from each of the plurality of mobile devices. Based on the analysis, the remote computing system may integrate the correlated received wireless signal fingerprint map model and visual feature point map model from each of the plurality of mobile devices to generate the global map of the first area (e.g., using global map integrated generator 170, or the like). [0070] According to some embodiments, the remote computing system may analyze the correlated first wireless signal fingerprint map model and first visual feature point map model, by: comparing the first wireless signal fingerprint map model with one or more portions of a previously generated global wireless signal fingerprint map using wireless signal fingerprint matching; in response to identifying a matching portion of the previously generated global wireless signal fingerprint map based on the comparison, comparing the first visual feature point map model with one or more portions of a previously generated visual feature point map corresponding to the identified matching portion of the previously generated global wireless signal fingerprint map using visual feature point matching (e.g., using feature matching system 175, or the like); and, in response to identifying a matching portion of the previously generated visual feature point map, extracting a regional visual feature point map model based on the identified matching portion of the previously generated visual feature point map, and sending the regional visual feature point map model to a positioning and navigation server (e.g., using positioning and navigation 185, or the like).

[0071] In some embodiments, in response to receiving the regional visual feature point map model, the positioning and navigation server may mount the received regional visual feature point map model to the SLAM system (e.g., SLAM 145c, or the like) to locate and navigate the first area, and concurrently send real-time local area pose information to the remote computing system. Herein, "pose information" may refer to a combination of position and orientation of an object and/or exterior orientation and translation of the object. In response to receiving the real-time local area pose information, the remote computing system may convert the real-time local area pose information into global pose coordinates using a global coordinate conversion system (e.g., using global coordinate converter 180, or the like). The remote computing system may distribute the global pose coordinates to one or more second mobile devices either within the first area or in proximity of the first area, to establish pose information sharing and linkage between the first mobile device and each of the one or more second devices, where the one or more second mobile devices may each be associated with one or more other users.

[0072] In another aspect, for loading a map model, a software application (e.g., software application 140, or the like) running on a first mobile device (e.g., mobile device 105 among mobile devices 105a-105n, or the like) that is located in a first area (e.g., first area 110, or the like) may generate a first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within, or in proximity to, the first area. The software application may generate a first visual feature point map model of at least a portion of the first area based on first image data received from the at least one first visual sensor, the first visual feature point map model being correlated with the first wireless signal fingerprint map model. The software application may send the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system (e.g., remote computing system 160, or the like) over a network (e.g., network(s) 165, or the like). The remote computing system may analyze the correlated first wireless signal fingerprint map model and first visual feature point map model, by: comparing the first wireless signal fingerprint map model with one or more portions of a previously generated global wireless signal fingerprint map using wireless signal fingerprint matching; in response to identifying a matching portion of the previously generated global wireless signal fingerprint map based on the comparison, comparing the first visual feature point map model with one or more portions of a previously generated visual feature point map corresponding to the identified matching portion of the previously generated global wireless signal fingerprint map using visual feature point matching (e.g., using feature matching system 175, or the like); and, in response to identifying a matching portion of the previously generated visual feature point map, extracting a regional visual feature point map model based on the identified matching portion of the previously generated visual feature point map, and sending the regional visual feature point map model as positioning and navigation data to the software application running on the first mobile device.

[0073] In some embodiments, sending the regional visual feature point map model as positioning and navigation data to the software application running on the first mobile device may comprise: the remote computing system sending the regional visual feature point map model to a positioning and navigation server (e.g., using positioning and navigation system 185, or the like); and, in response to receiving the regional visual feature point map model, the positioning and navigation server mounting the received regional visual feature point map model to a SLAM system (e.g., SLAM 145c, or the like) to locate and navigate the first area.

[0074] According to some embodiments, concurrent with mounting the received regional visual feature point map model to the SLAM system, the positioning and navigation server may send real-time local area pose information to the remote computing system. In response to receiving the real-time local area pose information, the remote computing system may convert the real-time local area pose information into global pose coordinates using a global coordinate conversion system (e.g., using global coordinate converter 180, or the like). The remote computing system may distribute the global pose coordinates to one or more second mobile devices either within the first area or in proximity of the first area, to establish pose information sharing and linkage between the first mobile device and each of the one or more second devices, where the one or more second mobile devices may each be associated with one or more other users.

[0075] By fusing wireless signal positioning technology and visual sensor positioning technology, and relying on back-end cloud services, the various embodiments can meet the requirements of lightweight mobile device performance, ultra-wide-range instant positioning and navigation within any environment (both indoor and outdoor, so long as wireless signal sources either are already present or are specifically placed (e.g., portable wireless signal sources, etc.)), and supporting multiple mobile devices to share real-time position information with each other. The system and method are suitable for navigation of smart mobile devices, single-person and multi-person entertainment, and other scenarios (including, but not limited to, teleconferencing or other meetings, gaming, education, search and rescue, etc.).

[0076] Fusion of wireless signal positioning technology and visual sensor positioning technology, through the advantages of these two technologies, compensate for each other's disadvantages. For example, positioning and navigation based on visual sensors alone may be greatly affected by light and cannot work in a dark environment. And, because massive image data needs to be processed, mobile devices cannot carry super large-scale visual feature point map models, and thus the positioning area is limited. The wireless signal positioning technology has the advantages of low power consumption, low delay, and large range, which can help the visual sensor positioning technology to draw larger-scale map models. However, the wireless signal positioning technology alone has the problems of being susceptible to interference from the surrounding environment and having poor accuracy. These disadvantages can be compensated by the advantages of the higher accuracy of the visual sensor positioning technology and the fewer interference factors from the surrounding environment. Moreover, both technologies have the advantage of low cost, which makes the architecture of the various embodiments low in cost and easy to implement.

[0077] On the other hand, the hardware performance of mobile devices is limited. The data capacity of the map model established by the visual sensor increases as the range increases. When facing large-scale indoor (or outdoor) positioning and navigation requirements, its huge map model data file cannot mount to the mobile device's memory. The back-end cloud service has unlimited computing power and storage capacity, and stores huge map model data files in the cloud and divides the map model data into scattered regional map model data. When the mobile device is in the corresponding area, the back-end cloud service finds the map model data of the corresponding area and sends it to the mobile device. Mobile devices can thus realize positioning and navigation through regional map model data.

[0078] After the mobile device achieves positioning through map model data and SLAM, it uploads its real-time coordinate location to the back-end cloud service. The back-end cloud service combined with the global map model data can accurately locate the current regional location of the mobile device and can share this location information with other mobile devices in the same scene, so as to realize the location information perception and linkage between the mobile devices.

[0079] With linkage between mobile devices associated with various different users within the same area but at different sub-areas with different poses (i.e., position and orientation of each user, or the like), common information may be shared. For example, virtual constructs observed by one mobile device associated with one person may be shared with another mobile device associated with another user (or multiple mobile devices associated with multiple users) who are in the same area (e.g., same room, or the like). The combination of wireless signal fingerprint map data, visual feature point map data, and pose information from each mobile device is analyzed by the remote computing system, which not only sends the regional map data to each mobile device in the area, but also data on the virtual constructs that may be shared as well as pose information of the other mobile devices (and users). Other data (e.g., status, communication, etc.) may also be shared using this linkage.

[0080] Thus, for implementations such as education, and instructor and the students may share a virtual or mixed reality environment based on the linkage achieved in accordance with the various embodiments to enhance the learning experience for the instructor as well as the students. For implementations related to teleconferencing or other meetings, one mobile device that is communicating with a user who is not present in the room may share through the linkage the video (and audio) of the physically absent user, such that other physically present users may also see through the linkage (via their respective mobile devices) the video (and audio) of the physically absent user), for instance. Similarly, for gaming implementations, experiences and content may be shared amongst mobile devices of the various users through the link (taking particular advantage of the pose information from each mobile device) to present to each user the corresponding perspectives of the simulated content in the shared environment, thereby enhancing game play and experience. In a search and rescue implementation, users may install (temporary) wireless signal sources on posts (that may be powered by solar cells and/or batteries, or other portable power sources) that broadcast wireless data described herein. In this manner, within a search zone, users (via their mobile devices) may share updated information and status of the search, as well as positioning and mapping data, thereby improving chances of finding missing (and perhaps injured) parties.

[0081] Thus, an advantage of the system and techniques described according to the various embodiments lies in the fusion of indoor (as well as outdoor) positioning and visual sensor positioning realized by utilizing wireless signal characteristics, with the large wireless signal positioning range being used to compensate for the limitation of the visual sensor positioning area. The advantages of high-precision and anti- environmental interference of the visual sensor's positioning compensate for the disadvantages of wireless signal positioning alone. An ultra-large-scale global map may be constructed through cloud services combined with a map data model composed of wireless signal characteristics and a feature point map model generated by visual sensors, while, at the same time, real-time mutual pose information is shared among multiple mobile devices. As such, low-cost, high-precision, ultra-large-scale indoor (and outdoor) multi-person interactive positioning (and navigation) may be achieved. The system is thus capable of ultra-large scale, not only for size of area, but also for the number of users (e.g., from less than 10, to 10's of users, 100's or users, 1000's of users, 10,000's of users, 100,000's of users, millions of users, or more). [0082] These and other functions of the system 100 (and its components) are described in greater detail below with respect to Figs. 2-4.

[0083] Fig. 2 is a schematic block flow diagram illustrating a non-limiting example 200 of a system for implementing multi-person SLAM linkage positioning and navigation, in accordance with various embodiments.

[0084] The system architecture of the various embodiments mainly include mobile clients and back-end cloud services running on mobile devices and may perform data exchange through a wireless network. There are two models used by the various embodiments for positioning and navigation: (A) Create Map Model and (B) Load Map Model.

[0085] The Create Map Model is responsible for creating and maintaining the map model in the scene. It generates the wireless signal fingerprint map model and the visual feature point map model by collecting the wireless signal fingerprint around the mobile device and the visual feature points from the visual sensor, and uploads the two map models to the back-end cloud service through the data forwarding module. When the mobile terminal cannot search for a map model of visual feature points matching the corresponding area through the background cloud service, a user can choose to enable this mode to create a local visual feature point map model of the area where the mobile device is located.

[0086] The characteristic information here includes, but is not limited to, the information of the wireless signal and the information transmitted by the signal, such as the name of the wireless network, the MAC address of the wireless signal transmitter, and/or the strength of the wireless signal, etc. Through this characteristic information, it is possible to quickly and roughly determine the current status of the mobile device, as well as the approximate location in the scene.

[0087] The Load Map Model is responsible for loading regional maps around the mobile device when it has moved into a new location (whose global map has been previously generated using wireless signal fingerprint map data and visual feature point map data from other mobile devices). In some cases, when the back-end cloud service searches for the received regional map model matching request, the matching search based on visual feature points can take some time and the amount of processed data is large. Here, the load map model gives priority to using wireless signal fingerprint for fast matching, thus locking the search interval in a short time, and then quickly finding the corresponding visual map model in this interval through visual feature point matching and returning the information and maps to the positioning navigation client. The advantage of this is that it reduces the time to search for local area map models and also reduces the amount of computing on the back-end cloud service, which is helpful for large-scale users to initiate regional map model search matching requests to the server with high concurrency.

[0088] After receiving the visual map model data from the back-end cloud service, the positioning and navigation client can mount it to the SLAM module to locate and navigate the corresponding area. At the same time, the positioning and navigation client will send real-time local area to pose information to the back-end cloud service through the data forwarding module.

[0089] After receiving the real-time local area pose information from the positioning and navigation client, the back-end cloud service may convert the local area pose into global pose coordinates through the global coordinate conversion module, and distributed to other positioning and navigation clients in real-time. In this way, the pose information sharing and linkage between multiple devices may be realized.

[0090] With reference to the non-limiting example of Fig. 2, system 200 may comprise system architecture for the Create Map Model 205 A, system architecture for the Load Map Model 205B, and system architecture for the back-end cloud service 260. In some embodiments, the system architecture for the Create Map Model 205A and the system architecture for the Load Map Model 205B may each include, without limitation, wireless signal fingerprint collection system 225, visual feature point collection system 230, wireless signal fingerprint map model creator 245 a, visual feature point map model creator 245b, SLAM 245c, and data forwarding system 290, which may correspond to wireless transceiver(s) 125, visual sensor(s) 130, wireless signal fingerprint map model creator 145a, visual feature point map model creator 145b, SLAM 145c, and wireless transceiver(s) 125, respectively of system 100 of Fig. 1, or the like, and the corresponding description thereof may apply to the corresponding components of the system architecture for the Create Map Model 205A or to the corresponding components of the system architecture for the Load Map Model 205B.

[0091] According to some embodiments, the system architecture for the Back-End Cloud Service 260 may include, without limitation, data forwarding system 290, global map integrated generator 270, feature matching system 275, and global coordinate converter 280, which may correspond to communications system of the remote computing system 160 (not shown in Fig. 1), global map integrated generator 170, feature matching system 175, and global coordinate converter 180, respectively of system 100 of Fig. 1, or the like, and the corresponding description thereof may apply to these components of the system architecture for the Back-End Cloud Service 260. The global wireless signal fingerprint map 295a and the global visual feature point map 295b may correspond to those described (although not shown) above with respect to Fig. 1. The positioning and navigation system 285 may correspond to positioning and navigation system 185 of Fig. 1, or the like, and the corresponding description thereof may apply to positioning and navigation system 285.

[0092] Referring to Fig. 2, for Create Map Model 205 A, the process may include, without limitation: (1) the wireless signal fingerprint collection system 225 obtaining or collecting wireless signal data based on data signals from wireless signal sources in the area, the wireless signal fingerprint map model creator 245a generating (or updating) a wireless signal fingerprint map model based on the collected wireless signal data, and sending the wireless signal fingerprint map model to the data forwarding system 290; (2) the visual feature point collection system 230 obtaining or capturing image (or video) data and extracting visual feature point data therefrom, the SLAM 245c using the extracted visual feature point data to locate (and navigate) the area (in some cases, encoding location information to the extracted visual feature point data), the visual feature point map model creator 245b generating (or updating) a visual feature point map model based on the extracted visual feature point data and the location information from SLAM 245 c, and sending the visual feature point map model to the data forwarding system 290; (3) the data forwarding system 290 forwarding the wireless signal fingerprint map model and the visual feature point map model to the global map integrated generator 270 of the Back-End Cloud Service 260; (4) the global wireless signal fingerprint map 295a being generated based on the wireless signal fingerprint map model; and (5) the global visual feature point map 295b being generated based on the visual feature point map model.

[0093] For Load Map Model 205B, the process may include, without limitation: (6) the wireless signal fingerprint collection system 225 obtaining or collecting wireless signal data based on data signals from wireless signal sources in the area, the wireless signal fingerprint map model creator 245 a generating a (temporary, current) wireless signal fingerprint map model based on the collected wireless signal data, and sending the (temporary, current) wireless signal fingerprint map model to the data forwarding system 290; (7) the visual feature point collection system 230 obtaining or capturing image (or video) data and extracting visual feature point data therefrom, the SLAM 245c using the extracted visual feature point data to locate (and navigate) the area (in some cases, encoding location information to the extracted visual feature point data), the visual feature point map model creator 245b generating a (temporary, current) visual feature point map model based on the extracted visual feature point data and the location information from SLAM 245c, and sending the (temporary, current) visual feature point map model to the data forwarding system 290; (8) the data forwarding system 290 forwarding the (temporary, current) wireless signal fingerprint map model and the (temporary, current) visual feature point map model to the feature matching system 275 of the Back-End Cloud Service 260; (9) the feature matching system 275 matching features of the (temporary, current) wireless signal fingerprint map model with the global wireless signal fingerprint map to extract a regional wireless signal fingerprint map and to determine a corresponding region of the global visual feature point map; (10) the feature matching system 275 matching features of the (temporary, current) visual feature point map model with the corresponding determined region of the global visual feature point map to generate a regional feature point map model; (11) the feature matching system sending the regional feature point map model to positioning and navigation system 285, which mounts the regional feature point map model as local area map 295 c to SLAM 245 c via data forwarding system 290; (12) the SLAM 245c using the local area map 295c to locate and navigate the area and to send real-time local area pose information to global coordinate converter 280 via data forwarding system 290; and (13) the global coordinate converter 280 converting the real-time local area pose information into global pose coordinates (in some cases, using the global visual feature point map 295b as an input) and sending the global pose coordinates to SLAM 245c (in some cases, as an updated or encoded local area map 295c). In this manner, pose information sharing and linkage among multiple mobile devices may be achieved. Although Fig. 2 depicts one system architecture for the Create Map Model 205A and one system architecture for the Load Map Model 205B, this is merely to simplify illustration of the processes involved in accordance with the various embodiments, and thus is not limited to one mobile device implementing Create Map Model and to one mobile device implementing Load Map Model. Rather, the various embodiments are scalable to any suitable number of mobile devices being used in Create Map Model mode, with more mobile devices doing so the better, as more complete (and accurate) real-time or near-real-time global maps 295a and 295b may be created (especially considering that wireless signal fingerprint maps may be affected by changes to the environment in the area (e.g., by movement of objects therein, etc.). Likewise, the various embodiments are scalable to any suitable number of mobile devices being used in Load Map Model mode, thereby serving and linking any suitable number of mobile devices in the area (e.g., to implement the linkages for one or more of the non-limiting situations described above with respect to Fig. 1, or other similar situations, etc.).

[0094] Figs. 3 A and 3B (collectively, "Fig. 3") are schematic diagrams illustrating various non-limiting examples 300 and 300' of wireless signal fingerprint maps and visual feature point maps that may be used when implementing multi-person SLAM linkage positioning and navigation, in accordance with various embodiments.

[0095] Generally, the data storage capacity of the visual feature map model in the same area is much larger than that of the wireless signal fingerprint map model. This means that the coverage area of the visual map model is much smaller than the area covered by the wireless signal fingerprint map model under the same storage capacity, and the visual map model needs to upload the model data to the back-end cloud service more frequently when the hardware resources of the mobile device are close to saturation. Therefore, the global map integration generation module of the back end cloud service can assemble and generate a global visual feature point map model according to the data relationship of the wireless signal fingerprint map model, thereby avoiding the problem that the performance of the mobile device hardware affects the creation of the visual feature point map model. [0096] In most cases, the mapping module will only be turned on briefly and only collects a small amount of wireless feature and visual feature point information and generates a small data volume. The map model may be uploaded to the back-end cloud service to initiate a map model search request for the corresponding area.

[0097] This is illustrated in Fig. 3A, in which the wireless signal fingerprint map may comprise a single map model within the first area 310, while the corresponding visual feature point map may require division into multiple portions or fragments to cover corresponding portions or fragments of the first area 310a-310n (in this case, three fragments are shown in Fig. 3A, although the number of fragments would depend on memory capacity of the mobile device).

[0098] With reference to Fig. 3B, a building is depicted as a non-limiting example of the area 310, as described herein. In this non-limiting example, the system may generate three wireless signal fingerprint maps 315 (or three sets of wireless signal fingerprint maps), one for each floor level (or storey or story) of the building, as wireless data signals may differ from floor level to floor level due to interference by components of the building between floor levels. For the reasons described above with respect to Fig. 3A, multiple the visual feature point maps 320 are generated for each floor level, each visual feature point map corresponding to a different portion (or sub-area) of each floor level. As depicted in the non-limiting example of area 310, a visual feature point map 320 may be generated for one of the 9 sub-areas (e.g., Areas A, B, C, D, E, F, G, H, and I) for each floor level.

[0099] For loading (and linking) purposes, a user moving into one of the floor levels of the building 310 (e.g., the 2^nd floor, or the like) into one of the sub-areas (e.g., Area I on the 2^nd floor, or the like) with a mobile device may send wireless signal fingerprint map model of the second floor along with visual feature point map model of Area I on the second floor to the remote computing system, which would first match the wireless signal fingerprint map model of the second floor with a portion of the previously generated global wireless signal fingerprint map of the building (in this case, the regional global wireless signal fingerprint map corresponding to the second floor of the building 305), and would subsequently match the visual feature point map model of Area I on the second floor with the previously generated global visual feature point map (which integrates the visual feature point maps of Areas A-I for each of all three floor levels). The remote computing system would then send the regional feature point map model to a positioning and navigation system, which would mount the regional feature point map model to a SLAM, which would concurrently send real-time local area pose information to the remote computing system to covert the real-time local area pose information into global pose coordinates, which are sent to the SLAM, thereby enabling sharing of pose information and linkage among multiple mobile devices.

[0100] Figs. 4A-4I (collectively, "Fig. 4") are flow diagrams illustrating a method 400 for implementing multi-person SLAM linkage positioning and navigation, in accordance with various embodiments. Method 400 of Fig. 4A continues onto one or more of Fig. 4D following the circular marker denoted, "A," Fig. 4E following the circular marker denoted, "B," Fig. 4F following the circular marker denoted, "C," and/or Fig. 4G following the circular marker denoted, "D," and returns to Fig. 4A following the circular marker denoted, "E." Method 400 of Fig. 4H continues onto Fig. 41 following the circular marker denoted, "F," and returns to Fig. 4H following the circular marker denoted, "G."

[0101] While the techniques and procedures are depicted and/or described in a certain order for purposes of illustration, it should be appreciated that certain procedures may be reordered and/or omitted within the scope of various embodiments. Moreover, while the method 400 illustrated by Fig. 4 can be implemented by or with (and, in some cases, are described below with respect to) the systems, examples, or embodiments 100, 200, 300, and 300' of Figs. 1, 2, 3 A, and 3B, respectively (or components thereof), such methods may also be implemented using any suitable hardware (or software) implementation. Similarly, while each of the systems, examples, or embodiments 100, 200, 300, and 300' of Figs. 1, 2, 3 A, and 3B, respectively (or components thereof), can operate according to the method 400 illustrated by Fig. 4 (e.g., by executing instructions embodied on a computer readable medium), the systems, examples, or embodiments 100, 200, 300, and 300' of Figs. 1, 2, 3A, and 3B can each also operate according to other modes of operation and/or perform other suitable procedures.

[0102] In the non-limiting embodiment of Fig. 4A, method 400, at block 402, may comprise generating or updating, using a software application running on a first mobile device that is located in a first area, a first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area. At collective block 408, method 400 may comprise generating or updating, using the software application, a first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor, the first visual feature point map model being correlated with the first wireless signal fingerprint map model (block 404), and sending, using the software application, the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system over a network (block 406).

[0103] Method 400 may further comprise, at block 410, receiving, using the remote computing system, the correlated first wireless signal fingerprint map model and first visual feature point map model. At block 412, method 400 may comprise analyzing, using the remote computing system, the correlated first wireless signal fingerprint map model and first visual feature point map model. Method 400, at block 414, may comprise, based on the analysis, integrating, using the remote computing system, the correlated received first wireless signal fingerprint map model and first visual feature point map model to generate a global map of the first area.

[0104] In some embodiments, the mobile device may include, without limitation, at least one of a smartphone, a tablet computer, an augmented reality ("AR") device, a virtual reality ("VR") device, a mixed reality ("MR") device, a portable gaming device, or a robot, and/or the like. In some cases, the remote computing system may include, but is not limited to, at least one of a server computer over the network, an image processing server, a graphics processing unit ("GPU") -based server, a positioning and mapping server, a machine learning system, an artificial intelligence ("AI") system, a deep learning system, a neural network, a convolutional neural network ("CNN"), a fully convolutional network ("FCN"), a cloud computing system, or a distributed computing system, and/or the like.

[0105] Merely by way of example, in some cases, the first area may include, without limitation, at least one of a room, a corridor, a courtyard, a floor level of a building, a building, a rooftop of a building, a balcony of a building, a patio of a building, a porch of a building, a residential facility, a business facility, a commercial facility, a parking facility, a storage facility, a facility operated by a public utility, a government facility, a military facility, an industrial facility, a transportation facility, a school facility, a school campus, a medical facility, a stadium, an athletics facility, an auditorium, a garden, a park, a zoo, a neighborhood, a city block of a municipality, a city center of a municipality, a municipality, a forest, a jungle, a field, agricultural land, a regional open space, a body of water, a waterway, an underwater area, an island region, a mountain region, a desert region, a rocky region, a sandy region, a terrestrial vehicle, an aquatic vehicle, an amphibious vehicle, an aircraft, a space vehicle, an aquatic facility, a space-based facility, or a portion of one of these areas, and/or the like.

[0106] According to some embodiments, the first signal data that is received from each of the at least one first wireless signal source may include, but is not limited to, at least one of Wi-Fi signal data, ultra- wideband ("UWB") signal data, ultra-wideband Doherty ("UWD") signal data, Bluetooth signal data, or Zig-Bee signal data, and/or the like. In some instances, the at least one first visual sensor may include, but is not limited to, at least one of a red-green-blue ("RGB") color camera, a wide-angle camera, an ultra- wide-angle camera, a fish-eye camera, or a telephoto camera, and/or the like.

[0107] In some embodiments, sending the correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network may comprise sending the correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network using a wireless network data transmission protocol, wherein the wireless network data transmission protocol comprises at least one of Wi-Fi protocol, Wi-Fi Direct protocol, other protocols under IEEE 802.11 standard, Bluetooth protocol, Zig-Bee protocol, other protocols under IEEE 802.15 standard, 4G broadband protocol, 4G LTE broadband protocol, 5G broadband protocol, or other cellular broadband protocol, and/or the like.

[0108] According to some embodiments, the first wireless signal fingerprint map model of the first area may be generated or updated based at least in part on characteristic information of the first signal data received from each first wireless signal source at a location at which the mobile device receives the first signal data. In some cases, the characteristic information of the first signal data may comprise at least one of name of a wireless network corresponding to said wireless signal source, a media access control ("MAC") address of said wireless signal source, or signal strength of said wireless signal source, and/or the like. [0109] Method 400 may continue onto one or more of the process at block 426 in Fig. 4D following the circular marker denoted, "A"; the process at block 440 in Fig. 4E following the circular marker denoted, "B"; the process at block 450 in Fig. 4F following the circular marker denoted, "C"; or the process at block 456 in Fig. 4G following the circular marker denoted, "D"; and/or may return to the process at block 402.

[0110] With reference to Fig. 4B, in some embodiments, generating or updating the first visual feature point map model of at least a portion of the first area (at block 404) may comprise: extracting, using the software application, feature point data from the received image data of the at least a portion of the first area, the feature point data comprising at least one of object feature point data or environmental feature point data (block 416); and implementing, using the software application, at least one of simultaneous localization and mapping ("SLAM") techniques or image recognition techniques on the extracted feature point data to generate the first visual feature point map model of the at least a portion of the first area (block 418).

[0111] Turning to Fig. 4C, according to some embodiments, generating or updating the first visual feature point map model of the at least a portion of the first area and sending the correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network (at collective block 408) may comprise repeating the following processes until the first visual feature point map model covers all portions of the first area covered by the first wireless signal fingerprint map model: generating or updating, using the software application, a first visual feature point map model of one portion of the first area based on image data received from the at least one first visual sensor for said portion of the first area (block 420); sending, using the software application, the correlated first wireless signal fingerprint map model and first visual feature point map model of said portion of the first area to the remote computing system over the network (block 422); and determining whether the first visual feature point map model covers all portions of the first area covered by the first wireless signal fingerprint map model (block 424). If it is determined that the first visual feature point map model does not cover all portions of the first area covered by the first wireless signal fingerprint map model, method 400 returns to the process at block 420, and the processes at blocks 420-424 are repeated until this condition has been met. [0112] At block 426 in Fig. 4D (following the circular marker denoted, "A"), method 400 may comprise: continually or periodically receiving, using a transceiver of the mobile device, updated first signal data from each of the at least one first wireless signal source within or in proximity to the first area; and based on a determination that the updated first signal data has changed compared with previously received updated first signal data, updating, using the software application, the first wireless signal fingerprint map model of the first area based on the updated first signal data (block 428). Method 400 may further comprise capturing, using at least one first visual sensor of the first mobile device, image data of objects within at least a portion of the first area (block 430); and based on a determination that the updated image data has changed compared with previously received updated image data, updating, using the software application, the first visual feature point map model of the at least a portion of the first area based on the updated image data (block 432). Method 400 may further comprise sending, using the software application, updated correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network (block 434); analyzing, using the remote computing system, the updated correlated first wireless signal fingerprint map model and first visual feature point map model (block 436); and based on the analysis, integrating, using the remote computing system, the correlated received first wireless signal fingerprint map model and first visual feature point map model to generate an updated global map of the first area (block 438). Method 400 may return to the process at block 402 in Fig. 4A following the circular marker denoted, "E."

[0113] At block 440 in Fig. 4E (following the circular marker denoted, "B"), method 400 may comprise: generating or updating, using the software application as the mobile device has moved to a second area, a second wireless signal fingerprint map model of the second area based on second signal data that is received from each of at least one second wireless signal source within or in proximity to the second area, wherein the second area is one of a different portion of the first area, an area in proximity and overlapping with the first area, or an area in proximity with yet separate from the first area, or the like; generating or updating, using the software application, a second visual feature point map model of at least a portion of the second area based on second image data received from the at least one first visual sensor, the second visual feature point map model being correlated with the second wireless signal fingerprint map model (block 442); and sending, using the software application, the correlated second wireless signal fingerprint map model and second visual feature point map model to the remote computing system over the network (block 444). Method 400 may further comprise analyzing, using the remote computing system, the correlated second wireless signal fingerprint map model and second visual feature point map model (block 446); and based on the analysis, integrating, using the remote computing system, the correlated received second wireless signal fingerprint map model and second visual feature point map model to generate a global map of a combination of the first area and the second area (block 448). Method 400 may return to the process at block 402 in Fig. 4A following the circular marker denoted, "E."

[0114] In some embodiments, the first mobile device may be associated with a first user, wherein the first mobile device may be among a plurality of mobile devices located within the first area, the plurality of mobile devices being associated with a plurality of users. In such a case, At block 450 in Fig. 4F (following the circular marker denoted, "C"), method 400 may comprise: receiving, using the remote computing system, correlated wireless signal fingerprint map model and visual feature point map model from each of the plurality of mobile devices; analyzing, using the remote computing system, the correlated wireless signal fingerprint map model and visual feature point map model from each of the plurality of mobile devices (block 452); and based on the analysis, integrating, using the remote computing system, the correlated received wireless signal fingerprint map model and visual feature point map model from each of the plurality of mobile devices to generate the global map of the first area (block 454). Method 400 may return to the process at block 402 in Fig. 4A following the circular marker denoted, "E."

[0115] In Fig. 4G (following the circular marker denoted, "D"), method 400 may comprise: analyzing, using the remote computing system, the correlated first wireless signal fingerprint map model and first visual feature point map model, by: comparing, using the remote computing system, the first wireless signal fingerprint map model with one or more portions of a previously generated global wireless signal fingerprint map using wireless signal fingerprint matching (block 456); in response to identifying a matching portion of the previously generated global wireless signal fingerprint map based on the comparison, comparing, using the remote computing system, the first visual feature point map model with one or more portions of a previously generated visual feature point map corresponding to the identified matching portion of the previously generated global wireless signal fingerprint map using visual feature point matching (block 458); and in response to identifying a matching portion of the previously generated visual feature point map, extracting, using the remote computing system, a regional visual feature point map model based on the identified matching portion of the previously generated visual feature point map, and sending, using the remote computing system, the regional visual feature point map model to a positioning and navigation server (block 460).

[0116] In some embodiments, method 400 may further comprise: in response to receiving the regional visual feature point map model, mounting, using the positioning and navigation server, the received regional visual feature point map model to a simultaneous localization and mapping ("SLAM") system to locate and navigate the first area, and concurrently sending, using the positioning and navigation server, real-time local area pose information to the remote computing system (block 462); in response to receiving the real-time local area pose information, converting, using the remote computing system, the real-time local area pose information into global pose coordinates using a global coordinate conversion system (block 464); and distributing, using the remote computing system, the global pose coordinates to one or more second mobile devices either within the first area or in proximity of the first area, to establish pose information sharing and linkage between the first mobile device and each of the one or more second devices (block 466), wherein the one or more second mobile devices may each be associated with one or more other users. [0117] Method 400 may return to the process at block 402 in Fig. 4A following the circular marker denoted, "E."

[0118] In another aspect, with maps having been previously created, mobile devices may load maps from the remote computing system. In that regard, with reference to Fig. 4H, method 400 may comprise: generating, using a software application running on a first mobile device that is located in a first area, a first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area (block 468); generating, using the software application, a first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor, the first visual feature point map model being correlated with the first wireless signal fingerprint map model (block 470); and sending, using the software application, the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system over a network (block 472). Method 400 may further comprise analyzing, using the remote computing system, the correlated first wireless signal fingerprint map model and first visual feature point map model, by: comparing, using the remote computing system, the first wireless signal fingerprint map model with one or more portions of a previously generated global wireless signal fingerprint map using wireless signal fingerprint matching (block 474); in response to identifying a matching portion of the previously generated global wireless signal fingerprint map based on the comparison, comparing, using the remote computing system, the first visual feature point map model with one or more portions of a previously generated visual feature point map corresponding to the identified matching portion of the previously generated global wireless signal fingerprint map using visual feature point matching (block 476); and in response to identifying a matching portion of the previously generated visual feature point map, extracting, using the remote computing system, a regional visual feature point map model based on the identified matching portion of the previously generated visual feature point map (block 478).

[0119] Method 400 may continue onto one or more of the process at block 480 in Fig. 41 following the circular marker denoted, "F," and/or may return to the process at block 468.

[0120] At block 480 in Fig. 41 (following the circular marker denoted, "F"), method 400 may comprise: sending, using the remote computing system, the regional visual feature point map model as positioning and navigation data to the software application running on the first mobile device.

[0121] In some embodiments, sending the regional visual feature point map model as positioning and navigation data to the software application running on the first mobile device (at block 480) may comprise: sending, using the remote computing system, the regional visual feature point map model to a positioning and navigation server (block 482); and in response to receiving the regional visual feature point map model, mounting, using the positioning and navigation server, the received regional visual feature point map model to a SLAM system to locate and navigate the first area (block 484).

[0122] According to some embodiments, method 400 may further comprise: concurrent with mounting the received regional visual feature point map model to the SLAM system, sending, using the positioning and navigation server, real-time local area pose information to the remote computing system (block 486); in response to receiving the real-time local area pose information, converting, using the remote computing system, the real-time local area pose information into global pose coordinates using a global coordinate conversion system (block 488); and distributing, using the remote computing system, the global pose coordinates to one or more second mobile devices either within the first area or in proximity of the first area, to establish pose information sharing and linkage between the first mobile device and each of the one or more second devices (block 490), wherein the one or more second mobile devices may each be associated with one or more other users.

[0123] Method 400 may return to the process at block 468 in Fig. 4H following the circular marker denoted, "G."

[0124] Examples of System and Hardware Implementation [0125] Fig. 5 is a block diagram illustrating an example of computer or system hardware architecture, in accordance with various embodiments. Fig. 5 provides a schematic illustration of one embodiment of a computer system 500 of the service provider system hardware that can perform the methods provided by various other embodiments, as described herein, and/or can perform the functions of computer or hardware system (i.e., mobile devices 105, 105a-105n, 205 A, and 205B, and remote computing systems 160 and 260, etc.), as described above. It should be noted that Fig. 5 is meant only to provide a generalized illustration of various components, of which one or more (or none) of each may be utilized as appropriate. Fig. 5, therefore, broadly illustrates how individual system elements may be implemented in a relatively separated or relatively more integrated manner.

[0126] The computer or hardware system 500 - which might represent an embodiment of the computer or hardware system (i.e., mobile devices 105, 105a- 105h, 205A, and 205B, and remote computing systems 160 and 260, etc.), described above with respect to Figs. 1-4 - is shown comprising hardware elements that can be electrically coupled via a bus 505 (or may otherwise be in communication, as appropriate). The hardware elements may include one or more processors 510, including, without limitation, one or more general-purpose processors and/or one or more special-purpose processors (such as microprocessors, digital signal processing chips, graphics acceleration processors, and/or the like); one or more input devices 515, which can include, without limitation, a mouse, a keyboard, and/or the like; and one or more output devices 520, which can include, without limitation, a display device, a printer, and/or the like.

[0127] The computer or hardware system 500 may further include (and/or be in communication with) one or more storage devices 525, which can comprise, without limitation, local and/or network accessible storage, and/or can include, without limitation, a disk drive, a drive array, an optical storage device, solid-state storage device such as a random access memory ("RAM") and/or a read-only memory ("ROM"), which can be programmable, flash-updateable, and/or the like. Such storage devices may be configured to implement any appropriate data stores, including, without limitation, various file systems, database structures, and/or the like. [0128] The computer or hardware system 500 might also include a communications subsystem 530, which can include, without limitation, a modem, a network card (wireless or wired), an infra-red communication device, a wireless communication device and/or chipset (such as a Bluetooth™ device, an 802.11 device, a WiFi device, a WiMax device, a WWAN device, cellular communication facilities, etc.), and/or the like. The communications subsystem 530 may permit data to be exchanged with a network (such as the network described below, to name one example), with other computer or hardware systems, and/or with any other devices described herein. In many embodiments, the computer or hardware system 500 will further comprise a working memory 535, which can include a RAM or ROM device, as described above.

[0129] The computer or hardware system 500 also may comprise software elements, shown as being currently located within the working memory 535, including an operating system 540, device drivers, executable libraries, and/or other code, such as one or more application programs 545, which may comprise computer programs provided by various embodiments (including, without limitation, hypervisors, VMs, and the like), and/or may be designed to implement methods, and/or configure systems, provided by other embodiments, as described herein. Merely by way of example, one or more procedures described with respect to the method(s) discussed above might be implemented as code and/or instructions executable by a computer (and/or a processor within a computer); in an aspect, then, such code and/or instructions can be used to configure and/or adapt a general purpose computer (or other device) to perform one or more operations in accordance with the described methods.

[0130] A set of these instructions and/or code might be encoded and/or stored on a non-transitory computer readable storage medium, such as the storage device(s) 525 described above. In some cases, the storage medium might be incorporated within a computer system, such as the system 500. In other embodiments, the storage medium might be separate from a computer system (i.e., a removable medium, such as a compact disc, etc.), and/or provided in an installation package, such that the storage medium can be used to program, configure, and/or adapt a general purpose computer with the instructions/code stored thereon. These instructions might take the form of executable code, which is executable by the computer or hardware system 500 and/or might take the form of source and/or installable code, which, upon compilation and/or installation on the computer or hardware system 500 (e.g., using any of a variety of generally available compilers, installation programs, compression/decompression utilities, etc.) then takes the form of executable code.

[0131] It will be apparent to those skilled in the art that substantial variations may be made in accordance with particular requirements. For example, customized hardware (such as programmable logic controllers, field-programmable gate arrays, application-specific integrated circuits, and/or the like) might also be used, and/or particular elements might be implemented in hardware, software (including portable software, such as applets, etc.), or both. Further, connection to other computing devices such as network input/output devices may be employed.

[0132] As mentioned above, in one aspect, some embodiments may employ a computer or hardware system (such as the computer or hardware system 500) to perform methods in accordance with various embodiments of the invention.

According to a set of embodiments, some or all of the procedures of such methods are performed by the computer or hardware system 500 in response to processor 510 executing one or more sequences of one or more instructions (which might be incorporated into the operating system 540 and/or other code, such as an application program 545) contained in the working memory 535. Such instructions may be read into the working memory 535 from another computer readable medium, such as one or more of the storage device(s) 525. Merely by way of example, execution of the sequences of instructions contained in the working memory 535 might cause the processor(s) 510 to perform one or more procedures of the methods described herein. [0133] The terms "machine readable medium" and "computer readable medium," as used herein, refer to any medium that participates in providing data that causes a machine to operate in some fashion. In an embodiment implemented using the computer or hardware system 500, various computer readable media might be involved in providing instructions/code to processor(s) 510 for execution and/or might be used to store and/or carry such instructions/code (e.g., as signals). In many implementations, a computer readable medium is a non-transitory, physical, and/or tangible storage medium. In some embodiments, a computer readable medium may take many forms, including, but not limited to, non-volatile media, volatile media, or the like. Non-volatile media includes, for example, optical and/or magnetic disks, such as the storage device(s) 525. Volatile media includes, without limitation, dynamic memory, such as the working memory 535. In some alternative embodiments, a computer readable medium may take the form of transmission media, which includes, without limitation, coaxial cables, copper wire, and fiber optics, including the wires that comprise the bus 505, as well as the various components of the communication subsystem 530 (and/or the media by which the communications subsystem 530 provides communication with other devices). In an alternative set of embodiments, transmission media can also take the form of waves (including without limitation radio, acoustic, and/or light waves, such as those generated during radio wave and infra-red data communications).

[0134] Common forms of physical and/or tangible computer readable media include, for example, a floppy disk, a flexible disk, a hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read instructions and/or code. [0135] Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to the processor(s) 510 for execution. Merely by way of example, the instructions may initially be carried on a magnetic disk and/or optical disc of a remote computer. A remote computer might load the instructions into its dynamic memory and send the instructions as signals over a transmission medium to be received and/or executed by the computer or hardware system 500. These signals, which might be in the form of electromagnetic signals, acoustic signals, optical signals, and/or the like, are all examples of carrier waves on which instructions can be encoded, in accordance with various embodiments of the invention.

[0136] The communications subsystem 530 (and/or components thereof) generally will receive the signals, and the bus 505 then might carry the signals (and/or the data, instructions, etc. carried by the signals) to the working memory 535, from which the processor(s) 505 retrieves and executes the instructions. The instructions received by the working memory 535 may optionally be stored on a storage device 525 either before or after execution by the processor(s) 510.

[0137] As noted above, a set of embodiments comprises methods and systems for implementing simultaneous location and mapping ("SLAM") functionalities, and, more particularly, to methods, systems, and apparatuses for implementing multi person SLAM linkage positioning and navigation, in some cases, for super large-scale scenes based on wireless signal characteristics and visual feature point information. Fig. 6 illustrates a schematic diagram of a system 600 that can be used in accordance with one set of embodiments. The system 600 can include one or more user computers, user devices, or customer devices 605. A user computer, user device, or customer device 605 can be a general purpose personal computer (including, merely by way of example, desktop computers, tablet computers, laptop computers, handheld computers, and the like, running any appropriate operating system, several of which are available from vendors such as Apple, Microsoft Corp., and the like), cloud computing devices, a server(s), and/or a workstation computer(s) running any of a variety of commercially-available UNIX™ or UNIX-like operating systems. A user computer, user device, or customer device 605 can also have any of a variety of applications, including one or more applications configured to perform methods provided by various embodiments (as described above, for example), as well as one or more office applications, database client and/or server applications, and/or web browser applications. Alternatively, a user computer, user device, or customer device 605 can be any other electronic device, such as a thin-client computer, Internet- enabled mobile telephone, and/or personal digital assistant, capable of communicating via a network (e.g., the network(s) 610 described below) and/or of displaying and navigating web pages or other types of electronic documents. Although the system 600 is shown with two user computers, user devices, or customer devices 605, any number of user computers, user devices, or customer devices can be supported.

[0138] Some embodiments operate in a networked environment, which can include a network(s) 610. The network(s) 610 can be any type of network familiar to those skilled in the art that can support data communications using any of a variety of commercially-available (and/or free or proprietary) protocols, including, without limitation, TCP/IP, SNA™, IPX™, AppleTalk™, and the like. Merely by way of example, the network(s) 610 (similar to network(s) 165 of Fig. 1, or the like) can each include a local area network ("LAN"), including, without limitation, a fiber network, an Ethernet network, a Token-Ring™ network, and/or the like; a wide-area network ("WAN"); a wireless wide area network ("WWAN"); a virtual network, such as a virtual private network ("VPN"); the Internet; an intranet; an extranet; a public switched telephone network ("PSTN"); an infra-red network; a wireless network, including, without limitation, a network operating under any of the IEEE 802.11 suite of protocols, the Bluetooth™ protocol known in the art, and/or any other wireless protocol; and/or any combination of these and/or other networks. In a particular embodiment, the network might include an access network of the service provider (e.g., an Internet service provider ("ISP")). In another embodiment, the network might include a core network of the service provider, and/or the Internet.

[0139] Embodiments can also include one or more server computers 615. Each of the server computers 615 may be configured with an operating system, including, without limitation, any of those discussed above, as well as any commercially (or freely) available server operating systems. Each of the servers 615 may also be running one or more applications, which can be configured to provide services to one or more clients 605 and/or other servers 615.

[0140] Merely by way of example, one of the servers 615 might be a data server, a web server, a cloud computing device(s), or the like, as described above. The data server might include (or be in communication with) a web server, which can be used, merely by way of example, to process requests for web pages or other electronic documents from user computers 605. The web server can also run a variety of server applications, including HTTP servers, FTP servers, CGI servers, database servers, Java servers, and the like. In some embodiments of the invention, the web server may be configured to serve web pages that can be operated within a web browser on one or more of the user computers 605 to perform methods of the invention.

[0141] The server computers 615, in some embodiments, might include one or more application servers, which can be configured with one or more applications accessible by a client running on one or more of the client computers 605 and/or other servers 615. Merely by way of example, the server(s) 615 can be one or more general purpose computers capable of executing programs or scripts in response to the user computers 605 and/or other servers 615, including, without limitation, web applications (which might, in some cases, be configured to perform methods provided by various embodiments). Merely by way of example, a web application can be implemented as one or more scripts or programs written in any suitable programming language, such as Java™, C, C#™ or C++, and/or any scripting language, such as Perl, Python, or TCL, as well as combinations of any programming and/or scripting languages. The application server(s) can also include database servers, including, without limitation, those commercially available from Oracle™, Microsoft™,

Sybase™, IBM™, and the like, which can process requests from clients (including, depending on the configuration, dedicated database clients, API clients, web browsers, etc.) running on a user computer, user device, or customer device 605 and/or another server 615. In some embodiments, an application server can perform one or more of the processes for implementing SLAM functionalities, and, more particularly, to methods, systems, and apparatuses for implementing multi-person SLAM linkage positioning and navigation, in some cases, for super large-scale scenes based on wireless signal characteristics and visual feature point information, as described in detail above. Data provided by an application server may be formatted as one or more web pages (comprising HTML, JavaScript, etc., for example) and/or may be forwarded to a user computer 605 via a web server (as described above, for example). Similarly, a web server might receive web page requests and/or input data from a user computer 605 and/or forward the web page requests and/or input data to an application server. In some cases, a web server may be integrated with an application server.

[0142] In accordance with further embodiments, one or more servers 615 can function as a file server and/or can include one or more of the files (e.g., application code, data files, etc.) necessary to implement various disclosed methods, incorporated by an application running on a user computer 605 and/or another server 615. Alternatively, as those skilled in the art will appreciate, a file server can include all necessary files, allowing such an application to be invoked remotely by a user computer, user device, or customer device 605 and/or server 615.

[0143] It should be noted that the functions described with respect to various servers herein (e.g., application server, database server, web server, file server, etc.) can be performed by a single server and/or a plurality of specialized servers, depending on implementation-specific needs and parameters.

[0144] In some embodiments, the system can include one or more databases 620a- 620n (collectively, "databases 620"). The location of each of the databases 620 is discretionary: merely by way of example, a database 620a might reside on a storage medium local to (and/or resident in) a server 615a (and/or a user computer, user device, or customer device 605). Alternatively, a database 620n can be remote from any or all of the computers 605, 615, so long as it can be in communication (e.g., via the network 610) with one or more of these. In a particular set of embodiments, a database 620 can reside in a storage-area network ("SAN") familiar to those skilled in the art. (Likewise, any necessary files for performing the functions attributed to the computers 605, 615 can be stored locally on the respective computer and/or remotely, as appropriate.) In one set of embodiments, the database 620 can be a relational database, such as an Oracle database, that is adapted to store, update, and retrieve data in response to SQL-formatted commands. The database might be controlled and/or maintained by a database server, as described above, for example.

[0145] According to some embodiments, system 600 may further comprise a first area 625 (similar to first area 110 of Fig. 1, or the like) in which the mobile device(s) 605a and/or 605b (similar to mobile devices 105, 105a-105n, 205 A, and 205B of Figs. 1 and 2, or the like) may be located. System 600 may further comprise one or more signal sources 630a-630n (collectively, "signal sources 630," "wireless signal sources 630," or the like; similar to wireless signal sources 115a-115n of Fig. 1, or the like) and object(s) 635 (similar to object(s) 120 of Fig. 1, or the like). Each mobile device 605 may comprise a wireless transceiver(s) 640 (similar to wireless transceiver(s) 125 of Fig. 1, or the like), a visual sensor(s) 645 (similar to visual sensor(s) 130 of Fig. 1, or the like), a processor(s) 650 (similar to processor(s) 135 of Fig. 1, or the like), software application 655 (similar to software application 140 of Fig. 1, or the like) running on the processor(s) 650, data storage device 660 (similar to data storage device 150 of Fig. 1, or the like), and display device(s) 665 (similar to display device(s) 155 of Fig. 1, or the like). A software application 650 may be run or executed on the processor(s) 650 of the mobile device 605 to implement multi-person SLAM linkage positioning and navigation, in some cases, for super large-scale scenes based on wireless signal characteristics and visual feature point information, as described herein, or the like. System 600 may further comprise a remote system 670 (similar to remote computing system 160 or back-end cloud service 260 of Fig. 1 or 2, or the like) and corresponding database(s) 675.

[0146] In an aspect, for creating a map model, a software application (e.g., software application 655, or the like) running on a first mobile device (e.g., mobile device 605 among mobile devices 605a and 605b, or the like) that is located in a first area (e.g., first area 625, or the like) may generate or update a first wireless signal fingerprint map model of the first area (e.g., using a wireless signal fingerprint map model creator (similar to wireless signal fingerprint map model creator 145a of Fig. 1, etc.) or the like) based on first signal data that is received from each of at least one first wireless signal source (e.g., wireless signal sources 630a-630n, or the like, via wireless transceiver(s) 640, or the like) within, or in proximity to, the first area. According to some embodiments, the first signal data that is received from each of the at least one first wireless signal source may include, but is not limited to, at least one of Wi-Fi signal data, ultra-wideband ("UWB") signal data, ultra-wideband Doherty ("UWD") signal data, Bluetooth signal data, or Zig-Bee signal data, and/or the like. [0147] The software application may generate or update a first visual feature point map model of at least a portion of the first area (e.g., using a visual feature point map model creator and/or SLAM (similar to visual feature point map model creator 145b and/or SLAM 145c of Fig. 1, etc.) or the like) based on first image data received from the at least one first visual sensor (e.g., visual sensor(s) 645 capturing images (and/or videos) of object(s) 635 within, or in line of sight of, the first area, or the like), the first visual feature point map model being correlated with the first wireless signal fingerprint map model. In some cases, the software application may send the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system over a network (e.g., remote computing system 670 via network(s) 610 via wireless transceiver(s) and any intermediate telecommunications relays systems (not shown), or the like).

[0148] The remote computing system may analyze the correlated first wireless signal fingerprint map model and first visual feature point map model. Based on the analysis, the remote computing system may integrate the correlated received first wireless signal fingerprint map model and first visual feature point map model to generate a global map of the first area (e.g., using a global map integrated generator (similar to global map integrated generator 170 of Fig. 1, etc.), or the like).

[0149] In some embodiments, sending the correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network may comprise sending the correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network using a wireless network data transmission protocol. In some cases, the wireless network data transmission protocol may include, without limitation, at least one of Wi-Fi protocol, Wi-Fi Direct protocol, other protocols under IEEE 802.11 standard, Bluetooth protocol, Zig-Bee protocol, other protocols under IEEE 802.15 standard, 4G broadband protocol, 4G LTE broadband protocol, 5G broadband protocol, or other cellular broadband protocol, and/or the like.

[0150] According to some embodiments, the first wireless signal fingerprint map model of the first area may be generated or updated based at least in part on characteristic information of the first signal data received from each first wireless signal source at a location at which the mobile device receives the first signal data. In some cases, the characteristic information of the first signal data may include, but is not limited to, at least one of name of a wireless network corresponding to said wireless signal source, a media access control ("MAC") address of said wireless signal source, or signal strength of said wireless signal source, and/or the like.

[0151] As the system is designed to enhance mapping and navigation within an area and not for static location determination or mapping, the system is capable of dynamic mapping and navigation as the user moves through the area. As such, according to some embodiments, as the mobile device moves to a second area, the software application may generate or update a second wireless signal fingerprint map model of the second area based on second signal data that is received from each of at least one second wireless signal source within, or in proximity to, the second area. Here, the second area may be one of a different portion of the first area, an area in proximity and overlapping with the first area, or an area in proximity with yet separate from the first area, or the like. The software application may generate or update a second visual feature point map model of at least a portion of the second area based on second image data received from the at least one first visual sensor, the second visual feature point map model being correlated with the second wireless signal fingerprint map model. The software application may send the correlated second wireless signal fingerprint map model and second visual feature point map model to the remote computing system over the network. The remote computing system may analyze the correlated second wireless signal fingerprint map model and second visual feature point map model. Based on the analysis, the remote computing system may integrate the correlated received second wireless signal fingerprint map model and second visual feature point map model to generate a global map of a combination of the first area and the second area.

[0152] In some embodiments, the first mobile device may be associated with a first user, where the first mobile device may be among a plurality of mobile devices (e.g., mobile devices 605a and 605b, or the like) located within the first area, the plurality of mobile devices being associated with a plurality of users. In such a case, the remote computing system may receive correlated wireless signal fingerprint map model and visual feature point map model from each of the plurality of mobile devices. The remote computing system may analyze the correlated wireless signal fingerprint map model and visual feature point map model from each of the plurality of mobile devices. Based on the analysis, the remote computing system may integrate the correlated received wireless signal fingerprint map model and visual feature point map model from each of the plurality of mobile devices to generate the global map of the first area.

[0153] In another aspect, for loading a map model, a software application (e.g., software application 655, or the like) running on a first mobile device (e.g., mobile device 605 among mobile devices 605a and 605b, or the like) that is located in a first area (e.g., first area 625, or the like) may generate a first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within, or in proximity to, the first area. The software application may generate a first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor, the first visual feature point map model being correlated with the first wireless signal fingerprint map model. The software application may send the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system (e.g., remote computing system 670, or the like) over a network (e.g., network(s) 610, or the like). The remote computing system may analyze the correlated first wireless signal fingerprint map model and first visual feature point map model, by: comparing the first wireless signal fingerprint map model with one or more portions of a previously generated global wireless signal fingerprint map using wireless signal fingerprint matching; in response to identifying a matching portion of the previously generated global wireless signal fingerprint map based on the comparison, comparing the first visual feature point map model with one or more portions of a previously generated visual feature point map corresponding to the identified matching portion of the previously generated global wireless signal fingerprint map using visual feature point matching (e.g., using a feature matching system (similar to feature matching system 175 of Fig. 1, etc.), or the like); and, in response to identifying a matching portion of the previously generated visual feature point map, extracting a regional visual feature point map model based on the identified matching portion of the previously generated visual feature point map, and sending the regional visual feature point map model as positioning and navigation data to the software application running on the first mobile device.

[0154] In some embodiments, sending the regional visual feature point map model as positioning and navigation data to the software application running on the first mobile device may comprise: the remote computing system sending the regional visual feature point map model to a positioning and navigation server (e.g., using a positioning and navigation system (similar to positioning and navigation system 185 of Fig. 1, etc.), or the like); and, in response to receiving the regional visual feature point map model, the positioning and navigation server mounting the received regional visual feature point map model to a SLAM system (similar to SLAM 145c of Fig. 1, or the like) to locate and navigate the first area.

[0155] According to some embodiments, concurrent with mounting the received regional visual feature point map model to the SLAM system, the positioning and navigation server may send real-time local area pose information to the remote computing system. In response to receiving the real-time local area pose information, the remote computing system may convert the real-time local area pose information into global pose coordinates using a global coordinate conversion system (e.g., using a global coordinate converter (similar to global coordinate converter 180 of Fig. 1, etc.), or the like). The remote computing system may distribute the global pose coordinates to one or more second mobile devices either within the first area or in proximity of the first area, to establish pose information sharing and linkage between the first mobile device and each of the one or more second devices, where the one or more second mobile devices may each be associated with one or more other users. [0156] These and other functions of the system 600 (and its components) are described in greater detail above with respect to Figs. 1-4.

[0157] While particular features and aspects have been described with respect to some embodiments, one skilled in the art will recognize that numerous modifications are possible. For example, the methods and processes described herein may be implemented using hardware components, software components, and/or any combination thereof. Further, while various methods and processes described herein may be described with respect to particular structural and/or functional components for ease of description, methods provided by various embodiments are not limited to any particular structural and/or functional architecture but instead can be implemented on any suitable hardware, firmware and/or software configuration. Similarly, while particular functionality is ascribed to particular system components, unless the context dictates otherwise, this functionality need not be limited to such and can be distributed among various other system components in accordance with the several embodiments. [0158] Moreover, while the procedures of the methods and processes described herein are described in a particular order for ease of description, unless the context dictates otherwise, various procedures may be reordered, added, and/or omitted in accordance with various embodiments. Moreover, the procedures described with respect to one method or process may be incorporated within other described methods or processes; likewise, system components described according to a particular structural architecture and/or with respect to one system may be organized in alternative structural architectures and/or incorporated within other described systems. Hence, while various embodiments are described with — or without — particular features for ease of description and to illustrate some aspects of those embodiments, the various components and/or features described herein with respect to a particular embodiment can be substituted, added and/or subtracted from among other described embodiments, unless the context dictates otherwise. Consequently, although several embodiments are described above, it will be appreciated that the invention is intended to cover all modifications and equivalents within the scope of the following claims.

Claims

WHAT IS CLAIMED IS:

1. A method for creating a map model, comprising: generating or updating, using a software application running on a first mobile device that is located in a first area, a first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area; generating or updating, using the software application, a first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor, the first visual feature point map model being correlated with the first wireless signal fingerprint map model; and sending, using the software application and over a network, the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system that is configured to generate a global map of the first area based on analysis and integration of the correlated first wireless signal fingerprint map model and first visual feature point map model.

2. The method of claim 1, wherein the mobile device comprises at least one of a smartphone, a tablet computer, an augmented reality (" AR") device, a virtual reality ("VR") device, a mixed reality ("MR") device, a portable gaming device, or a robot.

3. The method of any of claims 1-2, wherein the first area comprises at least one of a room, a corridor, a courtyard, a floor level of a building, a building, a rooftop of a building, a balcony of a building, a patio of a building, a porch of a building, a residential facility, a business facility, a commercial facility, a parking facility, a storage facility, a facility operated by a public utility, a government facility, a military facility, an industrial facility, a transportation facility, a school facility, a school campus, a medical facility, a stadium, an athletics facility, an auditorium, a garden, a park, a zoo, a neighborhood, a city block of a municipality, a city center of a municipality, a municipality, a forest, a jungle, a field, agricultural land, a regional open space, a body of water, a waterway, an underwater area, an island region, a mountain region, a desert region, a rocky region, a sandy region, a terrestrial vehicle, an aquatic vehicle, an amphibious vehicle, an aircraft, a space vehicle, an aquatic facility, a space-based facility, or a portion of one of these areas.

4. The method of any of claims 1-3, wherein the first signal data that is received from each of the at least one first wireless signal source comprises at least one of Wi-Fi signal data, ultra- wideband ("UWB") signal data, ultra- wideband Doherty ("UWD") signal data, Bluetooth signal data, or Zig-Bee signal data.

5. The method of any of claims 1-4, wherein the at least one first visual sensor comprises at least one of a red-green-blue ("RGB") color camera, a wide-angle camera, an ultra-wide-angle camera, a fish-eye camera, or a telephoto camera.

6. The method of any of claims 1-5, wherein sending the correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network comprises sending the correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network using a wireless network data transmission protocol, wherein the wireless network data transmission protocol comprises at least one of Wi-Fi protocol, Wi-Fi Direct protocol, other protocols under IEEE 802.11 standard, Bluetooth protocol, Zig-Bee protocol, other protocols under IEEE 802.15 standard, 4G broadband protocol, 4G LTE broadband protocol, 5G broadband protocol, or other cellular broadband protocol.

7. The method of any of claims 1-6, wherein the first wireless signal fingerprint map model of the first area is generated or updated based at least in part on characteristic information of the first signal data received from each first wireless signal source at a location at which the mobile device receives the first signal data, wherein the characteristic information of the first signal data comprises at least one of name of a wireless network corresponding to said wireless signal source, a media access control ("MAC") address of said wireless signal source, or signal strength of said wireless signal source.

8. The method of any of claims 1-7, wherein the first visual feature point map model of at least a portion of the first area is generated or updated by: extracting, using the software application, feature point data from the received image data of the at least a portion of the first area, the feature point data comprising at least one of object feature point data or environmental feature point data; and implementing, using the software application, at least one of simultaneous localization and mapping ("SLAM") techniques or image recognition techniques on the extracted feature point data to generate the first visual feature point map model of the at least a portion of the first area.

9. The method of any of claims 1-8, wherein generating or updating the first visual feature point map model of the at least a portion of the first area and sending the correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system over the network comprises repeating the following processes until the first visual feature point map model covers all portions of the first area covered by the first wireless signal fingerprint map model: generating or updating, using the software application, a first visual feature point map model of one portion of the first area based on image data received from the at least one first visual sensor for said portion of the first area; and sending, using the software application, the correlated first wireless signal fingerprint map model and first visual feature point map model of said portion of the first area to the remote computing system over the network.

10. The method of any of claims 1-9, further comprising: continually or periodically receiving, using a transceiver of the mobile device, updated first signal data from each of the at least one first wireless signal source within or in proximity to the first area; based on a determination that the updated first signal data has changed compared with previously received updated first signal data, updating, using the software application, the first wireless signal fingerprint map model of the first area based on the updated first signal data; capturing, using at least one first visual sensor of the first mobile device, image data of objects within at least a portion of the first area; based on a determination that the updated image data has changed compared with previously received updated image data, updating, using the software application, the first visual feature point map model of the at least a portion of the first area based on the updated image data; and sending, using the software application and over the network, updated correlated first wireless signal fingerprint map model and first visual feature point map model to the remote computing system, the remote computing system being further configured to generate an updated global map of the first area based on analysis and integration of the updated correlated first wireless signal fingerprint map model and first visual feature point map model.

11. The method of any of claims 1-10, further comprising: generating or updating, using the software application as the mobile device has moved to a second area, a second wireless signal fingerprint map model of the second area based on second signal data that is received from each of at least one second wireless signal source within or in proximity to the second area, wherein the second area is one of a different portion of the first area, an area in proximity and overlapping with the first area, or an area in proximity with yet separate from the first area; generating or updating, using the software application, a second visual feature point map model of at least a portion of the second area based on second image data received from the at least one first visual sensor, the second visual feature point map model being correlated with the second wireless signal fingerprint map model; and sending, using the software application and over the network, the correlated second wireless signal fingerprint map model and second visual feature point map model to the remote computing system, the remote computing system being further configured to generate a global map of a combination of the first area and the second area based on analysis and integration of the correlated second wireless signal fingerprint map model and second visual feature point map model.

12. A method for creating a map model, comprising: receiving, using a remote computing system and from a software application running on a first mobile device that is located in a first area, correlated first wireless signal fingerprint map model and first visual feature point map model, the software application being configured to generate or update the first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area and further configured to generate or update the first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor; analyzing, using the remote computing system, the correlated first wireless signal fingerprint map model and first visual feature point map model; and based on the analysis, integrating, using the remote computing system, the correlated received first wireless signal fingerprint map model and first visual feature point map model to generate a global map of the first area.

13. The method of claim 12, wherein the remote computing system comprises at least one of a server computer over the network, an image processing server, a graphics processing unit ("GPU") -based server, a positioning and mapping server, a machine learning system, an artificial intelligence ("AI") system, a deep learning system, a neural network, a convolutional neural network ("CNN"), a fully convolutional network ("FCN"), a cloud computing system, or a distributed computing system.

14. The method of any of claim 12 or 13, wherein the first mobile device is associated with a first user, wherein the first mobile device is among a plurality of mobile devices located within the first area, the plurality of mobile devices being associated with a plurality of users, wherein the method further comprises: receiving, using the remote computing system, correlated wireless signal fingerprint map model and visual feature point map model from each of the plurality of mobile devices; analyzing, using the remote computing system, the correlated wireless signal fingerprint map model and visual feature point map model from each of the plurality of mobile devices; and based on the analysis, integrating, using the remote computing system, the correlated received wireless signal fingerprint map model and visual feature point map model from each of the plurality of mobile devices to generate the global map of the first area.

15. The method of any of claims 12-14, further comprising: analyzing, using the remote computing system, the correlated first wireless signal fingerprint map model and first visual feature point map model, by: comparing the first wireless signal fingerprint map model with one or more portions of a previously generated global wireless signal fingerprint map using wireless signal fingerprint matching; in response to identifying a matching portion of the previously generated global wireless signal fingerprint map based on the comparison, comparing the first visual feature point map model with one or more portions of a previously generated visual feature point map corresponding to the identified matching portion of the previously generated global wireless signal fingerprint map using visual feature point matching; and in response to identifying a matching portion of the previously generated visual feature point map, extracting a regional visual feature point map model based on the identified matching portion of the previously generated visual feature point map, and sending the regional visual feature point map model to a positioning and navigation server.

16. The method of any of claims 15, further comprising: in response to receiving the regional visual feature point map model, mounting, using the positioning and navigation server, the received regional visual feature point map model to a simultaneous localization and mapping ("SLAM") system to locate and navigate the first area, and concurrently sending, using the positioning and navigation server, real time local area pose information to the remote computing system; in response to receiving the real-time local area pose information, converting, using the remote computing system, the real-time local area pose information into global pose coordinates using a global coordinate conversion system; and distributing, using the remote computing system, the global pose coordinates to one or more second mobile devices either within the first area or in proximity of the first area, to establish pose information sharing and linkage between the first mobile device and each of the one or more second devices, wherein the one or more second mobile devices are each associated with one or more other users.

17. A system for creating a map model, comprising: a first mobile device that is located in a first area, the first mobile device comprising: at least one first processor; and a first non-transitory computer readable medium communicatively coupled to the at least one first processor, the first non-transitory computer readable medium having stored thereon a software application comprising a first set of instructions that, when executed by the at least one first processor, causes the first mobile device to: generate or update a first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area; generate or update a first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor, the first visual feature point map model being correlated with the first wireless signal fingerprint map model; and send the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system over a network, the remote computing system being configured to generate a global map of the first area based on analysis and integration of the correlated first wireless signal fingerprint map model and first visual feature point map model.

18. The system of claim 17, further comprising: the remote computing system, comprising: at least one second processor; and a second non-transitory computer readable medium communicatively coupled to the at least one second processor, the second non- transitory computer readable medium having stored thereon computer software comprising a second set of instructions that, when executed by the at least one second processor, causes the remote computing system to: receive the correlated first wireless signal fingerprint map model and first visual feature point map model; analyze the correlated first wireless signal fingerprint map model and first visual feature point map model; and based on the analysis, integrate the correlated received first wireless signal fingerprint map model and first visual feature point map model to generate the global map of the first area.

19. A method for loading a map model, comprising: generating, using a software application running on a first mobile device that is located in a first area, a first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area; generating, using the software application, a first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor, the first visual feature point map model being correlated with the first wireless signal fingerprint map model; sending, using the software application and over a network, the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system that is configured to extract a regional visual feature point map model based on analysis of the correlated first wireless signal fingerprint map model and first visual feature point map model; and receiving, using the software application over the network, the regional visual feature point map model from the remote computing system.

20. A method for loading a map model, comprising: receiving, using a remote computing system and from a software application running on a first mobile device that is located in a first area, correlated first wireless signal fingerprint map model and first visual feature point map model, the software application being configured to generate the first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area and further configured to generate the first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor; and analyzing, using the remote computing system, the correlated first wireless signal fingerprint map model and first visual feature point map model, by: comparing the first wireless signal fingerprint map model with one or more portions of a previously generated global wireless signal fingerprint map using wireless signal fingerprint matching; in response to identifying a matching portion of the previously generated global wireless signal fingerprint map based on the comparison, comparing the first visual feature point map model with one or more portions of a previously generated visual feature point map corresponding to the identified matching portion of the previously generated global wireless signal fingerprint map using visual feature point matching; and in response to identifying a matching portion of the previously generated visual feature point map, extracting the regional visual feature point map model based on the identified matching portion of the previously generated visual feature point map, and sending the regional visual feature point map model as positioning and navigation data to the software application running on the first mobile device.

21. The method of claim 20, wherein sending the regional visual feature point map model as positioning and navigation data to the software application running on the first mobile device comprises: sending, using the remote computing system, the regional visual feature point map model to a positioning and navigation server; and in response to receiving the regional visual feature point map model, mounting, using the positioning and navigation server, the received regional visual feature point map model to a simultaneous localization and mapping ("SLAM") system to locate and navigate the first area.

22. The method of claim 20 or 21, further comprising: concurrent with mounting the received regional visual feature point map model to the SLAM system, sending, using the positioning and navigation server, real-time local area pose information to the remote computing system; in response to receiving the real-time local area pose information, converting, using the remote computing system, the real-time local area pose information into global pose coordinates using a global coordinate conversion system; and distributing, using the remote computing system, the global pose coordinates to one or more second mobile devices either within the first area or in proximity of the first area, to establish pose information sharing and linkage between the first mobile device and each of the one or more second devices, wherein the one or more second mobile devices are each associated with one or more other users.

23. A system for loading a map model, comprising: a first mobile device that is located in a first area, the first mobile device comprising: at least one first processor; and a first non-transitory computer readable medium communicatively coupled to the at least one first processor, the first non-transitory computer readable medium having stored thereon a software application comprising a first set of instructions that, when executed by the at least one first processor, causes the first mobile device to: generate a first wireless signal fingerprint map model of the first area based on first signal data that is received from each of at least one first wireless signal source within or in proximity to the first area; generate a first visual feature point map model of at least a portion of the first area based on first image data received from at least one first visual sensor, the first visual feature point map model being correlated with the first wireless signal fingerprint map model; send the correlated first wireless signal fingerprint map model and first visual feature point map model to a remote computing system over a network, the remote computing system being configured to extract a regional visual feature point map model based on analysis of the correlated first wireless signal fingerprint map model and first visual feature point map model; and receive the regional visual feature point map model from the remote computing system.

24. The system of claim 23, further comprising: the remote computing system, comprising: at least one second processor; and a second non-transitory computer readable medium communicatively coupled to the at least one second processor, the second non- transitory computer readable medium having stored thereon computer software comprising a second set of instructions that, when executed by the at least one second processor, causes the remote computing system to: receive the correlated first wireless signal fingerprint map model and first visual feature point map model; and analyze the correlated first wireless signal fingerprint map model and first visual feature point map model, by: comparing the first wireless signal fingerprint map model with one or more portions of a previously generated global wireless signal fingerprint map using wireless signal fingerprint matching; in response to identifying a matching portion of the previously generated global wireless signal fingerprint map based on the comparison, comparing the first visual feature point map model with one or more portions of a previously generated visual feature point map corresponding to the identified matching portion of the previously generated global wireless signal fingerprint map using visual feature point matching; and in response to identifying a matching portion of the previously generated visual feature point map, extracting the regional visual feature point map model based on the identified matching portion of the previously generated visual feature point map, and sending the regional visual feature point map model as positioning and navigation data to the software application running on the first mobile device.