WO2024025469A1

WO2024025469A1 - System for determining location of device and methods for forming and operating thereof

Info

Publication number: WO2024025469A1
Application number: PCT/SG2023/050517
Authority: WO
Inventors: Lihua Xie; He Huang; Xu Fang; Jianfei Yang
Original assignee: Nanyang Technological University
Priority date: 2022-07-28
Filing date: 2023-07-25
Publication date: 2024-02-01

Abstract

Various embodiments may provide a system for determining a location of a device. The system may include a backend server configured to receive device signals including inertial measurement unit (IMU) data from the device. The system may also include a plurality of Wi- Fi sensors configured to transmit received signal strength (RSS) signals including received signal strength (RSS) readings (also referred to as RSS fingerprints) to the backend server upon receiving the device signals from the device. The system may be configured to determine the location of the device based on the inertial measurement unit (IMU) data and the received signal strength (RSS) readings. The system may include a neural network configured to generate a posterior estimate based on the received signal strength (RSS) readings.

Description

SYSTEM FOR DETERMINING LOCATION OF DEVICE AND METHODS FOR FORMING AND OPERATING THEREOF

CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of priority of Singapore application No. 10202250617P filed July 28, 2022, the contents of it being hereby incorporated by reference in its entirety for all purposes.

TECHNICAL FIELD

[0002] Various embodiments of this disclosure may relate to a system for determining a location of a device. Various embodiments of this disclosure may relate to a method of forming a system for determining a location of a device. Various embodiments of this disclosure may relate to a method of operating a system for determining a location of a device.

BACKGROUND

[0003] Localization accuracy plays an important role in indoor location-based services, such as pedestrian navigation, customer flow analysis and health care. However, some existing indoor localization technologies are difficult or expensive to be deployed. For example, the widely used Global Positioning System (GPS) can only be applied to outdoor environments. Real-time simultaneous localization and mapping (SLAM) technologies have an unacceptable drift without odometer correction. The ultrawide band (UWB)-based localization suffers from None-Line-of-Sight (NLOS) issue. Vision-based localization has the problem of a small sensing range. Also, the performance of vision-based localization may be affected by NLOS issues.

[0004] An alternative method which utilizes the WiFi received signal strength (RSS) has attracted researchers' attention due to the popularity of smart devices and the prevalence of wireless network infrastructure. These characteristics bring about the properties of low cost, high scalability and pedestrian (user) compatibility. A pedestrian or user carrying a mobile phone is able to scan the WiFi Access Points (APs), while the installed APs can sniff the packets transmitted from the pedestrian or user’s mobile phone. RSS fingerprints obtained from either the scanned or sniffed results can be used for WiFi localization. WiFi based localization system includes two parts, i.e. the platform and the algorithm. The platform provides access between WiFi APs and mobile devices, while the algorithm processes the raw data and estimates the pedestrian or user’s position.

[0005] However, traditional WiFi based localization systems are difficult to be applied directly. Firstly, traditional WiFi sensing platforms are not able to provide high frequency sensing rates, and most platforms are not easy to install due to the cost and limitations of specific devices. Secondly, the algorithms can only estimate the user’s position based on WiFi RSS readings, whose localization accuracy is difficult to improve without fusion with other data of other sensors.

[0006] The problem of existing WiFi localization systems is that the systems are not flexible to deploy and combine with other sensors. FIG. 1 is a schematic illustrating an existing WiFi based localization method. The conventional WiFi sensing platforms usually utilize specific devices such as customized routers, Intel 5300 and Atheros Network Interface Cards (NICs) to get RSS fingerprints from received packets, and are not easy to be deployed, because the routers are expensive and need customized firmware configuration, while the Intel and Atheros NICs would need to be installed within a laptop. On the other hand, most existing algorithms for WiFi localization cannot fuse WiFi data with data from other sensors, and thus the localization accuracy is difficult to improve.

[0007] The most widely used scheme for WiFi localization includes a combination of a router based sensing platform and a RSS fingerprinting method.

[0008] A conventional router based sensing platform includes several parts: customized routers (serving as WiFi APs), the central router and the backend server. Although the RSS readings can be obtained through active scans of mobile device, the scan takes a relatively long time (about at least 2 seconds per scan) and consumes much power. As such, scanning is unable to provide high sensing rate and persistence for localization. As a result, most existing sensing schemes utilize routers to sniff the packets transmitted by the mobile device, which need customization on firmware. In this way, signals of the mobile device can be sniffed during transmission. The central router establishes a public WLAN, where all the mobile devices, APs and the backend server are connected, so that the APs could send sniffed data to the backend server in real time, and they can be kept synchronized via communication. The backend server receives the distributed RSS readings and synchronizes them into RSS fingerprints, which are used for implementation of localization algorithms. Nonetheless, this platform lacks the capability of large deployment due to the size and cost of the routers, as well as the excessive workload of such a configuration.

[0009] The RSS fingerprinting method includes phases: an offline phase and an online phase. In the offline phase, a survey of the reference sites is conducted to collect RSS fingerprints so that a database can be built. In the online phase, the measured RSS fingerprints will be compared to those in the database to select several reference points with the highest RSS similarities. Thereafter, the estimated position of the user can be calculated as the weighted sum of the selected points. However, RSS fingerprinting approaches suffer from noise, interference, shadowing and multipath effects, which introduce uncertainty into the RSS fingerprints and thus degrade the localization performance.

[0010] In summary, conventional router based sensing platforms and RSS fingerprinting methods can be used for WiFi localization in simple scenarios which do not have stringent requirements about localization accuracy and system deployment. However, an improved scheme is required for more complex scenarios with stringent requirements.

SUMMARY

[0011] Various embodiments may provide a system for determining a location of a device. The system may include a backend server configured to receive device signals including inertial measurement unit (IMU) data from the device. The system may also include a plurality of Wi-Fi sensors configured to transmit received signal strength (RSS) signals including received signal strength (RSS) readings (also referred to as RSS fingerprints) to the backend server upon receiving the device signals from the device. The system may be configured to determine the location of the device based on the inertial measurement unit (IMU) data and the received signal strength (RSS) readings. The system may include a neural network configured to generate a posterior estimate based on the received signal strength (RSS) readings.

[0012] Various embodiments may provide a method of forming a system for determining a location of a device. The method may include providing a backend server configured to receive device signals including inertial measurement unit (IMU) data from the device. The method may also include providing a plurality of Wi-Fi sensors configured to transmit received signal strength (RSS) signals including received signal strength (RSS) readings to the backend server upon receiving the device signals from the device. The system may be configured to determine the location of the device based on the inertial measurement unit (IMU) data and the received signal strength (RSS) readings. The system may include a neural network configured to generate a posterior estimate based on the received signal strength (RSS) readings.

[0013] Various embodiments may provide a method of operating a system for determining a location of a device. The method may include using a backend server configured to receive device signals including inertial measurement unit (IMU) data from the device. The method may also include using a plurality of Wi-Fi sensors to receive the device signals from the device, the plurality of Wi-Fi sensors configured to transmit received signal strength (RSS) signals including received signal strength (RSS) readings to the backend server upon receiving the device signals from the device. The system may be configured to determine the location of the device based on the inertial measurement unit (IMU) data and the received signal strength (RSS) readings. The system may include a neural network configured to generate a posterior estimate based on the received signal strength (RSS) readings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In the drawings, like reference characters generally refer to the same parts throughout the different views. The drawings are not necessarily drawn to scale, emphasis instead generally being placed upon illustrating the principles of various embodiments. In the following description, various embodiments of the invention are described with reference to the following drawings.

FIG. 1 is a schematic illustrating an existing WiFi based localization method.

FIG. 2 is a general illustration of a system for determining a location of a device according to various embodiments.

FIG. 3 is a general illustration of a method of forming a system for determining a location of a device according to various embodiments.

FIG. 4 is a general illustration of a method of operating a system for determining a location of a device according to various embodiments.

FIG. 5 is a schematic showing a framework of the localization system according to various embodiments.

FIG. 6 is a schematic showing a system for determining a location of a device 606 according to various embodiments. FIG. 7A shows a three-dimensional (3D) plot of intensity as a function of position along x-axis and position along y-axis illustrating the measured relative signal strength (RSS) distribution at access point (AP) 3 according to various embodiments.

FIG. 7B shows a three-dimensional (3D) plot of intensity as a function of position along x-axis and position along y-axis illustrating the predicted mean relative signal strength (RSS) distribution at access point (AP) 3 according to various embodiments.

FIG. 7C shows a three-dimensional (3D) plot of intensity as a function of position along x-axis and position along y-axis illustrating the measured relative signal strength (RSS) distribution at access point (AP) 6 according to various embodiments.

FIG. 7D shows a three-dimensional (3D) plot of intensity as a function of position along x-axis and position along y-axis illustrating the predicted mean relative signal strength (RSS) distribution at access point (AP) 6 according to various embodiments.

FIG. 8 is a schematic showing the determination of a location of a device according to various embodiments.

FIG. 9 is a table comparing an embodiment with existing localization methods.

DESCRIPTION

[0015] The following detailed description refers to the accompanying drawings that show, by way of illustration, specific details and embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. Other embodiments may be utilized and structural, logical, and electrical changes may be made without departing from the scope of the invention. The various embodiments are not necessarily mutually exclusive, as some embodiments can be combined with one or more other embodiments to form new embodiments.

[0016] Features that are described in the context of an embodiment may correspondingly be applicable to the same or similar features in the other embodiments. Features that are described in the context of an embodiment may correspondingly be applicable to the other embodiments, even if not explicitly described in these other embodiments. Furthermore, additions and/or combinations and/or alternatives as described for a feature in the context of an embodiment may correspondingly be applicable to the same or similar feature in the other embodiments.

[0017] In the context of various embodiments, the articles “a”, “an” and “the” as used with regard to a feature or element include a reference to one or more of the features or elements. [0018] In the context of various embodiments, the term “about” or “approximately” as applied to a numeric value encompasses the exact value and a reasonable variance, e.g. within 10% of the specified value.

[0019] As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items.

[0020] By “comprising” it is meant including, but not limited to, whatever follows the word “comprising”. Thus, use of the term “comprising” indicates that the listed elements are required or mandatory, but that other elements are optional and may or may not be present.

[0021] By “consisting of’ is meant including, and limited to, whatever follows the phrase “consisting of’. Thus, the phrase “consisting of’ indicates that the listed elements are required or mandatory, and that no other elements may be present.

[0022] Embodiments described in the context of one of the systems are analogously valid for the other system. Similarly, embodiments described in the context of a method are analogously valid for a system, and vice versa.

[0023] Various embodiments may address one of more issues facing existing localization schemes. Various embodiments may provide advantages, e.g. improved localization accuracy and/or greater ease of deployment over existing router based sensing platforms and RSS fingerprinting methods.

[0024] FIG. 2 is a general illustration of a system for determining a location of a device according to various embodiments. The system may include a backend server 202 configured to receive device signals including inertial measurement unit (IMU) data from the device. The system may also include a plurality of Wi-Fi sensors 204 configured to transmit received signal strength (RSS) signals including received signal strength (RSS) readings (also referred to as RSS fingerprints) to the backend server 202 upon receiving the device signals from the device. The system, e.g. the backend server 202, may be configured to determine the location of the device based on the inertial measurement unit (IMU) data and the received signal strength (RSS) readings. The system may include a neural network (also referred to as interference network) configured to generate a posterior estimate based on the received signal strength (RSS) readings.

[0025] In other words, the system may include a backend server 202 and a plurality of WiFi sensors 204 coupled to a backend server 202. The backend server 202 may be used to receive signals with inertial measurement unit (IMU) data from a device, while the plurality of Wi-Fi sensors 204 may transmit received signal strength (RSS) readings to the backend server 202 after receiving the device signals from the device via a sniffing process. The system may determine the location of the device based on the inertial measurement unit (IMU) data and the received signal strength (RSS) reading. The system may include a neural network configured to generate a posterior estimate based on the received signal strength (RSS) readings.

[0026] For avoidance of doubt, FIG. 2 illustrates the features of a system according to various embodiments, and is not intended to limit the shapes, sizes, arrangement, orientation etc. of the various features.

[0027] The device may be an electronic device. For instance, the device may be a mobile device, a tablet, a smart watch, or a tracking tag. The device may be carried by an user or pedestrian. Accordingly, determining a location of the device may also determine a location of the user or pedestrian.

[0028] In various embodiments, the system may also include the device.

[0029] In various embodiments, the plurality of WiFi sensors 204 may be configured to receive or sniff the device signals from the device. The plurality of WiFi sensors 204 may be configured to generate the RSS readings (or RSS fingerprints) based on the device signals (i.e. intensities of the device signals) received by different WiFi sensors 204 of the plurality of WiFi sensors 204, and may be further configured to transmit the received signal strength (RSS) signals including the received signal strength (RSS) readings to the backend server 202 upon receiving the device signals from the device. The RSS readings indicating the intensities of the device signals received by different WiFi sensors of the plurality of WiFi sensors 204 may collectively be used (as inputs by the neural network) to provide a posterior estimate of the location of the device 202. The firmware developed may enable the WiFi sensors 204 to sniff the passing packets (i.e. device signals) transmitted by the device (e.g. mobile phone) without privacy concerns.

[0030] In various embodiments, the plurality of Wi-Fi sensors 204 may be configured to communicate with one another, e.g. for synchronization to receive the device signals.

[0031] The inertia measurement unit (IMU) data may be generated by an IMU unit including a microelectromechanical system (MEMS). The IMU unit may, for instance, include IMU sensors such as a gyroscope and an accelerometer. The IMU data may indicate, for instance, a linear acceleration of the device and an angular rate of the device. [0032] In various embodiments, the backend server 202 or the device may include the neural network. The network may be a multilayer perceptron (MLP) network.

[0033] In various embodiments, the system, e.g. the backend server 202 or the device, may be further configured to generate a signal map, the signal map providing predicted RSS fingerprints, using Gaussian Process Regression (GPR) based on the posterior estimate. A polynomial surface function may be adopted to provide a mean of the predicted RSS readings, while a residual part of the predicted RSS readings may be estimated by the GPR. The GPR model may provide a loss containing gradient information for real-time position optimization. [0034] In various embodiments, the system, e.g. the backend server 202 or the device, may be further configured to determine a reconstruction loss based on the signal map and the received signal strength (RSS) readings.

[0035] In various embodiments, e.g. the backend server 202 or the device, may be configured to generate a prior estimate based on the inertial measurement unit (IMU) data (i.e. in t time step) and a previous optimized position (i.e. t-1 time step) via pedestrian dead reckoning (PDR) . The posterior estimate of the location of the device 202 by the neural network may occur in t time step. The calculations as described herein may be carried out iteratively, i.e. for each time step.

[0036] In various embodiments, the system, e.g. the backend server 202 or the device, may be further configured to generate a Kullback-Leibler (KL) divergence based on the prior estimate and the posterior estimate. The Kullback-Leibler (KL) divergence may compare a similarity between the prior estimate and the posterior estimate.

[0037] In various embodiments, the system, e.g. the backend server 202 or the device, may be further configured to determine a current or present optimized position of the device based on the Kullback-Leibler (KL) divergence and the reconstruction loss. The determination of the current or present optimized position of the device may involve summation of the Kullback- Leibler (KL) divergence and the reconstruction loss. The determination of current or present optimized position of the device may involve Adam algorithm.

[0038] In various embodiments, one or more parameters of the neural network may be configured to be changed based on the current or present optimized position of the device. The one or more parameters may reiteratively be changed or optimized based on the Kullback- Leibler (KL) divergence and the reconstruction loss using gradient-descent back propagation. As described above, various embodiments may use PDR and GPR, which may solely contribute to the loss terms for optimization. In various other embodiments, PDR and/or GPR may be replaced by any other suitable methods. Any parameter which can be updated according to the gradient-descent algorithm can be optimized or changed. For instance, in an MLP network, the weights of each neuron may be changed.

[0039] In various embodiments, the system or the backend server 202 may be configured to perform calculation or determination based on a logic implementing entity, e.g. software or algorithms present on the system or the backend server 202.

[0040] In various embodiments, each of the plurality of Wi-Fi sensors 204 may include an ESP32 microcontroller. However, it may be envisioned that each of the plurality of Wi-Fi sensors 204 may include any suitable microcontroller or communication circuit, such as standalone Wi-Fi enabled microcontrollers or microcontrollers with WiFi functions (e.g. Qualcomm chipset and microcontroller). In various embodiments, some developing toolkits may also be used as WiFi sensors 204.

[0041] In various embodiments, the plurality of Wi-Fi sensors 204 may be coupled to the backend server 202 via wired means (e.g. optical fibers or telecommunication wires), wireless means (e.g. Bluetooth or WiFi), or a combination of wired and wireless means.

[0042] The system may be referred to as an IMU-WiFi localization system.

[0043] FIG. 3 is a general illustration of a method of forming a system for determining a location of a device according to various embodiments. The method may include, in 302, providing a backend server configured to receive device signals including inertial measurement unit (IMU) data from the device. The method may also include, in 304, providing a plurality of Wi-Fi sensors configured to transmit received signal strength (RSS) signals including received signal strength (RSS) readings to the backend server upon receiving the device signals from the device. The system may be configured to determine the location of the device based on the inertial measurement unit (IMU) data and the received signal strength (RSS) readings. The system may include a neural network configured to generate a posterior estimate based on the received signal strength (RSS) readings.

[0044] In other words, the method may include forming a system including a backend server and a plurality of Wi-Fi sensors as described herein for determining a location of the device. [0045] For avoidance of doubt, FIG. 3 is not intended to limit the sequence of the steps. Step 302 may occur before, after or at the same time as step 304. [0046] In various embodiments, the system may be further configured to generate a signal map, the signal map providing predicted RSS fingerprints, using Gaussian Process Regression (GPR) based on the posterior estimate.

[0047] In various embodiments, the system may be further configured to determine a reconstruction loss based on the signal map and the received signal strength (RSS) readings.

[0048] In various embodiments, the system may be configured to generate a prior estimate based on the inertial measurement unit (IMU) data and a previous optimized position via pedestrian dead reckoning (PDR).

[0049] In various embodiments, the system may be further configured to generate a Kullback-Leibler (KL) divergence based on the prior estimate and the posterior estimate.

[0050] In various embodiments, the Kullback-Leibler (KL) divergence may compare a similarity between the prior estimate and the posterior estimate.

[0051] In various embodiments, the system may be further configured to determine a current optimized position of the device based on the Kullback-Leibler (KL) divergence and the reconstruction loss.

[0052] In various embodiments, one or more parameters of the neural network may be configured to be changed based on the current optimized position of the device.

[0053] In various embodiments, the neural network may be a multilayer perceptron (MLP) network.

[0054] In various embodiments, each of the plurality of Wi-Fi sensors may include an ESP32 microcontroller.

[0055] FIG. 4 is a general illustration of a method of operating a system for determining a location of a device according to various embodiments. The method may include, in 402, using a backend server configured to receive device signals including inertial measurement unit (IMU) data from the device. The method may also include, in 404, using a plurality of Wi-Fi sensors to receive the device signals from the device, the plurality of Wi-Fi sensors configured to transmit received signal strength (RSS) signals including received signal strength (RSS) readings to the backend server upon receiving the device signals from the device. The system may be configured to determine the location of the device based on the inertial measurement unit (IMU) data and the received signal strength (RSS) readings. The system may include a neural network configured to generate a posterior estimate based on the received signal strength (RSS) readings. [0056] In other words, various embodiments may involve using the system including the backend server and the plurality of Wi-Fi sensor to determine a location of the device based on a combination of inertial measurement unit (IMU) data and received signal strength (RSS) readings.

[0057] For avoidance of doubt, FIG. 4 is not intended to limit the sequence of the steps. Step 402 may occur before, after or at the same time as step 404.

[0058] Scalability and fusion compatibility are the main bottlenecks of the WiFi based localization systems, since scalability decides the scale of WiFi system in which more APs will improve the accuracy and serve more users at the same time, while accuracy can also be enhanced by sensor fusion. Various embodiments may tackle the scalability problem for the development of a sensing platform based on ESP32 chipsets. The tiny size of ESP32 chipsets and the firmware may enable the sensing platform to sniff and scan for RSS readings in the wireless traffic and send data to the backend server with a high rate. Moreover, various embodiments may provide higher localization accuracy compared to RSS fingerprinting methods by integrating the IMU and WiFi RSS data based on a variational inference algorithm. [0059] Various embodiments may relate to a WiFi sensing platform for smart devices. The sensors may be easy to install, and the developed firmware may enable the platform to obtain RSS readings flexibly. The sensing platform may also be suitable for sensor fusion, in which other types of sensors have the ability to access the system.

[0060] Various embodiments may relate to an algorithm for WiFi based localization using variational inference for optimization. The IMU and WiFi RSS data may be fused to enhance the localization robustness and accuracy. The RSS data may be used to optimize the estimate based on the reconstruction loss, while the IMU data may be translated into displacement information that infers the movement of the estimate.

[0061] Various embodiments may relate to a signal map based on GPR (Gaussian Process Regression) that is able to provide the reconstruction of RSS fingerprints for localization.

[0062] FIG. 5 is a schematic showing a framework of the localization system according to various embodiments. The idea of variational inference may be introduced into WiFi localization to estimate the user or pedestrian’s trajectory in two-dimensional (2-D) plane. In the position estimation process 502, PDR may be used to obtain a prior estimate, and variational inference may be applied to optimize the position based on the prior estimate and the real time RSS fingerprints (step 504). A MLP (Multilayer Perceptron) network may be incorporated to compute the posterior estimate (inferred position) given the RSS fingerprints (step 506), whose weights are updated according to the gradient-descent algorithm in real time. Besides, a GPR (Gaussian Process Regressor) may be designed to generate an RSS map (step 508), which provides predicted RSS fingerprints at the given position. The loss of optimization is evaluated as the log -likelihood of the measured RSS under the reconstructed distribution generated in the signal map, together with the Kullback-Leibler (KL) divergence between the prior and posterior estimates. By minimizing the total loss, the model can reach convergence, where the optimized results calculated by the system in this step may be taken as inputs provided to the PDR (“Posterior Estimate” as shown in FIG 5) in the next step. Further, parameters of the interference network may be updated based on the optimized results (shown as “Parameter Update” in FIG. 5). Finally, a filtering mechanism may be introduced to reduce the local minimum cases and improve the performance.

[0063] FIG. 6 is a schematic showing a system for determining a location of a device 606 according to various embodiments. The system may include a backend server 602 and a plurality of Wi-Fi sensors 604. The backend server 602 may be configured to receive device signals including inertial measurement unit (IMU) data from the device 606. The system may also include a plurality of Wi-Fi sensors 604 configured to transmit received signal strength (RSS) signals including received signal strength (RSS) readings (also referred to as RSS fingerprints) to the backend server 602 upon receiving the device signals from the device 606. The system, e.g. the backend server 602, may be configured to determine the location of the device 506 based on the inertial measurement unit (IMU) data and the received signal strength (RSS) readings.

[0064] The IMU sensor of the device 606 may generate IMU data. The IMU data may be sent to the network interface card (NIC) of the device 606. The device signals may be transmitted from the NIC of the device 506 to the backend server 602 and the plurality of WiFi sensors 604.

[0065] Each Wi-Fi sensor 604 of the system may utilize the commercial off-the-shelf (COTS) ESP32 WROVER-IE sensor with a connected IPEX antenna to enhance the signal quality. ESP32 series sensors are widely used in Internet of Things (loT) areas for various purposes because they consume less power and support all the necessary functions for WiFi and Bluetooth. A firmware may be developed based on FreeRTOS for ESP32. Passive sniffing may be implemented so that the regular data frames in the WiFi network can be sniffed, to obtain the RSS fingerprints. The device 606 may keep transmitting device signals containing the IMU data to the backend server 602. The wireless packets of the device signals may be sniffed by the plurality of WiFi sensors 604, and the plurality of WiFi sensors 604 may be configured to generate RSS readings (also referred to as RSS fingerprints) based on the intensities of the device signals received by different WiFi sensors 6604 of the plurality of WiFi sensors 604. The backend server 602 may receive the RSS readings from different WiFi sensors 604, and the IMU data from the device 602. The plurality of WiFi sensors 604 may be configured to generate the RSS readings based on the device signals emitted by the device 606, and may be further configured to transmit received signal strength (RSS) signals including the received signal strength (RSS) readings to the backend server 602.

[0066] After synchronization and denoising, the IMU data and the received signal strength (RSS) readings may be used for localization. There are two major advantages for passive sniffing on ESP32 based platform. First, all the WiFi sensors 604 may connect to the user's WiFi network, instead of requiring the device 606 of the user to connect to a new wireless local area network (WUAN,) which means that the user's daily work will not be affected. In addition, the plurality of sensors 604 may be able to communicate with one another under the same network to synchronize the sniffing frame from the device 606. Second, this system or platform can be easily deployed in various environments due to the sensor's tiny size of 55 mm x 28 mm x 17 mm.

[0067] The signal map may provide predicted RSS fingerprints at an arbitrary position, which is also referred to as reconstruction of measurement. Due to the presence of noise, interference and multipath effects, the ideal log-distance path loss model may not be able to predict the RSS distribution precisely. Instead, the nonparametric Gaussian Process Regression (GPR) may be employed to capture statistical features of the RSS distribution and predict RSS fingerprints. Nonetheless, original GPR assumes zero mean function, which is obviously impractical for RSS predictions. Thus, a polynomial surface function may be adopted to predict the mean of the RSS predictions, while the residual part of RSS predictions is estimated by the GPR.

[0068] FIG. 7A shows a three-dimensional (3D) plot of intensity as a function of position along x-axis and position along y-axis illustrating the measured relative signal strength (RSS) distribution at access point (AP) 3 according to various embodiments. FIG. 7B shows a three- dimensional (3D) plot of intensity as a function of position along x-axis and position along y- axis illustrating the predicted mean relative signal strength (RSS) distribution at access point (AP) 3 according to various embodiments. FIG. 7C shows a three-dimensional (3D) plot of intensity as a function of position along x-axis and position along y-axis illustrating the measured relative signal strength (RSS) distribution at access point (AP) 6 according to various embodiments. FIG. 7D shows a three-dimensional (3D) plot of intensity as a function of position along x-axis and position along y-axis illustrating the predicted mean relative signal strength (RSS) distribution at access point (AP) 6 according to various embodiments.

[0069] FIG. 8 is a schematic showing the determination of a location of a device according to various embodiments. The inference network may encode, in step 806, the WiFi RSS fingerprints (i.e. the received signal strength (RSS) readings) into an estimate, namely posterior estimate (or inferred position as indicated in FIG. 5), which may not be accurate initially, and may thus need to be updated to approximate the true state. The constructed signal map can provide the mapping relationship from the posterior estimate of the RSS readings to the position of the device in the form of conditional probability density. The constructed signal map may be generated using GPR. Therefore, the signal map can evaluate whether the posterior estimate match with the real time RSS readings (step 808a), and thus obtain the evaluation, which is called reconstruction loss. On the other hand, to combine IMU data or information, PDR (Pedestrian Dead Reckoning) approach is applied to a previous optimized position (step 802). When there is a new step detected, the PDR may generate an estimate (named as prior estimate) based on the previous optimized position (taken as posterior estimate of the current step) and displacement information translated from IMU data. The function of prior estimate may be to regularize the posterior estimate and avoid the posterior estimate from divergence since these two estimates should be close to each other in theory. The regularization term may generate the KL divergence (step 808b), which reflects the distance between two distributions. As a result, the total loss is the summation of the reconstruction loss and KL divergence, and may be minimized by the Adam algorithm (step 804). The optimization process in step 804 may update the weights of the inference networks according to the gradient descent mechanism, whose results will gradually approximate the true position. Although the computation workload is not heavy compared to many smart phone processors, it needs the operating system) (OS) to support the PyTorch framework, so the computation is conducted on the backend server in the current stage. Fortunately, more and more edge computing devices are developed which equip GPU computation and support the PyTorch framework. It is foreseen that various embodiments may be deployed on devices (e.g. mobile devices) in the future.

[0070] The tests were carried out in an office area of 20m x 17m. Six developed WiFi sensors or modules are placed in different fixed positions but at the same height. The firmware of these sniffers is developed to capture the packets transmitted from the user's device in the 2.4GHz WiFi traffic. Once the packets are obtained by the sniffers of the WiFi sensors or modules, they will be sent to the backend server at a rate of 2Hz, which means the backend server could get 2 samples per second. In this work, data at 32 reference points is collected with 8 RSS fingerprints for each point.

[0071] Various embodiments may be compared with the several existing methods: PDR (Pedestrian Dead Reckoning) method, WKNN (Weighted K Nearest Neighbors) method, UKF (Unscented Kalman Filter) based method and PF (Particle Filter) based method in accuracy, as presented in FIG. 9. FIG. 9 is a table comparing an embodiment with existing localization methods. The experiments are carried out using the same devices in the same indoor environment.

[0072] In order to validate the robustness of different methods, 24 trials are taken for each method in the same trajectory. RMSE (Root Mean Squared Error) is utilized to evaluate the localization accuracy.

[0073] It is obviously that the PDR method performs the worst because of the IMU drifting error that will increase along with time if not corrected. WKNN method is the most widely used RSS fingerprinting method, which requires less computation load, but the localization accuracy is also worse compared to the sensor fusion methods. UKF method, as an advanced version of Kalman filter that is usually used to solve linear estimation problems, can address the almost linear and slightly nonlinear problems. PF is applied to estimate the state with nonlinear equations, and usually performs better than the UKF, which can be validated from these results. Various embodiments may outperform all the above baselines, since various embodiments may make the best of IMU measured displacement and the RSS fingerprinting estimate into optimization, thereby improving the robustness and accuracy of localization.

[0074] Various indoor location based services provide basic capabilities of indoor localization. Various embodiments may provide accurate localization with requirement of computational resources applicable to edge computing devices such as NVIDIA Jetson series. The global Indoor Positioning and Navigation market was valued at $6.92 billion in 2020 and is projected to grow to $23.6 billion in 2025 at a CAGR of 27.9%. Developed countries in North America and Europe remain the largest markets for Indoor Positioning and Navigation Systems sales, accounting for a major share in the global market. The USA is the major market for Indoor Positioning and Navigation in Americas, and accounts around 40% share in the global market. The market in the USA was valued at $2.78 billion in 2020 and is anticipated to grow to $9.58 billion in 2025 at a CAGR of 28.1%. In the Asia Pacific market, China and Japan are the major customers. The Asia Pacific market is rapidly growing owing to the increasing applications of indoor positioning and navigation in aviation, healthcare, logistics etc., at a CAGR of 31.8% and is expected to reach $7.52 billion by 2025. The European market is expected to reach $8.75 billion in 2025 growing at a CAGR of 25.6%. The growing applications of indoor positioning and navigation systems and consumer demands are driving the European market. UK holds the largest share in the European market for indoor positioning and navigation, accounting around 25% share of the market and Germany comes next to the UK with a share of around 21%. Multiple industries benefit from indoor positioning and navigation technologies. Aviation industry uses these technologies for passenger services, such as navigation to designated lounges, baggage tracking, emergency and other airport-related services. Advertising industry has developed geo-fencing, location-based promotions and other applications related to E-commerce and digital retail. For the healthcare sector, hospitals use RFID and Wi-Fi to move patients to desired wards. Logistics uses these technologies for automating inventory control and asset tracking in warehouses. The transportation industry has many applications of these technologies in railway stations such as navigating to ticket counters, inquiry counters, lounges, automated teller machines (ATMs) etc. Geographically, North America has held the largest share in the Global Indoor Localization Market in the previous few years, owning to the surging investments by market players for developing several advanced technologies and offering various applications of indoor location solutions. In addition to this, the rising number of indoor location industry players also fuels the growth of the Indoor Location Market in North America. The launch of cloud techniques and loT systems helps companies automate their operations. The mounting applications of loT are also of great benefit for the expansion of the indoor localization market.

[0075] In this context, various embodiments may be beneficial to indoor location based services, especially for distributed systems that only have stringent deployment requirements and are in need of accurate localization. The cost for deploying the system is much reduced compared to conventional WiFi localization system, which decreases the cost for investors. Also, more functions may be added to conduct sensor fusion. Various embodiments may further stimulate the localization-related applications such as racking and navigation.

Claims

1. A system for determining a location of a device, the system comprising: a backend server configured to receive device signals including inertial measurement unit (IMU) data from the device; and a plurality of Wi-Fi sensors configured to transmit received signal strength (RSS) signals including received signal strength (RSS) readings to the backend server upon receiving the device signals from the device; wherein the system is configured to determine the location of the device based on the inertial measurement unit (IMU) data and the received signal strength (RSS) readings; and wherein the system comprises a neural network configured to generate a posterior estimate based on the received signal strength (RSS) readings.

2. The system according to claim 1, wherein the system is further configured to generate a signal map, the signal map providing predicted RSS fingerprints, using Gaussian Process Regression (GPR) based on the posterior estimate.

3. The system according to claim 2, wherein the system is further configured to determine a reconstruction loss based on the signal map and the received signal strength (RSS) readings.

4. The system according to claim 3, wherein the system is configured to generate a prior estimate based on the inertial measurement unit (IMU) data and a previous optimized position via pedestrian dead reckoning (PDR).

5. The system according to claim 4, wherein the system is further configured to generate a Kullback-Ueibler (KU) divergence based on the prior estimate and the posterior estimate. The system according to claim 5, wherein the Kullback-Leibler (KL) divergence compares a similarity between the prior estimate and the posterior estimate. The system according to claim 5 or claim 6, wherein the system is further configured to determine a current optimized position of the device based on the Kullback-Leibler (KL) divergence and the reconstruction loss. The system according to claim 7, wherein one or more parameters of the neural network are configured to be changed based on the current optimized position of the device. The system according to any one of claims 1 to 8, wherein the neural network is a multilayer perceptron (MLP) network. The system according to any one of claims 1 to 9, wherein each of the plurality of Wi-Fi sensors comprises an ESP32 microcontroller. A method of forming a system for determining a location of a device, the method comprising: providing a backend server configured to receive device signals including inertial measurement unit (IMU) data from the device; and providing a plurality of Wi-Fi sensors configured to transmit received signal strength (RSS) signals including received signal strength (RSS) readings to the backend server upon receiving the device signals from the device; wherein the system is configured to determine the location of the device based on the inertial measurement unit (IMU) data and the received signal strength (RSS) readings; and wherein the system comprises a neural network configured to generate a posterior estimate based on the received signal strength (RSS) readings. thod according to claim 11, wherein the system is further configured to generate a signal map, the signal map providing predicted RSS fingerprints, using Gaussian Process Regression (GPR) based on the posterior estimate. thod according to claim 12, wherein the system is further configured to determine a reconstruction loss based on the signal map and the received signal strength (RSS) readings. thod according to claim 13, wherein the system is configured to generate a prior estimate based on the inertial measurement unit (IMU) data and a previous optimized position via pedestrian dead reckoning (PDR). thod according to claim 14, wherein the system is further configured to generate a Kullback-Leibler (KL) divergence based on the prior estimate and the posterior estimate. thod according to claim 15, wherein the Kullback-Leibler (KL) divergence compares a similarity between the prior estimate and the posterior estimate. thod according to claim 15 or claim 16, wherein the system is further configured to determine a current optimized position of the device based on the Kullback-Leibler (KL) divergence and the reconstruction loss. thod according to claim 17, wherein one or more parameters of the neural network are configured to be changed based on the current optimized position of the device. The method according to any one of claims 11 to 18, wherein the neural network is a multilayer perceptron (MLP) network. The method according to any one of claims 11 to 19, wherein each of the plurality of Wi-Fi sensors comprises an ESP32 microcontroller. A method of operating a system for determining a location of a device, the method comprising: using a backend server configured to receive device signals including inertial measurement unit (IMU) data from the device; and using a plurality of Wi-Fi sensors to receive the device signals from the device, the plurality of Wi-Fi sensors configured to transmit received signal strength (RSS) signals including received signal strength (RSS) readings to the backend server upon receiving the device signals from the device; wherein the system is configured to determine the location of the device based on the inertial measurement unit (IMU) data and the received signal strength (RSS) readings; and wherein the system comprises a neural network configured to generate a posterior estimate based on the received signal strength (RSS) readings.