US20180328738A1 - Systems and methods for utilizing graph based map information as priors for localization using particle filter - Google Patents
Systems and methods for utilizing graph based map information as priors for localization using particle filter Download PDFInfo
- Publication number
- US20180328738A1 US20180328738A1 US15/726,371 US201715726371A US2018328738A1 US 20180328738 A1 US20180328738 A1 US 20180328738A1 US 201715726371 A US201715726371 A US 201715726371A US 2018328738 A1 US2018328738 A1 US 2018328738A1
- Authority
- US
- United States
- Prior art keywords
- particles
- processing unit
- computer
- central processing
- implemented method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
- G01C21/206—Instruments for performing navigational calculations specially adapted for indoor navigation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
- G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
- G01C21/1654—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments with electromagnetic compass
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
- G01S5/02—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves
- G01S5/0278—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves involving statistical or probabilistic considerations
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
- G01S5/02—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves
- G01S5/0294—Trajectory determination or predictive filtering, e.g. target tracking or Kalman filtering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/02—Services making use of location information
- H04W4/029—Location-based management or tracking services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/33—Services specially adapted for particular environments, situations or purposes for indoor environments, e.g. buildings
-
- H04W4/008—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/38—Services specially adapted for particular environments, situations or purposes for collecting sensor information
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/80—Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
Definitions
- the disclosed embodiments relate in general to localization and tracking systems and methods and, more specifically, to systems and methods for utilizing graph based map information as priors for localization using a particle filter.
- each of the above sensors has its own limitations in terms of noise, sampling rate, characteristics and/or dimensionality. This introduces new challenges in estimating the location of user or robot using data supplied by the aforesaid sensors. Therefore, in view of the aforesaid limitations of the conventional technology, new and improved systems and methods are needed for indoor localization and tracking applications.
- a computer-implemented method the method being performed in a computerized system incorporating a central processing unit, a localization signal receiver, a plurality of sensors, separate and distinct from the localization signal receiver, and a memory, the computer-implemented method involving: using the central processing unit to initialize a plurality of particles based on an information on a map graph; using the central processing unit to repeatedly execute a particle filter loop, wherein the particle filter loop includes: using the central processing unit to perform a motion update of the plurality of particles; using the central processing unit to perform a measurement update of the plurality of particles; and using the central processing unit to perform a resampling of the plurality of particles based on particle importance weights and the map graph information.
- the location of the computerized system is subsequently determined based on the plurality of particles.
- the localization signal receiver is a Bluetooth receiver and wherein the localization signal sources are Bluetooth low energy (BLE) beacons.
- BLE Bluetooth low energy
- the localization signal receiver is a radiofrequency receiver and wherein the localization signal sources are radiofrequency sources.
- the plurality of sensor readings includes magnetometer data.
- the plurality of sensor readings includes acceleration data.
- the plurality of sensor readings includes a three-dimensional spatial orientation data.
- the method further involves calculating an orientation of the computerized system using an attitude and heading reference system (AHRS) algorithm performed on at least some of the plurality of sensor readings.
- AHRS attitude and heading reference system
- the plurality of sensors are incorporated into an inertial measurement unit (IMU).
- IMU inertial measurement unit
- performing the measurement update involves calculating a distance between each of a plurality of particles and at least one localization signal source based on the received localization signal.
- performing the measurement update involves updating a plurality of particles using a confidence of a classifier.
- resampling the plurality of particles involves calculating a weight for each of a plurality of particles and replicating particles of the plurality of particles with higher weights.
- performing the resampling of the plurality of particles involves generating new particles in the plurality of particles based on particle importance weights and conditioning the newly generated particles on the map graph information.
- conditioning the newly generated particles on the map graph information involves projecting each of the newly generated particles on a corresponding closest edge of the map graph.
- the map graph includes a plurality of nodes interconnected by a plurality of edges.
- the nodes of the map graph are evenly spaced.
- the nodes of the map graph are arranged in a square grid.
- the nodes of the map graph are arranged in a triangular grid.
- the nodes of the map graph are evenly spaced inside a room of a building.
- the map graph includes a first plurality of nodes corresponding to an inside of at least two rooms of a building and a second plurality of nodes representing a navigation between the at least two rooms.
- the nodes of the map graph are automatically generated based on a location of a single initial node of the map graph within a room and coordinates of corners of the room.
- a non-transitory computer-readable medium embodying a set of computer-executable instructions, which, when executed in connection with a computerized system incorporating a central processing unit, a Bluetooth localization signal receiver, a plurality of sensors, separate and distinct from the Bluetooth localization signal receiver, and a memory, cause the computerized system to perform a method involving: using the central processing unit to initialize a plurality of particles based on an information on a map graph; using the central processing unit to repeatedly execute a particle filter loop, wherein the particle filter loop includes: using the central processing unit to perform a motion update of the plurality of particles; using the central processing unit to perform a measurement update of the plurality of particles; and using the central processing unit to perform a resampling of the plurality of particles based on particle importance weights and the map graph information.
- the location of the computerized system is subsequently determined based on the plurality of particles.
- a computerized system incorporating a central processing unit, a Bluetooth localization signal receiver, a plurality of sensors, separate and distinct from the Bluetooth localization signal receiver, and a memory, the memory storing a set of computer-readable instructions causing the system to perform a method involving: using the central processing unit to initialize a plurality of particles based on an information on a map graph; using the central processing unit to repeatedly execute a particle filter loop, wherein the particle filter loop includes: using the central processing unit to perform a motion update of the plurality of particles; using the central processing unit to perform a measurement update of the plurality of particles; and using the central processing unit to perform a resampling of the plurality of particles based on particle importance weights and the map graph information.
- the location of the computerized system is subsequently determined based on the plurality of particles.
- FIG. 1 illustrates an exemplary embodiment of a system based on which the described techniques could be implemented.
- FIG. 2 illustrates an exemplary block diagram of an operating sequence of an embodiment of a system for utilizing graph based map information as priors for localization using a particle filter.
- FIG. 3 illustrates the new particles generated from importance weight resampling being projected on the graph edge closest to the respective particles.
- FIG. 4( a ) illustrates square grids of navigation nodes inside three rooms.
- FIG. 4( b ) illustrates exemplary triangular grids of navigation nodes inside three rooms.
- FIG. 5 illustrates an exemplary embodiment of a computerized mobile system that could be used as the handheld client device in connection with the system illustrated in FIG. 1 .
- the described embodiments are generally related to systems and methods capable of accurate localization from noisy sensor measurement.
- accurate localization is a fundamental requirement for various applications, such as applications that require localization and tracking of a robot or a user in a given environment.
- many of the hardware sensors that are used for localization and tracking are very noisy, which results in a challenging signal processing problem.
- particle filter (PF) based localization algorithms have proven successful at effectively fusing various sensors outputs to create meaningful location predictions.
- the aforesaid particle filter algorithm is a Monte Carlo methodology to solve filtering problems for estimating the internal states in dynamical systems using incomplete and/or noisy sensor measurements.
- the modularity of the aforesaid particle filter algorithms makes them ideal for generating real time localization data using readings of multiple noisy sensors.
- a localization method wherein map information, represented as a connected graph, is incorporated within the particle filter based framework to initialize and update the states of the particles, which are conditioned on the graph information.
- the aforesaid graph comprises nodes and edges.
- the localization system fuses the graph information with sensor information within a particle filter framework.
- the aforesaid graph encapsulates the entire environment with nodes and edges, which represents valid space where user or robot can navigate through.
- the graph-based map is utilized in two different steps of the particle filter based localization method. The first time, the graph-based map is used during the initialization of particles and the second time it is used during the resampling of particles.
- the resampling of particles by the particle filter is conditioned on the importance weights as well as the closest edge in the map graph, wherein the new particles are projected on to that edge.
- conditioning the particles on both of these parameters allows the generation of new particles closest to the samples with highest confidence and further projecting them to closest edge in the graph.
- the described embodiment assists the particle filter based localization algorithm in improving its location estimate accuracy as it inherently utilizes areas where the user or robot can navigate.
- the described system and method fuses the graph based map information with readings from multiple sensors, which sense both the motion and environment to get better location estimate.
- the described methodology exploits the modularity of the aforesaid particle filter algorithm to fuse together multiple types of information.
- the aforesaid graph based map information is utilized during the resampling phase of the particle filter operation and the resampling of particles is done using two parameters, the importance weights resampling and projecting the new particles to the closest edge of the graph.
- the proposed fusion of graph-based map in the resampling phase provides a way to generate new particles, which are conditioned on both the importance weight samples and the closest edge to the importance weight sample.
- a graph map can be extrapolated from a single node and edge to multiple nodes and edges so that the map graph can be extended in places where there are insufficient nodes and edges to cover the entire space.
- the aforesaid graph map may be automatically extrapolated starting with a single node information and the coordinates of the corners of the specific room.
- Utilization of graph based map information intuitively provides the particle filter information of walls and other obstacles, which a user or robot cannot walk through. Further, initialization of particles around the graph also allows the particle filter to converge faster as compared to random distribution of particles. Lastly, incorporating the graph in the resampling phase of the particle filter allows the particles with highest weight to maintain its states and only project the new particles on the graph. This provides a more natural way of influencing the highest weight particles to move towards the graph as new particles are generated on the edges of the graph.
- FIG. 1 illustrates an exemplary embodiment of a system 100 based on which the described techniques could be implemented.
- the system 100 is implemented based on a mobile device 101 such as a smartphone.
- the system 100 uses data 106 generated by inertial measurement unit (IMU) sensors 102 and data 107 generated by the BLE receiver 103 , both of which are furnished to a localization module 104 executed by the processor of the mobile device 101 .
- the signal 107 of the BLE receiver 103 corresponds to multiple BLE beacons 105 disposed in the surroundings, such as in different rooms of a building.
- the localization module 104 uses a graph based map 108 to assist in determining the location information as will be described in detail below.
- the BLE scans consist of radio frequency (RF) signals emitted by different BLE beacons 105 present in the environment.
- RF radio frequency
- the IMU sensor 102 is a collection of 9-axis sensors, which provides 3-axis linear acceleration and angular velocity form accelerometer and gyroscope, respectively, as well as 3-axis magnetic field.
- each of these sensors is noisy and hence cannot be used directly to measure the orientation of the device 101 from raw sensor readings.
- the measurement bias of these sensors are time-varying and impacted by various factors, including mechanical stress on the sensor, ambient temperature and the misalignment of the sensor at the time of the installation.
- FIG. 2 illustrates an exemplary block diagram 200 of an operating sequence of an embodiment of a system for utilizing graph based map information as priors for localization using a particle filter.
- FIG. 2 shows a detailed flow of sensor information between each of the blocks in the block diagram 200 .
- the initialization of particles in step 201 is conditioned on the map information 202 available in form of graph.
- the resampling of the particles 205 is conditioned both on the importance of weights 206 and distance between edges of graph and resampled particles, see step 207 .
- the operating sequence shown in FIG. 2 will now be described in detail.
- an initialization step is performed.
- this step 201 involves initializing the particles, wherein the aforesaid initializing is conditioned on the map graph information 202 .
- the aforesaid utilization of the map graph (shown in FIGS. 4( a ) and 4( b ) ) allows the initialization of the particles to be constrained to the areas in which a user or robot is able to travel.
- a motion update step is performed. This step allows for the update of the particles based on a distance travelled by the user or the robot.
- the aforesaid motion update step involves a probabilistic motion update, wherein the localization system determines the orientation of the handheld device 101 using an attitude and heading reference system (AHRS) algorithm, well known in the art, and described in detail in S. O. Madgwick, “An efficient orientation filter for inertial and inertial/magnetic sensor arrays,” Rep. X-Io Univ. Bristol UK, 2010 , calculated from the IMU data, and sample data from the IMU to generate particles along the moving direction.
- the orientation of the mobile device 101 is calculated by fusing both the IMU and the magnetometer sensor data.
- a measurement update is performed. This block updates the likelihood of the particles using the signal strength received from the BLE beacons in the environment.
- the RSSI signals received from each BLE beacon are converted to distance using the Friis Free Space Model, well known in the art, and described in detail in “Friis Free Space Propagation Model,” GaussianWaves, 27 Sep. 2013. It should be noted that, in various alternative embodiments, the aforesaid BLE sensor may be replaced with any other type of sensor, which can sense the environment, including, without limitation, a laser range finder, an image sensor or an ultrasound distance sensor.
- an importance resampling is performed.
- This block uses the importance weight resampling technique to resample particles, see step 206 .
- the system may also utilize a geometric projection technique to condition the resampled particles to the closest edge of the map graph, see step 207 .
- the aforesaid steps 203 - 207 are repeatedly executed within a particle filter loop 208 .
- the new particles 301 generated from importance weight resampling are projected on the graph edge 302 closest to the respective particles.
- the resulting projection is designated in FIG. 3 with a numeral 303 . This allows generation of new particles around the edges of the map graph and at the same time maintaining the property of importance weights resampling.
- the geometric projection is applied only to the new generated particles thereby maintaining the states of the important particles around which the new particles are generated.
- the described localization system uses a grid of interconnected navigation nodes disposed inside each room of a building.
- the grid is aligned to the navigation node entering the room.
- FIG. 4( a ) illustrates an exemplary square grid of navigation nodes 401 inside three rooms connected by graph edges 402 . Specifically, this figure shows square grids with one meter spacing between nodes 401 .
- Nodes 403 in FIG. 4( a ) interconnected with edges 404 , indicate the navigation graph between rooms. In one embodiment, the latter nodes 403 and edges 404 of the graph shown in FIG. 4( a ) are constructed manually.
- FIG. 4( b ) illustrates an exemplary triangular grid of navigation nodes 401 inside the same three rooms.
- the triangular grid shown in FIG. 4( b ) makes the navigation between nodes 401 more natural.
- a similar effect could be achieved by adding diagonals to the grid in FIG. 4( a ) , but that would introduce edges of different lengths and require the addition of nodes in the center of squares where two edges would cross.
- other different geometric shapes and spacing may be chosen to fill the inside of the rooms.
- One example of such other shapes is a hexagonally shaped graph. Therefore, the concepts described herein are not limited to any specific graph structure.
- leaving rooms empty would produce inferior results, because it would introduce a strong bias towards the door in the room.
- the aforesaid map graph consisting of navigation nodes arranged in a square, triangular, hexagonal or any other appropriate patters may be generated automatically using a graph generation algorithms, which uses the coordinate information of the single node in the room and the coordinate information of the four corners of the room.
- the additional nodes in the room are automatically generated by way of extrapolation to cover the entire room with the map graph starting from the single (initial). This may be performed algorithmically and hence does not require any manual human involvement.
- utilizing the aforesaid graph map information constrains the initialization of particles to the nodes and edges of the graph.
- using the geometric projection of the newly resampled particles on to the closest edge in the graph further allows the future state estimates of the particles towards the edges of the graph, hence maintaining the probabilistic posterior update of the particles.
- utilizing the map graph information as a prior provides better estimation of the particle filter based localization.
- FIG. 5 illustrates an exemplary embodiment of a computerized mobile system 500 that could be used as the handheld client device 101 in connection with the system illustrated in FIG. 1 .
- the computerized mobile system 500 may be implemented within a form factor of a mobile computing device, such as a smartphone, a personal digital assistant (PDA), a tablet computer, or a smart watch, all of which are widely available commercially and are well known to persons of skill in the art.
- a mobile computing device such as a smartphone, a personal digital assistant (PDA), a tablet computer, or a smart watch, all of which are widely available commercially and are well known to persons of skill in the art.
- PDA personal digital assistant
- the computerized system 500 may include a data bus 504 or other interconnect or communication mechanism for communicating information across and among various hardware components of the mobile computerized system 500 , and a central processing unit (CPU or simply processor) 501 coupled with the data bus 504 for processing information and performing other computational and control tasks.
- Computerized system 500 also includes a memory 512 , such as a random access memory (RAM) or other dynamic storage device, coupled to the data bus 504 for storing various information as well as instructions to be executed by the processor 501 .
- the memory 512 may also include persistent storage devices, such as a magnetic disk, optical disk, solid-state flash memory device or other non-volatile solid-state storage devices.
- the memory 512 may also be used for storing temporary variables or other intermediate information during execution of instructions by the processor 501 .
- computerized system 500 may further include a read only memory (ROM or EPROM) 502 or other static storage device coupled to the data bus 504 for storing static information and instructions for the processor 501 , such as firmware necessary for the operation of the computerized system 500 , basic input-output system (BIOS), as well as various configuration parameters of the computerized system 500 .
- ROM or EPROM read only memory
- BIOS basic input-output system
- the computerized system 500 may incorporate a display device 509 , which may be also coupled to the data bus 504 , for displaying various information to a user of the computerized system 500 .
- the display device 509 may be associated with a graphics controller and/or graphics processor (not shown).
- the display device 509 may be implemented as a liquid crystal display (LCD), manufactured, for example, using a thin-film transistor (TFT) technology or an organic light emitting diode (OLED) technology, both of which are well known to persons of ordinary skill in the art.
- the display device 509 may be incorporated into the same general enclosure with the remaining components of the computerized system 500 .
- the display device 509 may be positioned outside of such enclosure.
- the computerized system 500 may further incorporate a GPS receiver 503 connected to the data bus 504 and configured to receive location information from one or more GPS satellites and transmit this information to the processor 501 via the data bus 504 .
- a GPS receiver 503 connected to the data bus 504 and configured to receive location information from one or more GPS satellites and transmit this information to the processor 501 via the data bus 504 .
- the computerized system 500 may incorporate one or more input devices, such as a touchscreen interface 510 for receiving tactile commands and a keyboard 506 , which all may be coupled to the aforesaid data bus 504 for communicating information, including, without limitation, user command selections to the processor 501 .
- input devices may include a system for tracking eye movements of the user (not shown), which may be used to indicate to the computerized system 500 the command selection by the user.
- the computerized system 500 may additionally include a location signal receiver 103 , such as a Bluetooth receiver, configured to perform scan for beacons 106 and supply scan data described above to the processor 501 via the data bus 504 .
- a location signal receiver 103 such as a Bluetooth receiver
- the IMU 102 may be also connected to the data bus 504 .
- the computerized system 500 may additionally include a communication interface, such as a network interface 505 coupled to the data bus 504 .
- the network interface 505 may be configured to establish a connection between the computerized system 500 and the Internet 524 using at least one of WIFI interface 507 and the cellular network (GSM or CDMA) adaptor 508 .
- the network interface 505 may be configured to provide a two-way data communication between the computerized system 500 and the Internet 524 .
- the WIFI interface 507 may operate in compliance with 802.11a, 802.11b, 802.11g and/or 802.11n protocols as well as Bluetooth protocol well known to persons of ordinary skill in the art.
- the WIFI interface 507 and the cellular network (GSM or CDMA) adaptor 508 send and receive electrical or electromagnetic signals that carry digital data streams representing various types of information.
- the aforesaid networking components may be used to establish a network data connection between the computerized system 500 and other network components.
- the Internet 524 typically provides data communication through one or more sub-networks to other network resources.
- the computerized system 500 is capable of accessing a variety of network resources located anywhere on the Internet 524 , such as web servers, other content servers as well as other network data storage resources.
- the computerized system 500 is configured send and receive messages, media and other data, including application program code, through a variety of network(s) including Internet 524 by means of the network interface 505 .
- the computerized system 500 when the computerized system 500 acts as a network client, it may request code or data for an application program executing on the computerized system 500 . Similarly, it may send various data or computer code to other network resources.
- the functionality described herein is implemented by computerized system 500 in response to processor 501 executing one or more sequences of one or more instructions contained in the memory 512 .
- Such instructions may be read into the memory 512 from another computer-readable medium.
- Execution of the sequences of instructions contained in the memory 512 causes the processor 501 to perform the various process steps described herein.
- hard-wired circuitry may be used in place of or in combination with software instructions to implement the embodiments of the invention.
- embodiments of the invention are not limited to any specific combination of hardware circuitry and software.
- computer-readable medium refers to any medium that participates in providing instructions to processor 501 for execution.
- the computer-readable medium is just one example of a machine-readable medium, which may carry instructions for implementing any of the methods and/or techniques described herein.
- Such a medium may take many forms, including but not limited to, non-volatile media and volatile media.
- non-transitory computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punchcards, papertape, any other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EPROM, a flash drive, a memory card, any other memory chip or cartridge, or any other medium from which a computer can read.
- Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to processor 501 for execution. For example, the instructions may initially be carried on a magnetic disk from a remote computer.
- a remote computer can load the instructions into its dynamic memory and send the instructions over the Internet 524 .
- the computer instructions may be downloaded into the memory 512 of the computerized system 500 from the foresaid remote computer via the Internet 524 using a variety of network data communication protocols well known in the art.
- the memory 512 of the computerized system 500 may store any of the following software programs, applications or modules:
- Operating system (OS) 513 which may be a mobile operating system for implementing basic system services and managing various hardware components of the computerized system 500 .
- Exemplary embodiments of the operating system 513 are well known to persons of skill in the art, and may include any now known or later developed mobile operating systems.
- Network communication module 514 for enabling network communications using one or more network interfaces described below.
- a localization application 515 may incorporate a set of software modules 516 - 519 , executed by the processor 501 of the computerized system 500 , which cause the computerized mobile system 500 to perform certain predetermined functions.
- the localization application modules 515 may include, for example, a particle initialization module 516 , which initializes particles using the aforesaid map graph information.
- the localization application modules 515 may incorporate a motion update module 517 and a measurement update module 518 , which perform functions 203 and 204 , respectively, shown in FIG. 2 .
- a resampling module 519 may also be deployed, which performs resampling of particles using the aforesaid importance weights as well as the aforesaid map graph information.
- Data storage 520 may be used, for example, for storing the graph based map information as well as IMU bias data 521 .
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Electromagnetism (AREA)
- Probability & Statistics with Applications (AREA)
- Navigation (AREA)
Abstract
Description
- This patent application claims the benefit of priority of and is a continuation-in-part of U.S. patent application Ser. No. 15/595,931 entitled “SYSTEM AND METHOD FOR CALIBRATION-LESSLY COMPENSATING BIAS OF SENSORS FOR LOCALIZATION AND TRACKING” and filed on May 15, 2017 (pending), the entire disclosure of which is incorporated by reference herein.
- The disclosed embodiments relate in general to localization and tracking systems and methods and, more specifically, to systems and methods for utilizing graph based map information as priors for localization using a particle filter.
- The spreading of personal communication systems into many public and private places, as well as the onset of new generation of smartphones, has enabled the development of a vast number of indoor positioning systems based on standard wireless communication technologies as described, for example, in Y. Gu, A. Lo, and I. Niemegeers, “A survey of indoor positioning systems for wireless personal networks,” IEEE Commun. Surv. Tutor., vol. 11, no. 1, pp. 13-32, First 2009. Various localization techniques have been developed and experimented using variety of sensors such as magnetometer, described, for example, in I. Vallivaara, J. Haverinen, A. Kemppainen, and J. Röning, “Magnetic field-based SLAM method for solving the localization problem in mobile robot floor-cleaning task,” in 2011 15th International Conference on Advanced Robotics (ICAR), 2011, pp. 198-203, inertial measurement unit (IMU) is described, for example, in F. Li, C. Zhao, G. Ding, J. Gong, C. Liu, and F. Zhao, “A Reliable and Accurate Indoor Localization Method Using Phone Inertial Sensors,” in Proceedings of the 2012 ACM Conference on Ubiquitous Computing, New York, N.Y., USA, 2012, pp. 421-430, radio frequency sensors (WiFi or Bluetooth low energy (BLE)) described in “Robotics Institute: Wifi localization and navigation for autonomous indoor mobile robots.” [Online]. Available: www.ri.cmu.edu/publication_view.html?pub_id=6819, imaging sensors described in N. Ravi, P. Shankar, A. Frankel, A. Elgammal, and L. Iftode, “Indoor Localization Using Camera Phones,” in Seventh IEEE Workshop on Mobile Computing Systems Applications (WMCSA'06 Supplement), 2006, pp. 49-49 as well as a fusion of these sensors as described in P. Mirowski, T. K. Ho, S. Yi, and M. MacDonald, “SignaISLAM: Simultaneous localization and mapping with mixed WiFi, Bluetooth, LTE and magnetic signals,” in International Conference on Indoor Positioning and Indoor Navigation, 2013, pp. 1-10.
- As would be appreciated by persons of ordinary skill in the art, each of the above sensors has its own limitations in terms of noise, sampling rate, characteristics and/or dimensionality. This introduces new challenges in estimating the location of user or robot using data supplied by the aforesaid sensors. Therefore, in view of the aforesaid limitations of the conventional technology, new and improved systems and methods are needed for indoor localization and tracking applications.
- The embodiments described herein are directed to methods and systems that substantially obviate one or more of the above and other problems associated with conventional localization and tracking systems.
- In accordance with one aspect of the inventive concepts described herein, there is provided a computer-implemented method, the method being performed in a computerized system incorporating a central processing unit, a localization signal receiver, a plurality of sensors, separate and distinct from the localization signal receiver, and a memory, the computer-implemented method involving: using the central processing unit to initialize a plurality of particles based on an information on a map graph; using the central processing unit to repeatedly execute a particle filter loop, wherein the particle filter loop includes: using the central processing unit to perform a motion update of the plurality of particles; using the central processing unit to perform a measurement update of the plurality of particles; and using the central processing unit to perform a resampling of the plurality of particles based on particle importance weights and the map graph information. The location of the computerized system is subsequently determined based on the plurality of particles.
- In one or more embodiments, the localization signal receiver is a Bluetooth receiver and wherein the localization signal sources are Bluetooth low energy (BLE) beacons.
- In one or more embodiments, the localization signal receiver is a radiofrequency receiver and wherein the localization signal sources are radiofrequency sources.
- In one or more embodiments, the plurality of sensor readings includes magnetometer data.
- In one or more embodiments, the plurality of sensor readings includes acceleration data.
- In one or more embodiments, the plurality of sensor readings includes a three-dimensional spatial orientation data.
- In one or more embodiments, the method further involves calculating an orientation of the computerized system using an attitude and heading reference system (AHRS) algorithm performed on at least some of the plurality of sensor readings.
- In one or more embodiments, the plurality of sensors are incorporated into an inertial measurement unit (IMU).
- In one or more embodiments, performing the measurement update involves calculating a distance between each of a plurality of particles and at least one localization signal source based on the received localization signal.
- In one or more embodiments, performing the measurement update involves updating a plurality of particles using a confidence of a classifier.
- In one or more embodiments, resampling the plurality of particles involves calculating a weight for each of a plurality of particles and replicating particles of the plurality of particles with higher weights.
- In one or more embodiments, performing the resampling of the plurality of particles involves generating new particles in the plurality of particles based on particle importance weights and conditioning the newly generated particles on the map graph information.
- In one or more embodiments, conditioning the newly generated particles on the map graph information involves projecting each of the newly generated particles on a corresponding closest edge of the map graph.
- In one or more embodiments, the map graph includes a plurality of nodes interconnected by a plurality of edges.
- In one or more embodiments, the nodes of the map graph are evenly spaced.
- In one or more embodiments, the nodes of the map graph are arranged in a square grid.
- In one or more embodiments, the nodes of the map graph are arranged in a triangular grid.
- In one or more embodiments, the nodes of the map graph are evenly spaced inside a room of a building.
- In one or more embodiments, the map graph includes a first plurality of nodes corresponding to an inside of at least two rooms of a building and a second plurality of nodes representing a navigation between the at least two rooms.
- In one or more embodiments, the nodes of the map graph are automatically generated based on a location of a single initial node of the map graph within a room and coordinates of corners of the room.
- In accordance with another aspect of the inventive concepts described herein, there is provided a non-transitory computer-readable medium embodying a set of computer-executable instructions, which, when executed in connection with a computerized system incorporating a central processing unit, a Bluetooth localization signal receiver, a plurality of sensors, separate and distinct from the Bluetooth localization signal receiver, and a memory, cause the computerized system to perform a method involving: using the central processing unit to initialize a plurality of particles based on an information on a map graph; using the central processing unit to repeatedly execute a particle filter loop, wherein the particle filter loop includes: using the central processing unit to perform a motion update of the plurality of particles; using the central processing unit to perform a measurement update of the plurality of particles; and using the central processing unit to perform a resampling of the plurality of particles based on particle importance weights and the map graph information. The location of the computerized system is subsequently determined based on the plurality of particles.
- In accordance with yet another aspect of the inventive concepts described herein, there is provided a computerized system incorporating a central processing unit, a Bluetooth localization signal receiver, a plurality of sensors, separate and distinct from the Bluetooth localization signal receiver, and a memory, the memory storing a set of computer-readable instructions causing the system to perform a method involving: using the central processing unit to initialize a plurality of particles based on an information on a map graph; using the central processing unit to repeatedly execute a particle filter loop, wherein the particle filter loop includes: using the central processing unit to perform a motion update of the plurality of particles; using the central processing unit to perform a measurement update of the plurality of particles; and using the central processing unit to perform a resampling of the plurality of particles based on particle importance weights and the map graph information. The location of the computerized system is subsequently determined based on the plurality of particles.
- Additional aspects related to the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Aspects of the invention may be realized and attained by means of the elements and combinations of various elements and aspects particularly pointed out in the following detailed description and the appended claims.
- It is to be understood that both the foregoing and the following descriptions are exemplary and explanatory only and are not intended to limit the claimed invention or application thereof in any manner whatsoever.
- The accompanying drawings, which are incorporated in and constitute a part of this specification exemplify the embodiments of the present invention and, together with the description, serve to explain and illustrate principles of the inventive concepts. Specifically:
-
FIG. 1 illustrates an exemplary embodiment of a system based on which the described techniques could be implemented. -
FIG. 2 illustrates an exemplary block diagram of an operating sequence of an embodiment of a system for utilizing graph based map information as priors for localization using a particle filter. -
FIG. 3 illustrates the new particles generated from importance weight resampling being projected on the graph edge closest to the respective particles. -
FIG. 4(a) illustrates square grids of navigation nodes inside three rooms. -
FIG. 4(b) illustrates exemplary triangular grids of navigation nodes inside three rooms. -
FIG. 5 illustrates an exemplary embodiment of a computerized mobile system that could be used as the handheld client device in connection with the system illustrated inFIG. 1 . - In the following detailed description, reference will be made to the accompanying drawing(s), in which identical functional elements are designated with like numerals. The aforementioned accompanying drawings show by way of illustration, and not by way of limitation, specific embodiments and implementations consistent with principles of the present invention. These implementations are described in sufficient detail to enable those skilled in the art to practice the invention and it is to be understood that other implementations may be utilized and that structural changes and/or substitutions of various elements may be made without departing from the scope and spirit of present invention. The following detailed description is, therefore, not to be construed in a limited sense. Additionally, the various embodiments of the invention as described may be implemented in the form of a software running on a general purpose computer, in the form of a specialized hardware, or combination of software and hardware.
- The described embodiments are generally related to systems and methods capable of accurate localization from noisy sensor measurement. As would be appreciated by persons of ordinary skill in the art, such accurate localization is a fundamental requirement for various applications, such as applications that require localization and tracking of a robot or a user in a given environment. As would be also appreciated by persons of ordinary skill in the art, many of the hardware sensors that are used for localization and tracking are very noisy, which results in a challenging signal processing problem.
- On the other hand, particle filter (PF) based localization algorithms, well known to persons of ordinary skill in the art, have proven successful at effectively fusing various sensors outputs to create meaningful location predictions. The aforesaid particle filter algorithm is a Monte Carlo methodology to solve filtering problems for estimating the internal states in dynamical systems using incomplete and/or noisy sensor measurements. As it is well known in the art, the modularity of the aforesaid particle filter algorithms makes them ideal for generating real time localization data using readings of multiple noisy sensors.
- In one or more embodiments, there is provided a localization method wherein map information, represented as a connected graph, is incorporated within the particle filter based framework to initialize and update the states of the particles, which are conditioned on the graph information. In one or more embodiments, the aforesaid graph comprises nodes and edges. The localization system fuses the graph information with sensor information within a particle filter framework. In one or more embodiments, the aforesaid graph encapsulates the entire environment with nodes and edges, which represents valid space where user or robot can navigate through. In various embodiments, the graph-based map is utilized in two different steps of the particle filter based localization method. The first time, the graph-based map is used during the initialization of particles and the second time it is used during the resampling of particles.
- In one embodiment, the resampling of particles by the particle filter is conditioned on the importance weights as well as the closest edge in the map graph, wherein the new particles are projected on to that edge. As would be appreciated by persons of ordinary skill in the art, conditioning the particles on both of these parameters allows the generation of new particles closest to the samples with highest confidence and further projecting them to closest edge in the graph. The described embodiment assists the particle filter based localization algorithm in improving its location estimate accuracy as it inherently utilizes areas where the user or robot can navigate.
- In accordance with one embodiment, the described system and method fuses the graph based map information with readings from multiple sensors, which sense both the motion and environment to get better location estimate. The described methodology exploits the modularity of the aforesaid particle filter algorithm to fuse together multiple types of information. In one or more embodiments, the aforesaid graph based map information is utilized during the resampling phase of the particle filter operation and the resampling of particles is done using two parameters, the importance weights resampling and projecting the new particles to the closest edge of the graph. The proposed fusion of graph-based map in the resampling phase provides a way to generate new particles, which are conditioned on both the importance weight samples and the closest edge to the importance weight sample. Further, we also provide methods using which a graph map can be extrapolated from a single node and edge to multiple nodes and edges so that the map graph can be extended in places where there are insufficient nodes and edges to cover the entire space. Specifically, in one or more embodiments, the aforesaid graph map may be automatically extrapolated starting with a single node information and the coordinates of the corners of the specific room.
- Utilization of graph based map information intuitively provides the particle filter information of walls and other obstacles, which a user or robot cannot walk through. Further, initialization of particles around the graph also allows the particle filter to converge faster as compared to random distribution of particles. Lastly, incorporating the graph in the resampling phase of the particle filter allows the particles with highest weight to maintain its states and only project the new particles on the graph. This provides a more natural way of influencing the highest weight particles to move towards the graph as new particles are generated on the edges of the graph.
-
FIG. 1 illustrates an exemplary embodiment of asystem 100 based on which the described techniques could be implemented. In one embodiment, thesystem 100 is implemented based on amobile device 101 such as a smartphone. Thesystem 100 usesdata 106 generated by inertial measurement unit (IMU)sensors 102 anddata 107 generated by theBLE receiver 103, both of which are furnished to alocalization module 104 executed by the processor of themobile device 101. Thesignal 107 of theBLE receiver 103 corresponds tomultiple BLE beacons 105 disposed in the surroundings, such as in different rooms of a building. In addition, thelocalization module 104 uses a graph basedmap 108 to assist in determining the location information as will be described in detail below. - The BLE scans consist of radio frequency (RF) signals emitted by
different BLE beacons 105 present in the environment. As would be appreciated by persons of ordinary skill in the art, the aforesaid BLE beacon signals are affected heavily by various environmental factors, which makes the localization problem challenging. Further, as it is known in the art, theIMU sensor 102 is a collection of 9-axis sensors, which provides 3-axis linear acceleration and angular velocity form accelerometer and gyroscope, respectively, as well as 3-axis magnetic field. As would be appreciated by persons of ordinary skill in the art, each of these sensors is noisy and hence cannot be used directly to measure the orientation of thedevice 101 from raw sensor readings. In addition, the measurement bias of these sensors are time-varying and impacted by various factors, including mechanical stress on the sensor, ambient temperature and the misalignment of the sensor at the time of the installation. -
FIG. 2 illustrates an exemplary block diagram 200 of an operating sequence of an embodiment of a system for utilizing graph based map information as priors for localization using a particle filter. In addition,FIG. 2 shows a detailed flow of sensor information between each of the blocks in the block diagram 200. As illustrated in this figure, the initialization of particles instep 201 is conditioned on themap information 202 available in form of graph. On the other hand, the resampling of theparticles 205 is conditioned both on the importance ofweights 206 and distance between edges of graph and resampled particles, seestep 207. The operating sequence shown inFIG. 2 will now be described in detail. - First, at
block 201, an initialization step is performed. In one or more embodiments, thisstep 201 involves initializing the particles, wherein the aforesaid initializing is conditioned on themap graph information 202. In one or more embodiments, the aforesaid utilization of the map graph (shown inFIGS. 4(a) and 4(b) ) allows the initialization of the particles to be constrained to the areas in which a user or robot is able to travel. - At
block 203 of the operating sequence shown inFIG. 2 , a motion update step is performed. This step allows for the update of the particles based on a distance travelled by the user or the robot. In various embodiments, the aforesaid motion update step involves a probabilistic motion update, wherein the localization system determines the orientation of thehandheld device 101 using an attitude and heading reference system (AHRS) algorithm, well known in the art, and described in detail in S. O. Madgwick, “An efficient orientation filter for inertial and inertial/magnetic sensor arrays,” Rep. X-Io Univ. Bristol UK, 2010, calculated from the IMU data, and sample data from the IMU to generate particles along the moving direction. In one or more embodiments, the orientation of themobile device 101 is calculated by fusing both the IMU and the magnetometer sensor data. - At
block 204 of the operating sequence shown inFIG. 2 , a measurement update is performed. This block updates the likelihood of the particles using the signal strength received from the BLE beacons in the environment. In one or more embodiments, the RSSI signals received from each BLE beacon are converted to distance using the Friis Free Space Model, well known in the art, and described in detail in “Friis Free Space Propagation Model,” GaussianWaves, 27 Sep. 2013. It should be noted that, in various alternative embodiments, the aforesaid BLE sensor may be replaced with any other type of sensor, which can sense the environment, including, without limitation, a laser range finder, an image sensor or an ultrasound distance sensor. - At
block 205 of the operating sequence shown inFIG. 2 , an importance resampling is performed. This block uses the importance weight resampling technique to resample particles, seestep 206. In various embodiments, in addition toimportance weight re-sampling 206, the system may also utilize a geometric projection technique to condition the resampled particles to the closest edge of the map graph, seestep 207. In the block diagram 200 shown inFIG. 2 , the aforesaid steps 203-207 are repeatedly executed within aparticle filter loop 208. - As illustrated in
FIG. 3 , atblock 207 of block diagram 200 shown inFIG. 2 , thenew particles 301 generated from importance weight resampling are projected on thegraph edge 302 closest to the respective particles. The resulting projection is designated inFIG. 3 with a numeral 303. This allows generation of new particles around the edges of the map graph and at the same time maintaining the property of importance weights resampling. In one or more embodiments, the geometric projection is applied only to the new generated particles thereby maintaining the states of the important particles around which the new particles are generated. - In one or more embodiments, the described localization system uses a grid of interconnected navigation nodes disposed inside each room of a building. The grid is aligned to the navigation node entering the room.
FIG. 4(a) illustrates an exemplary square grid ofnavigation nodes 401 inside three rooms connected by graph edges 402. Specifically, this figure shows square grids with one meter spacing betweennodes 401.Nodes 403 inFIG. 4(a) , interconnected withedges 404, indicate the navigation graph between rooms. In one embodiment, thelatter nodes 403 andedges 404 of the graph shown inFIG. 4(a) are constructed manually. -
FIG. 4(b) illustrates an exemplary triangular grid ofnavigation nodes 401 inside the same three rooms. The triangular grid shown inFIG. 4(b) makes the navigation betweennodes 401 more natural. A similar effect could be achieved by adding diagonals to the grid inFIG. 4(a) , but that would introduce edges of different lengths and require the addition of nodes in the center of squares where two edges would cross. In various embodiments, other different geometric shapes and spacing may be chosen to fill the inside of the rooms. One example of such other shapes is a hexagonally shaped graph. Therefore, the concepts described herein are not limited to any specific graph structure. On the other hand, it should be noted that leaving rooms empty would produce inferior results, because it would introduce a strong bias towards the door in the room. - In one or more embodiments, the aforesaid map graph consisting of navigation nodes arranged in a square, triangular, hexagonal or any other appropriate patters may be generated automatically using a graph generation algorithms, which uses the coordinate information of the single node in the room and the coordinate information of the four corners of the room. In one embodiment, the additional nodes in the room are automatically generated by way of extrapolation to cover the entire room with the map graph starting from the single (initial). This may be performed algorithmically and hence does not require any manual human involvement.
- As would be appreciated by persons of ordinary skill in the art, utilizing the aforesaid graph map information constrains the initialization of particles to the nodes and edges of the graph. In addition, using the geometric projection of the newly resampled particles on to the closest edge in the graph further allows the future state estimates of the particles towards the edges of the graph, hence maintaining the probabilistic posterior update of the particles. Further, being an additional information source to the noisy BLE and IMU data, utilizing the map graph information as a prior, provides better estimation of the particle filter based localization.
-
FIG. 5 illustrates an exemplary embodiment of a computerizedmobile system 500 that could be used as thehandheld client device 101 in connection with the system illustrated inFIG. 1 . In one or more embodiments, the computerizedmobile system 500 may be implemented within a form factor of a mobile computing device, such as a smartphone, a personal digital assistant (PDA), a tablet computer, or a smart watch, all of which are widely available commercially and are well known to persons of skill in the art. - The
computerized system 500 may include adata bus 504 or other interconnect or communication mechanism for communicating information across and among various hardware components of the mobilecomputerized system 500, and a central processing unit (CPU or simply processor) 501 coupled with thedata bus 504 for processing information and performing other computational and control tasks.Computerized system 500 also includes amemory 512, such as a random access memory (RAM) or other dynamic storage device, coupled to thedata bus 504 for storing various information as well as instructions to be executed by theprocessor 501. Thememory 512 may also include persistent storage devices, such as a magnetic disk, optical disk, solid-state flash memory device or other non-volatile solid-state storage devices. - In one or more embodiments, the
memory 512 may also be used for storing temporary variables or other intermediate information during execution of instructions by theprocessor 501. Optionally,computerized system 500 may further include a read only memory (ROM or EPROM) 502 or other static storage device coupled to thedata bus 504 for storing static information and instructions for theprocessor 501, such as firmware necessary for the operation of thecomputerized system 500, basic input-output system (BIOS), as well as various configuration parameters of thecomputerized system 500. - In one or more embodiments, the
computerized system 500 may incorporate adisplay device 509, which may be also coupled to thedata bus 504, for displaying various information to a user of thecomputerized system 500. In an alternative embodiment, thedisplay device 509 may be associated with a graphics controller and/or graphics processor (not shown). Thedisplay device 509 may be implemented as a liquid crystal display (LCD), manufactured, for example, using a thin-film transistor (TFT) technology or an organic light emitting diode (OLED) technology, both of which are well known to persons of ordinary skill in the art. In various embodiments, thedisplay device 509 may be incorporated into the same general enclosure with the remaining components of thecomputerized system 500. In an alternative embodiment, thedisplay device 509 may be positioned outside of such enclosure. - In one or more embodiments, the
computerized system 500 may further incorporate aGPS receiver 503 connected to thedata bus 504 and configured to receive location information from one or more GPS satellites and transmit this information to theprocessor 501 via thedata bus 504. - In one or more embodiments, the
computerized system 500 may incorporate one or more input devices, such as atouchscreen interface 510 for receiving tactile commands and akeyboard 506, which all may be coupled to theaforesaid data bus 504 for communicating information, including, without limitation, user command selections to theprocessor 501. In an alternative embodiment, input devices may include a system for tracking eye movements of the user (not shown), which may be used to indicate to thecomputerized system 500 the command selection by the user. - In one or more embodiments, the
computerized system 500 may additionally include alocation signal receiver 103, such as a Bluetooth receiver, configured to perform scan forbeacons 106 and supply scan data described above to theprocessor 501 via thedata bus 504. In addition, theIMU 102 may be also connected to thedata bus 504. - In one or more embodiments, the
computerized system 500 may additionally include a communication interface, such as anetwork interface 505 coupled to thedata bus 504. Thenetwork interface 505 may be configured to establish a connection between thecomputerized system 500 and theInternet 524 using at least one ofWIFI interface 507 and the cellular network (GSM or CDMA)adaptor 508. Thenetwork interface 505 may be configured to provide a two-way data communication between thecomputerized system 500 and theInternet 524. TheWIFI interface 507 may operate in compliance with 802.11a, 802.11b, 802.11g and/or 802.11n protocols as well as Bluetooth protocol well known to persons of ordinary skill in the art. In an exemplary implementation, theWIFI interface 507 and the cellular network (GSM or CDMA)adaptor 508 send and receive electrical or electromagnetic signals that carry digital data streams representing various types of information. For example, the aforesaid networking components may be used to establish a network data connection between thecomputerized system 500 and other network components. - In one or more embodiments, the
Internet 524 typically provides data communication through one or more sub-networks to other network resources. Thus, thecomputerized system 500 is capable of accessing a variety of network resources located anywhere on theInternet 524, such as web servers, other content servers as well as other network data storage resources. In one or more embodiments, thecomputerized system 500 is configured send and receive messages, media and other data, including application program code, through a variety of network(s) includingInternet 524 by means of thenetwork interface 505. In the Internet example, when thecomputerized system 500 acts as a network client, it may request code or data for an application program executing on thecomputerized system 500. Similarly, it may send various data or computer code to other network resources. - In one or more embodiments, the functionality described herein is implemented by
computerized system 500 in response toprocessor 501 executing one or more sequences of one or more instructions contained in thememory 512. Such instructions may be read into thememory 512 from another computer-readable medium. Execution of the sequences of instructions contained in thememory 512 causes theprocessor 501 to perform the various process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the embodiments of the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software. - The term “computer-readable medium” as used herein refers to any medium that participates in providing instructions to
processor 501 for execution. The computer-readable medium is just one example of a machine-readable medium, which may carry instructions for implementing any of the methods and/or techniques described herein. Such a medium may take many forms, including but not limited to, non-volatile media and volatile media. - Common forms of non-transitory computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punchcards, papertape, any other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EPROM, a flash drive, a memory card, any other memory chip or cartridge, or any other medium from which a computer can read. Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to
processor 501 for execution. For example, the instructions may initially be carried on a magnetic disk from a remote computer. Alternatively, a remote computer can load the instructions into its dynamic memory and send the instructions over theInternet 524. Specifically, the computer instructions may be downloaded into thememory 512 of thecomputerized system 500 from the foresaid remote computer via theInternet 524 using a variety of network data communication protocols well known in the art. - In one or more embodiments, the
memory 512 of thecomputerized system 500 may store any of the following software programs, applications or modules: - 1. Operating system (OS) 513, which may be a mobile operating system for implementing basic system services and managing various hardware components of the
computerized system 500. Exemplary embodiments of theoperating system 513 are well known to persons of skill in the art, and may include any now known or later developed mobile operating systems. - 2.
Network communication module 514 for enabling network communications using one or more network interfaces described below. - 3. A
localization application 515 may incorporate a set of software modules 516-519, executed by theprocessor 501 of thecomputerized system 500, which cause the computerizedmobile system 500 to perform certain predetermined functions. In one or more embodiments, thelocalization application modules 515 may include, for example, aparticle initialization module 516, which initializes particles using the aforesaid map graph information. In addition, thelocalization application modules 515 may incorporate amotion update module 517 and ameasurement update module 518, which performfunctions FIG. 2 . Finally, aresampling module 519 may also be deployed, which performs resampling of particles using the aforesaid importance weights as well as the aforesaid map graph information. - 4.
Data storage 520 may be used, for example, for storing the graph based map information as well as IMU biasdata 521. - Finally, it should be understood that processes and techniques described herein are not inherently related to any particular apparatus and may be implemented by any suitable combination of components. Further, various types of general purpose devices may be used in accordance with the teachings described herein. It may also prove advantageous to construct specialized apparatus to perform the method steps described herein. The present invention has been described in relation to particular examples, which are intended in all respects to be illustrative rather than restrictive. Those skilled in the art will appreciate that many different combinations of hardware, software, and firmware will be suitable for practicing the present invention. For example, the described software may be implemented in a wide variety of programming or scripting languages, such as Assembler, C/C++, Objective-C, perl, shell, PHP, Java, as well as any now known or later developed programming or scripting language.
- Moreover, other implementations of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. Various aspects and/or components of the described embodiments may be used singly or in any combination in systems and methods for utilizing graph based map information as priors for localization using particle filter. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
Claims (22)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/726,371 US11199411B2 (en) | 2017-05-15 | 2017-10-05 | Systems and methods for utilizing graph based map information as priors for localization using particle filter |
JP2018174514A JP7238305B2 (en) | 2017-10-05 | 2018-09-19 | Systems and methods, computer-implemented methods, programs, and systems that utilize graph-based map information as priors for localization using particle filters |
SG10201808599YA SG10201808599YA (en) | 2017-10-05 | 2018-09-28 | Systems and methods for utilizing graph based map information as priors for localization using particle filter |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/595,931 US10812877B2 (en) | 2017-05-15 | 2017-05-15 | System and method for calibration-lessly compensating bias of sensors for localization and tracking |
US15/726,371 US11199411B2 (en) | 2017-05-15 | 2017-10-05 | Systems and methods for utilizing graph based map information as priors for localization using particle filter |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/595,931 Continuation-In-Part US10812877B2 (en) | 2017-05-15 | 2017-05-15 | System and method for calibration-lessly compensating bias of sensors for localization and tracking |
Publications (2)
Publication Number | Publication Date |
---|---|
US20180328738A1 true US20180328738A1 (en) | 2018-11-15 |
US11199411B2 US11199411B2 (en) | 2021-12-14 |
Family
ID=64097718
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/726,371 Active 2038-11-12 US11199411B2 (en) | 2017-05-15 | 2017-10-05 | Systems and methods for utilizing graph based map information as priors for localization using particle filter |
Country Status (1)
Country | Link |
---|---|
US (1) | US11199411B2 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110763236A (en) * | 2019-10-28 | 2020-02-07 | 杭州电子科技大学 | Map matching positioning optimization method based on least square |
CN112082553A (en) * | 2020-07-24 | 2020-12-15 | 广州易来特自动驾驶科技有限公司 | Indoor positioning method and positioning device based on WIFI and laser radar and robot |
CN113008223A (en) * | 2021-03-03 | 2021-06-22 | 哈尔滨工程大学 | Seabed terrain synchronous positioning and mapping method based on particle filtering and mapping optimization |
CN114253511A (en) * | 2020-09-21 | 2022-03-29 | 成都睿芯行科技有限公司 | SLAM hardware accelerator based on laser radar and implementation method thereof |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7603329B2 (en) * | 2003-07-09 | 2009-10-13 | Raptor International, Inc. | Data set modification using training algorithm that grows the data set |
US7672911B2 (en) * | 2004-08-14 | 2010-03-02 | Hrl Laboratories, Llc | Graph-based cognitive swarms for object group recognition in a 3N or greater-dimensional solution space |
US7411167B2 (en) * | 2006-09-05 | 2008-08-12 | Honeywell International Inc. | Tracking a moving object from a camera on a moving platform |
WO2013055959A1 (en) * | 2011-10-11 | 2013-04-18 | Qualcomm Incorporated | System and/or method for pedestrian navigation |
US9361889B2 (en) | 2013-01-29 | 2016-06-07 | Qualcomm Incorporated | Landmark based positioning with verbal input |
US9491585B2 (en) | 2014-05-31 | 2016-11-08 | Apple Inc. | Location determination using dual statistical filters |
GB201500411D0 (en) | 2014-09-15 | 2015-02-25 | Isis Innovation | Determining the position of a mobile device in a geographical area |
US9709404B2 (en) | 2015-04-17 | 2017-07-18 | Regents Of The University Of Minnesota | Iterative Kalman Smoother for robust 3D localization for vision-aided inertial navigation |
RU2696603C1 (en) * | 2015-10-13 | 2019-08-05 | Сименс Акциенгезелльшафт | Method, apparatus and system for determining an internal location |
US11275149B2 (en) | 2016-03-18 | 2022-03-15 | Embarcadero Technologies, Inc. | Determining a location of an electronic device |
-
2017
- 2017-10-05 US US15/726,371 patent/US11199411B2/en active Active
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110763236A (en) * | 2019-10-28 | 2020-02-07 | 杭州电子科技大学 | Map matching positioning optimization method based on least square |
CN112082553A (en) * | 2020-07-24 | 2020-12-15 | 广州易来特自动驾驶科技有限公司 | Indoor positioning method and positioning device based on WIFI and laser radar and robot |
CN114253511A (en) * | 2020-09-21 | 2022-03-29 | 成都睿芯行科技有限公司 | SLAM hardware accelerator based on laser radar and implementation method thereof |
CN113008223A (en) * | 2021-03-03 | 2021-06-22 | 哈尔滨工程大学 | Seabed terrain synchronous positioning and mapping method based on particle filtering and mapping optimization |
Also Published As
Publication number | Publication date |
---|---|
US11199411B2 (en) | 2021-12-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10677883B2 (en) | System and method for automating beacon location map generation using sensor fusion and simultaneous localization and mapping | |
Xia et al. | Indoor localization on smartphones using built-in sensors and map constraints | |
US11199411B2 (en) | Systems and methods for utilizing graph based map information as priors for localization using particle filter | |
US10422849B2 (en) | Method and system for signal-based localization | |
US9568587B2 (en) | Tracking algorithm | |
US10117061B2 (en) | Systems and methods to track movement of a device in an indoor environment | |
US20130345967A1 (en) | Routability graph with predetermined number of weighted edges for estimating a trajectory of a mobile device | |
US10771983B2 (en) | System and method for optimizing placement and number of radio-frequency beacons to achieve better indoor localization | |
EP2878924A1 (en) | Method and system for automatically generating location signatures for positioning using inertial sensors | |
US10871547B1 (en) | Radiofrequency based virtual motion model for localization using particle filter | |
US10812877B2 (en) | System and method for calibration-lessly compensating bias of sensors for localization and tracking | |
Chen et al. | An indoor multi-source fusion positioning approach based on PDR/MM/WiFi | |
CN105891772A (en) | Navigation method and device | |
KR102095135B1 (en) | Method of positioning indoor and apparatuses performing the same | |
Jiang et al. | Robot-assisted smartphone localization for human indoor tracking | |
Štancel et al. | Indoor atlas service as a tool for building an interior navigation system | |
Ramesh et al. | Indoor Navigation using Augmented Reality for Mobile Application | |
Karabtcev et al. | BLE beacon interaction module and mobile application in the indoor-navigation system | |
JP7238305B2 (en) | Systems and methods, computer-implemented methods, programs, and systems that utilize graph-based map information as priors for localization using particle filters | |
Van de Wynckel et al. | Indoor positioning using the OpenHPS framework | |
Graichen et al. | A map framework using crowd-sourced data for indoor positioning and navigation | |
Lo et al. | Using intelligent mobile devices for indoor wireless location tracking, navigation, and mobile augmented reality | |
Monica et al. | An Optimization-Based Algorithm for Indoor Localization of JADE Agents. | |
Liu et al. | MagLoc-AR: Magnetic-based localization for visual-free augmented reality in large-scale indoor environments | |
US20230354258A1 (en) | Data processing method and apparatus |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
AS | Assignment |
Owner name: FUJI XEROX CO., LTD., JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PATEL, MITESHKUMAR;BIEHL, JACOB;GIRGENSOHN, ANDREAS;REEL/FRAME:046325/0831 Effective date: 20171002 |
|
AS | Assignment |
Owner name: FUJI XEROX CO., LTD., JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PATEL, MITESHKUMAR;BIEHL, JACOB;GIRGENSOHN, ANDREAS;REEL/FRAME:047230/0365 Effective date: 20171002 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
AS | Assignment |
Owner name: FUJIFILM BUSINESS INNOVATION CORP., JAPAN Free format text: CHANGE OF NAME;ASSIGNOR:FUJI XEROX CO., LTD.;REEL/FRAME:056392/0541 Effective date: 20210401 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |