IL281791B1 - Method for improving orientation using a hand-held smartphone having an integral GPS device - Google Patents

Description

Method for improving orientation using a hand-held smartphone having an integral GPS device DISCLAIMER It is to be noted that only subject matter embraced in the scope of the claims appended hereto, whether in the manner defined in the claims or in a manner similar thereto and involving the main features as defined in the claims, is intended to be included in the scope of the present invention, while subject matter described and exemplified to provide background and better understanding of the invention, is not intended for inclusion as part of the present invention.

FIELD OF THE INVENTION This invention relates to navigation systems and methods particularly for use by pedestrians.

BACKGROUND OF THE INVENTION With the advent of smart phones, GPS has become widely used by pedestrians and drivers as a principal method of navigation. Of high importance is the use of smartphones GPS by pedestrians in urban area, instead of the traditional paper maps.However, pedestrian use of GPS is inconvenient, difficult and sometimes impossible owing to the low positioning accuracy of mobile phones’ GPS systems, mainly in urban areas. Furthermore, owing to the low accuracy of mobile phone magnetometers, pedestrians do not have the orientation data needed for navigation.The main reason for the low accuracy of GPS systems in urban areas is the obstruction of the lines of sight to satellites due to high buildings. Furthermore, signals from satellites not having direct line of sight are rather reflected by the high buildings, received by the pedestrian’s GPS and wrongly interpreted to result in large errors in positioning readings. This phenomenon is referred to as "multiple path reflection".

Although multiple path reflection affects motor vehicles as well as pedestrians, it is less pronounced in motor vehicles because, since they are constantly on the move at significantly higher speeds than pedestrians, their respective satellite visibilities are constantly and rapidly changing and errors can therefore be smoothed. Also, since the location of cars and other road vehicles is restricted to roads, techniques such as "snap to map" and utilization of IMU data and other information are used. In addition, direction is known to high accuracy by the GPS motion and velocity.This drawback has become a major issue for companies like Uber and Lyft, where successful passenger pickup by a driver highly depends on the location accuracy reported by the passenger’s GPS. Driver and client may miss each other just because the client is across the street or across a junction.Attempts have been made to solve this issue, including 3D modelling of the surrounding buildings to ignore satellites having multiple reflection, phase analysis and statistical methods. However, to the best of our knowledge no practical, cost-effective solution has been proposed.There is a wealth of prior art relating to localization of autonomous vehicles and to optimization of localization strategies used by an autonomous vehicle based on a driving context, for example, the geographical region in which the autonomous vehicle is driving, the time of day, the speed of the autonomous vehicle, and so on. In conventional systems, a map database may be used to snap an initial location calculated from the navigation satellite system data to a physical geographical object, such as, a road, for a final output displayed by the navigation device. By such means disclosed, for example, in US20110257885 the coarse location provided by the GPS satellites may be significantly improved, sufficient for guiding vehicles whether autonomous or driven.WO 2018/222275 discloses vehicle technologies for improving positioning accuracy despite various factors that affect signals from navigation satellites. Such position­ing accuracy is increased via determining an offset and communicating the offset in various ways or via sharing of raw positioning data between a plurality of devices, where at least one knows its location sufficiently accurately, for use in differential algorithms.US20170307761 discloses a method of collaborative determination of position­ing errors of a satellite-based navigation system. Positioning receivers that are not very precise, such as smartphones, present in a geographical zone, of unknown precise position, can contribute to the production of precise atmospheric error corrections if the receivers are sufficiently numerous.US20180124572 relates to correcting GPS-based position information using local correction information in a network of moving things. If two receivers share the same set of visible satellites (i.e., are processing received signals from the same set of satellites in arriving at a positioning solution) the two receivers should experience similar positioning errors. The set of satellites used by a GNSS/GPS receiver in the calculation of geographic positioning information is dynamic and constantly changing due to a number of factors.US20190147610 in the name of Uber Technologies, Inc. discloses systems and methods for detecting and tracking objects based on sensor data received from one or more sensors and which is fed to one or more machine-learned models including one or more first neural networks configured to detect one or more objects based at least in part on the sensor data and one or more second neural networks configured to track the one or more objects over a sequence of sensor data.In summary, the prior art teaches techniques for correcting for satellite errors and recognizes that errors corrections associated with satellites that serves multiple receivers in the same geographic zone will be applicable to all receivers in that zone. The prior art also discloses collating data from vast numbers of vehicles traversing a geographic area and using the data received therefrom for improving a map that allows autonomous vehicles to navigate safely along the same roads. The prior art also addresses the need for driverless taxis to locate stationary passengers and provides techniques for navigation of moving pedestrians, recognizing the fact that conventional correction approaches applied to vehicles are not always applicable to pedestrians. The prior art discusses the use of deep learning and neural networks to improve map data.But there appears to be no suggestion in the art to process GPS positioning errors in bulk to create respective correction functions for multiple known locations as a function of the respective GPS coordinates and the satellite data by applying deep learning/machine learning techniques to multiple time samples.Nor has it been suggested to use error correction data associated with satellites in a geographic area to apply corresponding or derived corrections to a GPS navigation system of a pedestrian carrying on his or her person a mobile device such as a smartphone.

SUMMARY OF THE INVENTION .In accordance with the invention there is provided a method for improving orientation using a hand-held smartphone having an integral GPS device, the method comprising:determining a location of the smartphone using said GPS device;pointing a line of sight unit having an IMU to a distinct object in a real scene observed by the user;identifying an image of the object on a digital map image displayed on a screen of the smartphone;extracting a location of the object from a database of the digital map;calculating azimuth of the object relative to the location of the smartphone to allow a user holding the smartphone to orientate himself in space relative to the object.The method relies on first obtaining an accurate location of the smartphone using the GPS device: the higher the accuracy the more reliable will be the orientation. In some embodiments, location is obtained using a two-part procedure. The first part is a learning method that creates an error correction function for improving the accuracy of a GPS device in a region of interest, while the second part provides a method for improving the accuracy of GPS coordinates received in a region of interest, using the error correction function which has already been determined. Within the context of the description and appended claims, the term "GPS device" is used to refer to any device having a built-in GPS module. The GPS device is most typically a smartphone but may be any other suitable device configured for GPS positioning.In some embodiments, the invention provides a method for improving the accuracy of GPS coordinates received in a region of interest, for which a correction function has been obtained, the method comprising:inputting the GPS coordinates and associated satellite data to the correction function, so as to obtain a location with improved accuracy.The two procedures are mutually independent. Specifically, while the second procedure cannot be implemented until there has been derived an error correction function pertaining to a required region of interest, once the error correction function exists, the second procedure can be implemented in standalone manner. In this case, the two procedures are carried out sequentially. But they can also be carried out concurrently whereby the learning procedure is executed at the same time as implementing the second aspect either in respect of a different region of interest or to refine the error correction function by continually gathering new data and applying deep learning techniques to a continually expanding dataset.In some embodiments the multiple time samples are obtained by collecting time samples from motor vehicles driving in an extended urban area and having on-board GPS devices and enhanced accuracy positioning, each of the time samples consisting of the GPS coordinates and associated satellite data obtained by the on-board GPS devices and the enhanced accuracy location.Conceptually, the principle of the invention resides in repeating the above method for multiple locations over time in a required geographic area, so as to obtain multiple time samples for each location. The GPS coordinates are the coordinates of the GPS device typically obtained by triangulating between three or more satellites and represent coarse locations, the accuracy of which the invention serves to improve. The resulting data are stored in a database and analysed to derive a function for each coarse location that, when applied to GPS coordinates and using the satellite data obtained for the coarse location, corrects the GPS coordinates and yields a more precise location. The degree of precision may be improved by an order of magnitude, such that if the coarse location is correct to within 30 meters, the location as corrected by the invention will be accurate to within three meters or less. It will be understood that in practice it may not be possible to map every coordinate in space. But it is assumed that a sufficient number of points in close mutual proximity are mapped over time, so that the computed error correction function is equally applicable for location of any point in space that deviates slightly from the mapped points.It will be understood that each time sample may consist of different GPS coordinates and different satellite data. This implies that different GPS coordinates having different associated satellite data, may correlate to the same corrected location. The error correction function is derived such that regardless as to which set of GPS coordinates and associated satellite data it is applied, it will give the same corrected location.The GPS coordinates and the associated satellite data and location errors for multiple points in the region of interest are stored in a database for subsequent processing.

In some embodiments, the database is compiled based on data collected by a navigation system such as Waze™. Such systems in any case collect from vehicle GPS devices a coarse location, i.e. the associated GPS coordinates, and convey enhanced location data to the vehicles. The only additional information that is, therefore, required is the satellite data corresponding to each of the satellites from whose signals each coarse location is obtained. This information is, of course, known to the on-board GPS from which it can be collected and used to compile the database and derive correction functions for the present invention. In use, a coarse location received from a GPS device is then fed together with the satellite data and the enhanced location coordinates to the database. This database is used to derive a correction function using deep learning technique.A pedestrian carrying a smartphone having an in-built GPS device can, at best, obtain only a coarse location even if the smartphone uses a navigation system such as Waze™. As noted above, this is because enhanced accuracy is applied by the navigation system to vehicles, which are constrained to travel along defined paths cannot be applied to the pedestrian who are free to roam. Furthermore, as explained above, errors caused by multiple path reflection in urban areas impact more heavily on pedestrian devices, which are stationary or progress more slowly than they do on motor vehicles. However, in accordance with the invention, since the correction function is based on and applied to the coarse location, it may equally be applied to the GPS device in the pedestrian’s smartphone to derive a more accurate location. Likewise, techniques such as RADAR, LIDAR and other enhancements being developed for use by Advanced Driver Assistance Systems (ADAS) may be used to determine enhanced location accuracy. The correction function can be downloaded to the pedestrian’s smartphone from a remote server in communication with the pedestrian’s GPS device; or it may be stored in the pedestrian’s GPS device if there is sufficient memory.The error correction function receives as input a vector consisting of a coarse location corresponding to GPS coordinates and associated satellite data from which the GPS coordinates were obtained and produces as output a true location.

BRIEF DESCRIPTION OF THE DRAWINGS In order to understand the invention and to see how it may be carried out in practice, embodiments will now be described, by way of non-limiting example only, with reference to the accompanying drawings, in which: Fig. 1 is a pictorial representation of a system according to the invention; Fig. 2 shows schematically multiple time samples collected and stored randomly over time by vehicles for regions of interest that may be used to generate a database according to an embodiment of the invention; and Fig. 3 is a flow diagram showing the principal operations executed by a method according to an embodiment of the invention for improving passenger/driver encounter.

DETAILED DESCRIPTION OF EMBODIMENTS Fig. 1 is a pictorial representation of a system 10 according to one embodiment of the invention. The system 10 shows a pedestrian 11 located in a region of interest holding a smartphone 12 having a built-in GPS module that receives GPS data from at least three satellites 13, 13׳ and 13" and computes a coarse location in known manner using triangulation. Throughout this description and the appended claims, we will refer to the coarse location as the GPS coordinates of the GPS device. Vehicles 14, 14׳ that randomly drive through the region of interest likewise receive GPS data from satellites and determine respective coarse locations of the vehicles. However, the vehicles are equipped with navigation system such as WAZE™, SATNAV™ and the like which enhance accuracies in the GPS coordinates so that the correct or true locations of the vehicles are known. For the sake of clarity, by "true" or "correct" it is not intended to imply that the corrected locations are precise in absolute terms, but rather that they are significantly more accurate than the coarse locations based only on the coordinates. Anyone who has used a navigation system such as WAZE™ is well aware that they are instructed in advance to turn left at the next junction and when they are on top of the junction are instructed "turn left". It is this level of accuracy this renders navigation systems so reliable and user-friendly.The navigation systems in the vehicles enhances the accuracy for the coarse locations based solely on the satellite data, using auxiliary data, based for example, on accurate maps that have been pre-compiled and which allow the location of the vehicle to be corrected using known techniques such as snap to map.Likewise, techniques such as RADAR, LIDAR and other enhancements being developed for use by Advanced Driver Assistance Systems (ADAS) may be used to determine enhanced location accuracy.Although only two vehicles are shown in the figure, it is to be understood that in practice there are thousands of vehicles driving over time along charted routes such as highways, roads, streets and even off-road paths whose locations have been accurately mapped and are accessible to the vehicle navigation systems either because map data is pre-loaded or because they are able to access the map data on-line, typically over the Internet 15. The satellite data received by the vehicle navigation systems includes satellite azimuth and elevation of each associated satellite and Signal-to-Noise Ratio of a received signal from the associated satellite.The invention is based on collection and storage in a database of time samples consisting of coarse location, satellite data and true location, and deriving error correction functions using deep learning collated over time from motor traffic moving along charted routes in a region of interest wherein the GPS device is located. To this end, the invention comprises two distinct stages, which are now described.

Data gathering and learning: There are now described the principal operations carried out by the first procedure. GPS errors in specific known locations under various satellite and environ­mental conditions are collected over time, covering an extensive set of positioning errors and satellites data.Although the data can theoretically be collected by mapping locations manually in much the same way that ordnance survey maps were originally compiled, in order to obtain sufficient data quickly and automatically over an extended area, data may be collected from motor vehicles traveling within the desired area over a period of time. Fig. 2 shows schematically multiple time samples obtained randomly by motor vehicles passing known locations in a region of interest, whereby over an extended period of time, multiple time samples are collected for each known location. Thus, by way of simple illustration, there are shown three locations x1, x2 and x3 through which vehicles pass at random and convey the coarse GPS locations, the satellite data and the enhanced locations to the database server. It is seen that over a given time period, six vehicles designated v11… v61 pass through location x1; seven vehicles designated v12… v72 pass through location x2 and five vehicles designated v13… v53 pass through location x3. Although not important, it may be noted from Fig. 2 that the actual times of each sample may well vary for each known location. GPS navigation systems in the vehicles constantly receive the coarse GPS locations and satellite data and derive the enhanced locations. Likewise, although the vehicles have discrete designations, this is for the sake of clarity. In practice, any given vehicle passes through multiple locations and therefore not all the vehicles arriving at the three discrete locations x1, x2 and x3 are different. Indeed, it may also occur that a vehicle passes through the same known point more than once e.g. on an outward journey and on a return journey, or as a result of a repeat journey by the same vehicle in a subsequent time period. The collected information includes the coarse GPS location, the satellite status data and the true location. This allows the positioning error to be determined. Since most motor vehicles today have GPS navigation systems and motor vehicles cover most urban areas, data from a large region of interest can be collected over time. As a result, respective data corresponding to a vast number of locations in large urban areas are collected, including coarse and corrected positions, positioning errors, and associated satellite data. These data serve as input for deriving an error correction function using deep learning techniques. The error correction function is configured to receive as input a vector consisting of a coarse location corresponding to GPS coordinates and associated satellite data and to produce as output a true location. As more vehicles traverse known points within the region of interest, their data may be collected and stored thus allowing the error correction function to be further refined over time.

Estimating GPS deviation using Deep Neural Network (DNN)We assume that at a given point with a known location (x, y, z), we get an estimated location (x׳, y׳, z׳) produced by a GPS system. The difference (x-x׳ , y-y׳ , z-z׳) between the true location and the GPS-produced location is called the GPS-deviation. The task we consider is estimating the GPS-deviation in order to correct the location produced by the GPS system, and obtain the correct location as accurately as possible.The task can be approached using standard methods of training deep neural networks (DNNs). An example of training a DNN for the task will proceed as follows based on He, K, Zhang, X, Ren, S, Sun, J. 2015 "Deep Residual Learning for Image Recognition" arXiv:1512.03385 (He et al.) The input to the network is a vector of parameters produced by the GPS system (GPS data), including satellite positions, signal strengths, times of measurements and other possible parameters. For each given location, such data is collected at multiple times. The DNN output is the estimated deviation, for instance expressed as the probability distribution over a set of deviations. Training is done using a loss function, such as the cross entropy between the known deviation and the estimated deviation produced by the DNN. A possible state-of-the-art architecture for the task can be a ResNet DNN of sufficient depth as described by He et al, using standard training procedures, e.g. in terms of splitting the data into training and testing sets, searching for good training rates, using batch normalization, and the like. To get a sufficient amount of data for training the network, the training procedure may use data from multiple different locations at multiple times. It is important to note that though the method described by He et al. was applied to images, it is a general architecture that is equally applicable to the present invention.

Correction: The correction model is used to correct the GPS errors at any point, as a function of satellite data (as defined above). Thus, in use, a navigation system or an application in a GPS device such as a smartphone within a region of interest receives the coarse GPS location and associated satellite data, and obtains and applies the correction function to the coarse GPS coordinates in order to derive the corrected location.

Claims (2)

- 15 - CLAIMS:

1. Method for improving orientation using a hand-held smartphone having an integral GPS device, the method comprising: determining a location of the smartphone using said GPS device; pointing a line of sight unit having an IMU to a distinct object in a real scene observed by the user; identifying an image of the object on a digital map database image displayed on a screen of the smartphone; extracting a location of the object from a database of the digital map database; calculating azimuth of the object relative to the location of the smartphone to allow a user holding the smartphone to orientate himself in space relative to the object.

2. The method according to claim 1, wherein the digital map database is Google’s Street View™.