GB2510548A

GB2510548A - Personal navigation system

Info

Publication number: GB2510548A
Application number: GB201207030A
Authority: GB
Inventors: Ian Pratt; Shiyun Zhong
Original assignee: ZIGZAGEE Ltd
Current assignee: ZIGZAGEE Ltd
Priority date: 2012-02-08
Filing date: 2012-04-23
Publication date: 2014-08-13
Anticipated expiration: 2032-04-23
Also published as: GB201207030D0; GB201202167D0; GB2510548B

Abstract

A personal unit determines its distance from at least three nodes of a mesh network using the received signal strength of electromagnetic (radio or infra-red) or acoustic radiation, and hence its position. Alternatively or additionally, TDOA may be used. If insufficient nodes are available, the measurements are supplemented / replaced by sensor fusion and dead reckoning. This is used to determine position and thus to guide the user to a desired location in a building such as a library or shopping mall. The nodes may be incorporated in light switches, power sockets, lighting sockets, an annunciator or a security alarm panel. The system may control access through doors or barriers. A camera may record images which are geotagged (associated with position and orientation) and played back, or computer generated images may be used. The display may be ePaper. Communication with a site supervisor is possible. An alternative version directs a vehicle driver around a private enclosed site.

Description

Title: A navigation device employing meshed networking technologies with associated camera device

Background of the invention

RTLS (real time location systems) is a descriptor given to system that are able to locate objects, whether these be people, physical assets or intangible assets such as software. Being able to locate an object at any instance in time is a powerful tool which aids productivity and efficiency. Wireless technologies such as REID (radio frequency identification), WiMAX and other de facto technologies operating in the radio spectrum may be used to implement systems referred to as localization systems. Localization is the ability to locate an object in 3-dimensional space through the use of a propagating signal. This signal may be electromagnetic or sonic for example.

Field of the invention

The field of the invention is navigation and guidance applied to indoor applications. At the time of invention, outdoor navigation systems are commonplace. Global positioning systems (GPS) together with geographical information systems (GIS) provides a method for assisting navigation of road user and pedestrians. Commercial application of this technology has reached maturity Indoor navigation systems have not seen wide scale commercial exploitation. Outdoor systems generally rely on a trilateration or multilateration systems based upon the global positioning system (GPS) or equivalent system such as the Russian GLONASS or Europcan Union (Galileo) projects for example. Such systems operate on a line of sight basis, whcrc an object needs to be located through reception of synchroniscd, time-stamped signals.

Description of the related art

Localization systems for outdoor usage have reached commercial maturity. There are numerous patcnts published with claims made against various facets of trilateration and multilateration. These are two common methods for implementing localization systems. Other systems exploit the natural decay of a propagating signal in order to infer distance travelled through a medium through the measurement of the received signal strength and inferring the distance travelled.

ePaper is another technology that has received various patent claims. These relate to the use of ePaper in objects such as apparel which possesses dynamic features and replacements for Post-It notes in order to make them dynamic in nature. ePaper is now widely used in devices for reading documents, books, newspapers, etc.

Description

Navigation systems such as GPS (global position system), GLONASS, etc. are now ubiquitous.

They are based upon a series of satellites that are in orbit around the earth. They possess synchronised atomic clocks and emit time-stamped radio waves. Based upon the time of arrival of these radio waves at a receiver, the receivers position can be determined based upon the known speed of radio waves. Such a system does not work well indoors given the need for a line of sight to the orbiting satellites.

Thc invention described herein is a navigation device that can be used in indoor or closed environments, such as factory complexes, conference centres, hotels or University campuses for

example.

Thc invention consists of scvcral components. A plurality of fixcd nodcs arc located within thc building or location within which the navigation device will operate. The fixed nodes are the reference nodes for the operation of the overall system as shown in Illustration 3 [008]. The location of these nodes are known a priori and the mounting is implemented by fixing a device on a wall, ceiling, ceiling void or other location. Retrofit fixed nodes are available in light switches, electric sockcts as shown in Illustration 9 [059], [060]. Other control dcviccs such as wireless controlled valvcs and meters can also support thc nctwork of fixed nodcs. This mesh network supports both message passing as well as localization functions. Messages may be passed to support both data and control frmnctionality. Data messages could be text messages for example, whereas a control function would be changes to the device configuration for example. The localization function employs the fixed mesh network in order to ascertain the position of a mobile navigation device which constitutes a wandering node within the whole meshed networked system. A plurality of navigation devices orwandering nodes may be deployed within the overall system.

The navigation mechanism is achieved through a plurality of methods as shown in Illustration 3.

Localization of the mobile navigation device may be achieved by measuring the RSSI (received signal strength indication) of various transmitters as sensed by a single receiver. The transmitters arc rcfcrrcd to in this documcnt as thc fixcd nodc nctwork, whcrcas thc rcccivcr is thc wandcring, mobilc nodc. Thc clcctromagnctic signals uscd for communication bctwccn nodcs also providcs a mechanism to infer distance between nodes due to the inherent signal strength reduction due to radiation and absorption. As a consequence the RSSI infers distance travelled, which is used in a trilateration system to determine the location of the mobile node. The navigation mechanism is augmented through the use of sonic transmission and sensing. Sound waves propagate slower than clcctromagnctic wavcs, typically 300 ms-i in contrast to 300,000,000 ms-i. This cffcctivcly pcrmits nodc synchronisation via thc clcctromagnctic transmission. The crror in synchronisation bctwccn different nodes within the network is negligible when considering sound as the propagating medium since an error of 1 millisecond in the node clock will give rise to 300 ms-i 0.001 = 0.3m. This error level is acceptable in terms of localization accuracy. The 1 millisecond clock synchronisation error is readily achieved through message passing using the electromagnetic spectrum, which is used as the basis for message passing between nodes. This usc of the sonic spectrum forms the basis for a time of flight, trilatcration approach that infers distance travelled and as a consequence the location in 3D space. In another manifestation the sonic navigation system implements a multilateration approach whereby neighbouring fixed nodes are requested to emit a single tone. These tones are heard at the receiving node, where they are combined. The phase difference between the incoming signals is detected and used to infer a TDOA (time difference of arrival). This TDOA is used in conjunction with a hypcrbolic positioning algorithm to dctcrminc thc hypcrbolic shapc on which thc mobilc nodc is located. Using a similar approach with othcr ncighbouring nodcs rcsults in a scries of hyperboloids in 3-D space. The intersection of these shapes is used to determine the location of the mobile node. Use of multiple tones from the fixed node network is a technique which enables the receiver to eliminate one half of the hyperboloid during the computation of its location.

In order to assist the navigation process, the electromagnetic spectrum may also be employed.

Given the onerous timing requirements that would be associated with a trilateration-based solution that employs thc electromagnetic spectrum, prcferenee is given to a multilateration-based system.

This involves fixed nodes emitting frequencies such as 900 MHz, with a wavelength in free space of approximately 30cm. As the mobile node traverses 3D space it receives frequencies from the fixed node network. By receiving 900 MHz emitted from two nodes, the receiver is able to add received signals together and to note the phase difference between the incoming signals. This phase difference infers a TDOA (time difference of arrival). Phase detector circuity is included within the fixed node architecture. This phase detection aids localization since the observation of maxima and minima occurs over a well-defined interference map.

The overall localization process can draw upon any of the aforementioned approaches to assist in providing accurate location data for the mobile node. The aforementioned approaches are termed "closed" approaches since the mobile node relies upon sensing from a series of external devices. In addition to these "closed" approaches, the mobile node possesses its own inertial system that is used to assist the navigation and localization process in an "open" approach. The mobile node possesses accelerometers so that it can infer small-scale movements and orientation. In addition a magnetic bearing sensor is included so that the mobile node knows its orientation with respect to the Earth's magnetic field. The different mechanisms used to assist the navigation process are shown in Illustration 3.

A sensor fusion approach is used to assimilate the "open" and "closed" navigation methods with the ultimate goal of arriving at an accurate 3D location for the mobile node when it is moving around the fixed node network.

Upon entry to the network, a mobile node attempts to establish its location. The processes it employs are shown in the flowchart of Illustration 5. The mobile node first attempts to identify its neighbours [020]. If there are neighbours the RSSI (received signal strength indication) is used to derive the location in 3D space using trilateration [023]. If the location is deemed to be inaccurate a further process takes place to improve accuracy. This process consists of calling upon either ultrasonic or electromagnetic mechanisms to assist improvement of the accuracy [0271,[028],[029].

With the electromagnetic mechanism a multilateration approach is attempted. If the accuracy is deemed sufficient additional localization methods are not attempted. If the accuracy is still deemed poor an ultrasonic tweet is performed. Time of flight, trilateration and multilateration methods are employed on the emitted sonic tone. This will result in an accurate localization of the mobile node should the tone be received by the fixed node network. If the signal is received by too few nodes in the fixed node network, the amplitude of the emitted sonic signal is increased. If none of the assisting methods helps the localization of the mobile node, dead reckoning [0241 is employed using the on board inertial systems to deduce movements in position.

Audio communication between the mobile node and a master node as well as between nodes is facilitated through the radio wave baseband signal. These audio messages, together with text-based messages may be routed across the network.

The mobile node gives the user an audible and visual prompt to aid routing from the users existing position to a target position. The master node and its associated processing engine possess a map of the entire network over which the mobile node will traverse. This mapping data is shared with the mobile node. The mobile node is quite capable of autonomous routing but may also call upon the central processing engine to assist in routing.

The user is assisted to their target location through visual prompting as shown in Illustration 4 [011],[01 2],[01 3],[0 14],[01 51,[01 6],[01 7][0 18]. The visual guide to the user is implemented as either ePaper or liquid crystal display with back light. If the user wishes to travel, for example from the third floor to a first floor gym within a hotel complex or from a reading room to the "science fiction" section within a library, or to route from their current location to a food stall within a shopping mall the user walks up to a "near field communication" stand and presents their mobile node. Through contactless communication using "near field communication-NFC" the mobile node is programmed with the target destination. Illustration 2 shows an example of usage within a shopping mall. The user brings the navigation device up to a stand [007]. The user can select the desired destination, for example a specific shoe shop. The user then presents the navigation device to a landing spot on the stand. Contactless communication ensues through the NEC portal. This programs the navigation device with the target destination. The user hears audible and visual feedback that the destination has been accepted by the mobile node. The user is then prompted to walk or move in certain directions with visual and audible guidance instructions provided by the mobile node. Illustration 4 provides examples of the visual guidance that the user will see. The instructions could be to enter a lift or take the next left or right turning. The visual prompt is supplemented with an audible message to the same effect; it is possible to mute the audible output.

Illustration 5 shows the processes within the mobile navigation device or mobile node. A mobile node continually searches for neighbouring nodes [020]. Neighbouring nodes will generally be fixed nodes that are contained within the infrastructure of a building or privately contained enclosure. Neighbouring nodes will generally wander in and out as the mobile node traverses the building and as a result move in and out of range when using the electromagnetic spectrum. If no neighbouring nodes are available, the mobile node is forced to use sensor fusion in a "dead reckoning" [024] mode to calculate its current position; dead reckoning utilises accelerometer data and access to a real-time clock data. Once neighbouring nodes are available the sensor fusion process can update itself. The presence of neighbouring nodes affords the opportunity to perform localization based upon the electromagnetic spectrum. This localization process can acquire the RSSI (received signal strength indication) information which can be used to localize the mobile node [023]. Claim is also made to the use of trilateration and multilateration techniques employing the electromagnetic spectrum to assist the localization process. The difficulty when using the electromagnetic spectrum is the very high speed of propagation within a relatively small operating space. As such the preferred method is to use ultrasonic techniques to assist the localization process.

Illustration S shows the assistance being drawn from multilateration-based techniques, trilateration-based techniques and ultrasonic tweets (received amplitude strength) techniques to provide a more accurate estimate of current position [027],[028,[029]. As shown in the flowchart of IllustrationS, the entire process flow is needed to calculate and store the current position of the mobile node [030]. This position knowledge is of course obtained relative to the fixed node network.

Illustration 6 shows the data flow between various components within the overall invention. The mobile nodes within the mesh network possess memory to store both configuration data (eg.

silent mode, etc.) as well as a routing map [039], [040]. The routing map is used to aid the user towards their destination. Route determination is performed locally although assistance may be requested from the main control node or coordinator. The mesh network possesses a coordinator node along with a number of routing nodes and a number of mobile ("end" nodes). Illustration 6 shows the processes and data flows incorporated within a mobile node. One of the processes within the mobile node makes use of near field communications [033]. This provides a number of functions. Firstly it permits initialisation of a mobile node and one mechanism to upload routing maps to the mobile node. Secondly it permits programming of configuration data (eg. The mobile node can be used a eontaetless key) as well as a destination into the mobile node. The mobile node is thereafter capable of providing navigation through an indoor or privately enclosed environment to a destination.

The mobile node also possesses a number of mechanisms for message passing across the mesh network [036]. As shown in Illustration 6, the messaging system is capable of displaying text-based messages on the display of the device [0371. It is also capable of transporting text-based messages across the mesh network either between mobile nodes (if permitted by the configuration) or between the mobile mode and the central coordinator node. The vehicle used to transport messages is the radio component of the mesh network.

The navigation device has a speaker and microphone to enable low bandwidth communication with a central hub. This enables the user to listen and reply to messages received from the central hub. It also enables message passing between two wandering nodes if this option has been made available to the user. During the issue stage privilege settings are stored on the navigation device in order to assign communication privileges between various nodes (between those assigned to a group of friends within a hotel for example) Each mobile node has the option to interact with neighbouring nodes through the ultrasonic interface. This communication portal is used to pass information to the localization process. A mobile node can thereby use the ultrasonic signals to achieve localization based upon a number of scenarios: time of arrival, trilateration, mulilateration, etc. Each mobile node possesses local processing power that is capable of performing the localization analysis and mathematical computations associated with the localization process. The localization process can also call upon the mesh network itself as well as the electromagnetic spectrum to assist the localization process. This data communication channel is shown in Illustration 6. The mobile node can also request processing assistance from the coordinator node. The coordinator is equipped with intensive co-processing capability that may be used to assist each mobile node. The mechanism for assisting mobile nodes is also employed during the routing process shown in Illustration 6 [042]. Based upon the output of the localization process, the routing process is capable of ascertaining its current position relative to the series of fixed nodes distributed across the mesh network. The mobile node has already been pre-loaded with a routing map for the area over which the mobile node is expected to fraverse. The mobile node possesses local processing capability and is capable of deriving a route from the current position through to the destination. Where assistance is required with the routing process, the mobile node can request assistance from the central coordinator, which possesses extensive processing capability to assist routing. The routing map can possess features such as "no go areas", "preferred routing areas", "fire exit only", "key required", "muster point", "fixed reference node added", etc. These features may be updated live, with the coordinator sending out update requests to all mobile nodes. Generally the routing map will remain static orat least have minimal change over the course of its life.

Given that the mobile node knows its current location relative to a series of fixed nodes due to the localization process and the positions of the fixed nodes within the routing map arc also known, the mobile node is capable of knowing where it currently is within the routing map. In order to provide navigation guidance to the user, the orientation of the node needs to be ascertained. This orientation process is calculated within the "sensor fusion" process shown in Illustration 6 [0431. The Cartesian coordinates x,y,z have already been obtained through the localization process. In order to obtain orientation information 9, p, i4i the mobile node includes on board inclinometers and magnetometers. These components form an eCompass that provides the node with information in relation to how the user is currently holding the device. In order to ensure accuracy both in terms of localization and the associated acquisition of orientation information, the mobile node is equipped with a number of accelerometers. These accelerometers are inputs that are used as part of a sensor fusion system. The sensor fusion system amalgamates a number of inputs together with information relating to the dynamics of a person moving to provide an accurate and reliable estimate of current position and orientation. The accelerometers provide valuable local information that can be used to perform "dead reckoning" over short periods. This data is fused with the position data obtained from the localization process to provide an accurate estimate of current position. Likewise, when the eCompass is disturbed by magnetic materials within a building it's possible to correct this aberration through thc sensor fusion proccss shown in Illustration 6. Scnsor fusion is an attractivc technique since the dynamics of a person moving within a building are reasonably well bounded; a human generally walks at around 1 ms-i and the expected motion of the mobile node is captured within the sensor fusion process. Unorthodox events such as the user dropping the device, resulting in high accelerometer measurements can be corrected, as can events such as the user suddenly running or jumping on a motoriscd buggy; the sensor fusion system is capable of learning such behaviour.

The coordinator node is central to the whole mesh network. It is the portal whereby each of the mobile nodes can rcceivc generic updates. It is the portal whereby the user is generally issued with the mobile navigation device. During the issuing phase the mobile navigation device is programmed with the device configuration (eg. mobile node capable of messaging another node, silent mode, etc.) If the mobile node is a virgin device it can also be programmed with the routing map for area over which it is intended to operate.

The mesh network possesses a number of fixed nodes. These provide routing functions across the mesh network. They provide a framework whereby the mobile nodes or mobile navigation devices can communicate and interact with the mesh network. In this invention they arc contained within existing electrical fixtures within a building as shown in Illustration 9. In the example shown in Illustration 9, the mobile node is within range of 2 fixed nodes, in this instance a node contained within an electrical outlet socket [059] and and one contained within an electrical light switch fixture [060]. The mobile node is continually searching for local neighbours and once found, a localization process may be initiated. Although only 2 fixed nodes are currently within range of the mobile node, which would be insufficient to achieve a complete localization through received signal strength, trilateration or multilateration navigation is still possible due to the sensor fusion aspect of the design; the onboard accelerometers, magnetometer and routing mapping information are sufficient to progress the navigation until a larger number or alternative fixed nodes become available. It should be noted that even though only 2 fixed nodes are available it is still possible to locate the node accurately since there are rules that are contained within the sensor fusion model; a user can not suddenly jump between floors within a building for example. The historically accurate data together with the current, sensed data and a model of the system dynamics can be fused together to form a system that provides accurate navigation even during periods when the number of neighbouring fixed nodes is low.

It is clearly advantageous to have a large population of neighbouring nodes since the localization is greatly enhanced. However, it is quite acceptable to navigate through regions that possess a low number of neighbouring fixed nodes. The mobile navigation device is still able to present navigation information to the user since the eCompass and inclination measurements occur locally The device therefore knows its own orientation and can adjust the directions shown in Illustration 4 to ensure that the user is provide with accurate navigation data irrespective of the direction they are currently facing.

A number of forms of the navigation device are produced. These are a lapel-based device (Illustration 1), a wrist-mounted device (Illustration 7) and vehicle-mounted device (Illustration 8).

The device will only provide guidance when requested. In the form shown in Illustration 8 the device may be employed in a privately enclosed environment such as a building site or University campus for example. In the case of a building site, the device can provide audible and visual warning when approaching hazards. Height restrictions are common on building sites and drivers need to be aware of these dangers. The navigation device can give warning of this impending hazard based upon its knowledge of its current position within the route map.

Thc kcy fcaturcs of a form of thc mobilc dcvicc arc shown in Illustration 1. Thcsc arc audiblc uscr intcraction through spcakcrs & microphoncs [005], Light emitting diodcs indicating status [001], push-buttons for the user to initiate actions [003], liquid crystal displays, touchscreens and ePaper to facilitate visual interaction with the user [004]. A rotary selector is also a feature to facilitate user input [006].

An associatcd dcvicc that is part of this invcntion is a camera system as shown in Illustration 10.

This camera system also harnesses thc mesh network through thc tixed nodc nctwork. It uses this mesh network to perform localization in the same way that the mobile navigation does. The camera system also makes use of the multilateration, trilateration and positioning employing electromagnetic and ultrasonic mechanism to perform localization. The camcra systcm also posscsscs magnetometers and inclinometers in thc same way thc mobile navigation dcvicc [067].

The camera device is used as a recording mechanism to record a series of pictures as the operator traverses a route through the area to be mapped. Each of these pictures is geotagged, not only with x,y,z Cartesian position data but also the orientation 0, cp, tp, of the device when the picture was taken. This data can be used in playback mode on the mobile navigation node as it wanders through the mapped area. The overall camera system also employs the same Icalman filter-based sensor fusion system [068] that the mobile navigation device uses as shown in Illustration 11.

Thc overall systcm posscsscs learning fcaturcs as shown in Illustration 12. During thc acquisition phase, the camera acquires data as a user "programs" the system with data. The expected signal levels are noted and stored within a database. This database is subject to modification through a learning process as further users traverse the network. There will be a natural distribution of received signal levels. This statistical data is noted and uscd as part of a learning systcm [070] to refine the accuracy of the overall navigation system.