WO2021009752A1

WO2021009752A1 - Method and system for use in hybrid indoor navigation

Info

Publication number: WO2021009752A1
Application number: PCT/IL2020/050783
Authority: WO
Inventors: Yaron SHAVIT
Original assignee: Intraposition Ltd.
Priority date: 2019-07-14
Filing date: 2020-07-13
Publication date: 2021-01-21
Also published as: IL268045A

Abstract

A method and system for use in indoor navigation is disclosed. The method comprising: providing location data indicative of location of a selected tag unit and sensor data indicative of at least one of movement and orientation of a sensor unit, the sensor unit comprises at least one of inertial, magnetic and RF location sensors; using a computer processor and processing the location data and the sensor data for determining correlation between the location data and sensor data and for determining one or more user actions. The user actions and modes of movement comprise one or more of: common movement of said tag unit and said sensor unit, separate movement of said tag unit and said sensor unit, detachment event of said sensor unit from said tag unit, and reattachment event of said sensor unit and said tag unit to common movement.

Description

METHOD AND SYSTEM FOR USE IN HYBRID INDOOR NAVIGATION

TECHNOUOGICAU FIEUD

The present invention is in the field of indoor navigation techniques and is specifically useful for hybrid navigation using mobile electronic device sensors and external location data.

BACKGROUND

Determining location of a portable device allows navigation and providing location-based services to various users. Various location/positioning systems, such as global positioning systems (GPS) are available for use in different environments enabling to provide location-based data to users via their portable electronic device.

Indoor positioning systems provide for determining location of selected electronic units (e.g. tag units) within a selected region. Various indoor positioning systems utilize simple transmitting tag units and a server system that is configured for determining location of the tag units. Such a tag unit may, for example, be mounted on shopping carts or baskets for providing users with location-based information within a store. This enables an indoor positioning system to determine location of the tag units and thus of the various shopping carts that are in use by shoppers, or of those not in use. .

Another positioning technique uses inertial navigation sensors. This technique utilizes one or more accelerometers and/or magnetic sensors for determining changes in location of a device. Determining accumulated acceleration (e.g. by integration) enables to determine location of the device based on such inertial sensors. However, this technique may suffer from inaccuracies, limited reliability, and drift. GENERAL DESCRIPTION

There is a need in the art for a technique enabling hybrid positioning and navigation within a selected space. The present technique provides a method and a corresponding system configuration enabling a hybrid indoor positioning system using an external positioning system combined with one or more sensors associated with an electronic device. The system of the present technique may provide high positioning accuracy utilizing an external dedicated positioning system, but also the flexibility and relevancy of a hand held electronic device (e.g. mobile computer or mobile phone) which may generally be carried by a typical user. According to the present technique, a positioning system adapted for determining location of a plurality of tag units provides data on location of users or operators carrying the tag units within a selected region. For example, the tag units may be mounted on selected mobile elements such as shopping carts or baskets, karting and go-karts, toy vehicles etc., and used for monitoring location of the mobile elements within the selected region. Additionally, the present technique utilizes one or more electronic devices that may be carried by the users/operators, allowing interface with the user/operator for providing location-based information (e.g. via a user’s mobile computer or mobile phone).

The present technique utilizes correlation between location data of a selected tag unit (paired with the electronic device) and sensor data of the electronic device to determine one or more modes associated with relative motion of the user and the corresponding tag unit and user location. Thus, the present technique enables to determine the location of a user with high accuracy, based on location of the respective tag unit. Additionally, the present technique may allow monitoring user location when not in the vicinity of a respective tag unit. To this end the technique utilizes sensor data provided by the respective electronic device (carried by the user). In this connection, the electronic device as used herein, may generally include one or more sensors, or location identifying techniques, suitable to provide sensor data. Such sensors may include one or more accelerometers, magnetic field detectors, as well as location finding modules operable based on Wi-Fi or Bluetooth protocols. In this connection, the term sensor data as used herein relates to data on movement of the electronic device, or variation of movement thereof, as detected by one or more sensor or location sensing modules of the electronic device. Accordingly, the present technique utilizes processing of location data provided by an indoor positioning system and sensor data indicative of movement and/or orientation of an electronic device carried by the user for determining user location as well as for determining relative location and behavior of the user with respect to an associated tag unit. The processing allows to determine different movements modes of the user including common movement of the user with the tag unit (i.e. user is moving with the shopping cart, basket, vehicle etc.) as well as identifying a detachment event where the user leaves the tag unit at a certain location and moves around without it, and a reattachment event when the user goes back to move with the tag unit. Generally, the present technique also enables monitoring of a user’s movement without the tag unit to provide location data to the user when moving without the tag unit and to identify the reattachment event.

In some configurations, the tag units used in the positioning system may include one or more accelerometers or other location sensing devices, in addition to being operable within the positioning system. Such location sensors may also be used by the positioning system for determining location, movement, and general status of the tag unit. The term location data as used herein may include data on location of the tag unit and may also include data on sensor output from the tag unit, e.g. indicative of status of the tag unit. This may be used for detecting motion of the tag unit as well as for determining if a cart has tilted on its side, or rolled over, in some rare cases.

It should be noted that the term tag unit as used herein generally refers to an element, typically electronic element, configured to be used for determining location in the positioning system. More specifically, the tag unit may include a transmitting module (e.g. IR or RF transmitting module) and is configured to periodically transmit location signals allowing the respective positioning system (server) to determine location of the tag unit. Generally, as referred to herein, a tag unit allows the respective positioning system to identify its position within the selected space and may typically be assigned to a user’s electronic device using an assignment code (e.g. barcode or QR code) paired with the respective electronic device. More specifically, a user may operate an electronic device for pairing with a selected tag unit by scanning a code printed on, or in the vicinity of, the tag unit.

For example, the present technique may be used in grocery stores, where different shopping carts or baskets include tag units with a unique ID code on each tag. Users may use a handheld electronic device (e.g. smartphone or mobile computer) including selected software installed thereon for communication with the store positioning system to receive data on location of products with respect to the user’s location. The user may scan or enter an assignment code marked on the shopping cart for identifying and signing into the store network, and for assigning the tag unit mounted on the shopping cart to the user. Upon proper tag assignment and operation of the corresponding software (app) on the device, the user may receive input data indicative of his location in the store, location of selected items, and suggestion of path from current location to the location of selected items. Further, according to the present technique, the handheld electronic device may typically include a sensor unit comprising one or more sensors (e.g. accelerometers and/or magnetic sensors) providing data indicative of movement and/or direction of orientation of the electronic device. Readout data of the sensor unit may be indicative of the user’s orientation and field of view, i.e. if the user is facing the right shelf or the left shelf, etc. The readout data of the sensor unit associated with the handheld device further enables monitoring and directing user location when the tag unit (e.g. the shopping cart) is not moving with the user, such as when the shopping cart is left between aisles. To this end, the present technique utilizes input data in the form of position data of a selected tag unit and sensor data associated with output of the sensor unit to process the input data and determine one or more modes of movement of the user and respective operation mode. More specifically, the processing may involve determining correlations between the location data and the sensor data and using the determined correlation to select an operation mode based on the respective movement. The operation modes may comprise modes selected from: user moving with the cart, user leaving the cart (detachment event), and user returning to the cart (reattachment event). As indicated above, the location data may be formed by data on the location of the tag unit as determined by the positioning system. The location data may also include data indicative of movement of the tag unit, e.g. provided by one or more accelerometers of the tag unit. Accordingly, the term location data, as described herein, relates to location of the tag unit as determined by the indoor positioning system, and may also include accelerometer data indicative of movement of the tag unit.

The present technique may also determine the user location based on sensor data (e.g. using inertial navigation techniques based on the sensor data), e.g. when moving without the respective tag unit. Further, the technique may utilize the correlation between location data and sensor data for improving accuracy of user location determined based on the sensor data . More specifically, the sensor unit may typically include one or more accelerometers. Thus, determining user location based on the sensor data typically utilizes integration of data provided by the accelerometers and may accumulate errors (providing history data on previous movement and location of the electronic device). The correlation between the sensor unit and the location data (of the tag unit) may be used for determining improved location estimation based on integration of sensor data (e.g. by determining step length, reference position, reference orientation etc.) along history data including previously received sensor data. To this end, the present technique may determine and periodically update reference parameters such as step length, reference position or others, associated with sensor-based location data. Additionally, in some configurations, the technique may determine correction parameters in accordance with correlation between location of the user, determined by using location data, and from sensor data.

Thus, according to a broad aspect, the present invention provides a method for use in indoor navigation, the method comprising:

providing (or receiving) location data indicative of location of a selected tag unit and sensor data indicative of at least one of movement and orientation of a sensor unit, the sensor unit comprises at least one of inertial, magnetic and RF location sensors; using a computer processor and processing the location data and the sensor data for determining correlation between the location data and sensor data, and for determining one or more user actions comprising one or more of modes of movement comprising: common movement of said tag unit and said sensor unit, separate movement of said tag unit and said sensor unit, detachment event of said sensor unit from said tag unit, and reattachment event of said sensor unit and said tag unit to common movement.

The method may comprise operating of the sensor unit for obtaining respective sensor data.

According to some embodiments, the method may further comprise processing correlation between said location data and said sensor data and determining calibration data of said at least one of inertial, magnetic and RF location sensor, said calibration data indicative of one or more of reference position, reference orientation state and movement step size of said sensor data.

According to some embodiments, said determining one or more user actions may comprise determining correlation score indicative of relative movement of said tag unit and said sensor unit and determining between said one or more modes of movement in accordance with said correlation score.

According to some embodiments the method may further comprise determining location of detachment event of said sensor unit from said tag unit as a reference position for determining location of said sensor unit, and using said sensor data for determining location of said sensor unit while operating in separate movement mode.

The method may comprise processing said sensor data for determining user orientation data.

According to some embodiments the method may further comprise transmitting data on location of said sensor unit indicative of said sensor data, to one or more remote devices and receiving corresponding location-based data indicative of items within predetermined ranges from the determined location.

According to some embodiments, said receiving location data may comprise communicating with a location server and providing identification number of a selected tag unit for receiving said location data over a wireless communication protocol.

According to some embodiments, modes of movement may be determined in accordance with relative movement determined based on said location data and said sensor data.

According to some embodiments, the method may further comprise determining relative distance between said sensor unit and said tag unit, such that mode of common movement of said sensor unit and said tag unit is acceptable when said relative distance is below a predetermined threshold.

According to some embodiments, the location data may comprise at least one of data output of a positioning system indicative of location of said tag unit, and accelerometer data indicative of acceleration of the tag unit in one or more directions.

According to one other broad aspect, the present invention provides a computer program comprising computer program code means for performing all the steps defined by the above described method when said program is run on a computer. The computer program may preferably be embodied on a computer readable medium.

According to yet another broad aspect, the present invention provides a program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform a method for use in indoor navigation, the method comprising: providing location data indicative of location of a selected tag unit sensor data indicative of at least one of movement and orientation of a sensor unit, the sensor unit comprises at least one of inertial, magnetic and RF location sensors; using a computer processor and processing the location data and the sensor data for determining correlation between the location data and sensor data, and for determining one or more user actions comprising one or more of modes of movement: common movement of said tag unit and said sensor unit, separate movement of said tag unit and said sensor unit, detachment event of said sensor unit from said tag unit, and reattachment event of said sensor unit and said tag unit to common movement.

The program instructions may comprise instructions for operation of the sensor unit to obtain said sensor data.

According to yet another broad aspect, the present invention provides a computer program product comprising a computer useable medium having computer readable program code embodied therein for use in indoor navigation, the computer program product comprising:

computer readable program code for causing the computer to receive location data indicative of location of a selected tag unit, sensor data indicative of at least one of movement and orientation of a sensor unit, the sensor unit comprises at least one of inertial, magnetic and RF location sensors; using a computer processor and processing the location data and the sensor data for determining correlation between the location data and sensor data, and for determining one or more user actions comprising one or more of modes of movement: common movement of said tag unit and said sensor unit, separate movement of said tag unit and said sensor unit, detachment event of said sensor unit from said tag unit, and reattachment event of said sensor unit and said tag unit to common movement.

The program code may comprise instructions for operation of the sensor unit to obtain said sensor data.

According to yet another broad aspect, the present invention provides an electronic device comprising at least one processor, network communication module and sensor unit, said sensor unit comprising at least one of inertial, magnetic and RF location sensors for obtaining sensor data indicative of at least one of movement and orientation of said electronic device; said at least one processor is adapted for receiving location data indicative of location of a selected tag unit via a network connection link, for operating said sensor unit for determining sensor data indicative of at least one of movement and orientation of said electronic device, and for processing said location data and sensor data for determining correlation between the location data and sensor data, and for determining one or more user actions comprising one or more modes of movement: common movement of said tag unit and said electronic device, separate movement of said tag unit and said sensor unit, detachment event of said electronic device from said tag unit, and reattachment event of said electronic device and said tag unit to common movement.

According to some embodiments, the at least one processor may comprise: a location sensing module adapted for receiving sensor data indicative of at least one of movement and orientation variation of said electronic device and determining data on location variation of said electronic device;

a location correlation module adapted for receiving input data comprising at least data on location variation of said electronic device from the location sensing module and data location data indicative of location of said selected tag unit, and for processing input data for determining correlation between movement of the electronic device and said selected tag unit;

thereby enabling said at least one processor to determine between said one or more user actions.

According to some embodiments, the at least one processor may comprise a sensor data calibration module adapted for processing the sensor data in accordance with received location data for determining one or more calibration parameters providing reference parameters, thereby enabling to determine location of said electronic device based on the sensor data.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to better understand the subject matter that is disclosed herein and to exemplify how it may be carried out in practice, embodiments will now be described, by way of non-limiting example only, with reference to the accompanying drawings, in which:

Fig. 1 is a flow diagram illustrating the technique according to some embodiments of the present invention; Fig. 2 illustrates an electronic device according to some embodiments of the invention;

Fig. 3 illustrates the technique according to some embodiments of the present invention, employed within a selected region (e.g. store);

Fig. 4 exemplifies operation of the technique according to some embodiments thereof, for determining between modes of movement;

Figs. 5A to 5F illustrate different relative movements between a user and shopping cart that are detectable according to some embodiments of the invention; and

Figs. 6A to 6D show accelerometer sensor data along X, Y and Z and indication of tag unit movement respectively indicating common movement and detachment event.

DETAILED DESCRIPTION OF EMBODIMENTS

Reference is made to Fig. 1 illustrating by a flow chart operation of the present technique according to some embodiments thereof. As shown, the present technique may be implemented on a user’s electronic device (e.g. smartphone) operating a suitable software product and signed into the system. Further, the electronic device may generally be a handheld computing device and may typically include at least one processor, memory unit, network communication module, and user interface 700. The present technique may be used on an electronic device in the form of a software product executed by the processor(s) of the device and may utilize communication with a selected (e.g. remote) server via the network communication module. It should be noted that selected operations may be executed by the server and corresponding data may be transmitted to the electronic device, while certain other operations are performed in the electronic device. The technique may generally include pairing between a tag unit and electronic device of the user 1010. Generally, the user may initiate pairing by scanning a code marked on a tag unit and instructing the electronic device to communicate data indicative of the scanned code via a selected application to a respective server. The server may maintain a database including data on pairing of one or more electronic devices and corresponding one or more tag units, where generally each electronic device is paired to up to a selected number of tag units at each given time instance. The server may be located within or near the space and operated for monitoring location of the tag units, or it may be a remote server operating via communication network. After such assignment takes place, the technique includes periodically providing location data 1020 indicative of location of the tag unit from the server, and operating one or more sensors of the sensor unit (e.g. sensors of the electronic device) for providing sensor data 1030. The technique utilizes one or more processors operated for processing the location data and the sensor data 1040 either at the device end or by transmitting the sensor data to the server and processing at the server end. The processing may include determining one or more correlation functions indicating level of correlation between the location data and the sensor data 1050. Using the correlation functions, the present technique determines operation between one or more modes of movement 1060 selected from common movement state 1070, detachment event 1080, and reattachment event 1090. During common movement state, the present technique may utilize the location data for calibrating 1075 parameters associated with processing of the sensor data to thereby enable improved accuracy in determining location based on the sensor data. When a detachment event is detected, the present technique may utilize data on the recent location where the detachment event occurred, to determine reference parameters 1085 (e.g. reference position, orientation and step length) for use in determining location based on the sensor data 1088. Determining the location of the user away from the tag unit based on the sensor data enables the present technique to detect reattachment event 1090 when common movement state 1070 takes place again.

Generally, providing location data 1020 is associated with operation of the indoor positioning system. More specifically, as indicated herein, the positioning system is generally configured to determine location of the tag units using a selected number of signal receiving anchors positioned within the selected region.

Reference is made to Fig. 2 illustrating an electronic device 100, e.g. smartphone or handheld computer device, including memory unit 300, sensor unit 400, one or more processors 500, input/output communication module 600 and user interface module 700. The device 100 may be configured for operating computer software according to some embodiments of the present technique as described above.

To assign a tag unit, the device 100 may be operated by user input and transmit a pairing request 130 to a dedicated server (e.g. via communication network such as Wi-Fi or other wired or wireless communication network). The pairing request 130 may generally include data on identity of associated tag and of the electronic device 100. The device 100 may further be operated to be responsive for receiving tag location data 120 as well as additional data 140 from the server for determining location and additional information as mentioned above. The one or more processor unit(s) 500 generally include hardware and/or software modules including e.g. location sensing module 510, location correlation module 520 and mode selector 530 and may include additional modules such as location data module 540 illustrated in Fig. 2 to exemplify additional operation modules of the device 100. Additionally, the one or more processors may also include a sensor data calibration module 550.

The location sensing module 510 is configured to receive sensor data from the sensor unit 400 and to use one or more reference data (e.g. reference location, orientation etc.) stored in the memory unit 300, for determining variations in location of the electronic device 100 at selected time intervals (sampling rate). In some embodiments, the location sensing module 510 may be configured to operate in common movement mode (1070 in Fig. 1) for determining changes in location based on sensor data, or in detachment mode for determining actual location based on the sensor data (1088 in Fig. 1). In some other embodiments, the location sensing module 510 may operate in a single mode for determining data on location based on sensor data. The location sensing module 510 further provides the data on location or change thereof, to the location correlation module 520.

Generally, the location sensing module 510 is adapted for operating inertial navigation techniques. More specifically, the sensor unit 400 may include one or more accelerometer and/or magnetic sensors, providing data on variation in velocity of the electronic device 100 and orientation with respect to external magnetic fields. Accordingly, the location sensing module 510 may be adapted to receive sensor data from the sensor unit 400 and to determine data on location, or variation of location of the electronic device 100 by integration of the sensor data along time (using history data). To this end, the location sensing module 510 may utilize data indicative of reference location of the device 100 as well as additional calibration data (e.g. indicative of stride length, reference velocity etc.). Reference and calibration data, as well as general or processed sensor data may generally be stored in a memory unit 300 as history data and may be updated during common movement mode using location data provided by the positioning system in accordance with the location of the tag unit.

The location correlation module 520 is adapted for receiving location data 120 provided from the positioning system and indicative of location of a selected tag unit, as well as data on variations of location determined by the location sensing module 510, and for determining correlation between movement of the electronic device 100 and movement of the tag unit indicated from the location data 120. The location correlation module 520 provides data on correlation level between movement of the electronic device and the tag unit to the movement mode selector 530 for determining between one or more modes of movement and operating the electronic device accordingly. In some configurations of the present technique, the electronic device 100 may be adapted to transmit data on variation of location determined by the location sensing module 510 to the server allowing the server to operate location correlation processing at the server end. This may reduce processing required by the electronic device 100 to provide faster operation of the technique and reduced battery use of the handheld electronic device 100.

The movement mode selector 530 utilizes data on correlation between movement of the electronic device (determined based on sensor data) and movement of the associated tag unit (provided in the tag location data 120). Using the correlation data, the movement mode selector 530 determines mode of respective movement between the electronic device and the respective tag unit. Such modes of movement include: (i) the electronic device moves together with the tag unit, (ii) the electronic device moves separately from the tag unit. To determine shift between these modes of movement, the movement mode selector 530 also determines detachment and/or reattachment events between the electronic device and tag unit. For example, in cases where tag units are mounted on carts in a commercial environment (e.g. shopping carts in a store) common movement indicates that the user is walking with the cart, while separate movement may be associated with a situation when the user leaves the cart outside of a certain aisle or region and walks in the aisle of region without the cart. The movement mode of the electronic device 100 is used for determining operation of one or more additional modules of the electronic device, such as e.g. location data module 540, and the use of location data 120 and/or sensor data for determining device location as described in more detail further below.

During common movement mode, i.e. when the electronic device 100 (and the user) moves together with the tag unit, the one or more processors 500 may operate the sensor data calibration module 550 for determining and updating calibration parameters to improve determining of location based on the sensor data. Typically, the sensor data includes data on acceleration of the electronic device, and may also include data on variations of magnetic field (when the sensor unit 400 includes magnetic sensor(s)), enabling to determine location based on integration and variation in acceleration of the device 100. To this end, during common movement mode, the sensor data calibration module 550 may be adapted for receiving sensor data from the sensor unit 400 and location data 120 indicative of location of the tag, and for using the location data to optimize integration of the sensor data. More specifically, the sensor data may typically be indicative of acceleration in one or more axes associated with orientation of the electronic device. Accordingly, determining location of the electronic device based on the sensor data may include determining orientation of the electronic device, determining step count and/or integration of the acceleration data as determined in the sensor data. Generally, determining position based on such sensor data may require one or more additional parameters such as reference location of the device, reference velocity, step length of the user, reference orientation, and additional calibration parameters that may be used to improve accuracy of location based on the sensor data. Thus, the sensor data calibration module 550 may utilize provided data on tag location 120 along time of common movement mode, to determine and update such calibration parameters, and operate to store the calibration parameters in the memory unit 300 for use in separate movement mode.

As indicated above, the present technique may be used in commercial environments such as retail stores etc. An exemplary configuration of commercial environment and implementation of the present technique therein is illustrated in Fig. 3. Fig. 3 illustrates a region 5000 of a commercial environment (e.g. storage, store, warehouse, museum etc.) where a tag-based positioning system (e.g. indoor positioning system) is employed. The positioning system is exemplified as based on a plurality of receiving anchors 5030a to 5030d arranged in known locations and configured for receiving location signals from one or more tag units 5020 and providing data on the received signals to server system 1000 via communication arrangement 5010 (e.g. wired or wireless communication network). The region 5000 may include various physical elements 5050 such as shelves, cabinets and other elements that may carry objects of interest (and/or block free movement). A user may be using an electronic device 100 while in the region 5000 for obtaining data on items located in the region and navigation data within the region 5000. It should be noted that the electronic device 100 is illustrated out of the region 5000 for convenience of the illustration. In operation, the user generally moves with the electronic device 100 within the region 5000. The user may generally carry a tag unit 5020 for monitoring location thereof within the region 5000. The tag unit 5020 may be mounted on a cart/basket, name tag or other devices carried by the user to assist in determining location of the user within the region 5000. In some cases, due to existence of physical elements 5050 in the region 5000, the user may choose to leave the tag unit 5020 at certain locations while walking/moving within the region 5000. Typically, after a certain time, the user may return to the tag unit 5020 and continue moving within the region with the tag unit 5020. This may be, for example, when a shopping cart cannot move between aisles due to a narrow/crowded path, an audio device may be left outside of a quiet room in a museum, etc. The present technique provides for determining occurrence of such situations and allowing the user to maintain location tracking and receive location related information while not necessarily moving with the tag unit 5020.

To this end, the server 1000 is generally adapted for determining location of the tag unit 5020 and providing location data to the electronic device 100. The electronic device may use the location data to indicate location of the tag unit to the user, and, as illustrated in Fig. 3, providing tag location 1200 in the user interface (e.g. 700) of the electronic device 100. The electronic device 100 utilizes sensor unit thereof for providing sensor data associated with and indicating variation of location thereof for determining correlation function (i.e. level of correlation) between the location data and the sensor data. This may be performed by the processor of the device 100 or by the server 1000 as indicated above. The sensor data may indicate movement and/or orientation of the sensor device, e.g. in the form of steps, drift etc. Based on the correlation between the location data and the sensor data, the present technique can distinguish between one or more modes of movement and provide the user with relevant information via the user interface. For example, the correlation may indicate that the electronic device and the tag unit are moving together or separately. For example, determining correlation between the location data and sensor data may be associated with determining a correlation score indicating the similarity in movement patterns between the location data and sensor data. When the difference between movement patterns of the electronic device and the tag unit exceeds a selected threshold, i.e. the correlation score is below a selected detachment threshold, a detachment event may be identified. After a detachment event, location of the electronic device is determined (estimated) based on the sensor data. When the correlation score between the location data and sensor data is determined to exceed an attachment threshold, a reattachment event may be determined, returning to common movement mode. Generally, reattachment event may be determined when correlation is sufficiently high and estimated location of the electronic device is within a certain vicinity to the tag unit.

Accordingly, the present technique is capable of determining preliminary attachment of the tag unit 5020 to the electronic device 100 and common movement thereof, detachment of the electronic device 100 to move separately from the tag unit 5020, and a reattachment event, where the electronic device 100 returns to move with the tag unit 5020. Based on the selected mode of movement, the present technique may provide relevant data to the user, such as location thereof, location of selected items of shelves 5050, preferred path to certain location in the space 5000, location of the tag unit 5020, details on items in nearby shelves 5050 etc.

For example, the positioning system employed in region 5000 may be an Indoor Positioning System (IPS). In some examples, the IPS may utilize Wi-Fi, Ultra-Wide Band (UWB) or any other type of communication 5010 techniques enabling communication between the tag unit 5020, the anchors 5030a to 5030d and the server 1000. In some specific examples, an Ultra-Wide Band (UWB) communication network may be based on RF transmission in the range of 3.1 GHz to 10.6 GHz with a bandwidth of 500MHz or more.

In some configurations, the following elements may be used in the positioning system, including:

Anchors 5030a to 5030d are fixed units installed in known locations in the installation site. Anchors may be hardware units combining UWB and WIFI transceivers.

Tags 5020 may be mobile units that include a UWB transmitter, 3 -axis MEMS accelerometer, and may include other sensors.

Computing device /server 1000 formed by one or more computing units that can be physical on an on-site computer, remote computer, web server, or any other type of computing device.

WI-FI Infrastructure 5010 providing WI-FI communication in the IPS installation site that connects the anchors 5030a to 5030d and the computing devices/servers 1000. Electronic device 100, e.g. smartphone, that may be external to the system but run an end user application according to the present technique which is linked by network communication to the computing device/server 1000.

As indicated above, the technique is adapted for determining one or more modes of common movement states of the user with respect to correlation level of sensor data indicative of variation of the user's location and location data of the a tag unit (e.g. mounted on a shopping cart). Based on the correlation data, the technique is enabled for determining detachment and/or reattachment events, namely the system recognizes between common and separate movements of the user and the tag unit.

Generally, the tag unit and the electronic device may both be stationary or move together in common movement mode. In separate movement mode, the tag unit and the electronic device generally move in separate patterns. This may be associated with a certain distance between the electronic device and the tag unit, as well as in difference in speed and/or direction of movement. The present technique generally keeps determining correlation between the location data and sensor data in separate movement mode, and periodically determines a correlation score indicating relative movement variation. The technique may utilize data on movement, direction of movement, orientation, speed etc. for determining correlation score, and accordingly identify between the common or separate movement modes, as well as detachment and reattachment events. The technique may also utilize data on location of the user for providing relevant data to the user. The location of the user may be determined based on the sensor data, e.g. using step count and estimated step length, in separate movement mode, or based on location data in common movement mode.

Reference is made to Fig. 4 exemplifying a cycle between common and separate movement modes according to some embodiments of the present technique. As indicated above, an initialization and pairing 6010 action may be associated with signing into the system by transmitting data on a tag unit assignment code. In this non-limiting example, it is assumed for simplicity that the initialization and pairing is performed when the user is located next to the tag unit and common movement mode is assumed 6020. Location data indicative of location of the tag unit and sensor data indicative of movements of the electronic device are monitored to determine correlation between them 6030. In common movement mode, motion of the tag unit is generally correlated with movement of the electronic device provided by the sensor data. If the tag unit and the electronic device are determined to be in motion, common movement mode is maintained. Detachment between the electronic device and tag unit may be identified when a certain variation in the correlation is determined 6040 and the variation exceeds a detachment threshold level 6050. This may for example be the case when the location data indicates that the tag unit is stationary, while the sensor data shows movement of the electronic device 6045. If the correlation between location data and sensor data indicates that the distance exceeds the detachment threshold, the technique determines a detachment event 6060 (generally determined by the processor of the electronic device, or by a server operating the indoor positioning system as described above). After determining such a detachment event, the system operates in separate movement mode 6070 where location of the electronic device is directly determined (generally estimated) based on the sensor data 6080. This may be done based on determining the number and direction of steps from a determined reference location, and/or using integration of accelerometer data. Typically, the location data, indicative of location of the tag unit is still monitored to determine its location even if moved by other means that are not directly related to the paired user. The correlation between the location data and sensor data may still be periodically determined, such that when the correlation exceeds a reattachment threshold 6090, e.g. when the user goes back to the tag unit and moves together with the tag unit, a reattachment event can be determined 6100 returning to operating in common movement mode 6020.

Generally, reattachment event 6100 may be determined when correlation score, based on correlated movement of the electronic device and the tag unit, exceeds a selected reattachment threshold. Further, the present technique may operate in common movement mode 6020 as long the correlation between tag unit location and the sensor data is within selected common movement limits, indicative of distance between the electronic device and tag unit, direction of movement, speed etc. A high correlation score is indicative that the tag unit and the sensor data associated with the electronic device, are moving with substantially similar patterns.

In this connection, the tag units may also include accelerometer sensors allowing the tag to clearly indicate motion state thereof, e.g. indicate if the tag is“in motion” or “not in motion”. In such configurations, as indicated above, location data, provided by the system, may include data on location of the tag unit and data on motion status thereof. It should be understood that when not using accelerometer sensors within the tag unit, motion state of the tag may be determined based on history of location data, i.e. by determining derivatives of the location data for identifying if location of the tag unit has changed.

Reference is made to Figs. 5A-5F exemplifying several situations treated by the present technique by the abovementioned modes of movement. Fig. 5A exemplifies common movement of a user with a shopping cart carrying a tag unit 5020, Fig. 5B exemplifies a static attached state where common movement mode is maintained, Fig. 5C exemplifies attached state in which the user is rotating with respect to the tag unit while not exceeding detachment threshold, Fig. 5D exemplifies detachment event, in Fig. 5E the user is exemplified as walking separately from the cart 5020, the figure generally shows the user walking toward the cart and may exemplify a reattachment event, however such separate movement may be of a longer path before reaching reattachment event, and Fig. 5F exemplifies returning to common movement in the attached state.

As indicated above, the present technique may be advantageously implemented in shopping scenarios (e.g. grocery stores, malls etc.) where a user is using a personal hand held electronic device 100 (e.g. smartphone) for providing sensor data and a user interface, while using tag unit 5020 mounted on a shopping cart/basket for determining more accurate location data. In Fig. 5A, the user is moving together with the shopping cart along a selected direction, and location of the user may be monitored using location data of the tag unit. In Figs. 5B and 5C the shopping cart is stationary, e.g. the user is checking for items on a shopping list in Fig. 5B and turns to the relevant shelf for the desired items in Fig. 5C, while the cart is static. In Fig. 5D, the user is moving away from the shopping cart, and the sensor data indicates movement of the device 100 resulting in correlation level being reduced. In some configurations, a detachment event is detected when the location data and sensor data are non-correlated for a certain time exceeding a detachment threshold; in some embodiments such a detachment event is detected when a distance between the electronic device and tag unit is determined to be exceeding a distance detachment threshold. After detecting a detachment event, the user location may be determined based on sensor data. Generally, the shopping cart may be moved by other users of store employees and/or the user may return to close proximity to the tag unit, as shown in Fig. 5E. Typically, a reattachment event may be detected when the shopping cart is moving in correlation with the user, while at the same time the user is within a reattachment threshold distance. In some embodiments, a reattachment event may be determined when the location data and sensor data indicate movement with correlation exceeding a reattachment threshold, as exemplified in Fig. 5F. The examples of Figs. 5A to 5F are summarized in Table 1 for clarity.

Table 1

In order to distinguish between common and separate movement modes, as well as identify detachment and reattachment events, the present technique preferably utilizes processing of sensor data for determining data on movement, or at least variation of movement of the electronic device 100. Reference is made to Figs. 6A to 6D exemplifying correlation of the device and tag movement using accelerometer data. Figs. 6A to 6C show sensor data indicative of accelerometer output along X, Y and Z of the electronic device respectively. The graphs show accelerometer output (AccX, AccY and AccZ), filtered accelerometer output (butter) and detected peaks indicative of steps of the user (peaks). Fig. 6D shows movement data of a tag unit mounted on a shopping cart along tagX and tagY directions. In this example, a user is walking with the electronic device and pushing the cart in a direction corresponding to the negative Y axis. At time mark 73, the user stops and stands still for a few seconds. Around time mark 75, the user walks away without the cart. Movement of the cart is exemplified herein using the tag accelerometer but can also be determined by derivation of location data and/or based on one or more accelerometer associated therewith. Correlation between the common movements, and reduced correlation in the separate movement stage, are visible in this example, and are generally determined by computer processing according to the present technique. Thus, the present invention provides a technique enabling accurate detection of a user’s location within a selected region (e.g. indoor position) and enables providing the user with location-based information. The technique provides balance between accurate positioning that requires a dedicated location beacon or tag, and the flexibility of positioning, using standard sensors as used in typical portable electronic devices.

Claims

CLAIMS:

1. A method for use in indoor navigation, the method comprising:

providing location data indicative of location of a selected tag unit and sensor data indicative of at least one of movement and orientation of a sensor unit, the sensor unit comprises at least one of inertial, magnetic and RF location sensors; using a computer processor and processing the location data and the sensor data for determining correlation between the location data and sensor data and for determining one or more user actions comprising one or more modes of movement comprising: common movement of said tag unit and said sensor unit, separate movement of said tag unit and said sensor unit, detachment event of said sensor unit from said tag unit, and reattachment event of said sensor unit and said tag unit to common movement.

2. The method of claim 1, further comprising processing the correlation between said location data and said sensor data and determining calibration data of said at least one of inertial, magnetic and RF location sensor, said calibration data indicative of one or more of reference position, reference orientation state and movement step size of said sensor data.

3. The method of claim 1 or 2, wherein said determining one or more user actions comprises determining a correlation score indicative of relative movement of said tag unit and said sensor unit, and determining between said one or more modes of movement in accordance with said correlation score.

4. The method of any one of claims 1 to 3, further comprising determining the location of a detachment event of said sensor unit from said tag unit as a reference position for determining location of said sensor unit, and using said sensor data for determining location of said sensor unit while operating in separate movement mode.

5. The method of any one of claims 1 to 4, comprising processing said sensor data for determining user orientation data.

6. The method of any one of claims 1 to 5, further comprising transmitting data on location of said sensor unit indicative of said sensor data, to one or more remote devices, and receiving corresponding location-based data indicative of items within predetermined ranges from the determined location.

7. The method of any one of claims 1 to 6, wherein said receiving location data comprises communicating with a location server and providing identification number of a selected tag unit for receiving said location data over a wireless communication protocol.

8. The method of any one of claims 1 to 7, wherein said modes of movement are determined in accordance with relative movement determined based on said location data and said sensor data.

9. The method of any one of claims 1 to 8, further comprising determining relative distance between said sensor unit and said tag unit, such that mode of common movement of said sensor unit and said tag unit is acceptable when said relative distance is below a predetermined threshold.

10. The method of any one of claims 1 to 9, wherein said location data comprises at least one of data output of a positioning system indicative of location of said tag unit, and accelerometer data indicative of acceleration of the tag unit in one or more directions.

11. A computer program comprising computer program code means for performing all the steps of any of claims 1 to 10 when said program is run on a computer.

12. A computer program as claimed in claim 11 embodied on a computer readable medium.

13. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform a method for use in indoor navigation, the method comprising:

providing location data indicative of location of a selected tag unit sensor data indicative of at least one of movement and orientation of a sensor unit, the sensor unit comprises at least one of inertial, magnetic and RF location sensors; using a computer processor and processing the location data and the sensor data for determining correlation between the location data and sensor data, and for determining one or more user actions comprising one or more of modes of movement comprising: common movement of said tag unit and said sensor unit, separate movement of said tag unit and said sensor unit, detachment event of said sensor unit from said tag unit, and reattachment event of said sensor unit and said tag unit to common movement.

14. A computer program product comprising a computer useable medium having computer readable program code embodied therein for use in indoor navigation, the computer program product comprising:

computer readable program code for causing the computer to receive location data indicative of location of a selected tag unit, sensor data indicative of at least one of movement and orientation of a sensor unit, the sensor unit comprises at least one of inertial, magnetic and RF location sensors; using a computer processor and processing the location data and the sensor data for determining correlation between the location data and sensor data, and for determining one or more user actions comprising one or more modes of movement comprising: common movement of said tag unit and said sensor unit, separate movement of said tag unit and said sensor unit, detachment event of said sensor unit from said tag unit, and reattachment event of said sensor unit and said tag unit to common movement.

15. An electronic device comprising at least one processor, network communication module and sensor unit, said sensor unit comprising at least one of inertial, magnetic and RF location sensors for obtaining sensor data indicative of at least one of movement and orientation of said electronic device; said at least one processor is adapted for receiving location data indicative of location of a selected tag unit via a network connection link, for operating said sensor unit for determining sensor data indicative of at least one of movement and orientation of said electronic device, and for processing said location data and sensor data for determining correlation between the location data and sensor data, and for determining one or more user actions comprising one or more modes of movement: common movement of said tag unit and said electronic device, separate movement of said tag unit and said sensor unit, detachment event of said electronic device from said tag unit, and reattachment event of said electronic device and said tag unit to common movement.

16. The electronic device of claim 15, wherein said at least one processor comprises: location sensing module adapted for receiving sensor data indicative of at least one of movement and orientation variation of said electronic device and determining data on location variation of said electronic device;

location correlation module adapted for receiving input data comprising at least data on location variation of said electronic device from the location sensing module and data location data indicative of location of said selected tag unit, and for processing input data for determining correlation between movement of the electronic device and said selected tag unit;

17. The electronic device of claim 15 or 16, wherein said at least one processor comprises a sensor data calibration module adapted for processing the sensor data in accordance with received location data for determining one or more calibration parameters providing reference parameters, thereby enabling to determine location of said electronic device based on the sensor data.