WO2014195164A2

WO2014195164A2 - Localization system for monitoring objects in real-time

Info

Publication number: WO2014195164A2
Application number: PCT/EP2014/060775
Authority: WO
Inventors: Joachim METTER
Original assignee: Pps Gmbh
Priority date: 2013-06-04
Filing date: 2014-05-26
Publication date: 2014-12-11
Also published as: WO2014195164A3

Abstract

A localization system (10) for monitoring at least one object in real-timeis proposed. The localization system (10) comprises a plurality of localization units (12a, 12b, 2c, 12d, 12e) for generating a plurality of location data relating to the at least one object being monitored,and a control unit (14) for collecting the plurality of location data provided by the plurality of localization units (12a, 12b, 12c, 12d, 12e), wherein the control unit (14) is arranged to map at least one of the plurality of location data to derive a single position data of the at least one object being monitored.

Description

Localization system for monitoring objects in real-time

FIELD OF THE INVENTION

The present invention relates to a localization system for monitoring objects in real- time and a method for locating and monitoring objects in real-time.

TECHNICAL BACKGROUND

In many environments, such as for example an airport environment, a surveillance of objects may be an issue of high importance, and several localization technologies have been developed in order to localize, track and monitor objects of interest.

However, one major drawback of present localization technologies may be a lack of a fully comprehensive surveillance of all areas in the environment being monitored.

For example satellite-based localization technologies may ensure an accurate monitoring of objects moving in outdoor areas, whereas they may not be capable of monitoring objects moving in indoor areas. Vice versa, localization technologies developed for indoor monitoring may not be suited for a localization of objects moving in outdoor areas. SUMMARY OF THE INVENTION

It is an object of the present invention to overcome drawbacks and shortcomings of common approaches for a surveillance of an environment. Particularly, it may be an object of the present invention to provide a comprehensive system for locating, tracking and monitoring mobile and immobile objects, such as individuals, animals, semi-finished products or goods, in certain environments of interest, such as e.g. airports, hospitals, nursery schools, warehouses, stores, kindergartens, retirement or nursery homes, power plants, or factories.

The above-mentioned objects of the present invention may be achieved with the subject-matter of the independent claims. Advantageous embodiments are defined in the dependant claims and the following description.

According to a first aspect of the present invention, a localization system for monitoring at least one object in real-time in an environment of interest is proposed. The localization system comprises a plurality of localization units for generating a plurality of location data relating to the at least one object being monitored and a control unit for collecting the plurality of location data provided by the plurality of localization units.

By applying a plurality of localization units to track and monitor an object it may be ensured that even if some of the localization units may be standing idle or may be decommissioned for an instant, the object of interest may still be tracked by at least one other localization unit. This may allow keeping track of the object at any time. Furthermore, by generating a plurality of location data for each object being monitored an accuracy of the location data may be enhanced.

The control unit of the localization system is arranged to map at least one of the plurality of location data to derive a single position data of the at least one object being monitored. In this context the wording "to map location data" may not only comprise pinpointing a position on a map, but also analyzing or interpreting the location data provided by the localization units against a background of a physical map of the environment being monitored, and consequently deriving a preferably accurate single position of each object being monitored at each time step considered. Moreover, the expression "location data" refers to raw data provided by a localization unit. Such raw data may for instance be geographical coordinates or any other measure relating to a position of an object being monitored as provided by the corresponding localization unit. For example, a localization unit may comprise a mobile device attached to the object being monitored and antennas receiving signals from the mobile device. In this case the location data transmitted by the localization unit may be a distance of the mobile device to the antennas. In contrast, "single position data" may refer to coordinates or any other measure as derived by the control unit from the raw data of the plurality of localization units in the context of the environment being monitored. For instance the distance of the mobile device to the antennas may be analyzed by the control unit in the context of a specific map of the environment in order to derive a position data relating to the specific map which may be pinpointed on the map.

According to an embodiment of the present invention, the control unit of the localization system is arranged to analyze the at least one of the plurality of location data with respect to plausibility. This check in terms of plausibility may, for example, comprise accounting for any obstacle in the environment being monitored, such as e.g. walls of buildings, fences or other physical boundaries that cannot be trespassed by the object being monitored.

For this purpose, e.g. a model of buildings and other obstacles in the environment being under surveillance may be implemented in the control unit or may be accessed by the control unit to allow analyzing the location data with respect to plausibility. In case a location data is determined to be for instance inside a wall of a building, the control unit may consequently discard this location data due to implausibility, and only plausible location data are further considered to derive and map a single position data of the object being monitored. With this approach the plurality of location data provided by the plurality of localization units may be decreased, and in turn the control unit may only further process meaningful location data to derive and map the single position data of the object being monitored. This may advantageously decrease a computing time required to derive and map the single position data at each time step considered.

Apart from the above-mentioned plausibility check with respect to physical boundaries or obstacles in the environment being monitored, the control unit may also be arranged for checking plausibility of the location data with respect to time and speed of the object being monitored. For example, if at one time step a single position of the object was mapped at one end of the environment, and in a subsequent time step a location data suggests a position at an opposite end of the environment that may only had been reached if the object had moved with an unrealistic speed, the location data may be regarded implausible by the control unit and, thus, may be discarded.

It should be noted, however, that location data regarded implausible by the control unit do not necessarily have to be discarded. Such data may for instance still be further processed by the control unit, but with high uncertainty. According to an embodiment of the invention, the control unit of the localization system weights the location data with a quality factor, wherein the quality factor may account for a spatial resolution of the location data. For instance, this quality factor may be distributed in an interval of [0; 1], and all location data may be weighted linearly, i.e. with a linear weighting function. However, basically any other weighting, such as for example a logarithmic or an exponential weighting, i.e. a weighting with a logarithmic or exponential weighting function, may be used. In this context the wording "to weigh" may be understood as connecting a location data with a specific quality factor. Technically this connection may for instance be accomplished by a multiplication of a location data with the corresponding quality factor. For this purpose, an intrinsic quality factor of each localization unit may be implemented in the control unit or may be accessed by the control unit. It may also be that a localization unit transmits location data including metadata, such as for instance temporary signal strength of the localization unit at the location of the object being monitored and at the time step considered. Such metadata may thus contain information about a spatial resolution of the currently transmitted data, and the control unit may in turn be arranged and adapted for determining a quality factor for a specific location data from the metadata provided by a specific localization unit.

The prescribed concept of weighting location data with a quality factor may be advantageous for the derivation and the mapping of the single position of the object being monitored, since for example location data with higher accuracy may be considered more important by the control unit, and hence may have a higher (or lower) weight reflected by a higher (or lower) quality factor. By accounting for a hierarchy in the location data generated by the various localization units with respect to their quality and accuracy, the control unit may derive and map the single position of the object being monitored more precisely. According to an embodiment of the invention, the plurality of localization units of the localization system comprises localization units that are based on various localization or sensing technologies, such as for example GPS, D-GPS, Global Navigation Satellite System (GNSS), Automatic Dependent Surveillance-Broadcast (ADS-B), multi-lateration (MLAT) techniques, wireless local area network (WLAN), radar, near-range radar networks, ultra-wide band (UWB), Radio Frequency (RF) sensors, BlackFIR 2.4, BlackFIR 868, inertial sensors, closed circuit television (CCTV) with video analytics, magnetic flux sensors, infrared sensors, or any other sensing technology. Also localization units employing environmental sensors may be used. Potential candidates for environmental sensors may for example be sensors measuring an Oxygen level or an Oxygen saturation in certain rooms which may help to detect a presence of individuals. Other applicable environmental sensors may for example be air pressure sensors delivering information about an elevation of a point or area of interest, light barriers for identification of transits of individuals or objects of interest, or pressure sensors located e.g. in floors or furniture for detecting the presence of individuals. A combination of various localization or sensing

technologies and techniques may allow a more comprehensive monitoring of objects, because shortcomings of one technology, such as for instance a small range, may be compensated by another localization technology. Moreover, a combination of various sensing technologies may advantageously enhance the accuracy and flexibility of the localization system due to a possibly higher number of independently generated location data.

In an embodiment of the invention, at least one of the plurality of localization units of the localization system is radio-based. One major advantage of radio-based localization units may be that a rather large range may be covered, and thus objects moving in a rather large environment may be accurately monitored. Apart from this, radio-based localization units may operate at rather high speeds, i.e. they may transmit a current location of the object being monitored in rather small time steps. This may further improve the accuracy of tracking and monitoring in terms of time.

According to yet another embodiment of the invention, the localization system comprises at least one video-based localization unit. One major advantage of this embodiment may be, that the localization system may easily be embedded in a variety of environments that are already equipped with a video surveillance system, such as for example hospitals or airports. In turn, the localization system as described in previous and following sections may advantageously be complemented and supported by an existing video surveillance system without any additional costs for camera equipment. Furthermore, as most of the areas in such environments may already be covered by the video surveillance system, only in areas without video surveillance additional localization units, such as e.g. additional video cameras or localization units that are based on other technologies, may have to be installed. This may significantly reduce expenses required for a comprehensive and coherent monitoring of the complete environment. In an embodiment of the invention, the control unit of the localization system is arranged to match the plurality of location data provided by the plurality of localization units in terms of time and space.

Various localization units that may be based on various localization technologies may generate location data being heterogeneous in time. For some technologies and localization units a metering time period may be less than one second, for others it might be much longer. These time differences in the location data may have to be compensated by the control unit by timely matching the data. For the timely matching of data provided by different localization units for example the Network Time Protocol (NTP) may be used, which is a standardized networking protocol for clock synchronization between computer systems and which provides Coordinated Universal Time (UTC). However, if the objects being monitored are only moving at rather low speeds, small discrepancies and variations of the location data with respect to time may simply be neglected since at low speeds the total tracking accuracy may be dominated by the spatial resolution of the localization units. Small variations of the location data with respect to time may thus be neglected. For increasing the overall tracking accuracy and in case the objects being monitored are moving at rather high speeds, discrepancies and variations of the location data may be compensated by the control unit accounting for any latencies in a network the location data are transmitted by. The actual compensation of discrepancies and variations of data with respect to time may for example be achieved by taking an estimated speed of the objects being monitored and intrinsic timely inaccuracies of the location data into account that may result from an application of different localization technologies. Also a straightforward and basic comparison of all location data in terms of time with a reference system may be carried out by the control unit in order to timely match location data provided by different localization units.

However, since the control unit may compensate for time differences in location data the localization units themselves may not necessarily have to be synchronized in certain time intervals. Also drift effects with respect to time, i.e. a growing deviation of a time signal from a reference time signal, that may occur in certain localization units may be compensated by the control unit.

Besides the prescribed matching of the location data in terms of time, an additional matching in terms of space may enhance the flexibility of the localization system, because for example localization units operating in different coordinate systems may easily be combined within the same localization system. For instance, a localization unit operating in the World Geodetic System 1984 (WGS84), such as e.g. GPS-based localization units, may be combined with localization units operating in a metric coordinate system (such as e.g. a Cartesian coordinate system) and vice versa. This in turn means that the matching of the location data with respect to space may include a transformation of certain location data from one coordinate system to another one.

According to another embodiment of the invention, the control unit of the localization system is arranged to merge the plurality of location data provided by the plurality of localization units in order to derive the single position data of the at least one object being monitored. A merging of all location data considered plausible may significantly increase the accuracy and the spatial resolution of the localization system, since minor inaccuracies in certain location data of certain localization units may be balanced by others and, as a consequence, may cancel out.

It may be favored that the single position data at each time step as derived by the control unit by processing the location data of the various localization units are concisely displayed and presented for any personnel being in charge of the surveillance, such as e.g. airport authority, ground handlers or airlines. For this purpose, the localization system may be adapted to output the position data at arbitrary output terminals. Such output terminals may be, amongst others, a

Graphical User Interface (GUI) where the single position data may be displayed on a map or on several layers of a map, wherein each layer may contain certain information. For example, a layer of all buildings in an airport environment being monitored may be combined with a layer of all taxiways and a layer of passenger terminals. Moreover, the single position data of all monitored objects at each time step may also be stored in a database. This may allow the position data to be displayed at basically any place, e.g. via a local network or Internet access to the database. The single position data may also be distributed to and used by other systems, such as for example a video camera controlling system that may be able to use these data in order to automatically follow certain objects in certain areas.

Similar to the video camera controlling system, the output of the localization system may also be used and further processed by other external systems that may for instance serve to control certain processes related to the surveyed environment, such as e.g. fuelling an aircraft in an airport environment.

The prescribed flexibility in the output of position data of all objects being monitored may provide real-time information related to the surveillance of operational occurrences to any personnel, who in turn may get a reliable view of the overall situation. The localization system as proposed may therefore assist personnel in their daily decision-making process.

It should be noted here, that besides the output also the control unit itself may be configured and arranged in a decentralized manner to further contribute to the overall flexibility of the localization system. For this purpose, certain services or tasks accomplished by the control unit, such as for example the analysis of the location data with respect to plausibility, the matching of the location data in terms of time and space, or the merging of the location data, may be run as independent components that may be accessed by a network or via Internet. In other words, not all services and tasks of the control unit may have to be run on a central machine, and some services may even be run in a different continent. Two major advantages arise from this approach: First, services may be run in parallel on various machines providing a certain degree of redundancy and safety, because losing a few machines due to, for instance, network or hard disk failure may not pose a problem since other machines may be able to take over. Second, the localization system 10 may be scaled to an arbitrary extent, because adding a few machines may increase computing power. According to another embodiment of the invention, the control unit of the localization system accounts for at least one of a security and business regulation to analyze the single position data of the at least one object being monitored. This means a set of business and security or safety scenarios addressing operational requirements of the environment being surveyed may be incorporated into or may be accessed by the localization system. As a consequence, the localization system may be capable of analyzing certain predefined situations or scenarios and may take actions in response to certain business scenarios. It may, for example, be possible that certain restricted access areas may be locked for certain unauthorized individuals, whereas access may be granted for authorized individuals. The control unit of the localization system may therefore be capable of locking or unlocking certain doors, for instance.

The above-mentioned business and security regulations may comprise a variety of scenarios and regulations depending on the environment being monitored and the corresponding needs. An issue of importance in many environments may, for example, be a reliable alert system. Therefore, according to an embodiment of the invention, the control unit may trigger an alert upon an infringement of at least one of the security and business regulation. This may reduce manpower required for a broad surveillance of an environment, and consequently personnel expenses for the surveillance may be reduced. To further assist personnel responsible for surveying an environment, and to provide additional information on actions that may be required in response to an alert, the control unit may also be capable of triggering alerts with different levels of severity. This may further support the decision making process for personnel by providing a reliable view of the overall situation, whenever a safety or security event is reported.

In another embodiment of the invention, the control unit of the localization system calculates a movement path of an object being monitored in order to predict an infringement of the at least one of the security and business regulation. The calculation of a movement path may also comprise a determination of a speed of the object being monitored. This may be achieved by estimating the distance between at least two consecutive position data as derived by the control unit, and by taking the time interval between the at least two consecutive position data into account. As a consequence, the control unit may be able to identify certain movement patterns or events of objects being monitored.

One major advantage of the above-mentioned approach may be that an alert may be triggered before an actual violation or infringement of a security or business regulation might occur. Consequently, a personnel being in charge of surveying an environment may have the opportunity to react appropriately and take adequate actions in time. For example in case of an airport environment, including a runway or taxiway being monitored, accidents of aircrafts with ground vehicles or incursions of unauthorized personnel into restricted access areas may be avoided. According to another embodiment of the invention, the plurality of localization units of the localization system comprises at least one indoor and at least one outdoor localization unit. Thus, the localization system as proposed in previous and following sections may be operated in rather huge areas, such as for instance airport environments including terminal areas as well as taxiways and runways, or factories with indoor and outdoor areas. This may allow a comprehensive and coherent monitoring of objects in all areas of importance with a single localization system, and all information necessary for personnel being in charge of the surveillance may be gathered and displayed at any desired place. According to yet another embodiment of the invention, the control unit of the localization system is arranged to simultaneously process the plurality of location data provided by the indoor and the outdoor localization units in order to seamlessly monitor the at least one object being monitored both in an indoor and an outdoor area. Therefore, the localization system is capable of a seamless and gapless localization, tracking and monitoring of objects moving e.g. from indoors to outdoors or vice versa. In particular in busy environments, such as e.g. warehouses or factories, where it might be important that many movable objects may be

comprehensively tracked and monitored at the same time, it might be crucial to know the exact position of all objects at all times in order to effectively control, for example, a production process.

One aspect of the invention relates to a method for monitoring at least one object in real-time, wherein the method comprises a step of collecting at least one of a plurality of location data provided by at least one of a plurality of localization units. The wording "monitoring in real-time" may in this context be understood as

"monitoring as detailed and precisely as possible with respect to time". A metering time period, however, usually depends on intrinsic parameters of the localization units applied for monitoring. Preferably, this period may be in the order of one second or even below. Apart from the prescribed step of collecting location data, the method for monitoring further comprises a step of analyzing the at least one of the plurality of location data with respect to plausibility, a step of weighting the at least one of the plurality of location data with a quality factor that accounts for a spatial resolution of the at least one of the plurality of location data, and a step of mapping the at least one of the plurality of location data to derive a single position data of the at least one object being monitored. In this context "mapping location data" may not only comprise pinpointing a position on a map but also analyzing or interpreting the location data provided by the localization units against a background of a physical map of the environment being monitored. The above-mentioned method for monitoring objects further comprises a step of matching the plurality of the location data in terms of time and space, and a step of merging the plurality of location data provided by the plurality of localization units to derive the single position data of the at least one object being monitored. As described in the above in more detail, the matching of the location data in terms of time and space may enhance the flexibility of the localization system, because for example localization units operating in different coordinate systems or localization units transmitting data asynchronously may easily be combined within the same localization system. In addition, the merging of all location data considered plausible may significantly increase the accuracy and the spatial resolution of the localization system, since minor inaccuracies in certain location data of certain localization units may be balanced by others and, as a consequence, may cancel out.

Another aspect of the method for monitoring objects involves steps of accounting for at least one of a security and business regulation to analyze the plurality of location data, and triggering an alert upon an infringement of the at least one of the security and business regulation. Beyond that, a step of calculating a movement path of the at least one object being monitored in order to predict an infringement of the at least one of the security and business regulation may be involved in the method for monitoring objects.

This means a set of business and security or safety scenarios addressing operational requirements of the environment being surveyed may be taken into account in the method of monitoring objects. As a consequence, the localization system enforcing the prescribed method for monitoring may be capable of taking actions in response to a certain business or security scenario, such as for instance triggering an alert, granting or prohibiting access to restricted areas.

Further aspects of the invention relate to a computer program, which, when being executed by a processor, is adapted to carry out the steps of the method for the localization of objects in real-time, and a computer-readable medium, on which such computer program may be stored. A computer-readable medium may be a floppy disk, a hard disk, an USB (Universal Serial Bus) storage device, a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable

Programmable Read Only Memory) or a FLASH memory. A computer-readable medium may also be a data communication network, e.g. the Internet, which allows downloading a program code.

If technically possible but not explicitly mentioned, also combinations of embodiments of the invention described in the above and in the following may be embodiments of the method for the localization and the localization system.

Moreover, it has to be understood that features of the method as described in the above and in the following may be features of the localization system as described in the above and in the following. Vice versa, features of the localization system may be features of the method for the localization.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF DRAWINGS

Fig. 1 shows a schematic view of a localization system according to an embodiment of the invention. Fig. 2 shows a flow process chart of a method for the localization and monitoring of objects in real-time according to an embodiment of the invention. In principle, identical parts are provided with the same reference symbols in the figures.

Below, embodiments of the present invention are described in more detail with reference to the attached drawings.

DETAILED DESCRIPTION OF EMBODIMENTS

An objective of embodiments of the present invention may be to provide a comprehensive and coherent localization system for a real-time monitoring of a plurality of objects, such as for instance individuals, animals, semi- finished products or goods. The localization system proposed herein may be operated in a variety of environments of any size that may include indoor and outdoor areas, such as for example airports, hospitals, nursery schools, warehouses, stores, power plants, or factories. The localization system may also be installed and applied in kindergartens to monitor children. In particular because kindergartens may comprise indoor and outdoor areas, such as e.g. playgrounds, and because children may tend to constantly move from indoor to outdoor areas and vice versa, it may be difficult to keep track of all children in a surveyed kindergarten and to ensure that none of them may leave the kindergarten or get lost. Therefore, the localization system may help to

comprehensively keep track of all children at any time, and thus the system may help to ensure safety of all children with a minimum of personnel. The same may apply for instance to retirement or nursing homes, where elderly may be monitored, and where it may be important to ensure that no resident leaves the retirement home without permission. With the localization system proposed herein this task may be accomplished in a subtle way without generating a feeling of confinement for residents in a retirement home or for children in a kindergarten.

In the following paragraphs, a detailed description of an exemplary embodiment of the invention relating to the surveillance of an airport is given. Subsequently, certain aspects of other exemplary embodiments of the invention relating to livestock, production logistics, and stock logistics are briefly described. However, one skilled in the art will understand that all features described may equally apply to any other environment, in which a comprehensive monitoring of objects may be required.

Surveillance of an airport:

According to an embodiment of the invention, Fig. 1 shows a schematic view of a localization system 10 for monitoring objects in real-time in an airport environment. The localization system 10 comprises a plurality of localization units 12a, 12b, 12c, 12d, 12e for generating a plurality of location data relating to the objects being monitored, and a control unit 14 for collecting the plurality of location data provided by the plurality of localization units 12a, 12b, 12c, 12d, 12e. The control unit 14 is arranged to analyze and interpret the plurality of location data against a background of at least one map of the airport environment in order to derive a single position data of the objects being monitored, which can be displayed on the map at each time step.

For the surveillance of the airport environment, certain areas may be of particular importance, such as for example service roads, passenger terminals, taxiways, runways, or the apron area. The apron area is the area intended to accommodate aircrafts for purposes of embarkment, disembarkment, loading or unloading passengers, mail or cargo, fuelling and parking, or maintenance. Objects moving within these areas, and therefore objects being tracked and monitored by the localization system 10 may be for instance aircrafts, ground vehicles, staff, passengers, or any other mobile or immobile object. All areas and all objects of importance can be monitored with the localization system 10 at the same time.

In case of the surveillance of the airport, which is a rather complex environment, single localization technologies may not provide satisfactory and comprehensive performance in all areas of interest. Therefore, various localization technologies, including radio- and video-based technologies, are combined according to the exemplary embodiment and collaborate in order to provide a flexible localization system 10 suited for comprehensive and coherent real-time monitoring.

Apart from an operation of localization units 12a, 12b, 12c, 12d, 12e based on various technologies, an operation of localization units 12a, 12b, 12c, 12d, 12e suited for indoor and outdoor monitoring, respectively, can be required for the surveillance of an airport. The control unit 14, therefore, processes location data provided by both the indoor and the outdoor localization units in order to seamlessly monitor objects moving both in indoor and outdoor areas without any interruption or gaps in the tracking or monitoring.

A localization unit 12a appropriate for airport monitoring is the stand-alone Global Navigation Satellite System (GNSS). This satellite-based localization unit 12a is suited for outdoor monitoring and can be used together with a Wi-Fi communication device 12e to collect and transmit as frequently as possible (e.g. every second) the coordinates of an object being monitored, such as e.g. a vehicle's or aircraft's position. In addition, an Ultra Wide Band (UWB) system 12b can be operated providing immunity to multipath propagation and precision range measurement capabilities. The IEEE 802.15.4a UWB standard, for example, can implement precision location measurements when monitored objects are close to large metallic infrastructures, such as e.g. fences or hangars. Another localization unit 12c suited for airport monitoring is a Video Surveillance and Tracking System (VSTS). Such system can consist of multiple video cameras installed at predefined locations to preferably fully cover the area of interest with overlapping field of view, and it can be operated both in indoor and in outdoor areas. Video data collected by each camera can be processed by the control unit 14 to detect, track, locate, and monitor foreground objects within the area of interest. Also a system of Radio Frequency (RF) localization units 12d is suited for airport surveillance. Such system of RF localization units 12d can comprise mobile devices and antenna units equipped with smart-antennas mounted in the environment being surveyed. The RF system measures the position of a mobile device attached to an object being monitored, such as e.g. a vehicle, a passenger or staff in the area of interest. For example, a BlackFIR 2.4 or a BlackFIR 868 system may be applied as RF localization units 12d suited for both indoor and outdoor monitoring.

Apart from the above mentioned localization units 12a, 12b, 12c, 12d, 12e a variety of other localization units can be applied for a comprehensive monitoring of the airport. Amongst others, this may comprise GPS, D-GPS, or radar sensors, but also magnetic flux, infrared, inertial or even environmental sensors may be used (not explicitly shown in Fig. 1, but indicated by black dots).

A combination of various localization technologies, as described above, is likely to outperform any individual localization technology working alone, and shortcomings of one technology can be compensated by another technology. Therefore, the combination of various localization technologies, including video-based

technologies, such as e.g. the prescribed Video Surveillance and Tracking System, can ensure a homogeneous coverage of the full area of interest and can ensure that all objects being monitored are localized with at least one localization unit at any time.

An operation of a plurality of localization units 12a, 12b, 12c, 12d, 12e generating a plurality of location data relating to the objects being monitored sets certain demands on the control unit 14 and data processing techniques conducted by the control unit 14, in order to ensure a comprehensive monitoring in real-time, i.e. a monitoring in time intervals in the order of one second. All data transmitted by any localization unit 12a, 12b, 12c, 12d, 12e are therefore analyzed by the control unit 14 with respect to plausibility. This check in terms of plausibility can comprise accounting for any obstacle in the area of interest such as e.g. walls of buildings, fences or other physical boundaries that cannot be trespassed by the object being monitored. If, for example, a location data of one localization unit 12a, 12b, 12c, 12d, 12e of an airplane suggests a position inside a passenger terminal, whereas other localization units 12a, 12b, 12c, 12d, 12e indicate a position in the apron area, the location inside the terminal is likely to be erroneous, and thus is considered implausible by the control unit 14. As a consequence, the control unit 14 discards any location data regarded erroneous and analyzes and further processes only those location data considered plausible. For the analysis of the location data with respect to plausibility, for example, a model of all buildings and other obstacles in all areas of the airport being under surveillance can be implemented in the control unit 14 or can be accessed by the control unit 14. Apart from the plausibility check with respect to physical boundaries or obstacles, the control unit 14 is also arranged for checking plausibility of the location data with respect to time, and thus with respect to the speed of the objects being monitored. For example, if at one time step the location data of a ground vehicle indicates a position at one end of the airport and in a subsequent time step the location data indicates a position at the opposite end of the airport that may only had been reached if the vehicle had moved at an unrealistic speed, the location data is regarded implausible by the control unit 14 and is consequently discarded.

In order to make data generated by the various localization units 12a, 12b, 12c, 12d, 12e based on various localization technologies more comparable among one another, the control unit 14 weights the location data with a quality factor. For instance, this quality factor can be distributed in an interval of [0; 1], and all location data are weighted linearly, i.e. with a linear weighting function. However, basically any other weighting, such as for example a logarithmic or an exponential weighting (i.e. with a logarithmic or exponential weighting function), may be used. The quality factor can account for a spatial resolution of location data generated by various localization units 12a, 12b, 12c, 12d, 12e. For this purpose, an intrinsic quality factor of each localization unit 12a, 12b, 12c, 12d, 12e can be implemented in the control unit 14 or can be accessed by the control unit 14. Some localization units 12a, 12b, 12c, 12d, 12e can also transmit metadata in addition to the location data including information about the spatial resolution of currently transmitted location data, according to e.g. a temporary signal strength of the localization unit. The control unit can in turn determine a quality factor for the location data from the metadata provided by a localization unit 12a, 12b, 12c, 12d, 12e. The concept of weighting the location data with a quality factor can be advantageous for the derivation and the mapping of the single position of each object being monitored, because for example location data with higher accuracy can be considered more important by the control unit 14, and thus can have a higher (or lower) weight reflected by a higher (or lower) quality factor. By accounting for a hierarchy in the location data with respect to their quality and accuracy, the control unit 14 can derive and map the single position of each object being monitored more precisely. Hence, this approach contributes to increase localization accuracy, achieving a better coverage range with the same amount of technical equipment. In order to handle the plurality of location data provided by the plurality of localization units 12a, 12b, 12c, 12d, 12e for all objects being monitored in the airport environment, the control unit 14 is arranged to match the plurality of location data in terms of time and space. Location-based data tend to be strongly correlated in both time and space. For instance, position and speed data measured by one localization unit 12a based on a certain localization technology may be highly correlated to data collected by another adjacent localization unit 12b that may be based on a different technology. Similarly, readings observed at one time instant are highly indicative of the readings observed at the next time instant. This may be particularly relevant because for example airport stakeholders, such as e.g. airport authority, ground handlers, or airlines, may not be interested in individual readings in time or space, but rather in application level concepts of temporal and spatial granularities.

However, various localization units 12a, 12b, 12c, 12d, 12e, that are based on various localization technologies, can generate location data being heterogeneous in time. For some technologies and localization units, a metering time period may be less than one second, whereas it might be much longer for others. These time differences in the localization units' data need to be compensated by the control unit 14, wherein e.g. airport stakeholders' concerns in terms of a desired metering time period can be met.

Apart from the matching of location data in terms of time, a matching of the location data in terms of space can be required, particularly in the context of airport surveillance. The matching of location data in terms of space can enhance the flexibility of the localization system 10, because localization units 12a, 12b, 12c, 12d, 12e operating e.g. in different coordinate systems can easily be combined within the same localization system 10.

According to the exemplary embodiment, the satellite-based localization unit 12a operates in the World Geodetic System 1984 (WGS84) and is combined with localization units 12d operating in a metric coordinate system. This in turn requires the control unit 14 to transform location data from one coordinate system to another one in order to compare, match and simultaneously process the data from the different localization units 12a, 12d.

In order to display location information of all objects being monitored in the airport environment in a concise and clear manner, the control unit 14 of the localization system 10 is arranged to merge the plurality of location data provided by the plurality of localization units 12a, 12b, 12c, 12d, 12e to derive a single position data of each object being monitored. This means at each time step the plurality of location data are fused or condensed by the control unit 14 into a single position data. The merging of all location data considered plausible and matched with respect to time and space can significantly increase the accuracy, the spatial resolution, and the reliability of the localization system 10, since minor inaccuracies in certain location data of certain localization units 12a, 12b, 12c, 12d, 12e can be balanced by others and consequently can cancel out.

In particular in the exemplary embodiment of the airport surveillance, it can be favored that the single position data at each time step as derived by the control unit 14 by matching and merging the location data of the various localization units 12a, 12b, 12c, 12d, 12e are concisely displayed and presented for personnel being in charge of the surveillance, such as e.g. airport authority, ground handlers or airlines. For this purpose, the localization system 10 is adapted to output the position data at arbitrary output terminals 16a, 16b, 16c. An output terminal 16a is, for example, a Graphical User Interface (GUI) where the single position data is displayed on a map or on several layers of a map, wherein each layer may contain certain information. For example, a layer of all buildings in the airport environment being monitored may be combined with a layer of all taxiways and a layer of the apron. The single position data of all monitored objects at each time step can also be stored in a database 16b. This allows the position data to be displayed at basically any other place, e.g. via a local network or Internet access to the database 16b. The single position data can also be used by other systems, such as, for example, a video camera controlling system 16c that can use these data in order to automatically follow certain objects in certain areas. Similar to the video camera controlling system, the output of the localization system 10 can also be used and further processed by other external systems (not explicitly shown in Fig. 1, but indicated by black dots) that for instance may serve to control certain processes related to the surveyed environment, such as e.g. fuelling an aircraft.

The prescribed flexibility in the output of position data of all objects being monitored can provide real-time information related to the surveillance of operational occurrences to any personnel, who in turn can get a reliable view of the overall situation. The localization system 10 as proposed can therefore assist personnel in their daily decision-making process.

It should be noted here that besides the output, also the control unit 14 itself may be configured and arranged in a decentralized manner to further contribute to the overall flexibility of the localization system 10. For this purpose, certain services or tasks accomplished by the control unit 14, such as for example the analysis of the location data with respect to plausibility, the matching of the location data in terms of time and space, or the merging of the location data can be run as independent components that can be accessed by a network or via Internet. In other words, not all services and tasks of the control unit 14 have to be run on a central machine, and some services can even be run in a different continent. Two major advantages arise from this approach: First, services can be run in parallel on various machines providing a certain degree of redundancy and safety, because losing a few machines due to, for instance, network or hard disk failure may not pose a problem since other machines can take over. Second, the localization system 10 can be scaled to an arbitrary extent, because adding a few machines only increases computing power.

In particular in the complex airport environment, there is a need for a coordination of multiple activities occurring simultaneously. This can require a continuous control of all ground movements, in particular during taxi operations. A lack of context awareness and controllability can be a causal factor for business or security regulation infringements. Therefore, the control unit 14 of the localization system 10 accounts for security and business regulations to analyze the single position data of the objects being monitored. Furthermore, the control unit 14 is adapted for automatically triggering an alert upon an infringement of any security or business regulation. According to a severity level of an observed infringement of a security or business regulation, the control unit 14 can also trigger alerts with different levels of severity and report corresponding alert messages to any personnel being involved.

In order to predict an infringement of a security or business regulation, the control unit 14 can also calculate a movement path of the monitored objects and trigger an alert before an actual violation or infringement of a security or business regulation might occur, allowing personnel to take appropriate actions in time. The calculation of a movement path can also comprise a determination of a speed of the objects being monitored by estimating the distance between at least two consecutive position data and by taking the time interval between at least two consecutive position data into account.

Security or business regulations and scenarios taken into account by the control unit 14 may, for instance, comprise safety procedures for aircraft maneuvering, accounting for specified clearances and established procedures to enter, move within, or depart from apron areas. Moreover, an automatic identification or detection of situations or events related to speed limit thresholds, safety infringements, detection of incursions of unauthorized personnel into restricted access areas, or any other location-based occurrence related to airport resources, staff or passengers can be taken into account and surveyed by the control unit 14. Thus, the control unit 14 can trigger an alert or report a meaningful alert message associated with a detected conflict with a security or business regulation and, within an adequate time, can bring it to the attention of any personnel being in charge of the surveillance or even to the attention of a vehicle driver being involved. Such capabilities of a comprehensive alert system can, therefore, provide airport stakeholders with a new way to detect and analyze situations, events, or occurrences in real-time and can enable them to adjust control actions according to a severity level of an observed infringement of a security or business regulation. In order to further provide information and metadata about the objects being monitored, the localization system 10 can additionally be connected to external services and systems (not explicitly shown in Fig. 1). Hence, location-based data for each observed object can be coherently correlated with metadata from external services, such as e.g. an existing flight information data system. This can, for example, allow to receive airport operational data in order to obtain flight data in real-time.

Fig. 2 shows a flow process chart of a method for the localization and monitoring of objects in real-time according to an embodiment of the invention.

The method comprises a step S10, in which the control unit 14 of the localization system 10 collects location data provided by the plurality of localization units 12a, 12b, 12c, 12d, 12e; and a step S12, in which the control unit 14 analyzes the location data with respect to plausibility. All location data regarded implausible by the control unit 14 can be discarded in further steps of the method for the localization and monitoring of objects.

All location data regarded plausible by the control unit 14 are weighted with a quality factor in step SI 4, wherein the quality factor may account for a spatial resolution of the location data. In step SI 6, the control unit 14 of the localization system 10 matches the location data in terms of time and space; and in step SI 8, these timely and spatially matched location data are merged to derive a single position data of each object being monitored.

In step S20, the single position data of each object is mapped, wherein "mapped" in this context may not only comprise pinpointing the single position of each object on a map, but also analyzing or interpreting the position data against a background of a physical map.

Additionally, the control unit 14 of the localization system 10 can calculate a movement path for each object being monitored in step S22. The calculation of a movement path can also comprise a determination of a speed of each object being monitored, which can be achieved by estimating the distance between at least two consecutive position data as derived by the control unit 14, and by taking the time interval between at least two consecutive position data into account.

Furthermore in step S24, the control unit 14 can account for security and business regulations valid for the environment of interest in order to analyze the position data of each object being monitored.

In step S26, the control unit 14 of the localization system can trigger an alert upon an infringement of a security or business regulation, or upon an infringement of a security or business regulation predicted by accounting for the movement path calculated in step S22.

The functional modules or steps of the prescribed method may be implemented as programmed software modules or procedures, respectively; however, one skilled in the art will understand that the functional modules may be implemented fully or partially in hardware.

Surveillance of a livestock:

The localization system 10 as described in the above and in the following may also advantageously be applied for a surveillance of a livestock, wherein animals can be the objects of interest that are comprehensively monitored. In the livestock, the animals, such as e.g. cows or sheep, can be equipped with for instance Radio Frequency localization units 12d that can comprise mobile devices and antenna units equipped with smart antennas mounted in the environment being surveyed. Such RF system can measure the position of a mobile device attached to the animals. For example, a BlackFIR 2.4 or a BlackFIR 868 system may be applied as RF localization units 12d for both indoor and outdoor monitoring. In particular, due to the fact that permanent transitions of the animals between indoor and outdoor areas may have to be monitored, such RF localization units 12d may be suited. Also a Video Surveillance and Tracking System (VSTS) 12c can be used for the livestock surveillance, which can consist of multiple video cameras installed at predefined locations to preferably fully cover the area of interest with overlapping field of view. A VST system can also be operated both in indoor and in outdoor areas. The video data collected by each camera can be processed by the control unit 14 to detect, track, locate, and monitor animals moving in foreground within the area of interest. The control unit 14 can process location data provided by both the indoor and the outdoor localization units 12d, 12c to take care for a seamless and gapless localization, tracking, and monitoring between indoor and outdoor.

In addition, any other localisation and sensing technology may be used for the surveillance of the livestock. This may also include sensors for a continuous monitoring of various vital parameters, such as e.g. heart rate or blood pressure sensors, or sensors for detecting concentrations of certain nutrients in animals' blood. In this way, it may be possible to constantly observe and monitor the health of the animals.

Furthermore, the control unit 14 can account for any livestock-specific or company- specific business regulation, relating e.g. to animal welfare legislation or other established procedures in animal farming. Such company-specific business regulations can for example comprise certain thresholds for parameters of interest, such as a threshold for blood pressure or heart rate of the animals. These thresholds can be implemented in or can be accessed by the control unit 14 in order to trigger an alert if the parameter of interest of an animal exceeds or drops below the

corresponding threshold. According to an importance of a surveyed parameter also alerts with different levels of severity can be triggered by the control unit 14.

Also a sophisticated way of geo-fencing can be implemented in the localization system 10 for the livestock surveillance, including actions to be taken automatically by the control unit 14. For example animals can be equipped with electric shock devices, and the control unit 14 can automatically induce an electric shock to an animal crossing, passing by or coming close to a certain boundary of the

environment. The boundaries can also be set directly in the control unit 14 or they can be accessed by the control unit 14, e.g. by accessing a database.

Apart from the above-mentioned, also an analysis of herding behaviour may be carried out using the capabilities of the localization system 10 in the livestock. This can include a gathering of data relating to herding behaviour, such as feeding or nursing behaviour, or it can even include for example detection of sick animals when an animal behaves unnaturally and e.g. dissociates from the herd. Surveillance of production logistics:

Also in the field of production logistics the localization system 10 can

advantageously be operated, wherein the objects being monitored may for instance comprise components of a certain product, semi- finished goods or parts, devices, or any other item or object required or used in a manufacturing process, such as e.g. vehicles, turbines, or other parts of aircrafts.

Depending on the environment being monitored and depending on the characteristics of the production, a variety of localization units for indoor and outdoor monitoring may be operated for the surveillance of the manufacturing process. Potential candidates for localization units can for example be GPS, D-GPS, Global Navigation Satellite System (GNSS), Automatic Dependent Surveillance-Broadcast (ADS-B), multi-lateration (MLAT) techniques, wireless local area network (WLAN), radar, near-range radar networks, ultra- wide band (UWB), BlackFIR 2.4, BlackFIR 868, inertial sensors, closed circuit television (CCTV) with video analytics, magnetic flux sensors, or infrared sensors.

In particular in complex manufacturing processes, such as for instance in automotive or aviation industries, the efficiency of the process may be increased by the localization system 10, due to the fact that the control unit 14 is capable of accounting for business regulations when analyzing the position data provided by the localization units. Accordingly, the integration of business regulations in the localization system 10 may allow an automation of several important portions of the production process. Amongst others, this can comprise a correct assignment of parts to a production order, a consistent transfer of data relating to certain parts to production facilities and their controlling, confirmation of quality-related data to contracting IT systems, continuous provision of information about a status of an ongoing production process (e.g. car bodies in a finishing or assembling process) or about a status of customer sales in real-time. The localization system 10 can be arranged in a way that all data concerning the manufacturing process can be accessed by any personnel in real-time, for instance via a website.

Surveillance of stock logistics:

Also in the field of stock logistics the localization system (10) can be operated, wherein particularly the seamless and gapless monitoring of all objects of interest simultaneously in indoor and outdoor areas of the environment being monitored can be advantageous. The localization system 10 may, for instance, be used in depots, stock rooms, or production sites that may comprise outdoor areas.

Potential objects being monitored may, for example, be certain products, goods or devices, or any other item or object required or used at the site, such as e.g. floor- borne vehicles or industrial trucks.

Also in the surveillance of stock logistics, a variety of localization units for indoor and outdoor monitoring can be operated, such as for example GPS, D-GPS, Global Navigation Satellite System (GNSS), Automatic Dependent Surveillance-Broadcast (ADS-B), multi-lateration (MLAT) techniques, wireless local area network (WLAN), radar, near-range radar networks, ultra-wide band (UWB), BlackFIR 2.4, BlackFIR 868, inertial sensors, closed circuit television (CCTV) with video analytics, magnetic flux sensors, or infrared sensors.

By implementing business regulations defining, for example, a correct position of any product within a depot or stock room, it may be automatically ensured that e.g. all products or goods in the depot are at their desired place. Furthermore, by using appropriate localization technologies and localization units, such as e.g. IR or RF sensors, a manual scanning of palettes to track their location may be avoided. This can reduce personnel expenses, and a comprehensive profile of the stock can be generated automatically and constantly transferred to a service center in real-time. Also an ordering or replacement process for certain products may automatically be accomplished by the localization system 10 by implementing and defining corresponding business regulations and actions.

While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art and practising the claimed invention from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or controller, or other unit may fulfil the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. Localization system (10) for monitoring at least one object in real-time, the localization system (10) comprising:

a plurality of localization units (12a, 12b, 12c, 12d, 12e) for generating a plurality of location data relating to the at least one object being monitored; and

a control unit (14) for collecting the plurality of location data provided by the plurality of localization units (12a, 12b, 12c, 12d, 12e);

wherein the control unit (14) is arranged to map at least one of the plurality of location data to derive a single position data of the at least one object being monitored.

2. The localization system (10) according to claim 1, wherein the control unit (14) is arranged to analyze the at least one of the plurality of location data with respect to plausibility.

3. The localization system (10) according to claim 1 or 2, wherein the control unit (14) weights the at least one of the plurality of location data with a quality factor accounting for a spatial resolution of the at least one of the plurality of location data.

4. The localization system (10) according to any of the preceding claims, wherein the plurality of localization units (12a, 12b, 12c, 12d, 12e) comprises units based on various localization technologies.

5. The localization system (10) according to claim 4, wherein at least one of the plurality of localization units is radio-based.

6. The localization system (10) according to claim 4 or 5, wherein at least one of the plurality of localization units is video-based.

7. The localization system (10) according to any of the preceding claims, wherein the control unit (14) is arranged to match the plurality of location data provided by the plurality of localization units (12a, 12b, 12c, 12d, 12e) in terms of time and space.

8. The localization system (10) according to any of the preceding claims, wherein the control unit (14) is arranged to merge the plurality of location data provided by the plurality of localization units (12a, 12b, 12c, 12d, 12e) to derive the single position data of the at least one object being monitored.

9. The localization system (10) according to any of the preceding claims, wherein the control unit (14) accounts for at least one of a security and business regulation to analyze the single position data of the at least one object being monitored.

10. The localization system (10) according to claim 9, wherein the control unit (14) triggers an alert upon an infringement of the at least one of the security and business regulation.

11. The localization system (10) according to claim 9 or 10, wherein the control unit (14) calculates a movement path of the at least one object being monitored, in order to predict an infringement of the at least one of the security and business regulation.

12. The localization system according to any of the preceding claims, wherein the plurality of localization units (12a, 12b, 12c, 12d, 12e) comprises at least one indoor and at least one outdoor localization unit.

13. The localization system (10) according to claim 12, wherein the control unit (14) processes the plurality of location data provided by the at least one indoor and the at least one outdoor localization units to seamlessly monitor the at least one object being monitored in an indoor and an outdoor area.

14. Method for monitoring at least one object in real-time, the method comprising: collecting (S 10) at least one of a plurality of location data provided by at least one of a plurality of localization units (12a, 12b, 12c, 12d, 12e);

analyzing (SI 2) the at least one of the plurality of location data with respect to plausibility;

weighting (SI 4) the at least one of the plurality of location data with a quality factor that accounts for a spatial resolution of the at least one of the plurality of location data; and

mapping (S20) of the at least one of the plurality of location data to derive a single position data of the at least one object being monitored.

15. The method according to claim 14, further comprising steps of:

matching (SI 6) of the plurality of the location data in terms of time and space; and merging (SI 8) the plurality of location data provided by the plurality of localization units (12a, 12b, 12c, 12d, 12e) to derive the single position data of the at least one object being monitored.

16. The method according to any of claims 14 or 15, further comprising steps of: accounting for at least one of a security and business regulation (S24) to analyze the plurality of location data; and

triggering an alert upon an infringement of the at least one of the security and business regulation (S26).

17. The method according to any of claims 14, 15, or 16, further comprising a step of

calculating a movement path (S22) of the at least one object being monitored in order to predict an infringement of the at least one of the security and business regulation.

18. A computer program, which, when being executed by a processor, is adapted to carry out the steps of the method of one of claims 14, 15, 16, or 17.

19. A computer-readable medium, in which a computer program according to claim 18 is stored.