WO2023193923A1

WO2023193923A1 - Maritime traffic management

Info

Publication number: WO2023193923A1
Application number: PCT/EP2022/059327
Authority: WO
Inventors: Jordi DANIËLS; Edwin FLAPPER
Original assignee: Saab Technologies B.V.
Priority date: 2022-04-07
Filing date: 2022-04-07
Publication date: 2023-10-12

Abstract

The present invention relates to methods and systems for tracking vessels in maritime traffic management applications. The method comprises obtaining radar data comprising at least one radar image of a radar device arranged at a port. The radar data is associated with one or more scenario parameters indicative of a situational context of the radar data. The method further comprises selecting a set of machine-learning models out of a plurality of machine-learning models based on the one or more scenario parameters. Each machine-learning model of the plurality of machine-learning models is trained with training data related to a specific situational context, and the selected set of machine-learning models comprises an object detection machine-learning model, a feature extraction machine-learning model, and an object tracking machine-learning model. The method further comprises processing the radar data by means of the selected object detection machine-learning model in order to generate a first output indicative of one or more detected vessels in the at least one radar image. Moreover, the method comprises processing the first output by means of the selected feature extraction machine-learning model in order to generate a second output indicative of one or more predicted features of the one or more detected vessels in the at least one radar image. Further, the method comprises processing the second output by means of the selected object tracking machine-learning model in order to generate a third output indicative of an updated tracking state of each vessel of the one or more detected vessels in the at least one radar image.

Description

Title

MARTIME TRAFFIC MANAGEMENT

TECHNICAL FIELD

The present invention relates to the general field of Maritime Traffic Management (MTM) systems, and in particular to object detection and tracking in MTM applications.

BACKGROUND

Maritime traffic management system in harbours generally use radar technical and Automatic Identification System (AIS) to detect and track ships and other vessels. Additionally, video cameras may be employed, but these are mainly used for visual confirmation by the traffic management operators and not for automatic detection and tracking.

Radar systems can be described as object-detection systems that uses electromagnetic waves to determine the range, angle, or velocity of external objects. As mentioned, it may be used to detect and tracking ships, but are conventionally also used to detect aircraft, motor vehicles, weather formations, as well as terrain. Conventionally, a radar system includes a transmitter producing electromagnetic waves in the radio or microwaves domain, a transmitting antenna, a receiving antenna (often the same antenna is used for transmitting and receiving), a receiver and processor to determine properties of the object(s). Electromagnetic waves (pulsed or continuous) from the transmitter reflect off the object and return to the receiver, giving information about the object's location and speed.

In conventional target trackers for maritime applications that are based on radar data, various filtering techniques, such as Kalman filters and variations thereof, are used as the main tracking algorithm. Moreover, these filters are often combined in a multi-model (e.g. Bayesian networks or Interacting Multiple Models). With the statistic estimates of the model, a prediction is made for the new position of a vessel (x seconds into the future), when the new detection is made by the radar it is compared with the previously made prediction.

However, a drawback with such conventional filter-based target trackers is that it is problematic to update/alter one part of the model without negative impact on other parts. Moreover, other drawbacks with these conventional filter-based target trackers include difficulties to be tuned/adapted to new contexts/scenarios, low accuracy, high maintenance needs, complex architectures, and problems associated with unexpected data loss.

There is therefore a need in the art for new target detection and tracking solutions in maritime traffic management applications.

SUMMARY

It is therefore an object of the present invention to provide a computer-implemented method fortrackingvessels for maritime traffic management, a computer program product, a computer- readable storage medium, a system, a remote server, and a cloud environment, which alleviate all or at least some of the above-discussed drawbacks of presently known systems and methods.

It is also an object to provide an architecture for tracking targets, such as ships, for maritime applications based on radar images that is more accurate, more flexible, and more scalable than presently known systems and methods.

This object is achieved by means of a computer-implemented method for tracking vessels for maritime traffic management, a computer program product, a computer-readable storage medium, a system, a remote server, and a cloud environment as defined in the appended independent claims. The term exemplary is in the present context to be understood as serving as an instance, example or illustration.

According to a first aspect of the present invention, there is provided a computer-implemented method for tracking vessels for maritime traffic management (MTM) applications. The method comprises obtaining radar data comprising at least one radar image of a radar device arranged at a port. The radar data is associated with one or more scenario parameters indicative of a situational context of the radar data. The method further comprises selecting a set of machinelearning models out of a plurality of machine-learning models based on the one or more scenario parameters. Each machine-learning model of the plurality of machine-learning models is trained with training data related to a specific situational context, and the selected set of machine-learning models comprises an object detection machine-learning model, a feature extraction machine-learning model, and an object tracking machine-learning model. The method further comprises processing the radar data by means of the selected object detection machine-learning model in order to generate a first output indicative of one or more detected vessels in the at least one radar image. Moreover, the method comprises processing the first output by means of the selected feature extraction machine-learning model in order to generate a second output indicative of one or more predicted features of the one or more detected vessels in the at least one radar image. Further, the method comprises processing the second output by means of the selected object tracking machine-learning model in order to generate a third output indicative of an updated tracking state of each vessel of the one or more detected vessels in the at least one radar image.

According to a second aspect of the present invention, there is provided computer program product comprising instructions which, when the computer program is executed by a processing system, causes the processing system to carry out the method according to any one of the embodiments disclosed herein. With this aspect of the invention, similar advantages and preferred features are present as in the other aspects of the invention.

According to a third aspect of the present invention, there is provided a (non-transitory) computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a processing system, the one or more programs comprising instructions for performing the method according to any one of embodiments disclosed herein. With this aspect of the invention, similar advantages and preferred features are present as in the other aspects of the invention.

The term "non-transitory," as used herein, is intended to describe a computer-readable storage medium (or "memory") excluding propagating electromagnetic signals, but are not intended to otherwise limit the type of physical computer-readable storage device that is encompassed by the phrase computer-readable medium or memory. For instance, the terms "non-transitory computer readable medium" or "tangible memory" are intended to encompass types of storage devices that do not necessarily store information permanently, including for example, random access memory (RAM). Program instructions and data stored on a tangible computer-accessible storage medium in non-transitory form may further be transmitted by transmission media or signals such as electrical, electromagnetic, or digital signals, which may be conveyed via a communication medium such as a network and/or a wireless link. Thus, the term "non- transitory", as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM).

According to a fourth aspect of the present invention, there is provided a system for tracking vessels for maritime traffic management (MTM) applications. The system comprises control circuitry configured to obtain radar data comprising at least one radar image of a radar device arranged at a port. The radar data is associated with one or more scenario parameters indicative of a situational context of the radar data. The control circuitry is further configured to select a set of machine-learning models out of a plurality of machine-learning models based on the one or more scenario parameters. Each machine-learning model of the plurality of machine-learning models is trained with training data related to a specific situational context, and the selected set of machine-learning models comprises an object detection machine-learning model, a feature extraction machine-learning model, and an object tracking machine-learning model. The control circuitry is further configured to process the radar data by means of the selected object detection machine-learning model in order to generate a first output indicative of one or more detected vessels in the at least one radar image. Moreover, the control circuitry is configured to process the first output by means of the selected feature extraction machinelearning model in order to generate a second output indicative of one or more predicted features of the one or more detected vessels in the at least one radar image. Furthermore, the control circuitry is configured to process the second output by means of the selected object tracking machine-learning model in order to generate a third output indicative of an updated tracking state of each vessel of the one or more detected vessels in the at least one radar image. With this aspect of the invention, similar advantages and preferred features are present as in the other aspects of the invention.

According to a fifth aspect of the present invention, there is provided a remote server comprising a system for tracking vessels for maritime traffic management (MTM) applications according to any one of the embodiments disclosed herein. With this aspect of the invention, similar advantages and preferred features are present as in the other aspects of the invention.

According to a sixth aspect of the present invention, there is provided a cloud environment comprising one or more remote servers in accordance with any one of the embodiments disclosed herein. With this aspect of the invention, similar advantages and preferred features are present as in the other aspects of the invention.

Further embodiments of the invention are defined in the dependent claims. It should be emphasized that the term "comprises/comprising" when used in this specification is taken to specify the presence of stated features, integers, steps, or components. It does not preclude the presence or addition of one or more other features, integers, steps, components, or groups thereof.

An advantage of some embodiments is that the vessel tracking system can be updated to handle new situational contexts/scenarios without inadvertently impairing the performance of other parts of the system. Thereby providing for a more robust and scalable vessel tracking system.

An advantage of some embodiments is that quickly changing situational contexts are easily responded to, making the vessel tracking system more versatile and accurate in dynamic environments.

An advantage of some embodiments is that the vessel tracking system can be updated for performance improvement in an automated manner leading quicker and more cost effective software update cycles.

These and other features and advantages of the present invention will in the following be further clarified with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

Further objects, features and advantages of embodiments of the invention will appear from the following detailed description, reference being made to the accompanying drawings, in which:

Fig. 1 is a schematic flow chart representation of a method for tracking vessels for maritime traffic management in accordance with some embodiments.

Fig. 2 is a schematic block diagram representation of a system for tracking vessels for maritime traffic management in accordance with some embodiments. Fig. 3 are schematic block diagram representations of a system for tracking vessels for maritime traffic management, a remote server comprising such a system, and a cloud environment comprising one or more such remote servers, in accordance with some embodiments.

DETAILED DESCRIPTION

In the following detailed description, embodiments of the present invention will be described. However, it is to be understood that features of the different embodiments are exchangeable between the embodiments and may be combined in different ways, unless anything else is specifically indicated. Even though in the following description, numerous specific details are set forth to provide a more thorough understanding of the present invention, it will be apparent to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well known constructions or functions are not described in detail, so as not to obscure the present invention.

Those skilled in the art will appreciate that the steps, services and functions explained herein may be implemented using individual hardware circuitry, using software functioning in conjunction with a programmed microprocessor or general purpose computer, using one or more Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGA) and/or using one or more Digital Signal Processors (DSPs). It will also be appreciated that when the present disclosure is described in terms of a method, it may also be embodied in one or more processors and one or more memories coupled to the one or more processors, wherein the one or more memories store one or more programs that perform the steps, services and functions disclosed herein when executed by the one or more processors.

The herein proposed (marine) vessel tracking methods and systems utilizes a "distributed-AI" approach for providing a new and improved vessel tracker for maritime traffic management applications. In more detail, the tracking system as proposed herein, is built on a concept of using multiple machine-learning ML models, where each ML model has been trained for a specific situational context or scenario so to be able to provide accurate tracking in real-time or at least close to real-time in any situational context in a more cost-effective and less labor- intensive manner than previously known. The tracking system is based on a "modular" platform, which selects the most appropriate ML models for any given situation, and also enables for automatically updating the available ML models individually, thereby mitigating problems associated with complex and tedious updating processes where one fix at one part of the system leads to an impaired performance at another part of the system.

Moreover, the herein proposed methods and systems utilizes a three-stage vessel tracking setup, where a first ML model is used for object detection, whose output is fed as input to a second ML model that is used for feature extraction, and whose output in turn is fed as input to a third ML model is used for object tracking. Thereby any mistakes generated during the entire process can easily be contained and identified and subsequently handled without running the risk of inadvertently introducing errors in other parts of the system that are performing well. Moreover, during the training of the ML models noise may be introduced together with the input at each stage so to improve robustness of the cooperation between the ML models.

Fig. 1 is a schematic flowchart representation of a method S100 for tracking (marine) vessels (e.g. ships) for maritime traffic management (MTM) applications, in accordance with some embodiments. In particular some embodiments of the method S100 provides for target tracking in MTM applications with high accuracy for various situational contexts, i.e. for different scenarios.

The method S100 comprises obtaining S101 radar data comprising at least one radar image of a radar device arranged at a port, wherein the radar data is associated with one or more scenario parameters indicative of a situational context of the radar data. In more detail, the scenario parameters may for example include a location of the radar device (e.g. land-based or sea-based), a radar profile, a type of radar transceiver, a type of antenna/antennas of the radar device, range bins of the radar device, pulse length of the radar device, tracking area of the radar device, tracking range of the radar device, coverage of the radar device (e.g. single-/multi- sensor), and so forth. Thus, the radar data generated by a radar device at a first port has a first situational context, while the radar data generated by a different radar device at a second port may have a second situational context. The term obtaining is herein to be interpreted broadly and encompasses receiving, retrieving, collecting, acquiring, and so forth. The radar data may for example be in the form of Category 240 (CAT240) radar video data. The method S100 further comprises selecting S102 a set of machine-learning (ML) models out of a plurality of machine-learning models based on the one or more scenario parameters. Accordingly, each machine-learning model out of the plurality of machine-learning models is trained with training data related to a specific situational context. Moreover, the selected set of machine-learning models comprises an object detection machine-learning model, a feature extraction machine-learning model, and an object tracking machine-learning model.

The machine-learning models (may also be referred to as machine-learning algorithms, neural networks, and so forth) are implemented in some embodiments using publicly available suitable software development machine learning code elements, for example, such as those which are available in Python, Keras and TensorFlow or in any other suitable software development platform, in any manner known to be suitable to someone of ordinary skill in the art. Each machine-learning algorithm may be included in a machine-learning module comprises a body of computer code held in memory, which when executed, processes input data using a machinelearning model.

In other words, the method S100 comprises selecting S102 one ML model from each of three types or groups of ML models, namely, one object detection machine-learning model, one feature extraction machine-learning model, and one object tracking machine-learning model. Thus, in some embodiments, the plurality of ML models comprises a first machine-learning model group comprising a plurality of object detection machine-learning models, a second machine-learning model group comprising a plurality of feature extraction machine-learning models, and a third machine-learning model group comprising a plurality of tracking machinelearning models.

Moreover, in some embodiments, the selection S102 comprises extracting S107 a first set of machine-learning models based on the one or more scenario parameters, where the extracted first set of machine-learning models are machine-learning models trained with training data related to corresponding situational context as the situational context of the obtained radar data. Then, each machine-learning model of the extracted S107 first set of machine-learning models are tested S108 using earlier instances of the obtained radar data as input (i.e. historical radar data from the same or a corresponding radar device associated with the same situational context), and an accuracy score (may also be referred to as a performance score) of the output of each tested S108 machine-learning model is evaluated S109 in order to select a sub-set of machine-learning models associated with the highest accuracy scores in each machine-learning model group, the selected subset defining the selected set of machine-learning models. The "earlier instances" of the obtained radar data may for example be an aggregated set of radar images obtained from the radar device, or a set of radar data associated with the same one or more scenario parameters indicative of the (same) situational context. The evaluation S109 may further comprise comparing the output of each tested S108 machine-learning model against a ground-truth, which may be in the form of Automatic Identification System (AIS) data associated with the earlier instances of the obtained radar data. However, the ground-truth may alternatively, or additionally, comprise data from external sources, e.g. user feedback, camera images, etc.

In other words, one starts with the plurality of ML models for all of the situational context that are available. These models may then be filtered S107 based on some simple settings of the situational context of the obtained S101 radar data, e.g. based on some predefined rule-based logic. This may for example be that the obtained S101 radar data originates from an inland radar device, meaning that one can filter out all the ML models that are trained on training data originating from sea-based radars as these will not be applicable to the current situational context. The next step may then be to test all of the remaining ML models on a small dataset from the new situational context (e.g. on earlier instances of the obtained S101 radar data). Then the best models are selected as long as they score above a certain threshold, i.e. the accuracy score of their outputs is above a threshold. If they none of the available ML models meet that threshold, i.e. the obtained S101 radar data relates to a new situational context, one may instead use a set of "generic ML models" and employ transfer learning using a bigger dataset of this "new" situational context.

Further, the method S100 comprises processing S103 the obtained S101 radar data by means of the selected object detection machine-learning model in order to generate a first output indicative of one or more detected vessels in the at least one radar image. In some embodiments, the object detection machine-learning model is configured to generate a bounding box representation of each detected vessel in the at least one radar image, each bounding box representation comprising a confidence score for that bounding box. The object detection machine-learning models may for example be in the form of an artificial neural network such as a convolutional neural network (CNN). In some embodiments, the object detection machine-learning model is configured to generate a bounding polygon representation of each detected vessel in the at least one radar image, each bounding polygon representation comprising a confidence score for that bounding polygon. An advantage of using bounding polygons is that the accuracy of the overall system may be improved as overlapping between the detections in the radar images may be reduced.

In more detail, the selected object detection ML model may be used to detect vessels, such as ships, in radar video. As input, the selected object detection ML model may obtain a single radar image containing the latest rotation of the radar, which may further contain some "afterglow" of previous rotations in order to better identify moving objects. The term "afterglow" may be construed as a (decaying) trace indicative of one or more previous detections of an object in a radar image. The output of the selected object detection (i.e. the "first output" may contain one or more bounding boxes (or bounding polygon), where each bounding box contains a single identified vessel and a confidence score for that bounding box.

The method S100 further comprises processing S104 the first output by means of the selected feature extraction machine-learning model in order to generate a second output indicative of one or more predicted features of the one or more detected vessels in the at least one radar image. The one or more features may comprise at least one of a position of the one or more detected vessels, a size of the one or more detected vessels, a vessel type (i.e. a ship class) a speed of the one or more detected vessels, an orientation of the one or more detected vessels, and a course of the one or more detected vessels. Moreover, each of the predicted features may be further associated with a confidence score.

In more detail, in some embodiments the selected feature extraction model predicts one or more features of a single vessel/ship in a "cut-out" image based on the bounding box (or bounding polygon) for that vessel/ship. The feature extraction models may for example be in the form of convolutional neural networks (CNNs).

Further, the method S100 comprises processing S105 the second output by means of the selected object tracking machine-learning model in order to generate a third output indicative of an updated tracking state of each vessel of the one or more detected vessels in the at least one radar image. In some embodiments, the object tracking machine-learning model is configured to assign each vessel of the one or more detected vessel to an existing track or a new track based on the predicted features of the one or more detected vessels (i.e. the second output) and a previous tracking state of each vessel generated at a preceding iteration. The object tracking machine-learning model may for example be in the form of a Recurrent Neural Network (RNN).

A tracking state may in the presented context be understood as a temporal development of a position of a vessel and may include a heading, one or more previous positions, one or more current positions, and/or one or more predicted future positions of the vessel, within the range (i.e. coverage area) of the radar device. Thus, "tracking vessels" may be understood as the process of monitoring the movements of marine vessels within the coverage area of the radar device.

Further, in some embodiments, the method S100 comprises transmitting S106 a signal indicative of the updated tracking state to a control system. In other words, the updated tracking state may be transmitted to a Vessel Traffic Service/System (VTS) so that the tracks may be displayed to an operator (e.g. on a digital map).

An indication generally may explicitly and/or implicitly indicate the information it represents and/or indicates. Implicit indication may for example be based on position and/or resource containing the information. Explicit indication may for example be based on a parametrization with one or more parameters, and/or one or more index or indices, and/or one or more bit patterns representing the information.

Thus, in some embodiments, the selected object detection ML model is used to detect the various vessels that are residing within the coverage area of the radar device, the selected feature extraction ML model is used to predict features of the detected vessels so that they can be tracked by means of the object tracking ML model. In other words, the object detection ML model is responsible for pointing out which parts of the radar data is to be processed by the feature extraction ML model, whose output is used to be able to monitoring the individual movement of the vessels. Executable instructions for performing these functions are, optionally, included in a non- transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

Figs. 2 and 3 depict schematic block diagram representations of a system 300 for tracking vessels for maritime traffic management (MTM) applications in accordance with some embodiments. In general, Fig. 2 depicts the flow of information from the radar devices 231 to the visual representation of the tracked vessels at a suitable user-device 213 having a display. Accordingly, the system 300 comprises control circuitry 310 configured to perform the functions of the methods disclosed herein, where the functions may be included in a non-transitory computer- readable storage medium 320 or other computer program product 320 configured for execution by the control circuitry 310. However, in order to better elucidate the present invention, the control circuitry 310 is represented by "modules" or "engines" in Fig. 2, each of them linked to one or more specific functions. The processing system 300 may be comprised by a remote server 400, or realized as a distributed system (a so-called cloud computing system) 500 comprising a plurality of such remote servers 400.

The control circuitry 310 is configured to obtain radar data comprising at least one radar image of a radar device 231 arranged at a port, where the radar data is associated with one or more scenario parameters indicative of a situational context of the radar data. Furthermore, the control circuitry 310 is configured to select a set of ML models out of a plurality of ML models based on the one or more scenario parameters. Each ML model of the plurality of ML models is trained with training data related to a specific situational context. The selected set of ML models comprises an object detection machine-learning model 201, a feature extraction machinelearning model 202, and an object tracking machine-learning model 203.

Furthermore, the control circuitry is configured to process the radar data using the selected object detection ML model 201 in order to generate a first output indicative of one or more detected vessels in the one or more radar images. The output of the object detection ML model 201 is provided as input to the feature extraction ML model 202 in order to generate a second output indicative of one or more predicted features of the one or more detected vessels. Subsequently, the control circuitry 310 is configured to feed the output from the feature extraction ML model 202 as input to the selected object tracking ML model 203 in order to order to generate a third output indicative of an updated tracking state of each vessel of the one or more detected vessels in the at least one radar image.

The control circuitry 310 may further be configured to store the tracking states 210 of all of the detected vessels in the coverage area of the radar device in a suitable data repository. This data repository may accordingly contain all of the tracks, and their history (historical tracks) that are currently present in the coverage area of the radar device. Furthermore, in some embodiments, the object tracking machine-learning model 203 is configured to assign each vessel of the one or more detected vessel to an existing track or a new track based on the predicted features of the one or more detected vessels (i.e. the second output) and a previous tracking state of each vessel generated at a preceding iteration. In other words, the control circuitry 310 may be configured to process the second output and data comprising a previous tracking state of each vessel in the coverage area of the radar device using the selected object tracking ML model 203 in order to assign each vessel of the one or more detected vessel to an existing track or a new track.

Furthermore, if no updated tracking states are generated for any tracks comprised in the formed tracking states 210 for a predefined period of time, the control circuitry 310 may be configured to remove those tracks if the predefined period of time exceeds a threshold. In other words, if a "track" hasn't had any updates for a certain period of time, that track may be removed from the tracking states 210.

In other words, in some embodiments, the selected object detection ML model 201 is used to detect the (marine) vessels in the radar video generated by the radar device. As input the selected object detection ML model 201 obtains one or more radar images i.e. data of the one or more the latest rotations of the radar with some afterglow of previous rotations to more easily identify moving objects. In some embodiments the selected object detection ML model 201 obtains a single radar image as input. The output of the selected object detection ML model is a set of bounding boxes (or bounding polygons) where each bounding box contains a single identified (marine) vessel and a confidence score for that bounding box. Then, the selected feature extraction ML model 202 is used to predict one or more features of a single (marine) vessel in a cut-out image based on the bounding box for that vessel. The one or more features may include a position, size, speed, orientation and/or course of the vessel. Further, the selected object tracking ML model 203 assigns all of the vessels detected and parametrized by the previous ML models 201, 202 to existing or new tracks based on the predicted features and a previous tracking state of the vessels. Moreover, for all tracks that were not assigned a new detection the selected object tracking ML model may be configured to output a prediction of their new position (and extracted features) instead.

The system may further comprise a set of "generic" ML models 205 stored in a suitable data repository. The generic ML models 205 may be understood as generic object detection, feature extraction or object tracking ML models that have not been trained for a specific situational context. In some situations, the control circuitry may be configured to select a generic object detection, feature extraction or object tracking ML model if the one or more scenario parameters of the obtained radar data are indicative of a new situational context, for which not specific object detection, feature extraction or object tracking ML model is available. The system may further comprise a generic dataset 206 stored in a suitable data repository. The generic dataset 206 may for comprise radar data that is collected over time that contains data points from various situational contexts. Thus, the generic dataset 206 may be used to train the generic ML models 205 using a suitable machine-learning scheme so that the generic ML models 205 are trained to an adequate performance level or a broad range of situational contexts.

Moreover, the generic ML models 205 may be understood as base models for the specifically tuned ML models, which are formed from the generic models using transfer learning to any new situational contexts. Thus, the underlying model parameters (e.g. network weights/biases) of the generic ML models 205 form the initial model parameters (e.g. network weights/biases) of the specific ML models. The initial model parameters of the generic ML models 205 may be random or based on an off-the-shelf model (e.g. YOLOX or YOLOv4 for the object detection ML models). However, over time the generic ML models are updated based on the generic dataset, and therefore enable for a quicker tuning process for any new situational context.

Furthermore, the system may comprise a suitable data repository including historical predictions 207 (i.e. the output form the selected ML models 201, 202, 203) and the input radar image(s) associated with each output. In other words, the historical data 207 may comprise all the predictions made by the ML models 201, 202, 203 and the initial radar image(s) used as input. Accordingly, for the object detection ML model, the historical data 207 may comprise, for a single radar image, all of the predicted bounding boxes (or bounding polygons) of the detected vessels. For the feature extraction ML model the historical data 207 may comprise the predicted position, size, speed, orientation and course for each of the detected ships. For the object tracking ML model the historical data 207 may comprise, for each detected ship, an identifier for the track that ship is assigned to. In addition, for each track that was known but could not be found in the current radar image, where its position and features may be predicted based on the tracks from previous iterations. These predictions may also be included in the database containing the historical data 207.

Further, the system may comprise a performance evaluation engine 208 adapted to evaluate the performance/accuracy ofthe selected ML models 201, 202, 203 during deployment. In other words, the control circuitry may be further configured to evaluate the selected ML models 201, 202, 203. In more detail, the evaluation engine may be used to determine the performance of the selected ML models 201, 202, 203 based on two performance categories, namely execution process and accuracy. Moreover, the evaluation engine 208 may be configured to test and evaluate an accuracy score of an extracted set of ML models as described in the foregoing.

The control circuitry may be configured to determine a first set of performance metrics, pertaining to the execution process of the selected ML models 201, 202, 203. In more detail, the first set of performance metrics may include a number of CPU/GPU resources each of the selected ML models 201, 202, 203 used during processing of the input data, or the execution time of the selected ML models 201, 202, 203 for the processing of the input data. This evaluation may be advantageous since the tracking should desirably be performed in real-time or close to real-time, and a too long execution time would negatively affect the system's ability to provide a real-time or close to real-time tracking.

Furthermore, the control circuitry may be configured to determine a second set of performance metrics pertaining to the accuracy of the models. Here, the performance evaluation engine may be configured to compare the outputs (i.e. historical/previous predictions) of each of the selected ML models with external data 221 (e.g. Automatic Identification System (AIS) data, Global Navigation Satellite System (GNSS) data, Laser measurements, manually labelled data, and/or data from new and currently unapplied technologies) and user feedback in order to determine the second set of performance metrics. The user feedback may for example be in the form of direct feedback pertaining to the tracking state 210 supplied to a Vessel Tracking System/Service (VTS) 213 configured to provide a user interface for the user to monitor the tracked vessels. The user may further be provided with e.g. camera images via the user interface of the VTS whereupon the user can compare the supplied tracking state 210 with camera images in order to provide a feedback of the accuracy of the tracking state 210.

Moreover, the performance evaluation engine 208 may be configured to evaluate the performance of the selected object detection ML model 201 based on a computed mean Average Precision (mAP) metric. Furthermore, the performance evaluation engine 208 may be configured to evaluate the performance of the selected feature extraction ML model 202 based on a computed Root Mean Square Error (RMSE) metric. The RMSE metric may for example be computed based on a comparison between the predicted features as output by the selected feature extraction ML model 202 with external data after normalization of the values representing the extracted features. As mentioned, the selected object tracking ML model may be configured to provide an output in the form of future position prediction of the detected vessels and track assignment of the detected vessels. The future position prediction may be scored on RMSE after normalization. The track assignment may be scored using the percentage of tracks assigned correctly.

In addition to these "model specific" evaluation metrics, the performance evaluation engine may be configured to determine metrics regarding the performance of the whole system. Some example of these "system-level" performance metrics are: An estimate of how often a track gets lost, and an estimate of how many different identifiers a single track gets assigned to during its course (ideally this value should be 1).

Furthermore, the performance evaluation engine 208 may be configured to output data suitable for use a training data for updating the selected ML models 201, 202, 203. More specifically, the control circuitry may be configured to store a re-training dataset 209 comprising the data points of the mistakes the selected ML models 201, 202, 203 made as identified during the performance evaluation. Accordingly, this re-training dataset 209 may then be used to retrain the selected ML models (i.e. to update one or more model parameters (e.g. network weights or biases) of the selected ML models) in order to improve the performance of the selected ML models 201, 202, 203. In more detail, the system may further comprise a retraining engine 211 configured to update one or more model parameters of the selected ML models 201, 202, 203 based on the re-training dataset 209. For example, if an operator of the VTS 213 notices that a vessel is not detected, that a vessel is added to the wrong track, or that the system has made some other mistake, the operator can manually adjust this in the VTS 213. This adjustment is then passed on as user feedback and the performance evaluation engine 208 can process this user feedback in order to form a ground truth for the subsequent retraining of one or more of the selected ML models 201, 202, 203.

Moreover, before the ML models are subdued to an update, the updated model parameters may be tested, e.g. via shadow-mode testing, and verified to check if the performance is improved before the actual update is executed.

The system may further comprise an Application Programming Interface (API) so to allow for the obtaining of radar data from the radar devices 231, for external sources to provide the system with additional/new information 221, and for the ability to extract the performance metrics of the system as determined by the performance evaluation engine 208.

According to some embodiments, a computer program product comprises a tangible, or nontangible, computer readable medium such as, for example a universal serial bus (USB) memory, a plug-in card, an embedded drive or a read only memory (ROM). Fig. 3 illustrates an example computer readable medium 320 in the form of a compact disc (CD) ROM 350 having stored thereon a computer program comprising program instructions. The computer program is loadable into a data processor (e.g. control circuitry, data processing circuitry or a data processing unit) 310, which may, for example, be comprised in a general purpose computer. When loaded into the data processor, the computer program may be stored in a memory 320 associated with or comprised in the data processor. According to some embodiments, the computer program may, when loaded into and run by the data processor 310, cause execution of method steps according to, for example, any of the methods illustrated in Fig. 1 or otherwise described herein.

The present invention has been presented above with reference to specific embodiments. However, other embodiments than the above described are possible and within the scope of the invention. Different method steps than those described above, performing the method by hardware or software, may be provided within the scope of the invention. Thus, according to an exemplary embodiment, there is provided a non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer, the one or more programs comprising instructions for performing the method according to any one of the above-discussed embodiments. Alternatively, according to another exemplary embodiment a cloud computing system can be configured to perform any of the methods presented herein. The cloud computing system may comprise distributed cloud computing resources that jointly perform the methods presented herein under control of one or more computer program products.

Generally speaking, a computer-accessible medium may include any tangible or non-transitory storage media or memory media such as electronic, magnetic, or optical media— e.g., disk or CD/DVD-ROM coupled to computer system via bus. The terms "tangible" and "non-transitory," as used herein, are intended to describe a computer-readable storage medium (or "memory") excluding propagating electromagnetic signals, but are not intended to otherwise limit the type of physical computer-readable storage device that is encompassed by the phrase computer- readable medium or memory. For instance, the terms "non-transitory computer-readable medium" or "tangible memory" are intended to encompass types of storage devices that do not necessarily store information permanently, including for example, random access memory (RAM). Program instructions and data stored on a tangible computer-accessible storage medium in non-transitory form may further be transmitted by transmission media or signals such as electrical, electromagnetic, or digital signals, which may be conveyed via a communication medium such as a network and/or a wireless link.

The processor(s) 310 (associated with the system 300) may be or include any number of hardware components for conducting data or signal processing or for executing computer code stored in memory 320. The system 300 has an associated memory 320, and the memory 320 may be one or more devices for storing data and/or computer code for completing or facilitating the various methods described in the present description. The memory may include volatile memory or non-volatile memory. The memory 320 may include database components, object code components, script components, or any other type of information structure for supporting the various activities of the present description. According to an exemplary embodiment, any distributed or local memory device may be utilized with the systems and methods of this description. According to an exemplary embodiment the memory 320 is communicably connected to the processor 310 (e.g., via a circuit or any other wired, wireless, or network connection) and includes computer code for executing one or more processes described herein.

It should be noted that the word "comprising" does not exclude the presence of other elements or steps than those listed and the words "a" or "an" preceding an element do not exclude the presence of a plurality of such elements. It should further be noted that any reference signs do not limit the scope of the claims, that the invention may be at least in part implemented by means of both hardware and software, and that several "means" or "units" may be represented by the same item of hardware.

Although the figures may show a specific order of method steps, the order of the steps may differ from what is depicted. In addition, two or more steps may be performed concurrently or with partial concurrence. Such variation will depend on the software and hardware systems chosen and on designer choice. All such variations are within the scope of the invention. Likewise, software implementations could be accomplished with standard programming techniques with rule-based logic and other logic to accomplish the various connection steps, processing steps, comparison steps and decision steps. The above mentioned and described embodiments are only given as examples and should not be limiting to the present invention. Other solutions, uses, objectives, and functions within the scope of the invention as claimed in the below described patent embodiments should be apparent for the person skilled in the art.

Claims

1. A computer-implemented method (S100) for tracking vessels for maritime traffic management, the method comprising: obtaining (S101) radar data comprising at least one radar image of a radar device arranged at a port, wherein the radar data is associated with one or more scenario parameters indicative of a situational context of the radar data; selecting (S102) a set of machine-learning models out of a plurality of machine-learning models based on the one or more scenario parameters, wherein each machine-learning model of the plurality of machine-learning models is trained with training data related to a specific situational context, and wherein the selected set of machine-learning models comprises: an object detection machine-learning model, a feature extraction machine-learning model, and an object tracking machine-learning model; processing (S103) the radar data by means of the selected object detection machinelearning model in order to generate a first output indicative of one or more detected vessels in the at least one radar image; processing (S104) the first output by means of the selected feature extraction machinelearning model in order to generate a second output indicative of one or more predicted features of the one or more detected vessels in the at least one radar image; processing (S105) the second output by means of the selected object tracking machinelearning model in order to generate a third output indicative of an updated tracking state of each vessel of the one or more detected vessels in the at least one radar image.

2. The method (S100) according to claim 1, wherein the object detection machinelearning model is configured to generate a bounding polygon representation of each detected vessel in the at least one radar image, each bounding polygon representation comprising a confidence score for that bounding polygon.

3. The method (S100) according to any one of claims 1 or 2, wherein the one or more features comprises at least one of a position of the one or more detected vessels, a size of the one or more detected vessels, a speed of the one or more detected vessels, an orientation of the one or more detected vessels, a vessel type, and a course of the one or more detected vessels.

4. The method (S100) according to any one of claims 1 - 3, wherein the object tracking machine-learning model is configured to assign each vessel of the one or more detected vessel to an existing track or a new track based on the predicted features of the one or more detected vessels and a previous tracking state of each vessel generated at a preceding iteration.

5. The method (S100) according to any one of claims 1 - 4, wherein the plurality of machine-learning models comprises: a first machine-learning model group comprising a plurality of object detection machinelearning models, a second machine-learning model group comprising a plurality of feature extraction machine-learning models, and a third machine-learning model group comprising a plurality of object tracking machinelearning models.

6. The method (S100) according to claim 5, wherein the step of selecting the set of machine-learning models comprises: extracting (S107) a first set of machine-learning models based on the one or more scenario parameters, the extracted first set of machine-learning models being machine-learning models trained with training data related to corresponding situational context as the situational context of the obtained radar data; testing (S108) the each machine-learning model of the extracted first set of machinelearning models using earlier instances of the obtained radar data as input; evaluating (S109) an accuracy score of the output of each tested machine-learning model so to select a sub-set of machine-learning models associated with the highest accuracy scores in each machine-learning model group, the selected subset defining the selected set of machine-learning models.

7. The method (S100) according to any one of claims 1 - 6, further comprising: transmitting (S106) a signal indicative of the updated tracking state to a control system comprising a user interface.

8. A computer program product comprising instructions which, when the computer program is executed by a processing system, causes the processing system to carry out the method according to any one of claims 1 - 7.

9. A computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a processing system, the one or more programs comprising instructions for performing the method according to any one of claims 1 - 7.

10. A system for tracking vessels for maritime traffic management, the system comprising control circuitry configured to: obtain radar data comprising at least one radar image of a radar device arranged at a port, wherein the radar data is associated with one or more scenario parameters indicative of a situational context of the radar data; select a set of machine-learning models out of a plurality of machine-learning models based on the one or more scenario parameters, wherein each machine-learning model of the plurality of machine-learning models is trained with training data related to a specific situational context, and wherein the selected set of machine-learning models comprises: an object detection machine-learning model, a feature extraction machine-learning model, and an object tracking machine-learning model; process the radar data by means of the selected object detection machine-learning model in order to generate a first output indicative of one or more detected vessels in the at least one radar image; process the first output by means of the selected feature extraction machine-learning model in order to generate a second output indicative of one or more predicted features of the one or more detected vessels in the at least one radar image; process the second output by means of the selected object tracking machine-learning model in order to generate a third output indicative of an updated tracking state of each vessel of the one or more detected vessels in the at least one radar image.

11. The system according to claim 10, wherein the object detection machinelearning model is configured to generate a bounding polygon representation of each detected vessel in the at least one radar image, each bounding polygon representation comprising a confidence score for that bounding polygon.

12. The system according to any one of claims 10 or 11, wherein the one or more features comprises at least one of a position of the one or more detected vessels, a size of the one or more detected vessels, a speed of the one or more detected vessels, an orientation of the one or more detected vessels, and a course of the one or more detected vessels.

13. The system according to any one of claims 10 - 12, wherein the object tracking machine-learning model is configured to assign each vessel of the one or more detected vessel to an existing track or a new track based on the predicted features of the one or more detected vessels and a previous tracking state of each vessel generated at a preceding iteration.

14. The system according to any one of claims 10 - 13, wherein the plurality of machine-learning models comprises: a first machine-learning model group comprising a plurality of object detection machinelearning models, a second machine-learning model group comprising a plurality of feature extraction machine-learning models, and a third machine-learning model group comprising a plurality of object tracking machinelearning models.

15. The system according to claim 14, wherein the control circuitry is configured to: extract a first set of machine-learning models based on the one or more scenario parameters, the extracted first set of machine-learning models being machine-learning models trained with training data related to corresponding situational context as the situational context of the obtained radar data; test the each machine-learning model of the extracted first set of machine-learning models using earlier instances of the obtained radar data as input; evaluate an accuracy score of the output of each tested machine-learning model so to select a sub-set of machine-learning models associated with the highest accuracy scores in each machine-learning model group, the selected subset defining the selected set of machinelearning models.

16. The system according to any one of claims 10 - 15, wherein the control circuitry is further configured to: transmit a signal indicative of the updated tracking state to a control system comprising a user interface.

17. A remote server comprising the system according to any one of claims 10 - 16.

18. A cloud environment comprising one or more remote servers according to claim

17.