IL303531A

IL303531A - Computer-based prediction of a marine landing platform state

Info

Publication number: IL303531A
Application number: IL303531A
Authority: IL
Inventors: Rozenberg Ohad; Herzig Yoav
Original assignee: Israel Aerospace Ind Ltd; Rozenberg Ohad; Herzig Yoav
Priority date: 2023-06-06
Filing date: 2023-06-06
Publication date: 2025-01-01
Also published as: WO2024252393A1

Description

COMPUTER-BASED PREDICTION OF A MARINE LANDING PLATFORM STATE TECHNICAL FIELD The presently disclosed subject matter relates to the landing of an aircraft on a marine platform.

BACKGROUND Vertical take off and landing (VTOL) aircrafts are widely used today for various military and civilian operations that require landing in areas where traditional runways or landing strips are unavailable or cannot be used. For naval operations, the ability of VTOL aircrafts to take off and land on ships and other platforms at sea, without requiring a runway or airfield, is crucial. This capability is often important for successful completion of a mission, such as maritime security, search and rescue, surveillance, and others.

GENERAL DESCRIPTION Due to the dynamic and sometimes extreme conditions encountered at sea, which may involve for example strong waves and winds, it is difficult to maintain a steady landing platform for a VTOL aircraft. Specifically, the motion of marine landing platforms such as ships, caused by the waves, may result in a continuous change in the orientation of the deck, unpredictability affecting the required position and orientation of the VTOL aircraft relative to the deck during landing, thus compromising safe landing of the aircraft. For these reasons, landing a VTOL aircraft on a ship at sea is a complex and challenging task that requires precise control and coordination. Therefore, it is important to consider the effects of waves on the motion of a marine landing platform during landing and identify an appropriate platform state that allows safe landing of the VTOL aircraft on the platform in both autonomous landing and human controlled landing. – 2 – The presently disclosed subject matter includes a system and method designated for predicting the state of a marine landing platform. The term "platform state" as used herein refers to a plurality of degrees of freedom that define the state of a landing platform by position and orientation in three-dimensional space. There are six degrees of freedom (6 DOF), including three degrees of freedom that describe the position of an object in the latitude (X), longitude (Y), and altitude (Z) directions, and three degrees of freedom that describe the orientation of an object in terms of rotations around the X, Y, and Z axes, otherwise known as pitch, roll, and yaw (heading). It is noted that while the following description predominantly refers to a platform state defined by 6 DOF ("6 DOF platform state") this should not be construed as limiting, as defining a platform state with less degrees of freedom (e.g., 5 or degrees of freedom) is also contemplated within the scope of the presently disclosed subject matter.

The disclosed system includes one or more data acquisitioning devices configured to monitor the water around the marine landing platform and collect environmental data that includes information on incoming waves. The collected information is fed as input to a machine learning model trained to provide a predicted platform state of the marine landing platform (e.g., 6 DOF platform state) following impact of one or more incoming waves on the platform. Furthermore, the system is configured to obtain wave data pertaining to a sequence of incoming waves approaching the landing platform, calculate the predicted change to the 6 DOF platform state of the landing platform caused by the impact of individual waves in the sequence of incoming waves, and aggregate the changes caused by multiple waves in the sequence, to thereby provide prediction of waves impact on the 6 DOF platform state of the landing platform over a relatively long time-window.

Referring to Fig. 1 , it shows a schematic illustration of a VTOL aircraft 100 before and during landing on a ship 110 . While the presently disclosed subject is advantageous for the operation of any VTOL aircraft, Fig. 1 illustrates a non-limiting example of an aircraft that activates a VTOL propulsion unit before landing. One – 3 – example of an aircraft that activates VTOL propulsion before landing is an aircraft with a separate VTOL propulsion unit dedicated only for landing and take-off, that is engaged before landing, to provide vertical descent and landing capabilities to the aircraft. Another example is a tiltrotor aircraft which utilizes one or more rotor units comprising a powered rotor (sometimes called proprotors) and respective tiltable structures on which the rotor is mounted. The rotors are used for both lift and propulsion, wherein different effects are achieved when the rotors are tilted between a general vertical direction and a general horizontal direction, and in intermediate directions. Tilting of the rotors enables implementing, at certain times, vertical lift capability, which enables VTOL, and at other times, propellor propulsion usually associated with conventional fixed-wing aircraft. Often when a tiltrotor aircraft approaches a landing site such as a ship's deck, it initiates a transition process from a first flight mode, in which the tiltable rotors of the aircraft are directed to provide thrust in the general horizontal direction, to a second flight mode in which the tiltable rotors are directed in the general vertical direction and enable vertical landing on deck. The operation of both aircrafts mentioned above may involve an activation process (e.g., activation of a dedicated VTOL unit according to first example or transition from one mode to the other according to the second example) of the VTOL unit prior to landing that may take, in some examples, anywhere between several seconds up to several minutes.

In general VTOL is less efficient than fixed wing flight. Particularly, when a dedicated VTOL unit is engaged for landing (and take-off), the duration of its activation is often restricted due to the excessive consumption of resources associated with such operation. Furthermore, during VTOL the vehicle may be less stable and may be more vulnerable to winds. Therefore, it is advantageous to initiate activation of VTOL propulsion right before landing to keep its activation as short as possible. However, since safe landing can be executed only with a suitable platform state, prediction of a future platform state needs to be made for a sufficiently long prediction period that provides more time for completion of the activation process of the VTOL unit and – 4 – landing. These examples therefore demonstrate the need for a sufficiently long prediction period as disclosed herein. According to some non-limiting examples, the length of the prediction period is anywhere between 30 seconds to 3 minutes. According to other non-limiting examples, the length of the prediction period is anywhere between 30 seconds to 60 seconds.

According to one aspect of the presently disclosed subject matter there is provided a computer implemented method of predicting a state of a marine platform, the method comprising: using at least one data acquisition device for obtaining environmental data, including wave data of multiple waves approaching the platform; executing a learning procedure, comprising: for each wave of the multiple waves: processing the wave data to obtain respective wave parameters; following impact of the wave with the platform, recording, over a predefined period, starting from a time of impact, a series of data samples of the platform state for each one of multiple degrees of freedom (DOF), thereby generating a respective platform response vector; adding to a training dataset the respective wave parameters and the respective platform response vector; training a machine leaning model with the training dataset; executing a platform state prediction procedure, comprising: receiving wave data recorded by at least one data acquisition device over a certain detection range and obtaining a respective waves-snapshot; for each wave in the waves-snapshot: – 5 – processing the wave data to obtain respective wave parameters; determining a respective platform response vector, by applying the wave parameters to the ML model; and aggregating the platform response vectors of all waves in the respective waves-snapshot (and the corresponding current platform state) to thereby determine a predicted platform state following impact of all waves in the waves-snapshot.

In addition to the above features, the method according to this aspect of the presently disclosed subject matter can optionally comprise one or more of features (i) to (xix) listed below, in any technically possible combination or permutation: i. The computer implemented method comprising wherein the multiple DOF is 6 DOF. ii. The computer implemented method comprising: repeating the platform state prediction procedure, comprising: obtaining an updated wave-snapshot; and determining an updated predicted platform state following impact of all waves in the updated waves-snapshot. iii. The computer implemented method comprising: for each iteration of the platform state prediction procedure, determining whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, if so, terminating execution of the platform state prediction procedure. iv. The computer implemented method comprising: for each iteration of the platform state prediction procedure, determining whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, responsive to determining – 6 – that the predicted platform state is suitable for landing, transmitting landing instructions to the VTOL aircraft to synchronize landing with the predicted platform state. v. The computer implemented method comprising: determining whether the predicted platform state is suitable for landing the VTOL aircraft, and, responsive to determining that the predicted platform state is suitable for landing, transmitting landing instructions to the VTOL to synchronize landing with the predicted platform state. vi. The computer implemented method comprising: for each waves-snapshot captured at a time t, recording the corresponding current platform state of the platform at time t and aggregating the corresponding current platform state with the platform response vectors of all waves in the respective waves-snapshot. vii. The computer implemented method, wherein processing wave data to obtain respective wave parameters includes identifying individual waves and determining their respective wave parameters. viii. The computer implemented method, wherein executing a learning procedure comprises determining time of impact of each wave with the platform. ix. The computer implemented method, wherein environmental data further incudes wind data and wherein executing a learning procedure comprises: recording wind data; and adding the wind data to the training set. x. The computer implemented method, wherein executing the learning procedure comprises: applying a transformation function on each series of data samples to thereby transform the data samples from a time domain to a frequency – 7 – domain. xi. The computer implemented method, wherein the learning procedure comprises compressing the transformed data samples using quantization to obtain a finite set of quantized values. xii. The computer implemented method, wherein each platform response vector determined for each wave in the waves-snapshot during the platform state prediction procedure, includes transformed data samples, and where executing the platform state prediction comprises: applying an inverse transformation function on series of transformed data samples in each platform response vector. xiii. The computer implemented method, wherein recording a series of data samples of platform state for each one of multiple degrees of freedom is performed with the help of one or more positioning devices, including one or more of an inertial navigation system (INS) and a GPS. xiv. The computer implemented method comprising fixing the INS to a landing deck on the platform. xv. The computer implemented method, wherein the data acquisition device includes one or more of: RADAR, camera (e.g., 3D camera), and buoy. xvi. The computer implemented method further comprising monitoring at least one physical property of the platform and initiating the learning procedure responsive to receiving data indicative of a significant change in the at least one physical property. xvii. The computer implemented method wherein the platform is any one of: ship, boat, floating landing dock, or pad. xviii. The computer implemented method, wherein the ML model is a convolutional neural network. – 8 – xix. The computer implemented method, wherein the ML model provides a response function that does require, as input, information on physical properties of the platform.

According to another aspect of the presently disclosed subject matter there is provided a system mountable on a marine platform configured for predicting a state of a marine platform, the system comprising at least one data acquisition device operatively connected to at least one computer; the at least one data acquisition device is configured to capture environmental data, including wave data of multiple waves approaching the platform; the computer is configured to: execute a learning procedure, comprising: for each wave of the multiple waves: process the wave data to obtain respective wave parameters; following impact of the wave with the platform, record over a predefined period, starting from a time of impact, a series of data samples of platform state for each one of multiple degrees of freedom (DOF), thereby generating a respective platform response vector; add to a training dataset the respective wave parameters and the respective platform response vector; train a machine leaning model with the training dataset; execute a platform state prediction procedure, comprising: receive from the at least one data acquisition device wave data captured over a certain detection range and determine a respective waves- snapshot; – 9 – for each wave in the waves-snapshot: process the wave data to obtain respective wave parameters; determine a respective platform response vector, by applying the wave parameters to the ML model; and aggregate the platform response vectors of all waves in the respective waves- snapshot (and the corresponding current platform state) to thereby determine a predicted platform state following impact of all waves in the waves-snapshot. According to another aspect of the presently disclosed subject matter there is provided a computer program product comprising a computer readable storage medium retaining program instructions, which programs instructions when read by a processor, and causes the processor to perform the method according to the first aspect mentioned above.

According to another aspect of the presently disclosed subject matter there is provided a non-transitory program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform the method according to the first aspect mentioned above.

The system, the computer program product, and the non-transitory program storage device disclosed in accordance with the aspects of the presently disclosed subject matter detailed above can optionally comprise one or more of features (i) to (xix) listed above, mutatis mutandis, in any technically possible combination or permutation.

According to an additional aspect of the presently disclosed subject matter there is provided a computer program product comprising a computer readable storage medium retaining a program of instructions, which program of instructions when read by a computer processor causes the computer processor to perform a method, comprising: using at least one data acquisition device for obtaining environmental data, – 10 – including wave data of multiple waves approaching the platform; executing a platform state prediction procedure, comprising: receiving wave data recorded by at least one data acquisition device over a certain detection range and obtaining a respective waves-snapshot; for each wave in the waves-snapshot: processing the wave data to obtain respective wave parameters; determining a respective platform response vector, by applying the wave parameters to a machine learning model trained to receive wave parameters and determine data indicative of a respective platform response vector; and aggregating the platform response vectors of all waves in the respective waves-snapshot to thereby determine a predicted platform state following impact of all waves in the waves-snapshot.

Wherein it is further contemplated that the program of instructions retained on the computer program product, when read by a computer processor causes the computer processor to perform a method further comprising: executing a learning procedure, comprising: for each wave of multiple waves captured by the data acquisition device: processing the wave data to obtain respective wave parameters; following impact of the wave with the platform, recording, over a predefined period, starting from a time of impact, a series of data samples of a platform state for each one of multiple degrees of freedom (DOF) thereby generating a – 11 – respective platform response vector; adding to a training dataset the respective wave parameters and the respective platform response vector; and training the machine leaning model with the training dataset.

Wherein the presently disclosed subject matter further contemplates a system and a computer implemented method that comprise and/or implement the features set forth by the additional aspect above, mutatis mutandis.

BRIEF DESCRIPTION OF THE DRAWINGS In order to understand the invention and to see how it can be carried out in practice, embodiments will be described, by way of non-limiting examples, with reference to the accompanying drawings, in which: Fig. 1 is a schematic illustration of a landing scenario of a VTOL aircraft onboard a ship, in accordance with certain examples of the presently disclosed subject matter; Fig. 2 illustrates a block diagram of a system mountable on a marine vehicle, in accordance with certain examples of the presently disclosed subject matter; Fig. 3 illustrates a block diagram of a platform state prediction computer, in accordance with certain examples of the presently disclosed subject matter; Fig. 4 illustrates a flow-chart of operations carried out during ML training, in accordance with certain examples of the presently disclosed subject matter; Fig. 5 illustrates sampled data of 6 DOF platform state, in accordance with certain examples of the presently disclosed subject matter; and Fig. 6 illustrates a flow-chart of operations carried out during prediction of a platform state, in accordance with certain examples of the presently disclosed subject matter.

DETAILED DESCRIPTION In the drawings and descriptions set forth, identical reference numerals – 12 – indicate those components that are common to different embodiments or configurations. Elements in the drawings are not necessarily drawn to scale.

Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that, throughout the specification, discussions utilizing terms such as "executing", "adding", "recording", "processing", "training", "determining" or the like, include an action and/or processes of a computer that manipulate and/or transform data into other data, said data represented as physical quantities, e.g. such as electronic quantities, and/or said data representing the physical objects.

The terms "computer", "computer device" or the like, should be expansively construed to include any kind of hardware electronic device with at least one data processing circuitry (e.g., digital signal processor (DSP), a GPU, a TPU, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), microcontroller, microprocessor etc.). The processing circuitry can comprise, for example, one or more computer processors operatively connected to computer memory, loaded with executable instructions for executing operations as further described below.

Operations in accordance with the teachings herein may be performed by a computer specially constructed for the desired purposes, or by a general-purpose computer specially configured for the desired purpose by a computer program stored in a computer readable storage medium. (e.g., landing deck on ship) As used herein, the phrase "for example," "such as", "for instance" and variants thereof, describe non-limiting embodiments of the presently disclosed subject matter. Reference in the specification to "one case", "some cases", "other cases", or variants thereof, means that a particular feature, structure, or characteristic described in connection with the embodiment(s), is included in at least one embodiment of the presently disclosed subject matter. Thus, the appearance of the phrase "one case", "some cases", "other cases", or variants thereof, does not necessarily refer to the – 13 – same embodiment(s).

It is appreciated that certain features of the presently disclosed subject matter, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the presently disclosed subject matter, which are, for brevity, described in the context of a single embodiment, may also be provided separately, or in any suitable sub-combination.

In embodiments of the presently disclosed subject matter, fewer, more and/or different stages than those shown in Figs . 4and 6 may be executed. In embodiments of the presently disclosed subject matter, one or more stages in the illustrated figures may be executed in a different order, and/or one or more groups of stages may be executed simultaneously.

Figs . 2and 3schematically illustrate a system architecture in accordance with certain examples of the presently disclosed subject matter. Elements in Figs . 2and 3can be made up of any suitable combination of software and hardware and/or firmware that performs the functions as defined and explained herein. In examples of the presently disclosed subject matter, other than what is illustrated, the system may comprise fewer, more, and/or different elements than those shown in Figs . 2and 3 . Likewise, the division of the functionality of the disclosed system to specific parts, as described below with reference to Figs. 2and 3 , is provided by way of example, and other various alternatives are also construed within the scope of the presently disclosed subject matter. For example, while Figs. 2 and 3 show a single computer dedicated for both training the machine learning model and executing the trained model, in other examples two or more separate computers can be used, e.g., one for training the model, and the other for execution of the model.

The term "vertical takeoff and landing (VTOL) aircraft" as used herein should be broadly construed to include any aerial vehicle with vertical lift that can execute vertical landing, including both manned VTOL aircrafts and unmanned (autonomous) – 14 – VTOL aircrafts. VTOL aircraft include, for example, helicopters, tiltrotor aircraft (e.g., Bell Boeing V-22 Osprey and the Bell Nexus), drones of all kinds (e.g., multirotor drones), VTOL jets (e.g., Harrier Jump Jet and the F-35B Lightning II), flying cars (e.g., PAL-V Liberty), etc.

The terms "marine landing platform" or "landing platform" as used herein should be broadly construed to include any type of platform or vessel that floats on water and can be used as a landing surface for a VTOL aircraft. In some examples, a marine landing platform includes a landing deck and may include certain features that support VTOL, such as a non-skid surface, tie-down points, and deck lighting that facilitate nighttime landing. A marine landing platform can be, for example, a ship or other marine vehicle (typically comprising a designated landing deck), or a floating landing dock or pad. Marine landing platforms for VTOL aircrafts are used for various civilian applications, such as offshore oil and gas discovery and extraction, search and rescue missions, and emergency medical services.

Bearing the above in mind, attention is drawn to Fig. 2 , showing a schematic block diagram of a system mountable on a marine landing platform according to some examples of the presently disclosed subject matter. System 200 is configured in general to predict a future platform state, determine a time when the future platform state is suitable for safe landing of the VTOL aircraft, and coordinate the landing of the VTOL according to the determined time.

System 200 comprises one or more data acquisition devices 210 , including for example one or more of: camera system(s), a RADAR system(s), buoy(s) (e.g., wave monitoring buoys) etc., configured to monitor the waves around the platform and determine waves data including waves height (amplitude) and waves velocity (i.e., speed and direction). Data acquisition devices 210 may further include a wind sensor (anemometer) configured to determine wind velocity. The camera and/or radar can be positioned to capture images of waves as they approach. In some examples, a camera system consists of a set of multiple cameras (e.g., 3 cameras or more) for – 15 – obtaining 3-dimensional imaging of the waters. The camera can be for example a high-speed camera operatively connected to an image processing circuitry. In some examples, the camera can be equipped with a specialized lens that allows it to capture high-resolution images of waves, which can be processed as further described below.

System 200 also comprises positioning devices 220 capable of determining a platform state (6 DOF platform state) with respect to some reference frame, including for example one or more of inertial navigation system (INS), GPS receiver, and a cellular localization system. An INS is configured to continuously monitor the attitude of the platform, and, in some cases, it is also used for determining its location. According to some examples, the INS (or the inertial measurement unit of an INS, or some other altitude determining device) is mounted and fixed directly to a landing deck, which is the area dedicated for VTOL landing on the landing platform (e.g., landing deck on a ship), in order to have direct contact with the platform where the landing takes place, to enable direct monitoring of a 6 DOF platform state (or at least the 3 attitude components) at the place of landing, and thereby increase accuracy and reduce complexity of the 6 DOF data collection process.

System 200 further includes a platform state prediction computer 230 configured to predict future platform state of the landing platform, as further explained below. System 200 also comprises communication unit 240 configured to facilitate communicating, e.g., with the VTOL aircraft for sending to the aircraft landing instructions once a platform state suitable for landing is predicted.

Fig. 3 is a block diagram showing a more detailed description of components of platform state prediction computer 230 , according to some examples of the presently disclosed subject matter. The architecture in Fig. 3 is a general example which demonstrates various principles of the presently disclosed subject matter, while in practice it may vary in design and therefore should not be construed as limiting. Computer 230 can comprise one or more processing circuitries, each circuitry comprising one or more computer processors operatively connected to a computer – 16 – storage medium. The processing circuitries can be configured, for example, to execute functional modules in accordance with computer-readable instructions implemented on a non-transitory computer-readable storage medium. For simplicity, such functional modules are referred to hereinafter as components of the computer. A detailed description of the operations executed by platform state prediction computer 230 is disclosed below with reference to the following figures.

Proceeding to Fig. 4 , this shows a flowchart of operations carried out during a training procedure, in accordance with examples of the presently disclosed subject matter. For more clear understanding, operations in Fig. 4 (and Fig. 6 ) are described with reference to components of the system shown in Figs. 2 and 3 , however this should not be construed to limit the method to any specific system design or component. In the following description the term "ship" is used to demonstrate a marine landing platform for simplicity and by way of non-limiting example only.

Fig. 4 is related to the training of a machine learning model capable of predicting a future state of a marine landing platform in response to impact of waves. The training process can be initiated for example, prior to the landing of a VTOL aircraft, and in some cases may require a short initial training period (also referred to as calibration period). While additional training using additional training data may improve accuracy of the machine learning model output, the machine learning model can be used after the calibration period. By way of example, in the case of a ship, training can be initiated at the beginning of operation (e.g., following embarkment of ship from dock) and often the same model can be used throughout the mission while the ship is at sea (while optionally the model is being trained continuously to reinforce the model over time). In other examples, training can be initialized prior to landing. For example, training can be initiated in response to receiving a message at the ship indicating that a VTOL aircraft intends to land on the ship shortly (e.g., within the next 30-60 minutes). The calibration period can range from a few minutes (e.g., 10-minutes) to a few hours. – 17 – Data collection used for training of the ML model can be done in real-time during the training process and does not require to obtain a training dataset prior to initiation of training. As will be apparent from the following description, the machine learning model is trained using data collected in real-time by devices onboard the ship, including environmental data and platform state data, without any need for data pertaining to the geometric and other physical properties of the ship, such as length, type width, keel length, deck depth, overall weight (tonnage), planned velocity, and planned depth, which are used for determining a transfer function.

While the system and method disclosed herein do not rely on geometric/physical properties as part of the input used during training or execution of the ML model (as no transfer function is used), significant changes in variable physical properties, which occur after the training of the model, may influence the ship's response to the waves ("wave-response") and therefore also its prediction accuracy. Thus, in some cases, (e.g., following significant depletion in fuel levels or following loading of heavy cargo onto the ship) it may be advisable to re-train the model to accommodate changes in the ship's wave-response that may result from the change in physical properties. Since the initial training that is needed requires a relatively short calibration period, retraining of the model can be done relatively fast upon detection of a significant change in the ship's properties. In some examples, system 200 includes a gauge module (ship gauge 250 ) configured for monitoring certain physical properties of the ship (e.g., by directly measuring the properties or being connected to another system or sensor capable of providing data on the properties). If the gauge module receives information indicative of a significant change in a certain property of the ship (e.g., net weight), it may signal the platform state prediction computer 230 that retraining of the model is required, resulting in retraining the ML model.

At block 401 the water surrounding the ship is monitored using one or more data acquisition devices 210 , and raw environmental data is recorded. Environmental data includes wave data and possibly also wind data. Depending on the type of data – 18 – acquisition device, which is being used, the raw wave data may include corresponding output, such as imaging data output, RADAR data output, etc. As mentioned above, an acquisition device is positioned on the ship such that it can view and capture waves which are approaching the ship. In some examples, the data acquisition device is configured to capture wave data from multiple directions, for example by using multiple cameras pointed in different directions, or by using a pan and tilt mechanism allowing to change the viewing angle of the data acquisition device.

The raw wave data collected by the data acquisition devices is processed and individual waves are recognized (block 410 ). The respective wave parameters of each wave are determined (block 403 ). Wave parameters include wave height and wave velocity i.e., speed and direction. In some cases, wave parameters may further include wavelength and/or wave period. In the case of a camera system, the imaging output can be processed using various software such as edge detection, pattern recognition, and image segmentation, together with other dedicated software for wave identification and characterization. Alternatively, or additionally, machine learning models trained for determining wave parameters from wave data can be used. The obtained wave parameters can be stored in a computer data repository ( 350 ) operatively connected to computer 230 .

It is noted that wave data is captured and processed in real-time while the waves are approaching the ship. Identified waves are tracked to determine a respective time of impact with the ship, and, immediately following impact, the 6 DOF platform state of the platform is recorded (block 405 ). The time of impact can be determined, for example, by monitoring the wave's velocity and determining based on the wave velocity and its distance from the ship at the time of impact. Following impact, 6 DOF position and orientation of the ship is determined using appropriate navigation devices, such as INS and GPS.

Operations described with respect to block 410 , 40 3, and wave tracking in block 405 , can be executed for example by water state processing module 310 – 19 – configured to process the raw data received from data acquisition devices 210 .

In some examples, the 6 DOF platform state recorded for a certain wave includes a platform response vector. To this end, following the time of impact, the DOF position and orientation of the ship is sampled over a certain time window, to thereby obtain, for each wave, the respective platform response vector. The platform response vector includes a collection of a series of samples, recorded over the length of the time window, each series corresponding to one degree of freedom of a certain position or orientation, including a first series of samples of movement along the latitude (X) axis; a second series of samples of movement along the longitude (Y) axis, a third series of samples of movement along the altitude (Z) axis, a fourth series of samples for pitch, a fifth series of samples for roll, and a sixth series of samples yaw.

The length of the time window generally depends on the time between consecutive waves (wave period), but in some cases may be limited by some upper bound. Referring to Fig. 5 it shows a graph plotting an example of 6 series of sampled data, each corresponding to a certain axis of movement or axis of rotation of the ship. In the graph, series 1 represents sampled data of the height axis (up and down direction - altitude), series 2 represents sampled data of the sway axis (side to side direction - latitude), series 3 represents sampled data of the surge axis (forward backwards direction - longitude), series 4 represents sampled data of the delta yaw axis, series 5 represents sampled data of the pitch axis, and series 6 represents sampled data of the roll axis. The collection of series represent the platform response vector.

Operations described with respect to blocks 407 , 409 can be executed for example by platform state processing module 320 .

A transformation function is applied on the 6 DOF platform state for transforming the 6 DOF platform state from the time domain to the frequency domain (block 407 ). In some examples, data transformation is achieved by applying a Fourier Transform on each series in the respective response vector. – 20 – Transformation of the sampled data to the frequency domain can be performed for example by applying a discrete cosine transform (DCT) exemplified by the following expression.

Equation 1: u0001u0002= u0005 u0006u0007bt u0007u000bf ru000eu000f u0010u0011u0012 u0013u0014 +2u0018 u0019 u001a u001bu000eu001c u0019 = 0, … u0012 − 1 where u0001u0002 is the signal in the frequency domain and u0006u0007 is the signal in the time domain. u0012 is the number of samples in the series. As shown in Fig. 5 each sequence of samples is represented as a signal over time.

In some examples, the transformation output is compressed using quantization to obtain a finite set of quantized values, which helps to simplify and increase efficiency of the training of the machine learning model. Transformation and quantization can be executed for example by transformation module 330 .

The environmental data (including wav parameters) determined for a given wave are stored in association with the quantized values obtained from the corresponding 6 DOF platform state obtained for the same wave. This is repeated with many waves to thereby generate a training dataset, which can be stored in a computer data storage (block 409 ; computer data storage 350 ). The training dataset can also include the wind data that was measured at the time the wave data was recorded. In some examples, wind velocity, measured during the time of impact of a certain wave, is stored in the database in association with the wave. In some examples, wind data measured during a certain period (e.g., every 2 minutes) is associated with all waves that hit the ship during this period.

The machine learning model (ML) is trained using the training data stored in the storage (block 420 ). In some examples training can start in real-time as soon as training data is initially stored in the data storage and continue as more data is being processed and added to the storage. This is an advantage since it shortens the – 21 – calibration period required for a obtaining a valid model that can be used for predictions. In some examples the ML model is a Convolutional Neural Networks (CNNs) employing several hidden layers. Training can be executed for example by ML training module 340 .

Once the training process is complete (e.g., following a calibration period of 10 to 30 minutes) the trained ML model can be used for predicting the ship's state. By way of example, if 100 samples are sufficient for generating a valid model and considering 6 samples are recorded each minute (for each DOF), a 30-minute calibration period can suffice for training a valid ML model. The trained ML model provides a response function which provides a correlation between the recorded wave data (and wind data) and the respective platform response vector and does not require physical/geometric data of the platform as input during training and execution.

Turning to Fig. 6 , showing a flowchart of operations carried out during platform state prediction, according to examples of the presently disclosed subject matter. The platform state prediction procedure can follow the calibration procedure, which, as mentioned earlier, can be relatively short. However, training of the model can continue in parallel to prediction to enhance the model and improve its output.

At block 601the water surrounding the landing platform is monitored using one or more data acquisition devices 210 , and wave data of waves approaching the ship is recorded. As explained above with respect to Fig. 4 , the raw wave data collected by the data acquisition devices is processed, individual waves are recognized (block 610 ), and the respective wave parameters of each wave are determined, including at least wave height and velocity (block 603 ).

According to the presently disclosed subject matter, the data acquisition device 210 is configured and operable to obtain wave data of waves within a certain detection range from the ship. One non-limiting example of a detection range is between 50 to 2000 meters from the ship. Wave data captured within the detection – 22 – range is processed and individual waves are identified. If the field of view (FOV) of the data acquisitions device covers an area which is greater than the detection range, the raw wave data can be processed to extract therefrom a part that is equal to the detection range. If the field of view of the data acquisitions device covers an area which is smaller than the detection range, multiple frames covering the detection range can be captured. The data acquisition device output captured or otherwise obtained at a certain time t over the detection range, is referred to herein as "waves-snapshot".

The detection range, together with waves speed, define the prediction period, where a larger detection range and/or slower waves provide a longer prediction period, and smaller detection range and/or faster waves provide a shorter prediction period. Thus, the detection range can be adapted according to various constraints and needs. For example, if a longer prediction period is needed, the system can be configured to capture wave data over a longer detection range, and vice versa. In general, a good prediction period is one which is long enough to provide sufficient data for obtaining sufficiently accurate prediction of the platform state on one hand and is not too long, to avoid inaccuracies that may result from changes to the waves which occur while the waves are approaching the platform. In some examples, ML execution module 360 can be configured to receive input (e.g., from a pilot of an approaching aircraft, or from an operator onboard a ship where landing is about to occur) indicative of the desired prediction period and adapt the detection range according to the wave speed. As mentioned above, wave speed can be determined in real-time by processing the captured wave data.

The determined wave parameters are provided as input to the ML model, which provides output data indicative of corresponding 6 DOF platform state (block 605 ; e.g., by ML execution module 360 ). Assuming features in training dataset include quantized values of the transformed 6 DOF platform state (i.e., samples series of the respective wave response vector), the ML model output also includes quantized values of corresponding transformed 6 DOF platform state. In some examples, wind – 23 – data is also recorded and provided as input to the ML model, together with the wave parameters, as explained above with respect to training.

The obtained quantized values are processed using an inverse transformation function (e.g., inverse Discrete Cosine Transform (IDCT)) to thereby obtain the respective 6 DOF platform state e.g., the series of samples of the respective platform response vector (block 607 , e.g., by post processing ML output module 370 ). Thus, the ML model is used for determining the wave response vector of each wave within the detection range.

The ML model output of 6 DOF platform state corresponding to the waves detected within the detection range are aggregated with the current 6 DOF platform state to thereby obtain a predicted 6 DOF platform state over the distance of the prediction range (block 609 ; e.g., by future platform state prediction module 380 ). In other words, the predicted platform state is a 6 DOF platform state that the ship will assume in the future when all waves detected within the detection range at a certain time t have reached the ship. Assuming, for the sake of example, that the detection range is 750 meters, and that at time !f, 10 waves ("f… "#) are detected within the detection range (corresponding to a waves-snapshot at time !f), the aggregated effect of all 10 waves with the current 6 DOF platform state of the platform (i.e. the 6 DOF platform state at the time of data acquisition of the 10 waves), is calculated. The corresponding 6 DOF platform state values obtained for each of the 10 waves are aggregated together with the current state of the platform, providing the predicted DOF platform state following impact of all 10 waves with the ship. Notably, each wave induces a change to the ship's state that depends on its previous state before impact. By aggregating the platform state of each wave in a waves-snapshot with the previous platform state (starting from the respective current platform state), the accumulated change in the platform state over the course of the detection range can be determined.

The prediction procedure can be executed in a repetitive manner (as indicated – 24 – by the arrow connecting block 609 to block 610 ), processing updated waves-snapshots as new waves enter the waves detection range and are captured. Considering the next iteration at time !, the data acquisition output captured at that time would include a different set of waves (i.e., an updated waves-snapshot). Assuming, in the next iteration, there is a one wave difference from the previous iteration, such that wave "f has already hit the ship, and wave "f (11th wave) enters the detection range. In such a case, the new waves-snapshot would include waves "… ". The calculation is repeated, this time determining the predicted 6 DOF platform state following impact of waves "… " and using a different current 6 DOF platform state that corresponds to the time of acquisition of waves "… ". The updated waves-snapshot is processed to obtain a more updated predicted platform state following impact of the waves in the updated waves-snapshot. Thus, the disclosed system can continuously monitor the sea and provide updated predicted platform states, one after the other.

According to the disclosed subject matter, wave data of a series of waves within a detection range is obtained and translated to a corresponding future platform state the ship is predicted to assume following impact of the series of waves. Notably, each wave that hits the ship exerts certain forces on the ship that cause a certain movement to the ship that brings the ship to a certain 6 DOF platform state characterized by a certain position and orientation. The actual state of the ship following impact depends on the previous state of the ship. This is why, by aggregating the platform states, each resulting from the impact of a respective wave in the series of waves, and the corresponding current state, the future state of the ship, following impact of all waves in the series of waves, can be determined.

Once the future 6 DOF platform state of the ship has been predicted, it is determined whether this state is suitable for landing of the VTOL aircraft (block 611 ; e.g., by future platform state verification module 390 ). Future platform state verification module 390 can be configured to receive the future 6 DOF platform state of the ship and determine, based on the respective parameters (e.g., orientation and position) and other information such the type of VTOL aircraft, that is about to land, – 25 – and/or the topography characteristics (whether the water is shallow or deep) of the area surrounding the ship, and determine whether the predicted ship state is suitable for landing of the particular VTOL aircraft. In some examples, future platform state verification module 390 is configured to use a ML learning model trained for determining suitability of a certain 6 DOF platform state of a platform for landing. For example, the ML model can receive as input the future 6 DOF platform state of the platform predicted by system 200 , as well as additional information, such as the type of the VTOL, and provide a "confirmed landing" or "denied landing" output. In case landing is confirmed, landing instructions are transmitted to the VTOL aircraft, informing the aircraft the pending landing time, and causing it to prepare for landing (block 613 ; e.g., data is transmitted to VTOL by communication unit 240 ). Otherwise, the system continues to process environmental data, calculate future vehicles states, and search for a future platform state that is suitable for landing. In some examples, once the VTOL aircraft has landed, the prediction procedure is terminated until the need for a further landing arises. In some examples, the training procedure continues, even if landing has occurred and/or the prediction procedure is terminated, to further enhance the ML model and prepare of possible future landings.

Those skilled in the art will readily appreciate that various modifications and changes can be applied to the examples of the invention as hereinbefore described without departing from its scope, defined in and by the appended claims.

Claims

– 26 – CLAIMS

1. A computer implemented method of predicting a state of a marine platform, the method comprising: using at least one data acquisition device for obtaining environmental data, including wave data of multiple waves approaching the platform; executing a learning procedure, comprising: for each wave of the multiple waves: processing the wave data to obtain respective wave parameters; following impact of the wave with the platform, recording, over a predefined period, starting from a time of impact, a series of data samples of a platform state for each one of multiple degrees of freedom (DOF) thereby generating a respective platform response vector; adding to a training dataset the respective wave parameters and the respective platform response vector; training a machine leaning (ML) model with the training dataset; executing a platform state prediction procedure, comprising: receiving wave data recorded by at least one data acquisition device over a certain detection range and obtaining a respective waves-snapshot; for each wave in the waves-snapshot: processing the wave data to obtain respective wave parameters; determining a respective platform response vector, by applying the wave parameters to the ML model; and 25 – 27 – aggregating the platform response vectors of all waves in the respective waves-snapshot to thereby determine a predicted platform state following impact of all waves in the waves-snapshot.

2. The computer implemented method of claim 1, comprising: repeating the platform state prediction procedure, comprising: obtaining an updated waves-snapshot; and determining an updated predicted platform state following impact of all waves in the updated waves-snapshot.

3. The computer implemented method of claim 2, comprising: for each iteration of the platform state prediction procedure, determining whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, if so, terminating execution of the platform state prediction procedure.

4. The computer implemented method of claim 2, comprising: for each iteration of the platform state prediction procedure, determining whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, responsive to determining that the predicted platform state is suitable for landing, transmitting landing instructions to the VTOL to synchronize landing with the predicted platform state.

5. The computer implemented method of claim 1, comprising: determining whether the predicted platform state is suitable for landing a VTOL aircraft, and, responsive to determining that the predicted platform state is suitable for landing, transmitting landing instructions to the VTOL aircraft to synchronize landing with the predicted platform state.

6. The computer implemented method of any one of the preceding claims, wherein the multiple degrees of freedom include 6 degrees of freedom. – 28 –

7. The computer implemented method of any one of the preceding claims, comprising: for each waves-snapshot captured at a time t, recording a corresponding current platform state of the platform at time t and aggregating the corresponding current platform state with the platform response vectors of all waves in the respective waves-snapshot.

8. The computer implemented method of any one of the preceding claims, wherein processing wave data to obtain respective wave parameters includes identifying individual waves and determining their respective wave parameters.

9. The computer implemented method of any one of the preceding claims, wherein executing a learning procedure comprises determining time of impact of each wave with the platform.

10. The computer implemented method of any one of the preceding claims, wherein environmental data further incudes wind data, and wherein executing a learning procedure comprises: recording wind data, and adding the wind data to the training set.

11. The computer implemented method of any one of the preceding claims, wherein executing the learning procedure comprises: applying a transformation function on each series of data samples to thereby transform the data samples from a time domain to a frequency domain.

12. The computer implemented method of claim 11, wherein executing the learning procedure comprises: compressing the transformed data samples using quantization to obtain a finite set of quantized values.

13. The computer implemented method of any one of the preceding claims, wherein each platform response vector, determined for each wave in the – 29 – waves-snapshot during the platform state prediction procedure, includes transformed data samples, and where executing the platform state prediction comprises: applying an inverse transformation function on a series of transformed data samples in each platform response vector.

14. The computer implemented method of any one of the preceding claims, wherein recording a series of data samples of a platform state for each one of multiple degrees of freedom is performed with the help of, one or more positioning devices, including one or more of an inertial navigation system (INS) and a GPS.

15. The computer implemented method of claim 14 comprising fixing the INS to a landing deck on the platform.

16. The computer implemented method of any one of the preceding claims, wherein the data acquisition device includes one or more of RADAR, camera (e.g., 3D camera) and buoy.

17. The computer implemented method of any one of the preceding claims, further comprising monitoring at least one physical property of the platform and initiating the learning procedure responsive to receiving data indicative of a significant change in the at least one physical property.

18. The computer implemented method of any one of the preceding claims, wherein the platform is a ship.

19. The computer implemented method of any one of the preceding claims, wherein the ML model is a convolutional neural network.

20. The computer implemented method of any one of the preceding claims, wherein the ML model provides a response function that does require, as input, information on physical properties of the platform.

21. The computer implemented method of any one of the preceding claims wherein the multiple DOF include 6 DOF. – 30 –

22. A system mountable on a marine platform configured for predicting a state of a marine platform, the system comprising at least one data acquisition device operatively connected to at least one computer; the at least one data acquisition device is configured to capture environmental data, including wave data of multiple waves approaching the platform; the computer is configured to: execute a learning procedure, comprising: for each wave of the multiple waves: process the wave data to obtain respective wave parameters; following impact of the wave with the platform, record over a predefined period, starting from a time of impact, a series of data samples of a platform state for each one of multiple degrees of freedom (DOF) thereby generating a respective platform response vector; add to a training dataset the respective wave parameters and the respective platform response vector; train a machine leaning model with the training dataset; execute a platform state prediction procedure, comprising: receive, from the at least one data acquisition device, wave data captured over a certain detection range, and determine a respective waves-snapshot; for each wave in the waves-snapshot: process the wave data to obtain respective wave parameters; determine a respective platform response vector, by applying the wave parameters to the ML model; and – 31 – aggregate the platform response vectors of all waves in the respective waves-snapshot to thereby determine a predicted platform state following impact of all waves in the waves-snapshot.

23. The system of claim 22 wherein the at least one computer is configured to: execute the platform state prediction procedure repeatedly, comprising: obtaining an updated waves-snapshot; and determining an updated predicted platform state following impact of all waves in the updated waves-snapshot.

24. The system of claim 23, wherein the at least one computer is configured to: for each iteration of the platform state prediction procedure, determine whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, if so, terminate execution of the platform state prediction procedure.

25. The system of claim 23, wherein the at least one computer is configured to: for each iteration of the platform state prediction procedure, determine whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, responsive to determining that the predicted platform state is suitable for landing, transmit landing instructions to the VTOL aircraft to enable synchronization of landing with the predicted platform state.

26. The system of claim 22, wherein the at least one computer is configured to: determine whether the predicted platform state is suitable for landing a VTOL aircraft, and, responsive to determining that the predicted platform state is suitable – 32 – for landing, sending landing instructions to the VTOL aircraft to synchronize landing with the predicted platform state.

27. The system of any one of claims 22 to 26, wherein the multiple degrees of freedom include 6 degrees of freedom.

28. The system of any one of claims 22 to 27, wherein the at least one computer is configured to record, for each waves-snapshot captured at a time t, a corresponding current platform state of the platform at time t and aggregate the corresponding current platform state with the platform response vectors of all waves in the respective waves-snapshot.

29. The system of any one of claims 22 to 28, wherein processing wave data to obtain respective wave parameters includes identifying individual waves and determining their respective wave parameters.

30. The system of any one of claims 22 to 29, wherein executing a learning procedure comprises determining time of impact of each wave with the platform.

31. The system of any one of claims 22 to 30, wherein environmental data further incudes wind data, and wherein the at least one computer is further configured to add wind data to the training set.

32. The system of any one of claims 22 to 31, wherein the at least one computer is further configured, for executing the learning procedure, to: apply a transformation function on each series of data samples, to thereby obtain transform data samples from a time domain to a frequency domain.

33. The system of claim 32, wherein the at least one computer is further configured for executing a learning procedure to compress the transformed data samples, using quantization to obtain a finite set of quantized values.

34. The system of any one of claims 22 to 33, wherein each platform response vector, determined for each wave in the waves-snapshot during the platform state prediction procedure, includes transformed data samples, and wherein – 33 – the at least one computer is configured for executing platform state prediction to apply an inverse transformation function on a series of transformed data samples in each platform response vector.

35. The system of any one of claims 22 to 34 comprising at least one positioning device configured to provide a 6 DOF platform state, and wherein the recording a series of data samples of platform state for each one of multiple degrees of freedom is done with the help of the at least one positioning device.

36. The system of any one of claims 22 to 35, wherein at least one positioning device includes an attitude determination device that is fixed to a landing deck on the platform.

37. The system of any one of claims 22 to 36, wherein the data acquisition device includes one or more of RADAR, camera (e.g., 3D camera), and buoy.

38. The system of any one of claims 22 to 37 comprising a platform gauge configured to monitor at least one physical property of the platform, and wherein the at least one computer is configured to restart the learning procedure responsive to receiving data indicative of a significant change in the at least one physical property.

39. The system of any one of claims 22 to 38, wherein the platform is a ship.

40. The system of any one of claims 22 to 39, wherein the ML model provides a response function that does require, as input, information on physical properties of the platform.

41. A computer program product comprising a computer readable storage medium retaining a program of instructions, which program of instructions when read by a computer process causes the computer processor to perform a method according to any one of claims 1 to 21.

42. A non-transitory program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform – 34 – a method according to any one of claims 1 to 21.

43. A non-transitory program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform a method comprising: using at least one data acquisition device for obtaining environmental data, including wave data of multiple waves approaching a marine platform; executing a platform state prediction procedure, comprising: receiving wave data recorded by at least one data acquisition device over a certain detection range and obtaining a respective waves-snapshot; for each wave in the waves-snapshot: processing the wave data to obtain respective wave parameters; determining a respective platform response vector, by applying the wave parameters to a machine learning model trained to receive wave parameters and determine data indicative of a respective platform response vector; and aggregating the platform response vectors of all waves in the respective waves-snapshot to thereby determine a predicted platform state following impact of all waves in the waves-snapshot.

44. The non-transitory program storage device of claim 43, wherein the method further comprising: executing a learning procedure, comprising: for each wave of multiple waves captured by the data acquisition device: processing the wave data to obtain respective wave parameters; – 35 – following impact of the wave with the platform, recording, over a predefined period, starting from a time of impact, a series of data samples of a platform state for each one of multiple degrees of freedom (DOF) thereby generating a respective platform response vector; adding to a training dataset the respective wave parameters and the respective platform response vector; and training the machine leaning model with the training dataset.

45. The non-transitory storage device of claim 43, wherein the method comprising: repeating the platform state prediction procedure, comprising: obtaining an updated waves-snapshot; and determining an updated predicted platform state following impact of all waves in the updated waves-snapshot.

46. The non-transitory storage device of claim 45, comprising: for each iteration of the platform state prediction procedure, determining whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, if so, terminating execution of the platform state prediction procedure.

47. The non-transitory storage device of claim 45, comprising: for each iteration of the platform state prediction procedure, determining whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, responsive to determining that the predicted platform state is suitable for landing, transmitting landing instructions to the VTOL to synchronize landing with the predicted platform state.

48. The non-transitory storage device of claim 43, comprising: – 36 – determining whether the predicted platform state is suitable for landing a VTOL aircraft, and, responsive to determining that the predicted platform state is suitable for landing, transmitting landing instructions to the VTOL aircraft to synchronize landing with the predicted platform state.

49. The non-transitory storage device of any one of claims 43 to 48, wherein the multiple degrees of freedom include 6 degrees of freedom. For the Applicant, REINHOLD COHN AND PARTNERS By: