WO2024252393A1 - Computer-based prediction of a marine landing platform state - Google Patents
Computer-based prediction of a marine landing platform state Download PDFInfo
- Publication number
- WO2024252393A1 WO2024252393A1 PCT/IL2024/050553 IL2024050553W WO2024252393A1 WO 2024252393 A1 WO2024252393 A1 WO 2024252393A1 IL 2024050553 W IL2024050553 W IL 2024050553W WO 2024252393 A1 WO2024252393 A1 WO 2024252393A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- platform
- waves
- wave
- data
- landing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B63—SHIPS OR OTHER WATERBORNE VESSELS; RELATED EQUIPMENT
- B63B—SHIPS OR OTHER WATERBORNE VESSELS; EQUIPMENT FOR SHIPPING
- B63B79/00—Monitoring properties or operating parameters of vessels in operation
- B63B79/10—Monitoring properties or operating parameters of vessels in operation using sensors, e.g. pressure sensors, strain gauges or accelerometers
- B63B79/15—Monitoring properties or operating parameters of vessels in operation using sensors, e.g. pressure sensors, strain gauges or accelerometers for monitoring environmental variables, e.g. wave height or weather data
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B63—SHIPS OR OTHER WATERBORNE VESSELS; RELATED EQUIPMENT
- B63B—SHIPS OR OTHER WATERBORNE VESSELS; EQUIPMENT FOR SHIPPING
- B63B79/00—Monitoring properties or operating parameters of vessels in operation
- B63B79/40—Monitoring properties or operating parameters of vessels in operation for controlling the operation of vessels, e.g. monitoring their speed, routing or maintenance schedules
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C13/00—Surveying specially adapted to open water, e.g. sea, lake, river or canal
- G01C13/002—Measuring the movement of open water
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Definitions
- the presently disclosed subject matter relates to the landing of an aircraft on a marine platform.
- VTOL Vertical take off and landing
- mission such as maritime security, search and rescue, surveillance, and others.
- the presently disclosed subject matter includes a system and method designated for predicting the state of a marine landing platform.
- platform state refers to a plurality of degrees of freedom that define the state of a landing platform by position and orientation in three-dimensional space.
- degrees of freedom There are six degrees of freedom (6 DOF), including three degrees of freedom that describe the position of an object in the latitude (X), longitude (Y), and altitude (Z) directions, and three degrees of freedom that describe the orientation of an object in terms of rotations around the X, Y, and Z axes, otherwise known as pitch, roll, and yaw (where x, y and z refer to a certain reference frame e.g., Earth-Centered, Earth-Fixed).
- 6 DOF platform state 6 DOF platform state
- degrees of freedom e.g., 5 or 4 degrees of freedom
- the disclosed system includes one or more data acquisitioning devices configured to monitor the water around the marine landing platform and collect environmental data that includes information on incoming waves.
- the collected information is fed as input to a machine learning model trained to provide a predicted platform state of the marine landing platform (e.g., 6 DOF platform state) following impact of one or more incoming waves on the platform.
- the system is configured to obtain wave data pertaining to a sequence of incoming waves approaching the landing platform, calculate the predicted change to the 6 DOF platform state of the landing platform caused by the impact of individual waves in the sequence of incoming waves, and aggregate the changes caused by multiple waves in the sequence, to thereby provide prediction of waves impact on the 6 DOF platform state of the landing platform over a relatively long prediction period.
- FIG. 1 shows a schematic illustration of a VTOL aircraft 110 before and during landing on a ship 100. While the presently disclosed subject is advantageous for the operation of any VTOL aircraft, Fig. 1 illustrates a non-limiting example of an aircraft that activates a VTOL propulsion unit before landing.
- an aircraft that activates VTOL propulsion before landing is an aircraft with a separate VTOL propulsion unit dedicated only for landing and take-off, that is engaged before landing, to provide vertical descent and landing capabilities to the aircraft.
- a tiltrotor aircraft which utilizes one or more rotor units comprising a powered rotor (sometimes called proprotors) and respective tiltable structures on which the rotor is mounted.
- the rotors are used for both lift and propulsion, wherein different effects are achieved when the rotors are tilted between a general vertical direction and a general horizontal direction, and in intermediate directions. Tilting of the rotors enables implementing, at certain times, vertical lift capability, which enables VTOL, and at other times, propellor propulsion usually associated with conventional fixed-wing aircraft.
- a tiltrotor aircraft initiates a transition process from a first flight mode, in which the tiltable rotors of the aircraft are directed to provide thrust in the general horizontal direction, to a second flight mode in which the tiltable rotors are directed in the general vertical direction and enable vertical landing on deck.
- the operation of both aircrafts mentioned above may involve an activation process (e.g., activation of a dedicated VTOL unit according to first example or transition from one mode to the other according to the second example) of the VTOL unit prior to landing that may take, in some examples, anywhere between several seconds up to several minutes.
- VTOL is less efficient than fixed wing flight.
- the duration of its activation is often restricted due to the excessive consumption of resources associated with such operation.
- the vehicle may be less stable and may be more vulnerable to winds. Therefore, it is advantageous to initiate activation of VTOL propulsion right before landing to keep its activation as short as possible.
- safe landing can be executed only with a suitable platform state
- prediction of a future platform state needs to be made for a sufficiently long prediction period that provides more time for completion of the activation process of the VTOL unit and landing.
- a computer implemented method of predicting a state of a marine platform comprising: using at least one data acquisition device for obtaining environmental data, including wave data of multiple waves approaching the platform; executing a learning procedure, comprising: for each wave of the multiple waves: processing the wave data to obtain respective wave parameters; following impact of the wave with the platform, recording, over a predefined period, starting from a time of impact, a series of data samples of the platform state for each one of multiple degrees of freedom (DOF), thereby generating a respective platform response vector; adding to a training dataset the respective wave parameters and the respective platform response vector (or a derivative thereof, e.g., by applying DCT); training a machine leaning model with the training dataset; executing a platform state prediction procedure, comprising: receiving wave data recorded by at least one data acquisition device over a certain detection range and obtaining a respective waves-snapshot; for each wave in the waves-snapshot: processing the wave data to obtain respective wave parameters;
- the method according to this aspect of the presently disclosed subject matter can optionally comprise one or more of features (i) to (xx) listed below, in any technically possible combination or permutation: i.
- the computer implemented method comprising wherein the multiple DOF is 6 DOF.
- the computer implemented method comprising: repeating the platform state prediction procedure, comprising: obtaining an updated wave-snapshot; and determining an updated predicted platform state following impact of all waves in the updated waves-snapshot.
- the computer implemented method comprising: for each iteration of the platform state prediction procedure, determining whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, if so, terminating execution of the platform state prediction procedure. iv.
- the computer implemented method comprising: for each iteration of the platform state prediction procedure, determining whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, responsive to determining that the predicted platform state is suitable for landing, transmitting landing instructions to the VTOL aircraft to synchronize landing with the predicted platform state.
- the computer implemented method comprising: determining whether the predicted platform state is suitable for landing the VTOL aircraft, and, responsive to determining that the predicted platform state is suitable for landing, transmitting landing instructions to the VTOL to synchronize landing with the predicted platform state.
- the computer implemented method comprising: for each waves-snapshot captured at a time t, recording the corresponding current platform state of the platform at time t and aggregating the corresponding current platform state with the platform response vectors of all waves in the respective waves-snapshot.
- processing wave data to obtain respective wave parameters includes identifying individual waves and determining their respective wave parameters.
- executing a learning procedure comprises determining time of impact of each wave with the platform.
- environmental data further incudes wind data and wherein executing a learning procedure comprises: recording wind data; and adding the wind data to the training set.
- the computer implemented method wherein executing the learning procedure comprises: applying a transformation function on each series of data samples to thereby transform the data samples from a time domain to a frequency domain.
- the computer implemented method wherein the learning procedure comprises compressing the transformed data samples using quantization to obtain a finite set of quantized values.
- xii The computer implemented method, wherein each platform response vector determined for each wave in the waves-snapshot during the platform state prediction procedure, includes transformed data samples, and where executing the platform state prediction comprises: applying an inverse transformation function on series of transformed data samples in each platform response vector.
- the computer implemented method wherein recording a series of data samples of platform state for each one of multiple degrees of freedom is performed with the help of one or more positioning devices, including one or more of an inertial navigation system (INS) and a GPS.
- the computer implemented method comprising fixing the INS to a landing deck on the platform.
- the data acquisition device includes one or more of: RADAR, camera (e.g., 3D camera), acoustic sensor, and buoy.
- the computer implemented method further comprising monitoring at least one physical property of the platform and initiating the learning procedure responsive to receiving data indicative of a significant change in the at least one physical property.
- the computer implemented method wherein the platform is any one of: ship, boat, floating landing dock, or pad.
- the ML model is a convolutional neural network.
- the ML model provides a response function that does require, as input, information on physical properties of the platform.
- the training procedure is executed following embarkment of the marine platform or responsive to receiving information indicating that a VTOL aircraft is about to land on the marine platform.
- a system mountable on a marine platform configured for predicting a state of a marine platform
- the system comprising at least one data acquisition device operatively connected to at least one computer; the at least one data acquisition device is configured to capture environmental data, including wave data of multiple waves approaching the platform; the computer is configured to: execute a learning procedure, comprising: for each wave of the multiple waves: process the wave data to obtain respective wave parameters; following impact of the wave with the platform, record over a predefined period, starting from a time of impact, a series of data samples of platform state for each one of multiple degrees of freedom (DOF), thereby generating a respective platform response vector; add to a training dataset the respective wave parameters and the respective platform response vector; train a machine leaning model with the training dataset; execute a platform state prediction procedure, comprising: receive from the at least one data acquisition device wave data captured over a certain detection range and determine a respective waves- snapshot; for each wave in the waves-snapshot: process the wave data to
- a computer program product comprising a computer readable storage medium retaining program instructions, which programs instructions when read by a processor, and causes the processor to perform the method according to the first aspect mentioned above.
- a non-transitory program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform the method according to the first aspect mentioned above.
- the system, the computer program product, and the non-transitory program storage device disclosed in accordance with the aspects of the presently disclosed subject matter detailed above can optionally comprise one or more of features (i) to (xx) listed above, mutatis mutandis, in any technically possible combination or permutation.
- a computer program product comprising a computer readable storage medium retaining a program of instructions, which program of instructions when read by a computer processor causes the computer processor to perform a method, comprising: using at least one data acquisition device for obtaining environmental data, including wave data of multiple waves approaching the platform; executing a platform state prediction procedure, comprising: receiving wave data recorded by at least one data acquisition device over a certain detection range and obtaining a respective waves-snapshot; for each wave in the waves-snapshot: processing the wave data to obtain respective wave parameters; determining a respective platform response vector, by applying the wave parameters to a machine learning model trained to receive wave parameters and determine data indicative of a respective platform response vector; and aggregating the platform response vectors of all waves in the respective waves-snapshot to thereby determine a predicted platform state following impact of all waves in the waves-snapshot.
- the program of instructions retained on the computer program product when read by a computer processor causes the computer processor to perform a method further comprising: executing a learning procedure, comprising: for each wave of multiple waves captured by the data acquisition device: processing the wave data to obtain respective wave parameters; following impact of the wave with the platform, recording, over a predefined period, starting from a time of impact, a series of data samples of a platform state for each one of multiple degrees of freedom (DOF) thereby generating a respective platform response vector; adding to a training dataset the respective wave parameters and the respective platform response vector; and training the machine leaning model with the training dataset.
- DOF degrees of freedom
- FIG. 1 is a schematic illustration of a landing scenario of a VTOL aircraft onboard a ship, in accordance with certain examples of the presently disclosed subject matter;
- Fig. 2 illustrates a block diagram of a system mountable on a marine vehicle, in accordance with certain examples of the presently disclosed subject matter
- FIG. 3 illustrates a block diagram of a platform state prediction computer, in accordance with certain examples of the presently disclosed subject matter
- FIG. 4 illustrates a flow-chart of operations carried out during ML training, in accordance with certain examples of the presently disclosed subject matter
- Fig. 5 illustrates sampled data of 6 DOF platform state, in accordance with certain examples of the presently disclosed subject matter.
- Fig. 6 illustrates a flow-chart of operations carried out during prediction of a platform state, in accordance with certain examples of the presently disclosed subject matter.
- the processing circuitry can comprise, for example, one or more computer processors operatively connected to computer memory, loaded with executable instructions for executing operations as further described below.
- Operations in accordance with the teachings herein may be performed by a computer specially constructed for the desired purposes, or by a general-purpose computer specially configured for the desired purpose by a computer program stored in a computer readable storage medium, (e.g., landing deck on ship)
- a computer readable storage medium e.g., landing deck on ship
- the phrase “for example,” “such as”, “for instance” and variants thereof, describe non-limiting embodiments of the presently disclosed subject matter.
- Reference in the specification to “one case”, “some cases”, “other cases”, or variants thereof, means that a particular feature, structure, or characteristic described in connection with the embodiment(s), is included in at least one embodiment of the presently disclosed subject matter.
- the appearance of the phrase “one case”, “some cases”, “other cases”, or variants thereof, does not necessarily refer to the same embodiment(s).
- fewer, more and/or different stages than those shown in Figs. 4 and 6 may be executed.
- one or more stages in the illustrated figures may be executed in a different order, and/or one or more groups of stages may be executed simultaneously.
- Figs. 2 and 3 schematically illustrate a system architecture in accordance with certain examples of the presently disclosed subject matter. Elements in Figs. 2 and 3 can be made up of any suitable combination of software and hardware and/or firmware that performs the functions as defined and explained herein. In examples of the presently disclosed subject matter, other than what is illustrated, the system may comprise fewer, more, and/or different elements than those shown in Figs. 2 and 3. Likewise, the division of the functionality of the disclosed system to specific parts, as described below with reference to Figs. 2 and 3, is provided by way of example, and other various alternatives are also construed within the scope of the presently disclosed subject matter. For example, while Figs. 2 and 3 show a single computer dedicated for both training the machine learning model and executing the trained model, in other examples two or more separate computers can be used, e.g., one for training the model, and the other for execution of the model.
- VTOL aircraft should be broadly construed to include any aerial vehicle with vertical lift that can execute vertical landing, including both manned VTOL aircrafts and unmanned (autonomous) VTOL aircrafts.
- VTOL aircraft include, for example, helicopters, tiltrotor aircraft (e.g., Bell Boeing V-22 Osprey and the Bell Nexus), drones of all kinds (e.g., multirotor drones), VTOL jets (e.g., Harrier Jump Jet and the F-35B Lightning II), flying cars (e.g., PAL-V Liberty), etc.
- marine landing platform or “landing platform” as used herein should be broadly construed to include any type of platform or vessel that floats on water and can be used as a landing surface for a VTOL aircraft.
- a marine landing platform includes a landing deck and may include certain features that support VTOL, such as a non-skid surface, tie-down points, and deck lighting that facilitate nighttime landing.
- a marine landing platform can be, for example, a ship or other marine vehicle (typically comprising a designated landing deck), or a floating landing dock or pad.
- Marine landing platforms for VTOL aircrafts are used for various civilian applications, such as offshore oil and gas discovery and extraction, search and rescue missions, and emergency medical services.
- System 200 is configured in general to predict a future platform state, determine a time when the future platform state is suitable for safe landing of the VTOL aircraft, and coordinate the landing of the VTOL according to the determined time.
- System 200 comprises one or more data acquisition devices 210, including for example one or more of: camera system(s), a RADAR system(s), acoustic sensor (e.g., Acoustic Doppler Current Profiler (ADCP)) buoy(s) (e.g., wave monitoring buoys) etc., configured to monitor the waves around the platform and determine waves data including wave parameters such as, waves height (amplitude) and waves velocity (i.e., speed and direction).
- Data acquisition devices 210 may further include a wind sensor (anemometer) configured to determine wind velocity.
- the camera and/or radar can be positioned to capture images of waves as they approach.
- a camera system consists of a set of multiple cameras (e.g., 3 cameras or more) for obtaining 3-dimensional imaging of the waters.
- the camera can be for example a highspeed camera operatively connected to an image processing circuitry.
- the camera can be equipped with a specialized lens that allows it to capture high-resolution images of waves, which can be processed as further described below.
- System 200 also comprises positioning devices 220 capable of determining a platform state (6 DOF platform state) with respect to some reference frame, including for example one or more of inertial navigation system (INS), GPS receiver, and a cellular localization system.
- INS inertial navigation system
- An INS is configured to continuously monitor the attitude of the platform, and, in some cases, it is also used for determining its location.
- the INS (or the inertial measurement unit of an INS, or some other altitude determining device) is mounted and fixed directly to a landing deck, which is the area dedicated for VTOL landing on the landing platform (e.g., landing deck on a ship), in order to have direct contact with the platform where the landing takes place, to enable direct monitoring of a 6 DOF platform state (or at least the 3 attitude components) at the place of landing, and thereby increase accuracy and reduce complexity of the 6 DOF data collection process.
- a landing deck which is the area dedicated for VTOL landing on the landing platform (e.g., landing deck on a ship), in order to have direct contact with the platform where the landing takes place, to enable direct monitoring of a 6 DOF platform state (or at least the 3 attitude components) at the place of landing, and thereby increase accuracy and reduce complexity of the 6 DOF data collection process.
- System 200 further includes a platform state prediction computer 230 configured to predict future platform state of the landing platform, as further explained below.
- System 200 also comprises communication unit 240 configured to facilitate communicating, e.g., with the VTOL aircraft for sending to the aircraft landing instructions once a platform state suitable for landing is predicted.
- Fig. 3 is a block diagram showing a more detailed description of components of platform state prediction computer 230, according to some examples of the presently disclosed subject matter.
- the architecture in Fig. 3 is a general example which demonstrates various principles of the presently disclosed subject matter, while in practice it may vary in design and therefore should not be construed as limiting.
- Computer 230 can comprise one or more processing circuitries, each circuitry comprising one or more computer processors operatively connected to a computer storage medium.
- the processing circuitries can be configured, for example, to execute functional modules in accordance with computer-readable instructions implemented on a non-transitory computer-readable storage medium. For simplicity, such functional modules are referred to hereinafter as components of the computer.
- a detailed description of the operations executed by platform state prediction computer 230 is disclosed below with reference to the following figures.
- FIG. 4 shows a flowchart of operations carried out during a training procedure 400, in accordance with examples of the presently disclosed subject matter.
- operations in Fig. 4 are described with reference to components of the system shown in Figs. 2 and 3, however this should not be construed to limit the method to any specific system design or component.
- ship is used to demonstrate a marine landing platform for simplicity and by way of non-limiting example only.
- Fig. 4 is related to the training of a machine learning model capable of predicting a future state of a marine landing platform in response to impact of waves.
- the training process can be initiated for example, prior to the landing of a VTOL aircraft, and in some cases may require a short initial training period (also referred to as calibration period). While additional training (training over longer periods) using additional training data may improve accuracy and robustness of the machine learning model output, the machine learning model may provide sufficiently accurate output that can be used right after the calibration period. Longer training periods, possibility in a variety of wave conditions, provide a more robust model that enables ship state prediction in a greater diversity of wave conditions.
- training can be initiated at the beginning of operation (e.g., following embarkment of the ship from a dock) and often the same model can be used throughout the mission while the ship is at sea (while optionally the model is being trained continuously to reinforce the model over time).
- training can be initialized prior to landing.
- training can be initiated in response to receiving a message at the ship indicating that a VTOL aircraft intends to land on the ship shortly (e.g., within the next 30-60 minutes).
- the calibration period can range from a few minutes (e.g., 10-30 minutes) to a few hours.
- Data collection used for training of the ML model can be done in real-time during the training process and does not require to obtain a training dataset prior to initiation of training.
- the machine learning model is trained using data collected in real-time by devices onboard the ship, including environmental data and platform state data, without a need for data pertaining to the geometric and other physical properties of the ship, such as length, type width, keel length, deck depth, overall weight (tonnage), planned velocity, and planned depth, which are required for determining a transfer function.
- system 200 includes a gauge module (ship gauge 250) configured for monitoring certain physical properties of the ship (e.g., by directly measuring the properties or being connected to another system or sensor capable of providing data on the properties). If the gauge module receives information indicative of a significant change in a certain property of the ship (e.g., net weight), it may signal the platform state prediction computer 230 that retraining of the model is required, resulting in retraining the ML model.
- a gauge module shipment gauge 250
- the gauge module receives information indicative of a significant change in a certain property of the ship (e.g., net weight)
- the platform state prediction computer 230 may signal the platform state prediction computer 230 that retraining of the model is required, resulting in retraining the ML model.
- the water surrounding the ship is monitored using one or more data acquisition devices 210, and raw environmental data is recorded.
- Environmental data includes wave data and possibly also wind data.
- the raw wave data may include corresponding output, such as imaging data output, RADAR data output, etc.
- an acquisition device is positioned on the ship such that it can view and capture waves which are approaching the ship.
- the data acquisition device is configured to capture wave data from multiple directions, for example by using multiple cameras pointed in different (e.g., orthogonal) directions, or by using a stabilization and orientation control mechanisms such as gimbal or pan and tilt mechanism allowing to change the viewing angle of the data acquisition device.
- the raw wave data collected by the data acquisition devices is processed and individual waves are recognized (block 410).
- the respective wave parameters of each wave are determined (block 403).
- Wave parameters include wave height and wave velocity i.e., speed and direction.
- wave parameters may further include wavelength and/or wave period.
- the imaging output can be processed using various software such as edge detection, pattern recognition, and image segmentation, together with other dedicated software for wave identification and characterization.
- machine learning models trained for detecting waves and determining wave parameters from wave data can be used.
- the obtained wave parameters can be stored in a computer data repository (350) operatively connected to computer 230.
- wave data is captured and processed in real-time while the waves are approaching the ship.
- Identified waves are tracked to determine a respective time of impact with the ship, and, immediately following impact, the 6DOF platform state of the platform is recorded (block 405).
- the time of impact can be determined, for example, by monitoring the wave's velocity and determining based on the wave velocity and its distance from the ship at the time of impact.
- 6 DOF position and orientation of the ship is determined using appropriate navigation devices, such as INS and GPS.
- one or more crests or leading edges are identified and tracked as they approach the ship.
- the distance between such reference points and the ship is continuously monitored, along with the wave's velocity.
- Individual waves can be distinguished from one another by identifying the troughs or low points between consecutive crests.
- the distance between two adjacent troughs can be considered the wavelength, which helps separate one wave from the next. This allows for the tracking and analysis of individual waves within a continuous wave field.
- Operations described with respect to block 410, 403, and wave tracking in block 405, can be executed for example by water state processing module 310 configured to process the raw data received from data acquisition devices 210.
- the 6 DOF platform state recorded for a certain wave includes a platform response vector.
- the 6 DOF position and orientation of the ship is sampled over a certain time window, to thereby obtain, for each wave, the respective platform response vector.
- the platform response vector includes a collection of a series of samples, recorded over the length of the time window, each series corresponding to one degree of freedom of a certain position or orientation, including a first series of samples of movement along the latitude (X) axis; a second series of samples of movement along the longitude (Y) axis, a third series of samples of movement along the altitude (Z) axis, a fourth series of samples for pitch, a fifth series of samples for roll, and a sixth series of samples yaw.
- the length of the time window generally depends on the time between consecutive waves (wave period), but in some cases may be limited by some upper bound or may be otherwise selected.
- Fig. 5 it shows a graph plotting an example of 6 series of sampled data over a time window (x-axis), each corresponding to a certain axis of movement or axis of rotation of the ship.
- series 1 represents sampled data of the height axis (up and down direction - altitude)
- series 2 represents sampled data of the sway axis (side to side direction - latitude)
- series 3 represents sampled data of the surge axis (forward backwards direction - longitude)
- series 4 represents sampled data of the delta yaw axis
- series 5 represents sampled data of the pitch axis
- series 6 represents sampled data of the roll axis.
- the collection of series represents a platform response vector.
- Operations described with respect to blocks 407, 409 can be executed for example by platform state processing module 320.
- a transformation function is applied on the 6 DOF platform state for transforming the 6 DOF platform state from the time domain to the frequency domain (block 407).
- data transformation is achieved by applying a Fourier Transform on each series in the respective response vector.
- Transformation of the sampled data to the frequency domain can be performed for example by applying a discrete cosine transform (DCT) exemplified by the following expression.
- DCT discrete cosine transform
- Equation 1 where X k is the signal in the frequency domain and x n is the signal in the time domain. N is the number of samples in the series. As shown in Fig. 5 each sequence of samples is represented as a signal over time.
- the transformation output is compressed using quantization to obtain a finite set of quantized values, which helps to simplify and increase efficiency of the training of the machine learning model. Transformation and quantization can be executed for example by transformation module 330.
- the environmental data (including wave parameters) determined for a given wave are stored in association with the quantized values obtained from the corresponding 6 DOF platform state obtained for the same wave. This is repeated with many waves to thereby generate a training dataset, which can be stored in a computer data storage (block 409; computer data storage 350).
- the environmental data in the training dataset can also include the wind data that was measured at the time the wave data was recorded.
- wind velocity, measured during the time of impact of a certain wave is stored in the database in association with the wave.
- wind velocity measured during a certain period (e.g., every 2 minutes) is associated with all waves that hit the ship during this period.
- the machine learning model is trained using the training data stored in the storage (block 420).
- training can start in real-time as soon as training data is initially stored in the data storage and continue as more data is being processed and added to the storage. This is an advantage since it shortens the calibration period required for a obtaining a valid model that can be used for predictions.
- the ML model is a Convolutional Neural Networks (CNNs) employing several hidden layers.
- the DCT data (coefficients) can be formatted and normalized to be fitted to a CNN model.
- Training can be executed for example by ML training module 340.
- the trained model can be stored in a computer storage 355 accessible to ML execution model 360.
- prediction of ship state as disclosed herein, using the machine learning model as described above allows for the creation of a general and robust model that can predict the ship state across a wide variety of wave conditions.
- the model can extrapolate the ship state resulting from the impact of previously unseen waves. For instance, if the model is trained on weak and medium waves, it can still predict the ship state in strong waves and if the model is trained on weak and strong waves, it can still predict the ship state in medium waves. Weak, medium, and strong waves can be defined based on wave parameters.
- This special ability is achieved by the transformation of the 6 DOF platform state to the frequency domain and providing the transformed data as input to a Convolutional Neural Network (CNN) model, as it enables to determine a relationship between wave characteristics and ship state frequencies.
- CNN Convolutional Neural Network
- the trained ML model can be used for predicting the ship's state.
- the trained ML model provides a response function which provides a correlation between the recorded wave data (and possibly wind data) and the respective platform response vector and does not require physical/geometric data of the platform as input during training and execution.
- Fig. 6 showing a flowchart of operations carried out during platform state prediction procedure 600, according to examples of the presently disclosed subject matter.
- the platform state prediction procedure can follow the calibration procedure, which, as mentioned earlier, can be relatively short. However, training of the model can continue in parallel to prediction to enhance the model and improve its output.
- the water surrounding the landing platform is monitored using one or more data acquisition devices 210, and wave data of waves approaching the ship is recorded.
- the raw wave data collected by the data acquisition devices is processed, individual waves are recognized (block 610), and the respective wave parameters of each wave are determined, including at least wave height and velocity (block 603).
- the data acquisition device 210 is configured and operable to obtain wave data of waves within a certain detection range from the ship.
- a detection range is between 50 to 2000 meters from the ship.
- Wave data captured within the detection range is processed and individual waves are identified. If the field of view (FOV) of the data acquisitions device covers an area which is greater than the detection range, the raw wave data can be processed to extract therefrom a part that corresponds to the detection range. If the field of view of the data acquisitions device covers an area which is smaller than the detection range, multiple frames covering the detection range can be captured.
- the data acquisition device output captured or otherwise obtained at a certain time instance t over the detection range is referred to herein as "waves-snapshot".
- the detection range together with waves speed, define the prediction period - the length of time for which the prediction is made.
- a larger detection range and/or slower waves provide a longer prediction period, whereas a smaller detection range and/or faster waves provide a shorter prediction period.
- the detection range can be adapted to meet specific constraints and needs. For example, if a longer prediction period is needed, the system can be configured to capture wave data over a longer detection range, and the opposite in case a shorter prediction period is needed.
- a suitable prediction period should be long enough to gather sufficient data for accurately predicting the platform's state, yet short enough to avoid errors that may arise from changes in wave conditions while approaching the platform.
- ML execution module 360 can be configured to receive input (e.g., from a pilot of an approaching aircraft, or from an operator onboard a ship where landing is about to occur) indicative of the desired prediction period and adapt the detection range according to the wave speed.
- wave speed can be determined in real-time by processing the captured wave data.
- the determined wave parameters are provided as input to the ML model, which provides output data indicative of corresponding 6 DOF platform state (block 605; e.g., by ML execution module 360). If features in the training dataset include quantized values of the transformed 6 DOF platform state (i.e., samples series of the respective wave response vector), the ML model output also includes quantized values of corresponding transformed 6 DOF platform state. In some examples, wind data is also recorded and provided as input to the ML model, together with the wave parameters, as explained above with respect to training.
- the obtained quantized values are processed using an inverse transformation function (e.g., inverse Discrete Cosine Transform (IDCT)) to thereby obtain the respective 6 DOF platform state e.g., the series of samples of the respective platform response vector (block 607, e.g., by post processing ML output module 370).
- an inverse transformation function e.g., inverse Discrete Cosine Transform (IDCT)
- IDCT inverse Discrete Cosine Transform
- the ML model output of 6 DOF platform state corresponding to the waves detected within the detection range are aggregated with the current 6 DOF platform state (i.e., state of the platform at time t when the relevant waves-snapshot has been recorded) to thereby obtain a predicted 6 DOF platform state over the distance of the detection range (block 609; e.g., by future platform state prediction module 380).
- the predicted platform state is a 6 DOF platform state that the ship will assume in the future when all waves detected within the detection range at a certain time t have reached the ship. Assuming, for the sake of example, that the detection range is 750 meters, and that at time t 0 , 10 waves (w 0 ...
- each wave induces a change to the ship's state that depends on its previous state before impact.
- the prediction procedure can be executed in a repetitive manner (as indicated by the arrow connecting block 609 to block 610), processing updated waves-snapshots as new waves enter the waves detection range and are captured.
- the data acquisition output captured at that time would include a different set of waves (i.e., an updated waves-snapshot).
- an updated waves-snapshot Assuming, in the next iteration, there is a one wave difference from the previous iteration, such that wave w 0 has already hit the ship, and wave w 10 (11 th wave) enters the detection range.
- the new waves-snapshot would include waves w ... w 1:L .
- the calculation is repeated, thistime determiningthe predicted 6 DOF platform state following impact of waves w and using a different current 6 DOF platform state that corresponds to the time of acquisition of waves
- the updated waves-snapshot is processed to obtain a more updated predicted platform state following impact of the waves in the updated waves-snapshot.
- the disclosed system can continuously monitor the sea and provide updated predicted platform states, one after the other.
- wave data of a series of waves within a detection range is obtained and translated to a corresponding future platform state the ship is predicted to assume following impact of the series of waves.
- each wave that hits the ship exerts certain forces on the ship that cause a certain movement to the ship that brings the ship to a certain 6 DOF platform state characterized by a certain position and orientation.
- the actual state of the ship following impact depends on the previous state of the ship. This is why, by aggregating the platform states, each resulting from the impact of a respective wave in the series of waves, and the corresponding current state, the future state of the ship, following impact of all waves in the series of waves, can be determined.
- Future platform state verification module 390 can be configured to receive the future 6 DOF platform state of the ship and determine, based on the respective parameters (e.g., orientation and position) and other information such the type of VTOL aircraft, that is about to land, and/or the topography characteristics (whether the water is shallow or deep) of the area surrounding the ship, and determine whether the predicted ship state is suitable for landing of the particular VTOL aircraft.
- future platform state verification module 390 is configured to use a ML learning model trained for determining suitability of a certain 6 DOF platform state of a platform for landing.
- the ML model can receive as input the future 6 DOF platform state of the platform predicted by system 200, as well as additional information, such as the type of the VTOL, and provide a "confirmed landing" or "denied landing” output.
- landing instructions are transmitted to the VTOL aircraft, informing the aircraft the pending landing time, and causing it to prepare for landing (block 613; e.g., data is transmitted to VTOL by communication unit 240). Otherwise, the system continues to process environmental data, calculate future vehicles states, and search for a future platform state that is suitable for landing.
- the prediction procedure is terminated until the need for a further landing arises. In some examples, the training procedure continues, even if landing has occurred and/or the prediction procedure is terminated, to further enhance the ML model and prepare of possible future landings.
- Missile launch Predicting the resting period of a ship's motion can help determine the optimal window for launching missiles from a naval vessel, ensuring the stability and accuracy of the launch.
- Cargo transfer When transferring cargo between ships or from a ship to a smaller vessel, knowing the expected resting period can help plan and execute the operation more efficiently and safely.
- Personnel boarding Predicting the ship's motion resting period can assist in determining the most suitable time for personnel to board or disembark the vessel, reducing the risk of injury due to unstable platform conditions.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Theoretical Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Mechanical Engineering (AREA)
- Combustion & Propulsion (AREA)
- Data Mining & Analysis (AREA)
- Ocean & Marine Engineering (AREA)
- Artificial Intelligence (AREA)
- Chemical & Material Sciences (AREA)
- Hydrology & Water Resources (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Automation & Control Theory (AREA)
- Atmospheric Sciences (AREA)
- Environmental & Geological Engineering (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Traffic Control Systems (AREA)
Abstract
The presently disclosed subject matter includes a computer method and system for predicting the state of a marine platform. It involves generating platform response vectors based on environmental data and platform state data across multiple degrees of freedom (DOF) after wave impact and training a machine learning model using the multiple platform response vectors. During the prediction procedure, recorded wave data is used to create a waves-snapshot, process the wave data to determine wave parameters, apply these to the ML model to generate platform response vectors, and aggregate these vectors to predict the platform state following wave impact.
Description
COMPUTER-BASED PREDICTION OF A MARINE
LANDING PLATFORM STATE
TECHNICAL FIELD
The presently disclosed subject matter relates to the landing of an aircraft on a marine platform.
BACKGROUND
Vertical take off and landing (VTOL) aircrafts are widely used today for various military and civilian operations that require landing in areas where traditional runways or landing strips are unavailable or cannot be used. For naval operations, the ability of VTOL aircrafts to take off and land on ships and other platforms at sea, without requiring a runway or airfield, is crucial. This capability is often important for successful completion of a mission, such as maritime security, search and rescue, surveillance, and others.
GENERAL DESCRIPTION
Due to the dynamic and sometimes extreme conditions encountered at sea, which may involve for example strong waves and winds, it is difficult to maintain a steady landing platform for a VTOL aircraft. Specifically, the motion of marine landing platforms such as ships, caused by the waves, may result in a continuous change in the orientation of the deck, unpredictability affecting the required position and orientation of the VTOL aircraft relative to the deck during landing, thus compromising safe landing of the aircraft. For these reasons, landing a VTOL aircraft on a ship at sea is a complex and challenging task that requires precise control and coordination. Therefore, it is important to consider the effects of waves on the motion of a marine landing platform during landing and identify an appropriate platform state that allows safe landing of the VTOL aircraft on the platform in both autonomous landing and human controlled landing.
The presently disclosed subject matter includes a system and method designated for predicting the state of a marine landing platform. The term "platform state" as used herein refers to a plurality of degrees of freedom that define the state of a landing platform by position and orientation in three-dimensional space. There are six degrees of freedom (6 DOF), including three degrees of freedom that describe the position of an object in the latitude (X), longitude (Y), and altitude (Z) directions, and three degrees of freedom that describe the orientation of an object in terms of rotations around the X, Y, and Z axes, otherwise known as pitch, roll, and yaw (where x, y and z refer to a certain reference frame e.g., Earth-Centered, Earth-Fixed). It is noted that while the following description predominantly refers to a platform state defined by 6 DOF ("6 DOF platform state") this should not be construed as limiting, as defining a platform state with less degrees of freedom (e.g., 5 or 4 degrees of freedom) is also contemplated within the scope of the presently disclosed subject matter.
The disclosed system includes one or more data acquisitioning devices configured to monitor the water around the marine landing platform and collect environmental data that includes information on incoming waves. The collected information is fed as input to a machine learning model trained to provide a predicted platform state of the marine landing platform (e.g., 6 DOF platform state) following impact of one or more incoming waves on the platform. Furthermore, the system is configured to obtain wave data pertaining to a sequence of incoming waves approaching the landing platform, calculate the predicted change to the 6 DOF platform state of the landing platform caused by the impact of individual waves in the sequence of incoming waves, and aggregate the changes caused by multiple waves in the sequence, to thereby provide prediction of waves impact on the 6 DOF platform state of the landing platform over a relatively long prediction period.
Referring to Fig. 1, it shows a schematic illustration of a VTOL aircraft 110 before and during landing on a ship 100. While the presently disclosed subject is advantageous for the operation of any VTOL aircraft, Fig. 1 illustrates a non-limiting example of an aircraft that activates a VTOL propulsion unit before landing. One
example of an aircraft that activates VTOL propulsion before landing is an aircraft with a separate VTOL propulsion unit dedicated only for landing and take-off, that is engaged before landing, to provide vertical descent and landing capabilities to the aircraft. Another example is a tiltrotor aircraft which utilizes one or more rotor units comprising a powered rotor (sometimes called proprotors) and respective tiltable structures on which the rotor is mounted. The rotors are used for both lift and propulsion, wherein different effects are achieved when the rotors are tilted between a general vertical direction and a general horizontal direction, and in intermediate directions. Tilting of the rotors enables implementing, at certain times, vertical lift capability, which enables VTOL, and at other times, propellor propulsion usually associated with conventional fixed-wing aircraft.
Often when a tiltrotor aircraft approaches a landing site such as a ship's deck, it initiates a transition process from a first flight mode, in which the tiltable rotors of the aircraft are directed to provide thrust in the general horizontal direction, to a second flight mode in which the tiltable rotors are directed in the general vertical direction and enable vertical landing on deck. The operation of both aircrafts mentioned above may involve an activation process (e.g., activation of a dedicated VTOL unit according to first example or transition from one mode to the other according to the second example) of the VTOL unit prior to landing that may take, in some examples, anywhere between several seconds up to several minutes.
In general VTOL is less efficient than fixed wing flight. Particularly, when a dedicated VTOL unit is engaged for landing (and take-off), the duration of its activation is often restricted due to the excessive consumption of resources associated with such operation. Furthermore, during VTOL the vehicle may be less stable and may be more vulnerable to winds. Therefore, it is advantageous to initiate activation of VTOL propulsion right before landing to keep its activation as short as possible. However, since safe landing can be executed only with a suitable platform state, prediction of a future platform state needs to be made for a sufficiently long prediction period that provides more time for completion of the activation process of the VTOL unit and
landing. These examples therefore demonstrate the need for a sufficiently long prediction period as disclosed herein. According to some non-limiting examples, the length of the prediction period is anywhere between 30 seconds to 3 minutes. According to other non-limiting examples, the length of the prediction period is anywhere between 30 seconds to 60 seconds.
According to one aspect of the presently disclosed subject matter there is provided a computer implemented method of predicting a state of a marine platform, the method comprising: using at least one data acquisition device for obtaining environmental data, including wave data of multiple waves approaching the platform; executing a learning procedure, comprising: for each wave of the multiple waves: processing the wave data to obtain respective wave parameters; following impact of the wave with the platform, recording, over a predefined period, starting from a time of impact, a series of data samples of the platform state for each one of multiple degrees of freedom (DOF), thereby generating a respective platform response vector; adding to a training dataset the respective wave parameters and the respective platform response vector (or a derivative thereof, e.g., by applying DCT); training a machine leaning model with the training dataset; executing a platform state prediction procedure, comprising: receiving wave data recorded by at least one data acquisition device over a certain detection range and obtaining a respective waves-snapshot;
for each wave in the waves-snapshot: processing the wave data to obtain respective wave parameters; determining a respective platform response vector, by applying the wave parameters to the ML model; and aggregating the platform response vectors of all waves in the respective waves-snapshot (and the corresponding current platform state) to thereby determine a predicted platform state following impact of all waves in the waves-snapshot.
In addition to the above features, the method according to this aspect of the presently disclosed subject matter can optionally comprise one or more of features (i) to (xx) listed below, in any technically possible combination or permutation: i. The computer implemented method comprising wherein the multiple DOF is 6 DOF. ii. The computer implemented method comprising: repeating the platform state prediction procedure, comprising: obtaining an updated wave-snapshot; and determining an updated predicted platform state following impact of all waves in the updated waves-snapshot. iii. The computer implemented method comprising: for each iteration of the platform state prediction procedure, determining whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, if so, terminating execution of the platform state prediction procedure. iv. The computer implemented method comprising: for each iteration of the platform state prediction procedure, determining whether the predicted platform state or the updated predicted platform
state is suitable for landing a VTOL aircraft, and, responsive to determining that the predicted platform state is suitable for landing, transmitting landing instructions to the VTOL aircraft to synchronize landing with the predicted platform state. v. The computer implemented method comprising: determining whether the predicted platform state is suitable for landing the VTOL aircraft, and, responsive to determining that the predicted platform state is suitable for landing, transmitting landing instructions to the VTOL to synchronize landing with the predicted platform state. vi. The computer implemented method comprising: for each waves-snapshot captured at a time t, recording the corresponding current platform state of the platform at time t and aggregating the corresponding current platform state with the platform response vectors of all waves in the respective waves-snapshot. vii. The computer implemented method, wherein processing wave data to obtain respective wave parameters includes identifying individual waves and determining their respective wave parameters. viii. The computer implemented method, wherein executing a learning procedure comprises determining time of impact of each wave with the platform. ix. The computer implemented method, wherein environmental data further incudes wind data and wherein executing a learning procedure comprises: recording wind data; and adding the wind data to the training set. x. The computer implemented method, wherein executing the learning procedure comprises: applying a transformation function on each series of data samples to
thereby transform the data samples from a time domain to a frequency domain. xi. The computer implemented method, wherein the learning procedure comprises compressing the transformed data samples using quantization to obtain a finite set of quantized values. xii. The computer implemented method, wherein each platform response vector determined for each wave in the waves-snapshot during the platform state prediction procedure, includes transformed data samples, and where executing the platform state prediction comprises: applying an inverse transformation function on series of transformed data samples in each platform response vector. xiii. The computer implemented method, wherein recording a series of data samples of platform state for each one of multiple degrees of freedom is performed with the help of one or more positioning devices, including one or more of an inertial navigation system (INS) and a GPS. xiv. The computer implemented method comprising fixing the INS to a landing deck on the platform. xv. The computer implemented method, wherein the data acquisition device includes one or more of: RADAR, camera (e.g., 3D camera), acoustic sensor, and buoy. xvi. The computer implemented method further comprising monitoring at least one physical property of the platform and initiating the learning procedure responsive to receiving data indicative of a significant change in the at least one physical property. xvii. The computer implemented method wherein the platform is any one of: ship, boat, floating landing dock, or pad. xviii. The computer implemented method, wherein the ML model is a
convolutional neural network. xix. The computer implemented method, wherein the ML model provides a response function that does require, as input, information on physical properties of the platform. xx. The computer implemented method, wherein the training procedure is executed following embarkment of the marine platform or responsive to receiving information indicating that a VTOL aircraft is about to land on the marine platform.
According to another aspect of the presently disclosed subject matter there is provided a system mountable on a marine platform configured for predicting a state of a marine platform, the system comprising at least one data acquisition device operatively connected to at least one computer; the at least one data acquisition device is configured to capture environmental data, including wave data of multiple waves approaching the platform; the computer is configured to: execute a learning procedure, comprising: for each wave of the multiple waves: process the wave data to obtain respective wave parameters; following impact of the wave with the platform, record over a predefined period, starting from a time of impact, a series of data samples of platform state for each one of multiple degrees of freedom (DOF), thereby generating a respective platform response vector; add to a training dataset the respective wave parameters and the respective platform response vector;
train a machine leaning model with the training dataset; execute a platform state prediction procedure, comprising: receive from the at least one data acquisition device wave data captured over a certain detection range and determine a respective waves- snapshot; for each wave in the waves-snapshot: process the wave data to obtain respective wave parameters; determine a respective platform response vector, by applying the wave parameters to the ML model; and aggregate the platform response vectors of all waves in the respective waves- snapshot (and the corresponding current platform state) to thereby determine a predicted platform state following impact of all waves in the waves-snapshot.
According to another aspect of the presently disclosed subject matter there is provided a computer program product comprising a computer readable storage medium retaining program instructions, which programs instructions when read by a processor, and causes the processor to perform the method according to the first aspect mentioned above.
According to another aspect of the presently disclosed subject matter there is provided a non-transitory program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform the method according to the first aspect mentioned above.
The system, the computer program product, and the non-transitory program storage device disclosed in accordance with the aspects of the presently disclosed subject matter detailed above can optionally comprise one or more of features (i) to (xx) listed above, mutatis mutandis, in any technically possible combination or permutation.
According to an additional aspect of the presently disclosed subject matter
there is provided a computer program product comprising a computer readable storage medium retaining a program of instructions, which program of instructions when read by a computer processor causes the computer processor to perform a method, comprising: using at least one data acquisition device for obtaining environmental data, including wave data of multiple waves approaching the platform; executing a platform state prediction procedure, comprising: receiving wave data recorded by at least one data acquisition device over a certain detection range and obtaining a respective waves-snapshot; for each wave in the waves-snapshot: processing the wave data to obtain respective wave parameters; determining a respective platform response vector, by applying the wave parameters to a machine learning model trained to receive wave parameters and determine data indicative of a respective platform response vector; and aggregating the platform response vectors of all waves in the respective waves-snapshot to thereby determine a predicted platform state following impact of all waves in the waves-snapshot.
Wherein it is further contemplated that the program of instructions retained on the computer program product, when read by a computer processor causes the computer processor to perform a method further comprising: executing a learning procedure, comprising: for each wave of multiple waves captured by the data acquisition device: processing the wave data to obtain respective wave
parameters; following impact of the wave with the platform, recording, over a predefined period, starting from a time of impact, a series of data samples of a platform state for each one of multiple degrees of freedom (DOF) thereby generating a respective platform response vector; adding to a training dataset the respective wave parameters and the respective platform response vector; and training the machine leaning model with the training dataset.
Wherein the computer program product according to the additional aspect of the presently disclosed subject matter detailed above can optionally comprise one or more of features (i) to (xx) listed above, mutatis mutandis, in any technically possible combination or permutation.
Wherein the presently disclosed subject matter further contemplates a system and a computer implemented method that comprise and/or implement the features set forth by the additional aspect above, mutatis mutandis.
BRIEF DESCRIPTION OF THE DRAWINGS
In order to understand the invention and to see how it can be carried out in practice, embodiments will be described, by way of non-limiting examples, with reference to the accompanying drawings, in which:
Fig. 1 is a schematic illustration of a landing scenario of a VTOL aircraft onboard a ship, in accordance with certain examples of the presently disclosed subject matter;
Fig. 2 illustrates a block diagram of a system mountable on a marine vehicle, in accordance with certain examples of the presently disclosed subject matter;
Fig. 3 illustrates a block diagram of a platform state prediction computer, in accordance with certain examples of the presently disclosed subject matter;
Fig. 4 illustrates a flow-chart of operations carried out during ML training, in accordance with certain examples of the presently disclosed subject matter;
Fig. 5 illustrates sampled data of 6 DOF platform state, in accordance with certain examples of the presently disclosed subject matter; and
Fig. 6 illustrates a flow-chart of operations carried out during prediction of a platform state, in accordance with certain examples of the presently disclosed subject matter.
DETAILED DESCRIPTION
In the drawings and descriptions set forth, identical reference numerals indicate those components that are common to different embodiments or configurations. Elements in the drawings are not necessarily drawn to scale.
Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that, throughout the specification, discussions utilizing terms such as "executing", "adding", "recording", "processing", "training", "determining" or the like, include an action and/or processes of a computer that manipulate and/or transform data into other data, said data represented as physical quantities, e.g. such as electronic quantities, and/or said data representing the physical objects.
The terms "computer", "computer device" or the like, should be expansively construed to include any kind of hardware electronic device with at least one data processing circuitry (e.g., digital signal processor (DSP), a GPU, a TPU, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), microcontroller, microprocessor etc.). The processing circuitry can comprise, for example, one or more computer processors operatively connected to computer memory, loaded with executable instructions for executing operations as further described below.
Operations in accordance with the teachings herein may be performed by a
computer specially constructed for the desired purposes, or by a general-purpose computer specially configured for the desired purpose by a computer program stored in a computer readable storage medium, (e.g., landing deck on ship)
As used herein, the phrase "for example," "such as", "for instance" and variants thereof, describe non-limiting embodiments of the presently disclosed subject matter. Reference in the specification to "one case", "some cases", "other cases", or variants thereof, means that a particular feature, structure, or characteristic described in connection with the embodiment(s), is included in at least one embodiment of the presently disclosed subject matter. Thus, the appearance of the phrase "one case", "some cases", "other cases", or variants thereof, does not necessarily refer to the same embodiment(s).
It is appreciated that certain features of the presently disclosed subject matter, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the presently disclosed subject matter, which are, for brevity, described in the context of a single embodiment, may also be provided separately, or in any suitable subcombination.
In embodiments of the presently disclosed subject matter, fewer, more and/or different stages than those shown in Figs. 4 and 6 may be executed. In embodiments of the presently disclosed subject matter, one or more stages in the illustrated figures may be executed in a different order, and/or one or more groups of stages may be executed simultaneously.
Figs. 2 and 3 schematically illustrate a system architecture in accordance with certain examples of the presently disclosed subject matter. Elements in Figs. 2 and 3 can be made up of any suitable combination of software and hardware and/or firmware that performs the functions as defined and explained herein. In examples of the presently disclosed subject matter, other than what is illustrated, the system may comprise fewer, more, and/or different elements than those shown in Figs. 2 and 3.
Likewise, the division of the functionality of the disclosed system to specific parts, as described below with reference to Figs. 2 and 3, is provided by way of example, and other various alternatives are also construed within the scope of the presently disclosed subject matter. For example, while Figs. 2 and 3 show a single computer dedicated for both training the machine learning model and executing the trained model, in other examples two or more separate computers can be used, e.g., one for training the model, and the other for execution of the model.
The term "vertical takeoff and landing (VTOL) aircraft" as used herein should be broadly construed to include any aerial vehicle with vertical lift that can execute vertical landing, including both manned VTOL aircrafts and unmanned (autonomous) VTOL aircrafts. VTOL aircraft include, for example, helicopters, tiltrotor aircraft (e.g., Bell Boeing V-22 Osprey and the Bell Nexus), drones of all kinds (e.g., multirotor drones), VTOL jets (e.g., Harrier Jump Jet and the F-35B Lightning II), flying cars (e.g., PAL-V Liberty), etc.
The terms "marine landing platform" or "landing platform" as used herein should be broadly construed to include any type of platform or vessel that floats on water and can be used as a landing surface for a VTOL aircraft. In some examples, a marine landing platform includes a landing deck and may include certain features that support VTOL, such as a non-skid surface, tie-down points, and deck lighting that facilitate nighttime landing. A marine landing platform can be, for example, a ship or other marine vehicle (typically comprising a designated landing deck), or a floating landing dock or pad. Marine landing platforms for VTOL aircrafts are used for various civilian applications, such as offshore oil and gas discovery and extraction, search and rescue missions, and emergency medical services.
Bearing the above in mind, attention is drawn to Fig. 2, showing a schematic block diagram of a system mountable on a marine landing platform according to some examples of the presently disclosed subject matter. System 200 is configured in general to predict a future platform state, determine a time when the future platform
state is suitable for safe landing of the VTOL aircraft, and coordinate the landing of the VTOL according to the determined time.
System 200 comprises one or more data acquisition devices 210, including for example one or more of: camera system(s), a RADAR system(s), acoustic sensor (e.g., Acoustic Doppler Current Profiler (ADCP)) buoy(s) (e.g., wave monitoring buoys) etc., configured to monitor the waves around the platform and determine waves data including wave parameters such as, waves height (amplitude) and waves velocity (i.e., speed and direction). Data acquisition devices 210 may further include a wind sensor (anemometer) configured to determine wind velocity. The camera and/or radar can be positioned to capture images of waves as they approach. In some examples, a camera system consists of a set of multiple cameras (e.g., 3 cameras or more) for obtaining 3-dimensional imaging of the waters. The camera can be for example a highspeed camera operatively connected to an image processing circuitry. In some examples, the camera can be equipped with a specialized lens that allows it to capture high-resolution images of waves, which can be processed as further described below.
System 200 also comprises positioning devices 220 capable of determining a platform state (6 DOF platform state) with respect to some reference frame, including for example one or more of inertial navigation system (INS), GPS receiver, and a cellular localization system. An INS is configured to continuously monitor the attitude of the platform, and, in some cases, it is also used for determining its location. According to some examples, the INS (or the inertial measurement unit of an INS, or some other altitude determining device) is mounted and fixed directly to a landing deck, which is the area dedicated for VTOL landing on the landing platform (e.g., landing deck on a ship), in order to have direct contact with the platform where the landing takes place, to enable direct monitoring of a 6 DOF platform state (or at least the 3 attitude components) at the place of landing, and thereby increase accuracy and reduce complexity of the 6 DOF data collection process.
System 200 further includes a platform state prediction computer 230
configured to predict future platform state of the landing platform, as further explained below. System 200 also comprises communication unit 240 configured to facilitate communicating, e.g., with the VTOL aircraft for sending to the aircraft landing instructions once a platform state suitable for landing is predicted.
Fig. 3 is a block diagram showing a more detailed description of components of platform state prediction computer 230, according to some examples of the presently disclosed subject matter. The architecture in Fig. 3 is a general example which demonstrates various principles of the presently disclosed subject matter, while in practice it may vary in design and therefore should not be construed as limiting. Computer 230 can comprise one or more processing circuitries, each circuitry comprising one or more computer processors operatively connected to a computer storage medium. The processing circuitries can be configured, for example, to execute functional modules in accordance with computer-readable instructions implemented on a non-transitory computer-readable storage medium. For simplicity, such functional modules are referred to hereinafter as components of the computer. A detailed description of the operations executed by platform state prediction computer 230 is disclosed below with reference to the following figures.
Proceeding to Fig. 4, this shows a flowchart of operations carried out during a training procedure 400, in accordance with examples of the presently disclosed subject matter. For more clear understanding, operations in Fig. 4 (and Fig. 6) are described with reference to components of the system shown in Figs. 2 and 3, however this should not be construed to limit the method to any specific system design or component. In the following description the term "ship" is used to demonstrate a marine landing platform for simplicity and by way of non-limiting example only.
Fig. 4 is related to the training of a machine learning model capable of predicting a future state of a marine landing platform in response to impact of waves. The training process can be initiated for example, prior to the landing of a VTOL
aircraft, and in some cases may require a short initial training period (also referred to as calibration period). While additional training (training over longer periods) using additional training data may improve accuracy and robustness of the machine learning model output, the machine learning model may provide sufficiently accurate output that can be used right after the calibration period. Longer training periods, possibility in a variety of wave conditions, provide a more robust model that enables ship state prediction in a greater diversity of wave conditions.
By way of example, in the case of a ship, training can be initiated at the beginning of operation (e.g., following embarkment of the ship from a dock) and often the same model can be used throughout the mission while the ship is at sea (while optionally the model is being trained continuously to reinforce the model over time). In other examples, training can be initialized prior to landing. For example, training can be initiated in response to receiving a message at the ship indicating that a VTOL aircraft intends to land on the ship shortly (e.g., within the next 30-60 minutes). The calibration period can range from a few minutes (e.g., 10-30 minutes) to a few hours.
Data collection used for training of the ML model can be done in real-time during the training process and does not require to obtain a training dataset prior to initiation of training. As will be apparent from the following description, the machine learning model is trained using data collected in real-time by devices onboard the ship, including environmental data and platform state data, without a need for data pertaining to the geometric and other physical properties of the ship, such as length, type width, keel length, deck depth, overall weight (tonnage), planned velocity, and planned depth, which are required for determining a transfer function.
While the system and method disclosed herein do not rely on geometric/physical properties as part of the input used during training or execution of the ML model (as no transfer function is used), significant changes in variable physical properties, which occur after the training of the model, may influence the ship's response to the waves ("wave-response") and therefore also its prediction
accuracy. Thus, in some cases, (e.g., following significant depletion in fuel levels or following loading of heavy cargo onto the ship) it may be advisable to re-train the model to accommodate changes in the ship's wave-response that may result from the change in physical properties. Since the initial training that is needed requires a relatively short calibration period, retraining of the model can be done relatively fast upon detection of a significant change in the ship's properties. In some examples, system 200 includes a gauge module (ship gauge 250) configured for monitoring certain physical properties of the ship (e.g., by directly measuring the properties or being connected to another system or sensor capable of providing data on the properties). If the gauge module receives information indicative of a significant change in a certain property of the ship (e.g., net weight), it may signal the platform state prediction computer 230 that retraining of the model is required, resulting in retraining the ML model.
At block 401 the water surrounding the ship is monitored using one or more data acquisition devices 210, and raw environmental data is recorded. Environmental data includes wave data and possibly also wind data. Depending on the type of data acquisition device, which is being used, the raw wave data may include corresponding output, such as imaging data output, RADAR data output, etc. As mentioned above, an acquisition device is positioned on the ship such that it can view and capture waves which are approaching the ship. In some examples, the data acquisition device is configured to capture wave data from multiple directions, for example by using multiple cameras pointed in different (e.g., orthogonal) directions, or by using a stabilization and orientation control mechanisms such as gimbal or pan and tilt mechanism allowing to change the viewing angle of the data acquisition device.
The raw wave data collected by the data acquisition devices is processed and individual waves are recognized (block 410). The respective wave parameters of each wave are determined (block 403). Wave parameters include wave height and wave velocity i.e., speed and direction. In some cases, wave parameters may further include wavelength and/or wave period. In the case of a camera system, the imaging output
can be processed using various software such as edge detection, pattern recognition, and image segmentation, together with other dedicated software for wave identification and characterization. Alternatively, or additionally, machine learning models trained for detecting waves and determining wave parameters from wave data can be used. The obtained wave parameters can be stored in a computer data repository (350) operatively connected to computer 230.
It is noted that wave data is captured and processed in real-time while the waves are approaching the ship. Identified waves are tracked to determine a respective time of impact with the ship, and, immediately following impact, the 6DOF platform state of the platform is recorded (block 405). The time of impact can be determined, for example, by monitoring the wave's velocity and determining based on the wave velocity and its distance from the ship at the time of impact. Following impact, 6 DOF position and orientation of the ship is determined using appropriate navigation devices, such as INS and GPS.
According to some examples, to determine the time of impact, one or more crests or leading edges are identified and tracked as they approach the ship. The distance between such reference points and the ship is continuously monitored, along with the wave's velocity. By knowing the wave's velocity and its distance from the ship at various instances, the time of impact can be calculated.
Individual waves can be distinguished from one another by identifying the troughs or low points between consecutive crests. The distance between two adjacent troughs can be considered the wavelength, which helps separate one wave from the next. This allows for the tracking and analysis of individual waves within a continuous wave field. Operations described with respect to block 410, 403, and wave tracking in block 405, can be executed for example by water state processing module 310 configured to process the raw data received from data acquisition devices 210.
In some examples, the 6 DOF platform state recorded for a certain wave includes a platform response vector. To this end, following the identified time of
impact of a wave, the 6 DOF position and orientation of the ship is sampled over a certain time window, to thereby obtain, for each wave, the respective platform response vector. The platform response vector includes a collection of a series of samples, recorded over the length of the time window, each series corresponding to one degree of freedom of a certain position or orientation, including a first series of samples of movement along the latitude (X) axis; a second series of samples of movement along the longitude (Y) axis, a third series of samples of movement along the altitude (Z) axis, a fourth series of samples for pitch, a fifth series of samples for roll, and a sixth series of samples yaw.
The length of the time window generally depends on the time between consecutive waves (wave period), but in some cases may be limited by some upper bound or may be otherwise selected. Referring to Fig. 5 it shows a graph plotting an example of 6 series of sampled data over a time window (x-axis), each corresponding to a certain axis of movement or axis of rotation of the ship. In the graph, series 1 represents sampled data of the height axis (up and down direction - altitude), series 2 represents sampled data of the sway axis (side to side direction - latitude), series 3 represents sampled data of the surge axis (forward backwards direction - longitude), series 4 represents sampled data of the delta yaw axis, series 5 represents sampled data of the pitch axis, and series 6 represents sampled data of the roll axis. The collection of series represents a platform response vector.
Operations described with respect to blocks 407, 409 can be executed for example by platform state processing module 320.
A transformation function is applied on the 6 DOF platform state for transforming the 6 DOF platform state from the time domain to the frequency domain (block 407). In some examples, data transformation is achieved by applying a Fourier Transform on each series in the respective response vector.
Transformation of the sampled data to the frequency domain can be performed for example by applying a discrete cosine transform (DCT) exemplified by
the following expression.
Equation 1:
where Xk is the signal in the frequency domain and xn is the signal in the time domain. N is the number of samples in the series. As shown in Fig. 5 each sequence of samples is represented as a signal over time.
In some examples, the transformation output is compressed using quantization to obtain a finite set of quantized values, which helps to simplify and increase efficiency of the training of the machine learning model. Transformation and quantization can be executed for example by transformation module 330.
The environmental data (including wave parameters) determined for a given wave are stored in association with the quantized values obtained from the corresponding 6 DOF platform state obtained for the same wave. This is repeated with many waves to thereby generate a training dataset, which can be stored in a computer data storage (block 409; computer data storage 350). The environmental data in the training dataset can also include the wind data that was measured at the time the wave data was recorded. In some examples, wind velocity, measured during the time of impact of a certain wave, is stored in the database in association with the wave. In some examples, wind velocity measured during a certain period (e.g., every 2 minutes) is associated with all waves that hit the ship during this period.
The machine learning model (ML) is trained using the training data stored in the storage (block 420). In some examples training can start in real-time as soon as training data is initially stored in the data storage and continue as more data is being processed and added to the storage. This is an advantage since it shortens the calibration period required for a obtaining a valid model that can be used for predictions. In some examples the ML model is a Convolutional Neural Networks
(CNNs) employing several hidden layers. The DCT data (coefficients) can be formatted and normalized to be fitted to a CNN model. Training can be executed for example by ML training module 340. The trained model can be stored in a computer storage 355 accessible to ML execution model 360.
It is noted that prediction of ship state as disclosed herein, using the machine learning model as described above allows for the creation of a general and robust model that can predict the ship state across a wide variety of wave conditions. Once trained on a sufficiently diverse dataset of wave patterns, the model can extrapolate the ship state resulting from the impact of previously unseen waves. For instance, if the model is trained on weak and medium waves, it can still predict the ship state in strong waves and if the model is trained on weak and strong waves, it can still predict the ship state in medium waves. Weak, medium, and strong waves can be defined based on wave parameters.
This special ability is achieved by the transformation of the 6 DOF platform state to the frequency domain and providing the transformed data as input to a Convolutional Neural Network (CNN) model, as it enables to determine a relationship between wave characteristics and ship state frequencies.
This provides a technological advantage as the model remains effective even if sea states change and new, previously unencountered waves are present. It eliminates the need for retraining the model for specific sea conditions when sea conditions change, which is crucial because during retraining, the model is unavailable, and the ship state cannot be predicted and therefore it ensures continuous prediction ability without interruption, even if the sea conditions change.
Once the training process is complete (e.g., following a calibration period of 10 to 30 minutes) the trained ML model can be used for predicting the ship's state. By way of example, if 100 samples are sufficient for generating a valid model and considering 6 samples are recorded each minute (for each DOF), a 30-minute calibration period can suffice for training a valid ML model. The trained ML model
provides a response function which provides a correlation between the recorded wave data (and possibly wind data) and the respective platform response vector and does not require physical/geometric data of the platform as input during training and execution.
Turning to Fig. 6, showing a flowchart of operations carried out during platform state prediction procedure 600, according to examples of the presently disclosed subject matter. The platform state prediction procedure can follow the calibration procedure, which, as mentioned earlier, can be relatively short. However, training of the model can continue in parallel to prediction to enhance the model and improve its output.
At block 601 the water surrounding the landing platform is monitored using one or more data acquisition devices 210, and wave data of waves approaching the ship is recorded. As explained above with respect to Fig. 4 with respect to the training process, the raw wave data collected by the data acquisition devices is processed, individual waves are recognized (block 610), and the respective wave parameters of each wave are determined, including at least wave height and velocity (block 603).
According to the presently disclosed subject matter, the data acquisition device 210 is configured and operable to obtain wave data of waves within a certain detection range from the ship. One non-limiting example of a detection range is between 50 to 2000 meters from the ship. Wave data captured within the detection range is processed and individual waves are identified. If the field of view (FOV) of the data acquisitions device covers an area which is greater than the detection range, the raw wave data can be processed to extract therefrom a part that corresponds to the detection range. If the field of view of the data acquisitions device covers an area which is smaller than the detection range, multiple frames covering the detection range can be captured. The data acquisition device output captured or otherwise obtained at a certain time instance t over the detection range, is referred to herein as "waves-snapshot".
The detection range, together with waves speed, define the prediction period - the length of time for which the prediction is made. A larger detection range and/or slower waves provide a longer prediction period, whereas a smaller detection range and/or faster waves provide a shorter prediction period. Thus, the detection range can be adapted to meet specific constraints and needs. For example, if a longer prediction period is needed, the system can be configured to capture wave data over a longer detection range, and the opposite in case a shorter prediction period is needed. In general, a suitable prediction period should be long enough to gather sufficient data for accurately predicting the platform's state, yet short enough to avoid errors that may arise from changes in wave conditions while approaching the platform. In some examples, ML execution module 360 can be configured to receive input (e.g., from a pilot of an approaching aircraft, or from an operator onboard a ship where landing is about to occur) indicative of the desired prediction period and adapt the detection range according to the wave speed. As mentioned above, wave speed can be determined in real-time by processing the captured wave data.
The determined wave parameters are provided as input to the ML model, which provides output data indicative of corresponding 6 DOF platform state (block 605; e.g., by ML execution module 360). If features in the training dataset include quantized values of the transformed 6 DOF platform state (i.e., samples series of the respective wave response vector), the ML model output also includes quantized values of corresponding transformed 6 DOF platform state. In some examples, wind data is also recorded and provided as input to the ML model, together with the wave parameters, as explained above with respect to training.
The obtained quantized values are processed using an inverse transformation function (e.g., inverse Discrete Cosine Transform (IDCT)) to thereby obtain the respective 6 DOF platform state e.g., the series of samples of the respective platform response vector (block 607, e.g., by post processing ML output module 370). Thus, the ML model is used for determining the wave response vector of each wave within the detection range.
The ML model output of 6 DOF platform state corresponding to the waves detected within the detection range are aggregated with the current 6 DOF platform state (i.e., state of the platform at time t when the relevant waves-snapshot has been recorded) to thereby obtain a predicted 6 DOF platform state over the distance of the detection range (block 609; e.g., by future platform state prediction module 380). In other words, the predicted platform state is a 6 DOF platform state that the ship will assume in the future when all waves detected within the detection range at a certain time t have reached the ship. Assuming, for the sake of example, that the detection range is 750 meters, and that at time t0, 10 waves (w0 ... w9) are detected within the detection range (corresponding to a waves-snapshot at time t0), the aggregated effect of all 10 waves with the current 6 DOF platform state of the platform (i.e. the 6 DOF platform state at the time of data acquisition of the 10 waves), is calculated. The corresponding 6 DOF platform state values obtained for each of the 10 waves are aggregated together with the current state of the platform, providing the predicted 6 DOF platform state following impact of all 10 waves with the ship. Notably, each wave induces a change to the ship's state that depends on its previous state before impact. By aggregating the platform state of each wave in a waves-snapshot with the previous platform state (starting from the current platform state), the accumulated change in the platform state over the course of the detection range can be determined.
The prediction procedure can be executed in a repetitive manner (as indicated by the arrow connecting block 609 to block 610), processing updated waves-snapshots as new waves enter the waves detection range and are captured. Considering the next iteration at time t17 the data acquisition output captured at that time would include a different set of waves (i.e., an updated waves-snapshot). Assuming, in the next iteration, there is a one wave difference from the previous iteration, such that wave w0 has already hit the ship, and wave w10 (11th wave) enters the detection range. In such a case, the new waves-snapshot would include waves w ... w1:L. The calculation is repeated, thistime determiningthe predicted 6 DOF platform state following impact of waves w
and using a different current 6 DOF platform state that corresponds
to the time of acquisition of waves
The updated waves-snapshot is processed to obtain a more updated predicted platform state following impact of the waves in the updated waves-snapshot. Thus, the disclosed system can continuously monitor the sea and provide updated predicted platform states, one after the other.
According to the disclosed subject matter, wave data of a series of waves within a detection range is obtained and translated to a corresponding future platform state the ship is predicted to assume following impact of the series of waves. Notably, each wave that hits the ship exerts certain forces on the ship that cause a certain movement to the ship that brings the ship to a certain 6 DOF platform state characterized by a certain position and orientation. The actual state of the ship following impact depends on the previous state of the ship. This is why, by aggregating the platform states, each resulting from the impact of a respective wave in the series of waves, and the corresponding current state, the future state of the ship, following impact of all waves in the series of waves, can be determined.
Once the future 6 DOF platform state of the ship has been predicted, it is determined whether this state is suitable for landing of the VTOL aircraft (block 611; e.g., by future platform state verification module 390). Future platform state verification module 390 can be configured to receive the future 6 DOF platform state of the ship and determine, based on the respective parameters (e.g., orientation and position) and other information such the type of VTOL aircraft, that is about to land, and/or the topography characteristics (whether the water is shallow or deep) of the area surrounding the ship, and determine whether the predicted ship state is suitable for landing of the particular VTOL aircraft. In some examples, future platform state verification module 390 is configured to use a ML learning model trained for determining suitability of a certain 6 DOF platform state of a platform for landing. For example, the ML model can receive as input the future 6 DOF platform state of the platform predicted by system 200, as well as additional information, such as the type of the VTOL, and provide a "confirmed landing" or "denied landing" output. In case landing is confirmed, landing instructions are transmitted to the VTOL aircraft,
informing the aircraft the pending landing time, and causing it to prepare for landing (block 613; e.g., data is transmitted to VTOL by communication unit 240). Otherwise, the system continues to process environmental data, calculate future vehicles states, and search for a future platform state that is suitable for landing. In some examples, once the VTOL aircraft has landed, the prediction procedure is terminated until the need for a further landing arises. In some examples, the training procedure continues, even if landing has occurred and/or the prediction procedure is terminated, to further enhance the ML model and prepare of possible future landings.
It is noted that while the present application has been described in the context of VTOL, this should not be construed as limiting, as the prediction of marine platform state can likewise be determined and used for other purposes, such as missile launch, cargo transfer, and personnel boarding.
Examples:
Missile launch: Predicting the resting period of a ship's motion can help determine the optimal window for launching missiles from a naval vessel, ensuring the stability and accuracy of the launch.
Cargo transfer: When transferring cargo between ships or from a ship to a smaller vessel, knowing the expected resting period can help plan and execute the operation more efficiently and safely.
Personnel boarding: Predicting the ship's motion resting period can assist in determining the most suitable time for personnel to board or disembark the vessel, reducing the risk of injury due to unstable platform conditions.
These examples demonstrate the versatility of the marine platform state prediction method and its potential applications beyond VTOL operations.
Those skilled in the art will readily appreciate that various modifications and changes can be applied to the examples of the invention as hereinbefore described without departing from its scope, defined in and by the appended claims.
Claims
1. A computer implemented method of predicting a state of a marine platform, the method comprising: using at least one data acquisition device for obtaining environmental data, including wave data of multiple waves approaching the platform; executing a learning procedure, comprising: for each wave of the multiple waves: processing the wave data to obtain respective wave parameters; following impact of the wave with the platform, recording, over a predefined period, starting from a time of impact, a series of data samples of a platform state for each one of multiple degrees of freedom (DOF) thereby generating a respective platform response vector; adding to a training dataset the respective wave parameters and the respective platform response vector; training a machine leaning (ML) model with the training dataset; executing a platform state prediction procedure, comprising: receiving wave data recorded by at least one data acquisition device over a certain detection range and obtaining a respective waves-snapshot; for each wave in the waves-snapshot: processing the wave data to obtain respective wave parameters; determining a respective platform response vector, by applying the wave parameters to the ML model; and
aggregating the platform response vectors of all waves in the respective waves-snapshot to thereby determine a predicted platform state following impact of all waves in the waves-snapshot.
2. The computer implemented method of claim 1, comprising: repeating the platform state prediction procedure, comprising: obtaining an updated waves-snapshot; and determining an updated predicted platform state following impact of all waves in the updated waves-snapshot.
3. The computer implemented method of claim 2, comprising: for each iteration of the platform state prediction procedure, determining whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, if so, terminating execution of the platform state prediction procedure.
4. The computer implemented method of claim 2, comprising: for each iteration of the platform state prediction procedure, determining whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, responsive to determining that the predicted platform state is suitable for landing, transmitting landing instructions to the VTOL to synchronize landing with the predicted platform state.
5. The computer implemented method of claim 1, comprising: determining whether the predicted platform state is suitable for landing a VTOL aircraft, and, responsive to determining that the predicted platform state is suitable for landing, transmitting landing instructions to the VTOL aircraft to synchronize landing with the predicted platform state.
6. The computer implemented method of any one of the preceding claims, wherein the multiple degrees of freedom include 6 degrees of freedom.
7. The computer implemented method of any one of the preceding claims, comprising: for each waves-snapshot captured at a time t, recording a corresponding current platform state of the platform at time t and aggregating the corresponding current platform state with the platform response vectors of all waves in the respective waves-snapshot.
8. The computer implemented method of any one of the preceding claims, wherein processing wave data to obtain respective wave parameters includes identifying individual waves and determining their respective wave parameters.
9. The computer implemented method of any one of the preceding claims, wherein executing a learning procedure comprises determining time of impact of each wave with the platform.
10. The computer implemented method of any one of the preceding claims, wherein environmental data further incudes wind data, and wherein executing a learning procedure comprises: recording wind data, and adding the wind data to the training set.
11. The computer implemented method of any one of the preceding claims, wherein executing the learning procedure comprises: applying a transformation function on each series of data samples to thereby transform the data samples from a time domain to a frequency domain.
12. The computer implemented method of claim 11, wherein executing the learning procedure comprises: compressing the transformed data samples using quantization to obtain a finite set of quantized values.
13. The computer implemented method of any one of the preceding claims, wherein each platform response vector, determined for each wave in the
waves-snapshot during the platform state prediction procedure, includes transformed data samples, and where executing the platform state prediction comprises: applying an inverse transformation function on a series of transformed data samples in each platform response vector.
14. The computer implemented method of any one of the preceding claims, wherein recording a series of data samples of a platform state for each one of multiple degrees of freedom is performed with the help of, one or more positioning devices, including one or more of an inertial navigation system (INS) and a GPS.
15. The computer implemented method of claim 14 comprising fixing the INS to a landing deck on the platform.
16. The computer implemented method of any one of the preceding claims, wherein the data acquisition device includes one or more of RADAR, camera (e.g., 3D camera) and buoy.
17. The computer implemented method of any one of the preceding claims, further comprising monitoring at least one physical property of the platform and initiating the learning procedure responsive to receiving data indicative of a significant change in the at least one physical property.
18. The computer implemented method of any one of the preceding claims, wherein the platform is a ship.
19. The computer implemented method of any one of the preceding claims, wherein the ML model is a convolutional neural network.
20. The computer implemented method of any one of the preceding claims, wherein the ML model provides a response function that does require, as input, information on physical properties of the platform.
21. The computer implemented method of any one of the preceding claims wherein the multiple DOF include 6 DOF.
22. A system mountable on a marine platform configured for predicting a state of a marine platform, the system comprising at least one data acquisition device operatively connected to at least one computer; the at least one data acquisition device is configured to capture environmental data, including wave data of multiple waves approaching the platform; the computer is configured to: execute a learning procedure, comprising: for each wave of the multiple waves: process the wave data to obtain respective wave parameters; following impact of the wave with the platform, record over a predefined period, starting from a time of impact, a series of data samples of a platform state for each one of multiple degrees of freedom (DOF) thereby generating a respective platform response vector; add to a training dataset the respective wave parameters and the respective platform response vector; train a machine leaning model with the training dataset; execute a platform state prediction procedure, comprising: receive, from the at least one data acquisition device, wave data captured over a certain detection range, and determine a respective waves- snapshot; for each wave in the waves-snapshot: process the wave data to obtain respective wave parameters; determine a respective platform response vector, by applying the wave parameters to the ML model; and
aggregate the platform response vectors of all waves in the respective waves- snapshot to thereby determine a predicted platform state following impact of all waves in the waves-snapshot.
23. The system of claim 22 wherein the at least one computer is configured to: execute the platform state prediction procedure repeatedly, comprising: obtaining an updated waves-snapshot; and determining an updated predicted platform state following impact of all waves in the updated waves-snapshot.
24. The system of claim 23, wherein the at least one computer is configured to: for each iteration of the platform state prediction procedure, determine whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, if so, terminate execution of the platform state prediction procedure.
25. The system of claim 23, wherein the at least one computer is configured to: for each iteration of the platform state prediction procedure, determine whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, responsive to determining that the predicted platform state is suitable for landing, transmit landing instructions to the VTOL aircraft to enable synchronization of landing with the predicted platform state.
26. The system of claim 22, wherein the at least one computer is configured to: determine whether the predicted platform state is suitable for landing a VTOL aircraft, and, responsive to determining that the predicted platform state is suitable
for landing, sending landing instructions to the VTOL aircraft to synchronize landing with the predicted platform state.
27. The system of any one of claims 22 to 26, wherein the multiple degrees of freedom include 6 degrees of freedom.
28. The system of any one of claims 22 to 27, wherein the at least one computer is configured to record, for each waves-snapshot captured at a time t, a corresponding current platform state of the platform at time t and aggregate the corresponding current platform state with the platform response vectors of all waves in the respective waves-snapshot.
29. The system of any one of claims 22 to 28, wherein processing wave data to obtain respective wave parameters includes identifying individual waves and determining their respective wave parameters.
30. The system of any one of claims 22 to 29, wherein executing a learning procedure comprises determining time of impact of each wave with the platform.
31. The system of any one of claims 22 to 30, wherein environmental data further incudes wind data, and wherein the at least one computer is further configured to add wind data to the training set.
32. The system of any one of claims 22 to 31, wherein the at least one computer is further configured, for executing the learning procedure, to: apply a transformation function on each series of data samples, to thereby obtain transform data samples from a time domain to a frequency domain.
33. The system of claim 32, wherein the at least one computer is further configured for executing a learning procedure to compress the transformed data samples, using quantization to obtain a finite set of quantized values.
34. The system of any one of claims 22 to 33, wherein each platform response vector, determined for each wave in the waves-snapshot during the platform state prediction procedure, includes transformed data samples, and wherein
the at least one computer is configured for executing platform state prediction to apply an inverse transformation function on a series of transformed data samples in each platform response vector.
35. The system of any one of claims 22 to 34 comprising at least one positioning device configured to provide a 6 DOF platform state, and wherein the recording a series of data samples of platform state for each one of multiple degrees of freedom is done with the help of the at least one positioning device.
36. The system of any one of claims 22 to 35, wherein at least one positioning device includes an attitude determination device that is fixed to a landing deck on the platform.
37. The system of any one of claims 22 to 36, wherein the data acquisition device includes one or more of RADAR, camera (e.g., 3D camera), and buoy.
38. The system of any one of claims 22 to 37 comprising a platform gauge configured to monitor at least one physical property of the platform, and wherein the at least one computer is configured to restart the learning procedure responsive to receiving data indicative of a significant change in the at least one physical property.
39. The system of any one of claims 22 to 38, wherein the platform is a ship.
40. The system of any one of claims 22 to 39, wherein the ML model provides a response function that does require, as input, information on physical properties of the platform.
41. A computer program product comprising a computer readable storage medium retaining a program of instructions, which program of instructions when read by a computer processor causes the computer processor to perform a method according to any one of claims 1 to 21.
42. A non-transitory program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform
a method according to any one of claims 1 to 21.
43. A non-transitory program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform a method comprising: using at least one data acquisition device for obtaining environmental data, including wave data of multiple waves approaching a marine platform; executing a platform state prediction procedure, comprising: receiving wave data recorded by at least one data acquisition device over a certain detection range and obtaining a respective waves-snapshot; for each wave in the waves-snapshot: processing the wave data to obtain respective wave parameters; determining a respective platform response vector, by applying the wave parameters to a machine learning model trained to receive wave parameters and determine data indicative of a respective platform response vector; and aggregating the platform response vectors of all waves in the respective waves-snapshot to thereby determine a predicted platform state following impact of all waves in the waves-snapshot.
44. The non-transitory program storage device of claim 43, wherein the method further comprising: executing a learning procedure, comprising: for each wave of multiple waves captured by the data acquisition device: processing the wave data to obtain respective wave parameters;
following impact of the wave with the platform, recording, over a predefined period, starting from a time of impact, a series of data samples of a platform state for each one of multiple degrees of freedom (DOF) thereby generating a respective platform response vector; adding to a training dataset the respective wave parameters and the respective platform response vector; and training the machine leaning model with the training dataset.
45. The non-transitory storage device of claim 43, wherein the method comprising: repeating the platform state prediction procedure, comprising: obtaining an updated waves-snapshot; and determining an updated predicted platform state following impact of all waves in the updated waves-snapshot.
46. The non-transitory storage device of claim 45, comprising: for each iteration of the platform state prediction procedure, determining whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, if so, terminating execution of the platform state prediction procedure.
47. The non-transitory storage device of claim 45, comprising: for each iteration of the platform state prediction procedure, determining whether the predicted platform state or the updated predicted platform state is suitable for landing a VTOL aircraft, and, responsive to determining that the predicted platform state is suitable for landing, transmitting landing instructions to the VTOL to synchronize landing with the predicted platform state.
48. The non-transitory storage device of claim 43, comprising:
determining whether the predicted platform state is suitable for landing a VTOL aircraft, and, responsive to determining that the predicted platform state is suitable for landing, transmitting landing instructions to the VTOL aircraft to synchronize landing with the predicted platform state.
49. The non-transitory storage device of any one of claims 43 to 48, wherein the multiple degrees of freedom include 6 degrees of freedom.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| IL303531A IL303531A (en) | 2023-06-06 | 2023-06-06 | Computer-based prediction of a marine landing platform state |
| IL303531 | 2023-06-06 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2024252393A1 true WO2024252393A1 (en) | 2024-12-12 |
Family
ID=93795274
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/IL2024/050553 Ceased WO2024252393A1 (en) | 2023-06-06 | 2024-06-05 | Computer-based prediction of a marine landing platform state |
Country Status (2)
| Country | Link |
|---|---|
| IL (1) | IL303531A (en) |
| WO (1) | WO2024252393A1 (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107545250A (en) * | 2017-08-31 | 2018-01-05 | 哈尔滨工程大学 | A kind of ocean floating motion Real-time Forecasting System based on the remote sensing of wave image and artificial intelligence |
| US20210117796A1 (en) * | 2020-12-01 | 2021-04-22 | Harbin Engineering University | Ship Motion Prediction Method Based on Long Short-Term Memory Network and Gaussian Process Regression |
| CN114330828A (en) * | 2021-11-29 | 2022-04-12 | 北京遥测技术研究所 | A method for forecasting the rest period of ship motion |
-
2023
- 2023-06-06 IL IL303531A patent/IL303531A/en unknown
-
2024
- 2024-06-05 WO PCT/IL2024/050553 patent/WO2024252393A1/en not_active Ceased
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107545250A (en) * | 2017-08-31 | 2018-01-05 | 哈尔滨工程大学 | A kind of ocean floating motion Real-time Forecasting System based on the remote sensing of wave image and artificial intelligence |
| US20210117796A1 (en) * | 2020-12-01 | 2021-04-22 | Harbin Engineering University | Ship Motion Prediction Method Based on Long Short-Term Memory Network and Gaussian Process Regression |
| CN114330828A (en) * | 2021-11-29 | 2022-04-12 | 北京遥测技术研究所 | A method for forecasting the rest period of ship motion |
Also Published As
| Publication number | Publication date |
|---|---|
| IL303531A (en) | 2025-01-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11486346B1 (en) | Autonomous underwater beacon locator | |
| EP2506235B1 (en) | Methods and systems for predicting ship motion | |
| CA2904785C (en) | Systems and methods for deploying autonomous underwater vehicles from a ship | |
| KR101644589B1 (en) | Method and system for controlling the automatic landing/take-off of a drone on or from a circular landing grid of a platform, in particular a naval platform | |
| US9069376B2 (en) | Unpredictable vehicle navigation | |
| US10721461B2 (en) | Collaborative stereo system for three-dimensional terrain and object reconstruction | |
| EP3076259B1 (en) | A system and method for controlling rolling motion of a marine vessel | |
| Ha et al. | Radar based obstacle detection system for autonomous unmanned surface vehicles | |
| CN121201426A (en) | A deep-sea cross-domain unmanned aerial vehicle and its method for completing autonomous deep-sea exploration missions | |
| WO2024252393A1 (en) | Computer-based prediction of a marine landing platform state | |
| CN117647981B (en) | Unmanned ship intelligent navigation control method, device and equipment | |
| Riola et al. | The prediction of calm opportunities for landing on a ship: Aspects of the problem | |
| Koszelew et al. | Autonomous vessel with an air look | |
| Inzartsev et al. | Investigation of autonomous docking system elements for long term AUV | |
| Kim et al. | Operational concept and design of multiple AUVs for rapid underwater investigation in case of maritime accidents | |
| Ross et al. | Autonomous zero-visibility quadrotor landings towards persistent ship-based UAV ocean observations | |
| Riola et al. | Smoothing warships movements based on wavelets | |
| Capuozzo et al. | Heave Motion Estimation from IMU measurements in Hybrid Aerial-Amphibious Drones and Horizontal Take-Off Window Prediction | |
| Larkin et al. | Simulation validation for a unmanned semisubmersible vehicle | |
| JP2024109303A (en) | Flight Control System | |
| CN121385899A (en) | Intelligent unmanned ship for detecting fish shoal in lakes and reservoirs and fish shoal detection method | |
| TWM680930U (en) | Unsteady image-based ranging system | |
| Williams IV | Design and experimental evaluation of an autonomous surface craft to support AUV operations | |
| JP2025129789A (en) | Monitoring system, monitoring method, and monitoring program | |
| Trimble et al. | Reacquisition and imaging of mine-like targets in very shallow water using the CetusII AUV and MIRIS high-resolution sonar |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 24818918 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
