IT202000028616A1 - ASSISTANCE SYSTEM FOR VESSELS DOCKING IN THE PORT - Google Patents

Description

Description accompanying the patent application for an industrial invention entitled:

TITLE

ASSISTANCE SYSTEM FOR VESSELS DOCKING MANEUVERS IN

PORT

Scope of the invention

The present invention relates to a highly innovative distributed system for managing parking areas.

In particular, the invention refers to a highly innovative distributed system capable of monitoring the stretches of water in front of the boat docking area in order to identify the boat object (for example ship, boat, etc.) and give assistance to the same during the entry or exit operation from the docking area assigned to the vessel.

Brief outline of the prior art

How ? known, in the port areas there are docking areas which are landing places for boats.

The vessel approaching the port area maneuvers the vessel to bring it to the assigned docking area.

Operations and manoeuvres, in a port area, are particularly complex as spaces are limited and the tonnages of many vessels are high.

Especially in the case of large boats, for example ships in general, the maneuvers are complex as the ship is? equipped with great inertia so that a change of direction to reverse course during a maneuver requires rather long response times from the ship.

AND? therefore it is important to make a steering correction as soon as possible.

In the current state of the art, there are no known systems of assistance at landing towards one's own docking area and the whole? entrusted to the instrumentation on board the ship and to the capacity? and obviously experience of the captain and crew.

However, unfortunately, events of naval disasters due to incorrect maneuvers are well known, with large vessels, for example ships, crashing into the dock, causing enormous structural damage and creating a danger to human lives.

There? can? could also be due to the taking over, in the maneuvering area, of some further vessel which transits erroneously in the area and which perhaps is displayed late by the crew.

If the maneuver is corrected too late there ? the risk that it will not end correctly.

Summary of the invention

AND? therefore, the object of the present invention is to provide an assistance system during the docking maneuvers of a vessel which allows the above mentioned technical drawbacks to be overcome.

Particularly ? The object of the present invention is to provide an assistance system which allows the craft to land in its own docking area in complete safety, even in the event of sudden entry into the maneuvering area of further obstacles.

These and other objects are obtained with an arrangement according to claim 1.

This assistance arrangement, to assist a vessel during an approach operation to a docking area, includes:

Image acquisition means (1) configured to acquire a succession of images at different instants (t0..tn);

A data processing system (10) in communication with said image acquisition means, said data processing system being programmed to process the image sequences taken by said image acquisition means and comprising:

An image analysis module (20) programmed to distinguish, for each image acquired at a certain instant (t), the subjects (200, 300) present in the image with respect to a background and to identify a possible boat among said subjects ( 300) approaching fixed or mobile obstacles (200) present in the image;

A calculation module (30) configured to receive the data relating to the images processed by said image analysis module and create, instant by instant (t), a predetermined optimal approach path of the boat (300) which ? processed through the determination of the spatial position occupied in each image by the identified vessel (300) and by any obstacles (200) appearing in said images in such a way as to result in a dynamically modifiable path in real time according to a possible change in the number and/or the position of the obstacles detected between an image at one instant and the image at a subsequent instant;

A video system to be set up for use on the approaching vessel, said video system displaying the sequence of images framed in real time by said image acquisition means, said images being displayed on said processed approach path and being identified instant by instant (t1?tn) the position (P1..Pn) occupied by the boat, and in which an alarm is generated when said detected position of the boat is outside the bounding limits of said elaborated approach path.

In this way all the aforesaid technical drawbacks are easily resolved.

In particular, thanks to an appropriate acquisition of images and their appropriate processing, you can generate in real time an approach path that can? be viewed in a shot arranged on board the vessel itself.

In fact, the system works, in some respects, in a similar way to the car parking assistance system in which a camera acquires the image. In the case of the present invention, however, the image is processed by creating an optimal path which is highlighted on the image and whose geometry depends on the presence or absence of obstacles, some of which may be fixed (for example other anchored boats) and others may be mobile (a boat entering the parking area). Each sequence of images is therefore processed continuously and, for each processing, any obstacles are identified. Therefore, an optimal path corresponds to each image. The sequence of images creates the sequence of the optimal route and therefore its possible evolution during the maneuver time.

For example, the image at? instant t=0 will be? analyzed to determine the vessel and obstacles in order to extrapolate the relative distances between them and build an approach path_0.

The next image at the instant t=1 is processed in the same way and if nothing ? changed about the obstacles return? an approach path_1 for t=1 equal to the path determined at the instant t=0.

If at the instant t=2 a new obstacle occurs (for example a boat enters the area framed by the cameras) then the analysis will detect? a new obstacle and the processing of the path could result in an approach path_2 at the instant t=2 different from the previous one.

In this way, in real time and instant by instant, an optimal approach path is built that can be change dynamically according to the real obstacles present and this path is returned on a video installed on board the vessel that will visualize? on the shot taken by the camera your position and the optimal route, which you can? change during the maneuver.

This assistance system therefore helps to visualize an approach path dynamically.

If you go off the optimal path, an alarm is triggered.

Advantageously, the said image analysis module can? understand a ?Computer Vision? to neural networks of the ?Deep Learning? typology.

This system, right? in itself pre-existing, it allows you to easily determine the subjects present in the shot, ie? obstacles and vessel, distinguishing the possible vessel from the possible obstacles. The Deep Learning neural network allows you to train the system by refining it over time.

Advantageously, said image acquisition means (1) and said data processing system (10) can be of the "Embedded" type.

There? makes the system very compact and therefore makes it suitable for easy installations.

Advantageously, said image acquisition means (1) can be in the form of one or more? cameras.

AND? Also described here is a landing assistance method for assisting a vessel during an approach operation in the parking space of a port, the method comprising the following steps:

Acquisition of a sequence of images at successive instants framing an approach area for a vessel;

Processing of the images in such a way as to distinguish for each image at a predetermined instant any vessel approaching any static or dynamic obstacles (200) present in said image;

Generation instant by instant (t) of a predetermined optimal approach path of the boat which ? processed through the determination of the spatial position occupied in the image by the identified vessel (300) and by any obstacles (200) appearing in said images in such a way as to result in a dynamically modifiable path in real time according to a variation of the number and /o the position of the obstacles detected between an image at one instant and the image at a subsequent instant;

Display on board the vessel, on a special video system, of the sequence of images framed in real time by said image acquisition means, on said images being displayed the said processed approach path and being identified instant by instant (t1?tn) the position (P1..Pn) occupied by the vessel with respect to said elaborated approach path;

Generation of an alarm when said detected position of the vessel moves outside the perimeter delimiting said approach path.

Advantageously, image processing can? be carried out with the ?Computer Vision? to neural networks of the ?Deep Learning? typology.

Advantageously, the images can be acquired continuously.

Brief description of the drawings

Further characteristics and the advantages of the present system, according to the invention, will be more clearly with the following description of some of its embodiments, given by way of non-limiting example, with reference to the attached drawings, in which:

- Figure 1 schematically shows a television camera 1 arranged to frame a docking area 2, the television camera being in communication with a computer system which can? preferably be integrated into the camera system itself (?embedded? system);

- Figure 2 again schematises the video camera which has a certain framing angle and which frames an approaching vessel 300 along a trajectory and in which, by way of example, obstacles 200 (fixed or mobile) are shown;

- Figure 3 schematises the embedded system 10 which comprises an image acquisition system, for example one or more? video cameras, an image analysis module 20 of the type for example computer vision and a calculation module 30;

- Figure 3A and figure 3B schematize the operation to which ? used the Computer Vision module, i.e. the identification of the objects present instant by instant in the framed image;

- Figure 3C refers to an elaboration of the data extrapolated from two successive images in which the objects are identified in order to determine the trajectory which instant by instant ? held by the vessel identified with respect to the other objects present;

- Figure 4 shows a corridor which delimits an ideal trajectory calculated by the computer as a function of the objects present in the image as well as the approaching vessel and shows an example of a trajectory which is kept by the vessel in such a way as to show, by way of example , the case in which the vessel leaves the ideal trajectory (in this case at point P3); - Figure 5 shows the guidance image which is sent to a screen arranged on the approaching vessel which uses this assistance system;

Description of some preferred embodiments

According to the invention ? therefore a system of the ?embedded? typology is envisaged.

An ?embedded? system, in itself well known, ? a system that falls into the category of IoT (Internet of Things) as it incorporates all the hardware and software components necessary to perform specific tasks and ? able to process large quantities locally? data (eg: images and / or video streams) without the need? to transmit them via the Internet to a server and then process them. So everything is processed ?On site?.

The processed data can then be transmitted, providing the system with ?emedded? with known wireless data transmission systems.

In this way there is a considerable saving in space and optimization of data processing.

The system object of the invention ? therefore preferably of type ?embedded? even if a non-embedded system pu? Be expected.

In all of the aforesaid cases, the system provides image acquisition means 1, for example in the form of one or more? cameras.

Said television cameras are positioned in predetermined points on a pier 100 or a mooring area for boats in general (a wharf or the like) so as to frame a certain body of water 2 in front of a landing area of the boat.

The video camera system, as later described in the continuation of the present description, includes a data processing system and an image analysis system in order to be able to carry out the operations described.

The embedded system in which the cameras include hardware and software related to data processing, cio? image processing and data flow processing/sending ? favorite.

However, for how already? said, these software and hardware systems could still be separated from the cameras and located in different points.

According to figure 1 ? therefore schematically represented a landing area 100, or an area of a pier 100, of a wharf or area in general which represents a docking area for a vessel, for example a ship or any boat.

In accordance with the invention, figure 1 therefore schematizes a video camera 1 which frames a certain area in front of the landing place 100 and this area is schematized with a cone 2.

Again as shown in figure 1, the camera? in communication with a processor 3 which, as stated above, in the preferred configuration of the invention, can? be a system integrated with the camera which is therefore part of the so-called ?embedded? system.

Figure 2 therefore schematizes a vessel 300 and, by way of example, obstacles 200 are shown along the approach path which can be any type of object present in the body of water, for example other vessels that stop in the area, poles emerging from the water, etc., both fixed and mobile.

In the non-limiting example of figure 2, a succession of such obstacles 200 present both on the right and on the left of the boat have been represented to define a fixed entrance corridor. AND? clear that the arrangement and the number of obstacles shown in figure 2 ? only indicative and not limiting for the present invention.

The camera (one or more than one) ? therefore arranged in such a way as to have a predetermined framing which entirely frames a possible landing area for a vessel.

The image is therefore processed in real time, ie? instant by instant, in order to determine an optimal approach path and identify the vessel in the image.

The whole ? therefore elaborated in the following way.

As schematized in figure 3, the system 10, preferably of the "embedded" type, therefore includes camera(s), an image analysis module and a calculation module.

The image analysis module ? preferably of the type ?Computer Vision? who ? therefore equipped with a special image analysis software per se already? known and commercially available in various versions.

For example, at the Hardware level ? You can use Nvidia's Jetson series SOM system.

There are several OEMs (Original Equipment Manufacturing) that build embedded systems using these SOMs (System On Modules).

As far as the software part is concerned, Object Detection techniques can be used that use Deep Learning type neural networks (e.g. Yolo).

Deep learning, therefore neural networks, are systems in themselves already? known and, as mentioned above, already? on the market and therefore will not be further detailed here.

The image analysis module, preferably comprising programming of the Computer Vision type with Deep Learning type neural networks, ? therefore able to analyze the image instant by instant and therefore provide for each instant the subjects present in the image (vessel, possible obstacles, etc.).

The image analysis system serves to process the image and therefore be able to identify the objects present in the shot, as schematically indicated in figure 3A.

The identification of the objects, and therefore the analysis of the image, takes place at every moment and therefore every ?frame? of image instant by instant (from t0 to tn).

Figure 3B therefore shows an image acquired at any instant (t) relating to the example of figure 2 and thus highlights the vessel, at a predetermined instant of image acquisition, and any obstacles.

The computer vision software ? therefore programmed for the purpose of recognition in the image of the objects present in the area of the image, in this case vessel and obstacles.

By carrying out an analysis of images in succession, it is possible to carry out a dynamic analysis, i.e. report changes to the evolving scenario.

As shown in figure 3C, the identification of the objects in the images allows the calculation module to be able to process instant by instant (therefore for each image taken) the position occupied by the vessel during the approach with respect to any obstacles present in the scene (for example a boat that cuts the path suddenly).

In fact, figure 3C schematizes an example of a succession of two images (IMM.1) and (IMM.2), it being understood that during an approach maneuver the sequence of images requires a much higher number of acquisitions. top of the two example images shown in Figure 3C.

However, referring to the example of two images in figure 3C, each image is processed by the image analysis module which identifies the objects. Each image processed by the image analysis module, or in any case the identification data of the detected subjects, are further processed by the calculation module which, once the objects are known, determines the distances between them, in particular the distance between the identified vessel and any obstacles identified.

Therefore, the example of figure 3C shows two images in two successive instants (t1, t2) it is well understood that, as mentioned, the acquisitions are much more? numerous therefore with images in instants (t1, t2?tn) and the obstacles present between one instant and the next can be different, if the obstacles are for example dynamic (ie mobile).

The image analysis module therefore processes (n) images in (n) different instants in order to extrapolate for each image at a certain instant (t) the vessel and obstacles.

These images at the instants (tn) which have been processed, and in which the objects present are therefore identified, thanks to this processing, are sent to the calculation module.

Object identification implies that the system recognizes the moving vessel from other objects (eg fixed obstacles, anchored vessels).

In this way, the calculation module elaborates an optimal approach path and geo-locates the vessel instant by instant along the route.

Pi? in particular, the knowledge of the position of the boat and of the obstacles in each image, allows to build an approach path for each image which can be modify according to changes in position and/or number of obstacles present on a subsequent image.

The trajectories (that is, the path) are therefore calculated starting from the information that comes from the combination of the outputs of the object detection module, which through an analysis and appropriate training carried out beforehand on the types of objects present in the reference context,? able to support the tracing engine in the most? timely.

The calculation module that receives the information concerning the position instant by instant of the vessel with respect to the surrounding environment therefore calculates, instant by instant, the trajectory that the vessel is following.

The system ? it is also able to collect trajectory statistics over time, providing real-time feedback on the possible presence of obstacles or sudden danger situations.

As shown in figure 3C, in fact, are two different instants indicated in which the vessel passes from position P1 to position P<1>? and the calculated distances (a<1>, d<2 >a?1..d1?1, a1-1 etc.) of the vessel from the surrounding obstacles that have been detected in the two images are highlighted.

The calculation module, therefore, elaborates an ideal trajectory in function of the recognized obstacles which ? schematically represented in figure 4 with a filled corridor.

The same calculation processor which receives the images instant by instant from the image processing module calculates the position occupied by the vessel during its advance and easily verifies whether, as in figure 4, the vessel maintains the optimal trajectory or not.

In the example of figure 4, in fact, a position (P3) is highlighted in which the vessel is? out of the ideal corridor.

The ideal path can change instant by instant as a function of new obstacles that occur in the image for which the image in figure 4 ? to be understood in an optimal path which occurs at a certain instant t1 and which path could change at a different subsequent instant t(1+n), with n>0.

The elaboration of the optimal path, which ? function of the obstacles detected, it is sent via screen to the vessel itself, for example with a sending in mode? using, for example, the internet network or other wireless data transmission systems.

The boat, equipped with a receiving screen, as shown in figure 5, will display? the camera image on which the ideal path delimited by lines is constructed (for example two parallel lines as shown in figure 5).

The vessel in the image is then displayed instant by instant and if the vessel goes outside the limits of the ideal trajectory, the system sounds an alarm requesting a correction that can also be accompanied by a vocal indication.

Figure 5 shows, by way of example, a fielded area which represents the ideal trajectory and a further fielded area outside the latter which represents an area outside the trajectory.

So, according to a system operation flowchart:

One or more? cameras are placed to acquire images over time relating to a ?parking? area? for a boat.

Each image is processed with a ?Computer Vision? and Deep Learning neural system that allows to determine the subjects framed in the images, for example an approaching boat and other obstacles present in the frame.

The computer vision system? therefore able to verify if in the framed image relative to a ?parking? ? approaching a vessel and which obstacles other than the vessel are present.

Each image processed, instant by instant, is sent to the processor module which, depending on the obstacles present, processes the received image at every instant, creating an ideal dynamic approach path for the vessel.

The vessel is geo-localized with respect to the ideal route and the ideal route is dynamically modified according to the outcome of the processing of the images coming from the image processing module.

The computer verifies instant by instant the position occupied by the vessel and verifies whether the vessel maintains the calculated optimal trajectory.

When a vessel position outside the limits of the calculated optimal route is detected, an alarm is triggered, for example of an audible type.

All this therefore translates into a real-time video screen that receives the boat which displays its geo-location on its screen with respect to an ideal approach path which obviously is not? static but can? change and ? therefore dynamic.

The ideal path can be delimited by lines of a certain color and if the boat geo-located with respect to this route goes outside the optimal route, then an alarm is generated.

In this way the boat has a landing aid that helps to avoid fixed or dynamic obstacles that may potentially be in the approach path.

The understanding of the framed scene? in real time (?real time?). This means that all objects in the scene are recognized continuously. Therefore, if a boat approaches in my maneuvering space, the system detects it and signals it.

In a further variant of the invention it is possible to provide for the integration of additional proximity sensors? aimed at providing support information to the main image analysis module in case of poor visibility conditions? due to adverse weather or contingent conditions.

Claims (8)

1. An arrangement for assisting a craft during an approach operation at a landfall area, the arrangement comprising: - Image acquisition means (1) configured to acquire a succession of images at different instants (t0..tn); - A data processing system (10) in communication with said image acquisition means, said data processing system being programmed to process the image sequences taken by said image acquisition means and comprising: - An image analysis module (20) programmed to distinguish, for each image acquired at a certain instant (t), the subjects (200, 300) present in the image with respect to a background and identify a possible vessel among said subjects (300) approaching fixed or mobile obstacles (200) present in the image; - A calculation module (30) configured to receive the data relating to the images processed by said image analysis module and create, instant by instant (t), a predetermined optimal approach path of the boat (300) which ? elaborated through the determination of the spatial position occupied in the image by the identified vessel (300) and by any obstacles (200), in such a way as to result in a dynamically modifiable path in real time according to a possible variation of the number and/or of the position of the obstacles detected between an image at one instant and the image at a subsequent instant; - A video system to be set up for use on the approaching boat, said video system displaying the sequence of images framed in real time by said image acquisition means, said images being displayed on said processed approach path and being identified instant by instant (t1?tn) the position (P1..Pn) occupied by the boat, and in which ? a programmed alarm is provided to be activated when said detected position of the boat is outside the limits delimiting said elaborated approach path. 2. The arrangement according to claim 1, wherein said image analysis module comprises a ?Computer Vision? to neural networks of the ?Deep Learning? typology. 3. The arrangement, according to claim 1 or 2, wherein said image acquisition means (1) and said data processing system (10) are of the "Embedded" type. 4. The arrangement, according to one or more of the preceding claims, wherein said image acquisition means (1) are in the form of one or more? cameras. 5. The arrangement, according to one or more of the preceding claims, in which one or more? proximity sensors?. 6. A method of assisting a craft during an approach operation into a port parking space, the method comprising the following steps: - Acquisition of a sequence of images at successive instants framing an approach area for a boat; - Processing of the images in such a way as to distinguish for each image at a predetermined instant any vessel approaching any static or dynamic obstacles (200) present in said image; - Generation instant by instant (t) of a predetermined optimal approach path of the boat which ? processed through the determination of the spatial position occupied in the image by the identified vessel (300) and by any obstacles (200) appearing in said images in such a way as to result in a dynamically modifiable path in real time according to a variation of the number and /o the position of the obstacles detected between an image at one instant and the image at a subsequent instant; - Display on board the boat, on a special video system, of the sequence of images framed in real time by said image acquisition means, on said images being displayed the said processed approach path and being identified instant by instant (t1?tn) the position (P1..Pn) occupied by the boat with respect to said elaborated approach path; - Generation of an alarm when said detected position of the vessel moves outside the perimeter delimiting said approach path. 7. The method, according to claim 6, in which the image processing is performed with the ?Computer Vision? to neural networks of the ?Deep Learning? typology. 8. The method according to claim 6, wherein the images are acquired continuously.