KR20210078589A - Apparatus and method for berthing assistance by using unmanned aerial vehicle - Google Patents

Abstract

The present invention relates to a port berthing assistance apparatus using an unmanned aerial vehicle and a method thereof, which can provide continuous information in the form of an around view of a ship and an entire image of a ship's periphery by using a plurality of unmanned aerial vehicles. According to one embodiment of the present invention, the port berthing assistance apparatus using an unmanned aerial vehicle receives a plurality of reference distance images obtained by photographing a reference distance measurement plate using the plurality of unmanned aerial vehicles, receives a plurality of image data photographed outside a ship using the plurality of unmanned aerial vehicles, calculates offset information for each of the plurality of unmanned aerial vehicles by using the plurality of reference distance images and the actual distances of the reference distance measurement plate, and forms an around-view image by synthesizing the plurality of image data using the offset information.

Description

Apparatus and method for port berthing support using unmanned aerial vehicles {APPARATUS AND METHOD FOR BERTHING ASSISTANCE BY USING UNMANNED AERIAL VEHICLE}

The present invention relates to a port berthing support apparatus, and in particular, port berthing support using an unmanned aerial vehicle that can provide continuous information in the form of an Around View of the entire image of a ship and the periphery of the vessel using a plurality of unmanned aerial vehicles It relates to an apparatus and method.

In general, unmanned aerial vehicles (aka drones) manufactured for military missions for the purpose of reconnaissance, surveillance, precision strike, etc. have recently been used for disaster monitoring, goods transport, video shooting, disaster relief, etc. Depending on the method, demand and utilization are also increasing in the private sector.

In particular, interest in unmanned aerial vehicles has been rapidly increasing in recent years, and the application fields are limitless, so the advanced aviation industry and the information technology (IT) industry around the world are spurring investment and R&D for technology development.

Unmanned aerial vehicles can observe areas that are difficult for humans to access, and have the advantage of being able to perform precise observations with excellent visibility, especially due to low-altitude flight.

When a vessel is docked in a port, the pilot boards the vessel, guides the vessel to a safe waterway, and guides the vessel to anchor safely. In this case, the berthing of the ship is performed using the pilot's experience and information such as wind speed, wind direction, weather, and current provided by the ship operating system. Therefore, there is a need for a means capable of providing vessel operation information, information on vessels or facilities located near the vessel, and collision-inducing information in real time in the form of an image for safer and more rapid berthing of a vessel in a port.

The present invention provides an apparatus and method for supporting port berthing using an unmanned aerial vehicle that can provide continuous information in the form of an around view of a ship and the entire image of the ship's periphery using a plurality of unmanned aerial vehicles.

Port berthing support apparatus using an unmanned aerial vehicle according to an embodiment of the present invention, one or more processors; and one or more memories in which instructions for causing the one or more processors to perform operations are stored when executed by the one or more processors, wherein the one or more processors take a reference distance measuring plate using a plurality of unmanned aerial vehicles. Receive a plurality of reference distance images, receive a plurality of image data photographed outside the ship using a plurality of unmanned aerial vehicles, and use the plurality of reference distance images and the actual distance of the reference distance measurement plate to obtain the plurality of An around-view image may be formed by calculating offset information for each unmanned aerial vehicle, and synthesizing the plurality of image data using the offset information.

Port berthing support apparatus using an unmanned aerial vehicle according to an embodiment of the present invention, one or more processors; and one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, wherein the one or more processors include a plurality of instructions for each of a plurality of unmanned aerial vehicles located outside the ship. Receive location data and a plurality of image data, obtain a plurality of pieces of reference location information for each of the plurality of unmanned aerial vehicles, and based on a difference between the plurality of reference location information and the plurality of location data, the plurality of unmanned aerial vehicles An around-view image may be formed by calculating offset information for at least a part of an airplane, and synthesizing the plurality of image data using the offset information.

As an embodiment, the one or more processors may calculate a distance between an obstacle outside the vessel and the vessel using the around-view image, and form an alarm message when the calculated distance is less than or equal to a threshold value.

In one embodiment, the one or more processors receive a facility image for a berthing facility of a port to which the vessel intends to dock, and use the facility image to form information on an expected berthing area of the vessel, the expected berthing area Information on the movement path of the vessel may be calculated using the information on .

As an embodiment, the plurality of image data may include infrared information.

A port berthing support method using an unmanned aerial vehicle according to an embodiment of the present invention comprises the steps of: receiving a plurality of reference distance images obtained by photographing a reference distance measuring plate using a plurality of unmanned aerial vehicles; Receiving a plurality of image data taken outside the ship using a plurality of unmanned aerial vehicles; calculating offset information for each of the plurality of unmanned aerial vehicles by using the plurality of reference distance images and the actual distances of the reference distance measurement plate; and synthesizing the plurality of image data using the offset information to form an around-view image.

A port berthing support method using an unmanned aerial vehicle according to an embodiment of the present invention includes: receiving a plurality of location data and a plurality of image data for each of a plurality of unmanned aerial vehicles located outside a ship; obtaining a plurality of pieces of reference location information for each of the plurality of unmanned aerial vehicles; calculating offset information for at least some of the plurality of unmanned aerial vehicles based on a difference between the plurality of pieces of reference location information and the plurality of pieces of location data; and synthesizing the plurality of image data using the offset information to form an around-view image.

In an embodiment, calculating a distance between an obstacle outside the vessel and the vessel using the around-view image; and forming an alarm message when the calculated distance is less than or equal to a threshold value.

In one embodiment, the method comprising: receiving a facility image for a berthing facility of a port that a vessel intends to dock; forming information on the expected berthing area of the vessel by using the facility image; and calculating information on the movement path of the vessel by using the information on the expected berthing area.

As an embodiment, the plurality of image data may include infrared information.

According to embodiments of the present invention, it may be possible to provide an image formed using a plurality of unmanned aerial vehicles to a user, thereby providing an around view, which is continuous information on a precise vessel and the periphery of the vessel. In addition, by using the ship's around view, it is possible to determine the operation information of the ship, the information of other ships located in the vicinity of the ship, the collision-inducing information, and the like, thereby helping the ship to safely and quickly proceed with berthing at the port facility.

1 is an exemplary diagram showing the configuration of a port berthing support apparatus according to an embodiment of the present invention.
2 is an exemplary diagram showing the configuration of a port berthing support environment using an unmanned aerial vehicle according to an embodiment of the present invention.
3 is an exemplary view showing the configuration of a mobile console according to an embodiment of the present invention.
4 and 5 are flowcharts showing a procedure of a method for supporting berthing in a port using an unmanned aerial vehicle according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. The present invention may be embodied in many different forms and is not limited to the embodiments described herein.

1 is an exemplary diagram showing the configuration of a port berthing support apparatus according to an embodiment of the present invention.

As shown in FIG. 1 , the port berthing support device 100 may include a mobile console 110 , a plurality of unmanned aerial vehicles 120 and a storage device 130 .

The mobile console 110 may be used for the purpose of controlling a plurality of unmanned aerial vehicles 120 , and by synthesizing images transmitted from a plurality of unmanned aerial vehicles 120 , the port berthing support device 100 is mounted around the vessel. It is possible to form an Around View Image and/or an infrared image of Also, the mobile console 110 may further include a display unit DU for displaying an around-view image or an infrared image according to a user's selection.

The mobile console 110 may determine the number of unmanned aerial vehicles 120 to be used for port berthing by receiving information on the ship alignment and size from personnel performing port berthing tasks, such as pilots. According to an embodiment, the number of unmanned aerial vehicles 120 to be used for berthing in a port may be determined using a separate determination program.

The harbor berthing support device 100 may be installed in a place that can be safely boarded in the wheelhouse or the wheelhouse front of the ship for berthing possessed by personnel performing port berthing tasks. The personnel performing the port berthing service turn on the power of the mobile console 110 after the installation of the port berthing support device 100, and turn on the power of a plurality of unmanned aerial vehicles 120 to turn on the mobile console Control of a plurality of unmanned aerial vehicles 120 using 110 may be started.

Before the unmanned aerial vehicle 120 takes off, the reference distance measuring plate RM may be installed in a safe place near the take-off location of the unmanned aerial vehicle 120 . Each of the plurality of unmanned aerial vehicles 120 that have taken off may photograph the pre-installed reference distance measuring plate RM, and transmit the captured reference distance image to the moving console 110 . The mobile console 110 provides offset information (images) between the lengths included in the received plurality of reference distance images and the actual lengths of the reference distance measurement plate RM and the lengths measured in the plurality of reference distance images. distance correction value) can be calculated. Thereafter, the user who checks the around-view image can recognize the actual length by forming the around-view image using the calculated offset information for the images photographed by the plurality of unmanned aerial vehicles 120 .

The mobile console 110 may form an around-view image by synthesizing the images received from each of the plurality of unmanned aerial vehicles 120 using the offset formed in each of the plurality of unmanned aerial vehicles 120 .

The mobile console 110 receives a plurality of position data and a plurality of image data for each of a plurality of unmanned aerial vehicles, obtains a plurality of reference position information for each of a plurality of unmanned aerial vehicles, and a plurality of reference position information and a plurality of Offset information for at least some of the plurality of unmanned aerial vehicles 120 may be calculated based on a difference from the position data of , and a plurality of image data may be synthesized using the offset information to form an around-view image.

2 is an exemplary diagram showing the configuration of a port berthing support environment using an unmanned aerial vehicle according to an embodiment of the present invention.

As shown in Figure 2, the vessel (EV) that was sailing performs berthing at the berthing facility (BF) for anchoring when it arrives at the port. Personnel performing the port berthing task may be installed in a place that can board safely in the wheelhouse or the wheelhouse front of the ship (EV) for berthing with the port berthing support device 100 . The personnel performing the port berthing service turn on the power of the mobile console 110 after the installation of the port berthing support device 100, and turn on the power of a plurality of unmanned aerial vehicles 120 to turn on the mobile console Control of a plurality of unmanned aerial vehicles 120 using 110 may be started. Thereafter, the plurality of unmanned aerial vehicles 120 may move to a preset location. According to an embodiment, the plurality of unmanned aerial vehicles 120 may photograph the outside of the vessel at the left and right sides of the bow of the vessel (EV), the left and right sides of the stern of the vessel, and the center of the bow, but the plurality of unmanned airplanes 120 is The moving position is not limited thereto. In addition, the plurality of unmanned aerial vehicles 120 may be provided in various numbers of at least four or more capable of measuring the overall appearance of the vessel (EV).

The plurality of unmanned aerial vehicles 120 may move to a preset position and photograph the exterior of the vessel at the corresponding position. According to one embodiment, in the image of the outside of the vessel taken by the plurality of unmanned aerial vehicles 120, the vessel BS that is already moored at the berthing facility BF, or another vessel that attempts to anchor the berthing facility BF. (FS), etc. images of obstacles may be included.

The plurality of unmanned aerial vehicles 120 may include a position measuring unit 122 and a photographing unit 124 . The position measuring unit 122 may measure the position of the unmanned aerial vehicle 120 to form position data. According to an embodiment, the unmanned aerial vehicle 120 may include a device such as a global positioning system (GPS) or a global navigation satellite system (GLONASS), but is not limited thereto.

The photographing unit 124 may form image data by photographing the exterior of the ship. According to an embodiment, the photographing unit 124 may include, but is not limited to, a 3D camera, a first person view (FPV) camera, an infrared camera, and the like. According to another embodiment, the photographing unit 124 may form image data including infrared information. In this way, since the image data includes infrared information, it is possible to provide clear and continuous information about the outside of the ship even at night to help the berthing facility (BF) of the ship (EV).

The mobile console 110 mounted on the ship (EV) receives a plurality of reference distance images obtained by photographing the reference distance measuring plate using a plurality of unmanned aerial vehicles (120), and uses a plurality of unmanned aerial vehicles (120) to Receives a plurality of image data photographed outside the ship, calculates offset information for each of a plurality of unmanned aerial vehicles 120 using a plurality of reference distance images and the actual distance of the reference distance measurement plate, and uses the offset information A plurality of image data may be synthesized to form an around-view image.

In addition, the mobile console 110 receives a plurality of image data formed in a plurality of unmanned aerial vehicles 120 , acquires a plurality of reference images for each of a plurality of image data, a plurality of reference images and a plurality of image data Offset information for at least some of the plurality of unmanned aerial vehicles 120 may be calculated based on the difference between , and the plurality of image data may be synthesized using the offset information to form an around-view image.

In addition, the mobile console 110 receives a plurality of location data formed in a plurality of unmanned aerial vehicles 120 , obtains a plurality of reference location information for each of the plurality of unmanned aerial vehicles 120 , and a plurality of reference location information and calculating offset information for at least some of the plurality of unmanned aerial vehicles 120 based on a difference from the plurality of location data, and synthesizing a plurality of image data using the offset information to form an around-view image.

The vessel EV may include a plurality of sensor units 140 . The sensor unit 140 may calculate the distance from the vessel (EV) to the object by using a time difference between a signal transmitted to an object located around the vessel (EV) and a signal received after being reflected from the object (EV). have. Also, the sensor unit 140 may generate a warning signal when the distance from the vessel EV to the specific object is within a threshold value (eg, 50m, 30m, 10m, etc.). According to an embodiment, the sensor unit 140 may include an infrared sensor, an ultrasonic sensor, and the like, but is not limited thereto.

The mobile console 110, when receiving a warning signal from the sensor unit 140, limits the output of the engine of the ship (EV) or sets the engine to stop to prevent the ship (EV) from collided with objects located nearby can be prevented

3 is an exemplary view showing the configuration of a mobile console according to an embodiment of the present invention.

As shown in FIG. 3 , the mobile console 110 may include one or more processors 112 , one or more memories 114 and/or transceivers 116 . As an embodiment, at least one of these components of the mobile console 110 may be omitted, or other components may be added to the mobile console 110 . Additionally (additionally) or alternatively (alternatively), some of the elements may be implemented as an integrated or singular or plural entity. At least some of the components inside and outside the mobile console 110 are connected to each other through a bus, a general purpose input/output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI). , data and/or signals may be exchanged.

As an embodiment, the mobile console 110 receives a plurality of reference distance images obtained by photographing a reference distance measuring plate using a plurality of unmanned aerial vehicles 120 , and uses a plurality of unmanned aerial vehicles 120 to view the outside of the ship. Receives a plurality of photographed image data, calculates offset information for each of a plurality of unmanned aerial vehicles 120 using a plurality of reference distance images and actual distances of a reference distance measurement plate, and uses the offset information to calculate a plurality of images The data may be synthesized to form an around-view image.

As another embodiment, the mobile console 110 receives a plurality of image data photographed outside the ship using a plurality of unmanned aerial vehicles 120 , and obtains a plurality of reference images for each of the plurality of image data, Calculate offset information for at least some of the plurality of unmanned aerial vehicles 120 based on the difference between the reference image and the plurality of image data, and synthesize a plurality of image data using the offset information to form an around-view image. can

As another embodiment, the mobile console 110 receives a plurality of location data and a plurality of image data for each of a plurality of unmanned aerial vehicles 120 located outside the ship, and receives a plurality of unmanned aerial vehicles 120 for each Acquire a plurality of reference position information, calculate offset information for at least some of a plurality of unmanned aerial vehicles based on a difference between the plurality of reference position information and a plurality of position data, and synthesize a plurality of image data using the offset information to form an around-view image.

The one or more processors 112 may control at least one component of the mobile console 110 connected to the processor 112 by driving software (eg, commands, programs, etc.). In addition, the processor 112 may perform various operations, processing, data generation, processing, etc. related to the present invention. In addition, the processor 112 may load data or the like from the memory 114 or store it in the memory 114 .

As described above, the one or more processors 112 may receive a plurality of image data photographed outside the ship from the plurality of unmanned aerial vehicles 120 through the transceiver 116 . The plurality of image data is formed in each of the plurality of unmanned aerial vehicles 120 , and may be formed by the photographing unit 124 provided in each of the plurality of unmanned aerial vehicles 120 , and may include infrared information. According to an embodiment, the plurality of unmanned aerial vehicles 120 may be set to photograph the outside of the vessel in the left and right sides of the bow of the vessel, the left and right sides of the stern of the vessel, and the center of the bow.

In addition, the one or more processors 112 may receive a plurality of location data for each of the plurality of unmanned aerial vehicles 120 from the plurality of unmanned aerial vehicles 120 through the transceiver 116 . A plurality of location data is formed in each of the plurality of unmanned aerial vehicles 120 , and may be formed by the location measurement unit 122 provided in each of the plurality of unmanned aerial vehicles 120 . According to an embodiment, the location measuring unit 122 may include a device such as a global positioning system (GPS) or a global navigation satellite system (GLONASS), but is not limited thereto.

Also, the one or more processors 112 may acquire a plurality of reference images for each of the plurality of image data from the one or more memories 114 . According to an embodiment, the one or more memories 114 may store an image of a port frequently used by the ship (EV) as a reference image. One or more processors 112 fetch a reference image from one or more memories 114 when the ship (EV) attempts to dock at the port, and based on the difference with the image data captured by the plurality of unmanned aerial vehicles (120) Thus, offset information for at least some of the plurality of unmanned aerial vehicles 120 may be calculated. According to an embodiment, when the reference image and the image data include an image of a portion of a vessel, the one or more processors 112 may be configured to select the image of the portion of the vessel in the reference image and the image of the portion of the vessel in the image data. Offset information may be calculated using the difference.

According to another embodiment, when the reference image and the image data include an image of a facility outside the ship, the one or more processors 112 use the difference between the facility image in the reference image and the facility image in the image data to obtain offset information can be calculated. For example, facilities outside the ship may include, but are not limited to, berthing facilities (BF), locks, anchors, and the like.

The one or more processors 112 may form an around-view image by synthesizing a plurality of image data using the calculated offset information. In addition, the one or more processors 112 calculate the distance between the vessel (EV) and an obstacle (eg, another vessel (BS) anchored at the berthing facility (BF)) outside the vessel using the formed around view image, and , an alarm message may be formed when the calculated distance is less than or equal to a threshold value (eg, 25m, 15m, 5m, etc.). According to one embodiment, the alarm message may include a phrase such as “Please be careful that there is a vessel moored 15m ahead of port port”, but the content of the alarm message is not limited thereto.

In addition, the one or more processors 112 receive, from one or more memories 114, a facility image for the berthing facility of the port that the vessel intends to dock, and information on the expected berthing area of the vessel by using the received facility image. It is formed, and information on the movement path of the vessel can be calculated by using the information on the formed expected berthing area. According to an embodiment, the information on the movement path may include information on the steering direction and steering angle of the steering gear, the movement speed of the ship, and the like, but is not limited thereto.

The one or more memories 114 may store various data. Data stored in the memory 114 is data acquired, processed, or used by at least one component of the mobile console 110 , and may include software (eg, commands, programs, etc.). Memory 114 may include volatile and/or non-volatile memory. In the present invention, commands or programs are software stored in the memory 114 , and various functions so that the operating system, applications and/or applications for controlling the resources of the mobile console 110 can utilize the resources of the mobile console 110 . It may include middleware that provides the application to the application.

The one or more memories 114 include a plurality of image data, a plurality of reference images, a plurality of position data, offset information, an alarm message, a plurality of facility images, information on the movement path of the ship, the ship (EV) and the ship ( EV) information about the distance to an external object can be stored. In addition, the one or more memories 114 may store instructions that, when executed by the one or more processors 112 , cause the one or more processors 112 to perform operations.

In one embodiment, the mobile console 110 may further include a transceiver 116 . The transceiver 116 may perform wireless or wired communication between the plurality of unmanned aerial vehicles 120 , the sensor unit 140 , and/or other devices. For example, the transceiver 116 may include enhanced Mobile Broadband (eMBB), Ultra Reliable Low-Latency Communications (URLLC), Massive Machine Type Communications (MMTC), long-term evolution (LTE), LTE Advance (LTE-A), UMTS (Universal Mobile Telecommunications System), GSM (Global System for Mobile communications), CDMA (code division multiple access), WCDMA (wideband CDMA), WiBro (Wireless Broadband), WiFi (wireless fidelity), Bluetooth (Bluetooth), NFC ( Near field communication), a global positioning system (GPS), or a global navigation satellite system (GNSS) may perform wireless communication. For example, the transceiver 116 may perform wired communication according to a method such as universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard232 (RS-232), or plain old telephone service (POTS). have.

As an embodiment, the one or more processors 112 may control the transceiver 116 to obtain information from the plurality of unmanned aerial vehicles 120 and the sensor unit 140 . Information obtained from the unmanned aerial vehicle 120 and the sensor unit 140 may be stored in one or more memories 114 . As an embodiment, the information obtained from the plurality of unmanned aerial vehicles 120 includes a plurality of image data formed by the position measurement unit 122 of the plurality of unmanned aerial vehicles 120 and the image data formed by the photographing unit 124, , the information obtained from the sensor unit 140 may include information on a distance between the vessel EV and an object located outside the vessel EV.

As an embodiment, the mobile console 110 may be a device of various types. For example, the mobile console 110 may be a portable communication device, a computer device, or a device according to a combination of one or more of the devices described above. However, the mobile console 110 of the present invention is not limited to the above-described devices.

Various embodiments of the mobile console 110 according to the present invention may be combined with each other. Each embodiment may be combined according to the number of cases, and the embodiment of the mobile console 110 made in combination also falls within the scope of the present invention. In addition, the internal/external components of the mobile console 110 according to the present invention described above may be added, changed, replaced, or deleted according to embodiments. In addition, the above-described internal/external components of the mobile console 110 may be implemented as hardware components.

4 and 5 are flowcharts showing a procedure of a method for supporting berthing in a port using an unmanned aerial vehicle according to an embodiment of the present invention.

Although process steps, method steps, algorithms, etc. are described in a sequential order in the flowcharts of FIGS. 4 and 5 , such processes, methods, and algorithms may be configured to operate in any suitable order. In other words, the steps of the processes, methods, and algorithms described in various embodiments of the invention need not be performed in the order described herein. Also, although some steps are described as being performed asynchronously, in other embodiments some of these steps may be performed concurrently. Further, the exemplification of a process by description in the drawings does not imply that the exemplified process excludes other changes and modifications thereto, and that the illustrated process or any of its steps is not one of the various embodiments of the present invention. It is not meant to be essential to one or more, nor does it imply that the illustrated process is preferred.

As shown in FIG. 4 , in step S410 , a plurality of reference distance images obtained by photographing a reference distance measuring plate using a plurality of unmanned aerial vehicles are received. For example, referring to FIGS. 1 to 3 , when the vessel (EV) attempts to berth in the port, the personnel performing the port berthing service carry the port berthing support device 100 and the vessel (EV) for berthing. It can be installed in the wheelhouse or in a safe place in front of the wheelhouse. After installation of the port berthing support device 100, the power of the mobile console 110 is turned on, and the power of the plurality of unmanned aerial vehicles 120 is turned on to turn on the plurality of unmanned aerial vehicles using the mobile console 110. You can start manipulating (120). Before the unmanned aerial vehicle 120 takes off, the reference distance measuring plate RM may be installed in a safe place near the take-off location of the unmanned aerial vehicle 120 . Each of the plurality of unmanned aerial vehicles 120 that have taken off may photograph the pre-installed reference distance measuring plate RM, and transmit the captured reference distance image to the moving console 110 .

In step S420 , a plurality of image data obtained by photographing the outside of the ship using a plurality of unmanned aerial vehicles is received. For example, referring to FIGS. 1 to 3 , a plurality of unmanned aerial vehicles 120 moving to a preset position forms image data by photographing an image of the outside of the vessel (EV) at the corresponding position, and is used for movement. The console 110 may receive the formed image data from a plurality of unmanned aerial vehicles 120 .

In step S430 , offset information for each of the plurality of unmanned aerial vehicles is calculated using the plurality of reference distance images and the actual distances of the reference distance measurement plate. For example, referring to FIGS. 1 to 3 , the one or more processors 112 of the mobile console 110 include a plurality of reference distance images and a reference received from a plurality of unmanned aerial vehicles 120 through the transceiver 116 . Offset information for each of the plurality of unmanned aerial vehicles 120 may be calculated by using the actual distance difference of the distance measuring plate RM. According to an embodiment, when the calculated offset value is greater than or equal to the threshold value, the one or more processors 112 may stop the subsequent procedure and generate an alarm message such as “Cannot form an around view.”

In step S440, an around-view image is formed by synthesizing a plurality of image data using the calculated offset information. For example, referring to FIGS. 1 to 3 , the one or more processors 112 of the mobile console 110 may synthesize a plurality of image data using the offset information calculated in step S430 to form an around-view image. can

As shown in FIG. 5 , in step S510 , a plurality of location data for each of a plurality of unmanned aerial vehicles located outside the ship and a plurality of image data formed in a plurality of unmanned aerial vehicles are received. For example, referring to FIGS. 1 to 3 , when the vessel (EV) attempts to berth in the port, the personnel performing the port berthing service carry the port berthing support device 100 and the vessel (EV) for berthing. It can be installed in the wheelhouse or in a safe place in front of the wheelhouse. After installation of the port berthing support device 100, the power of the mobile console 110 is turned on, and the power of the plurality of unmanned aerial vehicles 120 is turned on to turn on the plurality of unmanned aerial vehicles using the mobile console 110. You can start manipulating (120). Each of the plurality of unmanned aerial vehicles 120 may move to a preset position, and each of the plurality of unmanned aerial vehicles 120 forms position data at the corresponding position by using the position measurement unit 122 and the photographing unit 124 . is used to form image data by taking an image of the outside of the ship, and the mobile console 110 may receive a plurality of location data and a plurality of image data from a plurality of unmanned aerial vehicles (120).

In step S520, a plurality of pieces of reference location information for each of a plurality of location data is obtained. For example, referring to FIGS. 1 to 3 , a plurality of reference location information for each of a plurality of location data acquired from a plurality of unmanned aerial vehicles 120 may be stored in one or more memories 114 , and one The above processor 112 may obtain a plurality of pieces of reference location information from one or more memories 114 . According to an embodiment, the plurality of reference location information may include location information of a port where the vessel EV attempts to berth, but is not limited thereto.

In step S530 , offset information for at least some of the plurality of unmanned aerial vehicles is calculated based on the difference between the plurality of pieces of reference location information and the plurality of location data. For example, referring to FIGS. 1 to 3 , one or more processors 112 of the mobile console 110 , a plurality of reference location information fetched from one or more memories 114 and a plurality of unmanned aerial vehicles 120 . Offset information for at least some of the plurality of unmanned aerial vehicles 120 may be calculated based on a difference from the received plurality of location data. According to an embodiment, when the calculated offset value is greater than or equal to the threshold value, the one or more processors 112 may stop the subsequent procedure and generate an alarm message such as “Cannot form an around view.”

In step S540, an around-view image is formed by synthesizing a plurality of image data using the calculated offset information. For example, referring to FIGS. 1 to 3 , the one or more processors 112 of the mobile console 110 may synthesize a plurality of image data using the offset information calculated in step S530 to form an around-view image. can

Various embodiments of the present invention may be implemented as software in a machine-readable storage medium. The software may be software for implementing various embodiments of the present invention. Software can be inferred from various embodiments of the present invention by programmers in the art. For example, software may be a program containing machine-readable instructions (eg, code or code segments). The device is a device capable of operating according to a command called from a storage medium, and may be, for example, a computer. As an embodiment, the device may be a mobile console 110 according to embodiments of the present invention. As an embodiment, the processor of the device may execute the called command to cause device components to perform a function corresponding to the command. In one embodiment, the processor may be one or more processors 112 in accordance with embodiments of the present invention. The storage medium may refer to any type of recording medium in which data that can be read by a device is stored. The storage medium may include, for example, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like. In one embodiment, the storage medium may be one or more memories 114 . As an embodiment, the storage medium may be implemented as a distributed form in a computer system connected to a network or the like. The software may be distributed, stored, and executed in a computer system or the like. The storage medium may be a non-transitory storage medium. A non-transitory storage medium refers to a medium that exists regardless of whether data is semi-permanently or temporarily stored, and does not include a transitory propagated signal.

In the above, the present invention has been described with reference to the embodiments shown in the drawings. However, the present invention is not limited thereto, and various modifications or other embodiments within the scope equivalent to the present invention are possible by those of ordinary skill in the art to which the present invention pertains. Accordingly, the true scope of protection of the present invention should be defined by the following claims.

100: port berthing support device 110: mobile console
120: drone 130: storage device
140: sensor unit 112: processor
114: memory 116: transceiver
122: position measuring unit 124: photographing unit
RM: Reference distance measuring plate DU: Display
BS, FS, EV: Ship

Claims (10)

As a port berthing support device using an unmanned aerial vehicle,
one or more processors; and
and one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to perform an operation,
The one or more processors,
Receives a plurality of reference distance images of the reference distance measurement plate using a number of unmanned aerial vehicles,
Receives a number of image data taken outside the ship using a number of unmanned aerial vehicles,
Calculating offset information for each of the plurality of unmanned aerial vehicles by using the plurality of reference distance images and the actual distances of the reference distance measurement plate,
A port berthing support device using an unmanned aerial vehicle, which forms an around-view image by synthesizing the plurality of image data using the offset information. As a port berthing support device using an unmanned aerial vehicle,
one or more processors; and
and one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to perform an operation,
The one or more processors,
Receive a plurality of location data and a plurality of image data for each of a plurality of unmanned aerial vehicles located outside the ship,
Obtaining a plurality of reference location information for each of the plurality of unmanned aerial vehicles,
Calculating offset information for at least some of the plurality of unmanned aerial vehicles based on a difference between the plurality of reference location information and the plurality of location data,
A port berthing support device using an unmanned aerial vehicle, which forms an around-view image by synthesizing the plurality of image data using the offset information. 3. The method according to claim 1 or 2,
The one or more processors,
Calculating the distance between the obstacle outside the vessel and the vessel using the around-view image,
A port berthing support device using an unmanned aerial vehicle that forms an alarm message when the calculated distance is less than or equal to a threshold. 3. The method according to claim 1 or 2,
The one or more processors,
Receive a facility image of the berthing facility of the port where the vessel intends to dock,
Using the facility image to form information on the expected berthing area of the vessel,
A port berthing support device using an unmanned aerial vehicle for calculating information on a movement path of the vessel by using the information on the expected berthing area. 3. The method according to claim 1 or 2,
The plurality of image data includes infrared information, port berthing support device using an unmanned aerial vehicle. As a method of supporting port berthing using an unmanned aerial vehicle,
receiving a plurality of reference distance images obtained by photographing a reference distance measuring plate using a plurality of unmanned aerial vehicles;
Receiving a plurality of image data taken outside the ship using a plurality of unmanned aerial vehicles;
calculating offset information for each of the plurality of unmanned aerial vehicles by using the plurality of reference distance images and the actual distances of the reference distance measurement plate; and
Comprising the step of synthesizing the plurality of image data using the offset information to form an around-view image, a port berthing support method using an unmanned aerial vehicle. As a method of supporting berthing in a port using an unmanned aerial vehicle,
Receiving a plurality of location data and a plurality of image data for each of a plurality of unmanned aerial vehicles located outside the ship;
obtaining a plurality of pieces of reference location information for each of the plurality of unmanned aerial vehicles;
calculating offset information for at least some of the plurality of unmanned aerial vehicles based on a difference between the plurality of pieces of reference location information and the plurality of pieces of location data; and
Comprising the step of synthesizing the plurality of image data using the offset information to form an around-view image, a port berthing support method using an unmanned aerial vehicle. 8. The method according to claim 6 or 7,
calculating a distance between an obstacle outside the vessel and the vessel by using the around-view image; and
Port berthing support method using an unmanned aerial vehicle, further comprising the step of forming an alarm message when the calculated distance is less than or equal to a threshold value. 8. The method according to claim 6 or 7,
Receiving a facility image for a docking facility of a port to which the vessel intends to dock;
forming information on the expected berthing area of the vessel by using the facility image; and
Port berthing support method using an unmanned aerial vehicle, further comprising the step of calculating information on the movement path of the vessel by using the information on the expected berthing area. 8. The method according to claim 6 or 7,
The plurality of image data includes infrared information, port berthing support method using an unmanned aerial vehicle.

Images