KR20240059205A - Drone for ship guidance, ship guidance system and method there of - Google Patents

Abstract

The present invention relates to an unmanned aerial vehicle for guiding ships that monitors the expected path of a ship using an unmanned aerial vehicle and guides the path during berthing and navigation of a ship, and a ship guide system and method for guiding a ship using the same. The unmanned aerial vehicle uses video images. It includes an imaging unit, a sound wave measurement unit, and a control unit, and the ship guide system guides the ship including an unmanned aerial vehicle and a server.

Description

Unmanned aerial vehicle for ship guidance, ship guide system and ship guide method using the same {DRONE FOR SHIP GUIDANCE, SHIP GUIDANCE SYSTEM AND METHOD THERE OF}

The present invention relates to an unmanned aerial vehicle for ship guidance that monitors the expected path of a ship using an unmanned aerial vehicle and guides the path of a ship during berthing and navigation, and a ship guide system and ship guide method using the same.

In the operation, berthing, and berthing of ships, there is a risk of accidents occurring due to the operator's carelessness, and in many cases, such carelessness is mainly caused by the fact that it is difficult to visually check the situation around the ship or in the port.

Additionally, there may be cases where the seafloor topography on the chart is different from the actual seafloor topography. In the case of existing ships, there are many cases where sonar devices such as sonar are not equipped. In this case, it is difficult to understand the marine environment, including the seabed topography, without sonar devices, so navigation relies on the pilot's knowledge and experience. There are many cases where this happens. And, even in these situations, careless driving may occur.

In particular, in the situation of berthing, the ship must be maintained parallel to the port or quay wall, but it is difficult to accurately determine information such as the angle and speed of the ship, so it is common for multiple tugboats and navigators to be mobilized to berth.

Therefore, there is a need for a ship assistance system that does not depend on the operator's experience in operating, berthing, and berthing of a ship, and can safely operate, berth, and berth without the help of a tugboat or the like.

Korean Patent Publication No. 10-2011-0025398 (published on March 10, 2011)

The present invention is intended to solve the above problems, and its purpose is to provide a ship guide system and method for assisting the navigation and berthing of ships using an unmanned aerial vehicle that acquires image information and sound wave information.

In order to achieve the above object, the present invention provides an unmanned aerial vehicle and a ship guide system formed as follows.

The unmanned aerial vehicle according to an embodiment of the present invention includes an image capture unit for capturing marine images, a sound wave emitting unit for firing sound waves toward the sea surface, and a sound wave recognition unit for recognizing that the sound waves are reflected from the sea floor, A sound wave measuring unit that acquires sound wave information of the seabed topography, an information transmitting unit that transmits the image information captured by the image capture unit and the sound wave information to a server, and the information obtained by the information transmitting unit from the server is processed and derived. It includes an information receiving unit that receives position supplementation information based on the maritime information and the preset ship route, and a control unit that controls the flight state including the position and height of the unmanned aerial vehicle based on the maritime information or the position supplementation information. It may further include a digital recorder unit that stores information obtained from the image capture unit and the sound wave measurement unit.

A ship guide system according to an embodiment of the present invention includes an unmanned aerial vehicle that acquires maritime information of the expected path of a ship, and a server that analyzes and processes the information acquired by the unmanned aerial vehicle, and the server provides the obtained information. It includes a first processing unit that collects and analyzes information, and a second processing unit that supplementally calculates the flight state of the unmanned aerial vehicle using the obtained information. The unmanned aerial vehicle has a flight path controlled based on a route search mode that searches for an expected route during a ship's voyage, a control unit that supplements the flight path according to information processed by the server, and an image that captures images of the ocean. It includes a photographing unit and a sound wave irradiation unit that irradiates sound waves toward the seafloor, a sound wave recognition unit that recognizes that the sound waves are reflected from the seafloor, and further includes a sound wave measurement unit that acquires sound wave information, and the route search mode is, It moves through a reconnaissance area set at a preset reconnaissance distance in the direction of the traveling direction and a preset reconnaissance angle, and can acquire image information and sound wave information through the image capture unit and the sound wave measurement unit.

Through the above structure, the present invention has the advantage of helping to search the expected path of the ship in advance and secure a wide field of view when berthing the ship, preventing ship accidents from occurring, and reducing manpower during navigation and berthing. there is.

It has the advantage of being able to respond immediately if there is a problem with the ship's route through the server.

Figure 1 is a schematic diagram of a ship guide system including an unmanned aerial vehicle for ship guide in one embodiment of the present invention.
Figure 2 is a schematic diagram showing an area where an unmanned aerial vehicle obtains maritime information in a route search mode according to an embodiment of the present invention.
Figure 3 is a flow chart in the route search mode according to an embodiment of the present invention.
Figure 4 is a diagram of an image taken by an unmanned aerial vehicle in a dual-eye search mode according to an embodiment of the present invention. (a) is a state in which top view image information of a ship has been acquired, and (b) is a state in which an object is detected. (c) is a drawing of the image in alignment with the ship.
FIG. 5 is a block diagram of content corresponding to (a), (b), and (c) of FIG. 4 in the disjunctive eye search mode according to an embodiment of the present invention.
Figure 6 is a flowchart in the disjunctive eye search mode according to an embodiment of the present invention.
Figure 7 is a block diagram showing a ship guide method according to an embodiment of the present invention.

Hereinafter, specific embodiments of the present invention will be described with reference to the attached drawings. However, the spirit of the present invention is not limited to the presented embodiments, and those skilled in the art who understand the spirit of the present invention may add, change, or delete other components within the scope of the same spirit, or create other degenerative inventions or this invention. Other embodiments that are included within the scope of the inventive idea can be easily proposed, but it will also be said that this is included within the scope of the inventive idea of the present application.

In addition, in describing the present invention, '~ part' or 'unit' may be used in various ways, for example, a processor, program instructions executed by the processor, software module, microcode, computer program product, logic circuit, application-specific It can be implemented by an integrated circuit, firmware, etc.

The contents of the method disclosed in the embodiments of the present application may be directly implemented with a hardware processor, or may be implemented and completed through a combination of hardware and software modules among the processors. Software modules may be stored in conventional storage media such as random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. The storage medium is located in a memory, and the processor reads the information stored in the memory and combines it with the hardware to complete the content of the method described above. To prevent duplication, detailed description is omitted here.

In the implementation process, each content of the above-described method can be completed by a logical integrated circuit of hardware among processors or instructions in the form of software. The contents of the method disclosed in the embodiments of the present application may be directly implemented with a hardware processor, or may be implemented and completed through a combination of hardware and software modules among the processors. Software modules may be stored in conventional storage media such as random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. The storage medium is located in a memory, and the processor reads the information stored in the memory and combines it with the hardware to complete the content of the method described above.

That is, those skilled in the art will know that each exemplary unit and algorithm step described in the embodiments disclosed herein can be combined and realized by electronic hardware or a combination of computer software and electronic hardware. You can. Whether these functions are performed by hardware or software is determined by the specific application and design constraints of the technical solution. Those skilled in the art may implement the described functionality using different methods for each particular application, but such implementation should not be considered beyond the scope of the present application.

In some embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiment described above is merely illustrative. For example, the division of the units is only a kind of logical function division, and in actual implementation, other division methods may exist, for example, a plurality of units. Alternatively, the assembly may be combined or integrated into another system, or some features may be ignored or not performed. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be an indirect coupling or communication connection through some interface, device or unit and may be electrical, mechanical or other form.

Units described above as separate parts may be physically separated, and parts displayed as units may or may not be physical units, that is, they may be located in one place or distributed across a plurality of network units. Some or all of the units may be selected according to actual demand to realize the purpose of the scheme of this embodiment.

That is, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist alone, or two or more than two units may be integrated into one unit.

If the above function is implemented in the form of a software function unit and sold or used as an independent product, it may be stored in a single computer-readable storage medium. Based on this understanding, the part of the technical solution of this application that is essentially or a contribution to the prior art or a part of the technical solution may be implemented in the form of a software product, and the computer software product is stored in a storage medium. , including some instructions to cause one computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The above-described storage media includes various media that can store program code, such as USB memory, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or CD-ROM. Includes.

Figure 1 briefly shows an unmanned aerial vehicle for guiding a ship and a ship guide system including the same according to an embodiment of the present invention.

In the present invention, the ship 1 may include a location recognition unit 11 and a ship route control unit 12. The location recognition unit 11 is used to determine the current location during navigation and may be a system such as GPS. The ship route control unit 12 is a part that controls the route of the ship 1. It may be directly controlled by the server 3 described below, but is controlled by the ship route control unit 12 from inside the ship 1. The route in (1) may be controlled.

The unmanned aerial vehicle (2) for guiding a ship that assists in route navigation or berthing of a ship (1) according to an embodiment of the present invention includes a portion for actual measurement of the image capture unit (21) and the sound wave measurement unit (22). A digital recorder unit 23 that stores information, an information transmitter 24 that transmits measured information to the outside, an information receiver 25 that receives externally processed information, etc., and the flight path and flight of the unmanned aerial vehicle (2). It may include a control unit 26 that controls status, etc.

The unmanned aerial vehicle (2) referred to in the present invention includes all objects that fly without a person directly aboard, and may generally refer to a drone.

The video capture unit 21 can capture high-resolution images. The image capture unit 21 may use a conventional imaging device or camera. In addition, correction means, etc. may be additionally provided to prevent image shaking. The video capture unit 21 may use a fisheye lens that shoots at 360° to shoot at various angles, and a plurality of shooting equipment is installed to capture at least the top view of the ship 1 and the unmanned aerial vehicle 2. It may be equipped to photograph the front. In addition, since it is intended to be used on the ocean, a separate anti-corrosion means may be provided to prevent corrosion due to salt, or a corrosion-prevention layer may be provided on the surface of the unmanned aerial vehicle (2).

The sound wave measurement unit 22 includes a sound wave irradiation unit 22a that irradiates sound waves toward the seafloor, and a sound wave recognition unit 22b that recognizes the reflected sound waves when the irradiated sound waves are reflected by terrain in the water. . Through this, sound wave information of the seafloor topography can be obtained. A unique sound wave signal is generated and fired toward the sea surface, and the fired sound wave is recognized or detected by the sound wave recognition unit 22b. These sound wave signals can be analyzed using information about the irradiated time and received time or the speed at which the sound wave propagates in water.

The digital recorder unit 23 can store image information and sound wave information obtained from the image capture unit 21 and the sound wave measurement unit 22. It is initially stored in the digital recorder unit 23 and then transmitted to the server 3 in batches, which can help prevent data damage and efficiently process data.

The digital recorder unit 23 may be provided in the server 3 in addition to the unmanned air vehicle 2 or in a server other than the server 3.

The information transmission unit 24 can transmit the image information and the sound wave information captured by the image capture unit 21 to the server 3. Information can be transmitted to the server 3 according to all common methods.

The information receiving unit 25 receives maritime information derived by processing the information processed by the server 3 and location supplementary information about the location of the unmanned aerial vehicle 2 based on the preset expected path of the ship 1. You can. The unmanned aerial vehicle (2) can receive location supplementary information from the server (3) through any method available to the public.

In particular, by applying IoT technology, communication between the server 3 and the unmanned aerial vehicle 2, marine structures, and mobile floating objects may be possible.

When operating the ship 1, the position supplementary information may change the expected path of the ship 1 depending on the presence or absence of obstacle objects in the sea and the seabed, or the expected path may be maintained as is. This may be information about the return location (ro) to which the unmanned aerial vehicle (2) should return, considering the path of the ship (1) and the speed of the unmanned aerial vehicle (2).

Alternatively, in the case where the ship 1 is not adjacent, the unmanned aerial vehicle 2, which photographs a certain part of the ship 1 in the image information according to the movement of the ship 1, is aligned with the ship 1. This may be information about the sort position. That is, based on the expected path of the ship 1, the position or path of the unmanned aerial vehicle 2 may be modified or supplemented according to the position supplementation information.

The control unit 26 can control the flight state including the position and height of the unmanned aircraft 2. This control can be modified and supplemented by maritime information or location supplementation information.

Hereinafter, in the ship guide system or ship guide method, the description of the unmanned air vehicle will refer to the contents of the unmanned air vehicle for ship guide (2) described above.

A ship guide system that guides ship navigation or berthing using an unmanned aerial vehicle according to an embodiment of the present invention includes an unmanned aerial vehicle (2) and a server (3).

The unmanned aerial vehicle (2) can acquire maritime information about the ship's expected route in advance. The maritime information acquired by the unmanned aerial vehicle 2 includes image information. And additionally, it may be sound wave information. Additionally, the unmanned aerial vehicle may be the unmanned aerial vehicle 2 for a ship guide described above.

The unmanned aerial vehicle 2 may be divided into modes for controlling the flight state depending on the operating state of the ship 1.

The server 3 can process and analyze the information acquired by the unmanned air vehicle 2, calculate and transmit information to supplement the control of the flight path and flight state of the unmanned air vehicle 2.

The server 3 includes a first processing unit 31 that collects and analyzes the information acquired by the unmanned air vehicle 2, and a second processing unit that supplements and calculates the flight state of the unmanned air vehicle 2 with the acquired information ( 32) may be included.

The first processing unit 31 and the second processing unit 32 may be configured separately, or may be arbitrary areas divided into one processing unit to arbitrarily distinguish each processing.

The server 3 may be equipped with a chart storage unit 33. The chart storage unit 33 is a part where existing charts are stored, and digital charts may be managed.

Additionally, the server 3 may be provided with a server receiving unit 34 and a server transmitting unit 35. Information transmitted from the unmanned aerial vehicle 2 or the ship 1 can be received through the server receiver 34. Additionally, the unmanned aerial vehicle 2, ship 1, or display unit 4 may further include a server transmission unit 35 for transmitting information processed by the server 3.

The present invention may further include a display unit 4. It can further include a display unit 4 to output or display information processed by the server 3 and provide the effect of allowing a user or administrator to easily see the information transmitted through the display unit 4. Additionally, by displaying various information at once on the display unit 4, you can check information without missing it.

Figure 2 schematically shows the flight path of an unmanned aerial vehicle in route search mode. Figure 3 shows a flow chart according to one embodiment of the present invention.

The flight path of the unmanned aerial vehicle 2 may be controlled based on a route search mode that searches for an expected route while the ship 1 is sailing.

The route search mode is a flight mode for guiding the ship's route by having the unmanned aerial vehicle (2) search in advance for the expected route of the vessel (1) and photographing and measuring information to determine whether there will be any problems in operating on the expected route. .

Referring to FIG. 2, the route search mode moves to a preset reconnaissance area and can acquire image information and sound wave information through the image capture unit 21 and the sound wave measurement unit 22. The preset reconnaissance area may be an area comprised of the moving direction of the ship 1, a preset reconnaissance angle (θ), and a preset reconnaissance distance (r).

As an example, an area having a reconnaissance angle θ, which is a specific angle based on the forward direction of the expected path of the ship 1, can be detected in advance. If the preset reconnaissance angle (θ) is set to 30° and the reconnaissance distance (r) is preset to 3km, the forward direction of the expected path of the ship based on the current position of the ship (1) is set to 0°, and the upper angle and lower angle are set to 0°. It can be configured to move the area so that the angles of relief are each 30° and a fan-shaped area with a radius of 3 km. However, the return location (ro) may be moved to the position where the ship (1) will be located at the time of return based on the current position of the ship (1). As another example, after moving along the expected path to the return position (ro) expected to be the return point after reconnaissance of the unmanned aerial vehicle (2) at a certain distance among the reconnaissance distance (r), for example, on the expected path before the change, As shown, based on the current position of the ship (1), the forward direction of the ship's expected path is set to 0°, and the upper and lower angles are each 30° and are configured to scout a distance corresponding to an arc with a radius of 3 km. It may be possible. That is, firstly, based on the current position of the ship (1), move a certain distance along the expected path of the ship, and secondly, based on the current position of the ship (1), set the forward direction of the expected path to 0° and set the upper angle or Move to a point where the lower angle is 30° and the distance is 3km. Third, based on the current position of the ship (1), set the forward direction of the expected path to 0° and the upper or lower angle is 30° and the distance is 3km. You can move along. Finally, at the time of return, the unmanned aerial vehicle (2) can move the estimated location of the ship (1) to the return location (ro).

In the route search mode, it is desirable to acquire image information or sound wave information while maintaining a constant height, and as an example, it can be acquired by maintaining an altitude of 80 m from the ship 1. However, there may be cases where it is difficult to maintain a constant height due to obstacles in the sky or weather conditions, or it may be advantageous to take pictures while changing the height, and it is not limited to taking pictures at a constant height.

In the route search mode, the unmanned aerial vehicle 2 moves in a certain area as described above. While moving through the area, maritime image information can be acquired through maritime photography using a camera, and information on the topography of the seafloor can be obtained by examining sound waves in the sound wave measurement unit 22 and measuring the time for the returning sound waves. And the acquired information may be stored in the digital recorder unit 23, which is a separate storage area.

Image information and sound wave information can be transmitted from the information transmission unit 24 to the server 3. Information can be received from the server receiver 34 within the server 3. The first processing unit 31 receives image information and sound wave information and can determine maritime information on the ship's expected route.

The first processing unit 31 distinguishes objects existing in the sea through the image information, determines whether the object is an obstacle on the expected path of the ship, and determines the measured seafloor topography through the sound wave information. , Compare the seabed topography information of the chart stored in the chart storage unit 33 and the measured seafloor topography to determine whether they are the same, but only if there are no obstacles in the sea and the seafloor topography is the same, the ship (1) The path can be maintained as the expected path. Also, if an obstacle exists on the expected path of the ship, the path of the ship 1 can be changed. Alternatively, if the seafloor topography is not the same, the seafloor topography information in the chart storage unit 33 can be stored as the measured seafloor terrain, and the route of the ship 1 can be changed.

For example, object detection using deep learning can be performed through image information. Afterwards, when moving to the expected route, if an obstacle object existing on the route is recognized, the route of the ship 1 can be changed.

Objects recognized as objects may include ships 1, buoys, terrain, ports, buildings, etc. Hereinafter, analysis of images obtained mainly through visible light cameras will be explained, but the analysis is not limited thereto.

Additionally, image information may be recognized according to segmentation, which indicates which object each pixel in the image corresponds to. Additionally, characteristics corresponding to pixels for each object are labeled or assigned in advance, so that the object can be recognized according to this prior information and segmentation.

By comparing the recognized object with the path of the ship 1, it can be determined whether it is an obstacle object. If there is an obstacle, the path of the ship 1 is changed. If there is no obstacle, the path of the ship 1 is maintained, but a result of whether the path is changed can be derived based on the sound wave information.

The above segmentation and determination of whether a fault object exists can also be derived through an artificial neural network. Types of artificial neural networks include a convolution neural network (CNN), which extracts features using filters, and a recurrent neural network (RNN), which has a structure in which the output of a node is fed back to the input. Various structures such as restricted Boltzmann machine (RBM), deep belief network (DBN), generative adversarial network (GAN), and relational networks (RN) can be applied and have limitations. It doesn't exist. Before using an artificial neural network, a learning step is necessary.

Alternatively, it can be trained using an artificial neural network. Artificial neural networks can be learned through various methods such as supervised learning, unsupervised learning, reinforcement learning, and imitation learning.

For example, the sound wave information may show the seafloor topography according to the transmission time of the sound wave by the first processing unit 31. By comparing the directly measured seafloor topography with the existing chart previously stored in the chart storage unit 33 present in the server 3, it can be determined whether the current topography and the topography according to the chart are the same. Identical as referred to in the present invention, in addition to being completely identical, may also include matching to the extent that there is no problem such as collision of the vessel 1 when the vessel 1 moves along the expected path.

If the seafloor topography stored in the chart storage unit 33 and the measured seafloor topography are the same, the existing expected route is maintained, and if they are not the same, the ship route is changed. In this case, the process of storing and updating the actually measured seafloor topography in the chart storage unit 33 may be further included. That is, the seafloor topography information of the chart storage unit 33 can be stored as the measured seabed terrain and the route of the ship 1 can be changed.

The image information and sound wave information may be processed separately or simultaneously by the first processing unit 31, but at least one of the cases where it is determined that an obstacle object exists based on the image information or the seafloor topography is determined to be different based on the sound wave information. In the case of It can be controlled to keep the path as expected.

And the unmanned air vehicle 2 has a return position (ro) calculated by the second processing unit 32, and the server 3 directly controls the unmanned air vehicle 2 or sends the unmanned air vehicle 2 to the second processing unit ( The flight path may be supplemented and controlled to move to the return position (ro) by the control unit 26 provided in the unmanned air vehicle 2 by transmitting the calculation result by 32).

The second processing unit 32 calculates the return position ro based on the processing result of the first processing unit 31.

As an example, the second processing unit 32 obtains GPS location information from the location recognition unit 11 provided on the ship 1 and analyzes the expected path of the ship and the speed of the unmanned aerial vehicle 2. Thus, the return position of the unmanned aerial vehicle (2) can be calculated. And, when the acquisition of information on the reconnaissance area is completed, the unmanned aerial vehicle (2) receives the value from the second processing unit (32) by the information receiving unit (25), and returns the value by the control unit (26). The flight path can be supplemented to move to position ro.

If the ship's route is maintained, the return position ( ro), the flight path of the unmanned aerial vehicle (2) can be supplemented.

If the ship (1) changes its route, the time to move from the search end position of the unmanned aerial vehicle (2) to a point on the changed route of the ship (1), and the time to move to a point on the changed route of the ship (1) The flight path of the unmanned aerial vehicle (2) can be supplemented by calculating the point where the travel time is the same as the return position (ro) of the unmanned aerial vehicle (2).

In addition, in the case where the expected route of the ship 1 is changed, the unmanned aerial vehicle 2 has a second processing unit to supplement the route with the route for the search before providing information on the return location (ro) to search for a new route. The calculated result can also be derived from (32).

In addition to the above-mentioned contents, the second processing unit 32 may include supplementing, maintaining, and changing the flight path of the unmanned aerial vehicle 2 in consideration of the ship path.

FIG. 4 is a diagram illustrating a series of processes for recognizing and aligning objects from shooting information captured by an unmanned aerial vehicle in a disjunctive eye search mode according to an embodiment of the present invention, and FIG. 5 is a distal eye search mode of FIG. 4 This is a block diagram explaining the control of an unmanned aerial vehicle. Figure 6 shows a flow chart in the case of the berthing search mode of the ship guide system according to an embodiment of the present invention.

The berthing search mode is used to prevent accidents with ports or adjacent ships (1) when berthing a ship (1) by securing a field of view that the ship (1) cannot see from the top of the ship (1), thereby preventing accidents in advance. This is a mode to prevent.

In the berthing search mode, the flight path of the unmanned air vehicle 2 is controlled based on the berthing search mode, which searches for the berthing path of the ship 1 expected at the time of berthing of the ship 1, and the unmanned air vehicle 2 ) can be controlled to acquire image information through the image capture unit 21 that captures a top view image of the vessel 1 while moving at a preset height in parallel with the movement path of the vessel 1 there is.

The unmanned aerial vehicle 2 may be further equipped with a control unit 26 to supplement the flight path with the information processed by the server 3.

As an example, in the disjunctive eye search mode, the first processing unit 31 distinguishes objects existing at sea through the image information, sets the largest detected ship 1 among the objects as the main ship, and sets the main ship. From data including image information and the speed of the unmanned aerial vehicle 2, data of the main ship including the proceeding angle and speed of the main ship can be calculated based on the eye or berthing object.

The first processing unit 31 classifies objects in the image information in the disjunctive eye search mode and sets the largest object among the recognized objects as the main vessel. The main vessel becomes the vessel (1) to be guided. When berthing, a top view of the vessel (1) is obtained from directly above, and other objects that are difficult to see within the vessel (1) and when the vessel (1) is controlled to move along the expected path may cause an accident. Check to see if it doesn't happen. Alternatively, an accident can be prevented in advance by obtaining a view from directly above the ship 1 and using it as a direct reference when the operator is berthing.

Figure 4(a) shows that the unmanned aerial vehicle 2 captured image information directly above the ship 1. And Figure 4 (b) can be set to recognize the largest recognized ship (1) as the main ship using object detection technology through deep learning in the same way as described in the route search mode. It shows that the part shown in the yellow box is set as the main ship. According to (c) of FIG. 4, this is image information in a state in which the main ship is set with the unmanned aerial vehicle 2 aligned with the ship 1. That is, the unmanned aerial vehicle 2 can be controlled to be positioned at the alignment position of the ship 1 due to the first processing unit 31 and the second processing unit 32.

When the second processing unit 32 converts the image information into an image and the image appears in the shape of a square with a first horizontal length and a second length that is relatively shorter or equal to the first length, The vessel alignment position of the unmanned aerial vehicle 2 is such that the length of the vessel 1 from the head to the stern of the main vessel and the width of the vessel 1 of the main vessel are located at the center of the first length and the second length, respectively. can be calculated. In addition, the ship alignment position is transmitted by the server transmitter 35 provided in the server 3, and the information receiver 25 provided in the unmanned air vehicle 2 and receives information receives the ship alignment position. And, the flight path can be supplemented so that the control unit 26 moves the unmanned aerial vehicle 2 to the ship alignment position. The flight path can be controlled by the control unit 26 of the unmanned air vehicle 2 by transmitting route supplementation information from the second processing unit 32. In other words, as the unmanned aerial vehicle (2) is located in a position aligned with the ship (1), it provides information with a wide field of view that could not be seen from the ship (1), and uses this to guide the operation of the ship (1). The position of the unmanned aerial vehicle 2 is maintained along the path of the ship 1, and the berthing search mode may be released when the berthing of the ship 1 is completed.

In addition, since the height of the unmanned air vehicle 2 must be above a certain level to secure a view that is difficult to see within the ship 1, the height of the unmanned air vehicle 2 may be preset. And if the height is not reached, the height of the unmanned aerial vehicle 2 can be controlled to change.

Hereinafter, the ship guide method will be described. In case the ship guide method has the same name or mentioned process and effect as the above-mentioned unmanned aerial vehicle or ship guide system, the above-mentioned content is cited.

Figure 7 shows a flow chart for a ship guide method according to an embodiment of the present invention.

According to an embodiment of the present invention, in a method for monitoring and guiding the expected path of a ship, a ship measurement information acquisition step (S10), an information processing step (S20), an information determination step (S30), and a ship path It includes a correction step (S40) and an unmanned aerial vehicle path correction step (S50).

In the measurement information acquisition step (S10), a preset reconnaissance range that can be formed by the reconnaissance distance (r), which is a certain distance from the expected path of the ship and a specific point of the expected path, and the reconnaissance angle (θ), which is a certain angle, is determined by the unmanned aerial vehicle. As (2) moves, image information of the sea is acquired by the image capture unit 21 provided in the unmanned air vehicle 2, and sound wave information of the sea floor is acquired by the sound wave measurement unit 22 provided in the unmanned air vehicle 2. This is the stage of acquiring.

In the information processing step (S20), the acquired image information is received by the server 3, and the object in the image information is recognized through object recognition using deep learning by the first processing unit 31 provided in the server 3. It may include image information processing to recognize and distinguish, and sound wave information processing to receive the acquired sound wave information to the server 3 and calculate the seafloor topography with the sound wave information by the first processing unit 31. there is.

In the information determination step (S30), the presence or absence of an obstacle that may be an obstacle to the movement of the ship 1 is determined from the processed image information, and the chart provided in the server 3 is determined from the processed sound wave information. This is a step of comparing the seafloor terrain stored in the storage unit 33 with the calculated seafloor terrain.

In the ship path modification step (S40), in the first processing unit 31, if the obstacle object exists or the stored seabed terrain is different from the calculated seabed terrain, the expected path of the ship is changed, and the obstacle object If does not exist and the stored seafloor topography and the calculated seafloor terrain are the same, this is a step of maintaining the expected path of the ship 1 as is.

In the unmanned aerial vehicle path correction step (S50), the server 3 acquires the location information of the ship 1 by the location recognition unit 11 provided on the ship 1, and sends the information to the server 3. This is a step of calculating the return position (ro) of the unmanned aerial vehicle (2) by calculating the location information of the ship (1) and the speed of the unmanned aerial vehicle (2) by the provided second processing unit (32).

In addition, a display step (not shown) may be further included to display the information calculated by the first processing unit 31 or the second processing unit 32 on a separate display unit 4. .

Through the above steps, the unmanned aerial vehicle can scout the expected route of the ship in advance and guide the ship, providing the effect of preventing ship accidents in advance.

In the above, the present invention has been described focusing on the embodiments, but the present invention is not limited to the above-described embodiments, and may be modified and implemented by those skilled in the art without changing the technical spirit of the present invention as claimed in the claims. Of course.

1: Ship 2: Unmanned Aerial Vehicle
3: Server 4: Display unit
11: Location recognition unit 12: Ship route control unit
21: video recording unit 22: sound wave measurement unit
23: digital recorder unit 24: information transmission unit
25: information receiving unit 26: control unit
31: first processing unit 32: second processing unit
33: chart storage unit 34: server reception unit
35: Server transmitter r: Reconnaissance distance
ro: return position θ: reconnaissance angle
S10: Measurement information acquisition step S20: Information processing step
S30: Information judgment step S40: Ship route correction step
S50: Unmanned aerial vehicle path correction step

Claims (14)

In the unmanned aerial vehicle that assists ship route navigation or berthing,
Video recording department that captures marine video;
A sound wave measurement unit comprising a sound wave emitting unit that fires a sound wave toward the sea surface, a sound wave recognition unit that recognizes that the sound wave is reflected from the sea floor, and acquiring sound wave information of the sea floor topography;
an information transmission unit that transmits the image information captured by the image capture unit and the sound wave information to a server;
an information receiving unit that receives location supplementary information from the server based on maritime information derived by processing the information acquired by the information transmitting unit and a preset vessel route; and
a control unit that controls the flight state of the unmanned aerial vehicle based on the maritime information or the location supplementary information; An unmanned aerial vehicle for ship guidance, including a.
According to paragraph 1,
An unmanned aerial vehicle for guiding a ship, further comprising a digital recorder unit that stores information obtained from the image capture unit and the sound wave measurement unit.
In a ship guide system that guides ship navigation or berthing using an unmanned aerial vehicle,
It includes an unmanned aerial vehicle that acquires maritime information about the expected path of the ship, and a server that analyzes and processes the information acquired by the unmanned aerial vehicle,
The server is,
a first processing unit that collects and analyzes the obtained information;
A ship guide system including a second processing unit that supplementally calculates the flight state of the unmanned aerial vehicle using the acquired information.
According to paragraph 3,
The unmanned aerial vehicle,
The flight path is controlled based on the route search mode, which searches for the expected route while the ship is sailing.
a control unit that supplements the flight path according to the information processed by the server;
A video recording department that captures images of the ocean; and
It further includes a sound wave irradiation unit that irradiates sound waves toward the seafloor, a sound wave recognition unit that recognizes that the sound waves are reflected from the seafloor, and a sound wave measurement unit that acquires sound wave information,
The route search mode is,
Moves the reconnaissance area set at the preset reconnaissance distance in the direction of the ship's direction of travel and the preset reconnaissance angle,
A ship guide system that acquires image information and sound wave information through the image capture unit and the sound wave measurement unit.
According to paragraph 4,
The second processing unit,
Obtaining GPS location information from the location recognition unit provided on the vessel,
Calculate the return location of the unmanned aerial vehicle by analyzing the expected path of the ship and the speed of the unmanned aerial vehicle,
The unmanned aerial vehicle,
When information acquisition of the reconnaissance area is completed, the value from the second processing unit is received by the information receiving unit, and the flight path is supplemented so that the control unit moves to the return location.
According to clause 4,
The server further includes a chart storage unit in which charts are stored,
The unmanned aerial vehicle further includes an information transmission unit that transmits information obtained through the image capture unit and the sound wave measurement unit to the server,
The server further includes a server receiver that receives the information,
The first processing unit,
Distinguish between objects existing at sea through the image information and determine whether the object is an obstacle on the expected path of the ship,
Identify the measured seafloor topography through the sound wave information, and compare the seafloor topography information of the chart stored in the chart storage unit with the measured seafloor topography to determine whether they are the same,
A ship guide system that maintains the path of the ship as the expected path when there are no obstacles in the sea and the seafloor topography is the same.
According to clause 6,
If the seafloor topography is not the same,
Saving the seafloor topography information of the chart storage unit as the measured seafloor topography,
A ship guide system that changes the route of the ship.
According to clause 6,
A ship guide system that changes the path of the ship when an obstacle exists in the ship's expected path.
According to paragraph 3,
The unmanned aerial vehicle,
The flight path is controlled based on the berthing search mode, which searches for the expected berthing path of the ship when berthing,
a control unit that supplements the flight path according to the information processed by the server;
It further includes a video capture unit that takes a top view image of the ship,
The above-mentioned eye search mode is,
A ship guide system in which the image capture unit acquires image information while moving at a preset height parallel to the movement path of the vessel.
According to clause 9,
The unmanned aerial vehicle further includes an information transmission unit that transmits information obtained through the image capture unit to the server,
The server further includes a server receiver that receives the information,
The first processing unit,
Recognize an object from the image information,
Among the objects, the largest detected vessel is set as the main vessel,
A ship guide system that calculates data of the main ship including the moving angle and speed of the main ship based on the eye or berthing object from data including the image information and the speed of the unmanned aerial vehicle.
According to clause 10,
The second processing unit,
When the image information is converted to an image and the image appears in the shape of a square with a first length and a second length,
Calculate the vessel alignment position of the unmanned aerial vehicle so that the vessel length from the head to the stern of the main vessel and the vessel width of the main vessel are located at the center of the first length and the second length,
The ship alignment position is transmitted by a server transmitter provided in the server,
An information receiver provided in the unmanned aerial vehicle to receive information receives the vessel alignment position,
A ship guide system in which the flight path is supplemented so that the control unit moves the unmanned aerial vehicle to the ship alignment position.
According to paragraph 3,
A ship guide system further comprising a display unit that outputs information processed by the server.
In the method of monitoring and guiding the expected route of a ship,
As the unmanned aerial vehicle moves through a preset reconnaissance range from the expected path of the ship and a specific point on the expected path, image information of the sea is acquired with an image capture unit provided on the unmanned aerial vehicle, and the seabed is measured with a sound wave measurement unit provided on the unmanned aerial vehicle. Obtaining sound wave information;
The acquired image information is received by a server, and the first processing unit provided in the server determines whether there is an obstacle object that may be an obstacle to the movement of the vessel through object recognition using deep learning,
Receiving the acquired sound wave information to the server, calculating a seabed topography using the sound wave information by the first processing unit, and comparing the calculated seabed topography with the seabed topography stored in the chart storage unit provided in the server. ; and
In the first processing unit, if the obstacle object exists or the stored seafloor terrain and the calculated seafloor terrain are different, the expected path of the ship is changed,
maintaining the expected path of the ship when the obstacle object does not exist and the stored seafloor terrain and the calculated seafloor terrain are the same; Ship guide method including.
According to clause 13,
The server acquires location information of the ship by a location recognition unit provided on the ship,
A ship guide method further comprising calculating the return location of the unmanned aerial vehicle by calculating the location information of the vessel and the speed of the unmanned aerial vehicle by a second processing unit provided in the server.

Images