KR20230051012A - Management system of lighters wharf - Google Patents

Abstract

A lighters wharf management system is disclosed. According to one embodiment of the present invention, the lighters wharf management system may comprise: a vessel identification unit which detects an image of a vessel by receiving and analyzing real-time captured images from a photography unit installed at the entrance to the lighters wharf; a location information acquisition unit which receives location information of the vessel from an automatic vessel identification device or a fishing vessel location transmission device mounted on the vessel; a database which stores information matching the image of the vessel and the location information of the vessel, information of the detected vessel, and the image of the vessel; and a monitoring unit which checks and monitors arrival/departure information and berthing information of the vessel from the database, loads and checks the information of the vessel stored in the database, and analyzes event information of the vessel. The monitoring unit may be configured to: receive map data from a map server where the map data of the lighters wharf is stored; display the image of the vessel at a point on the map data corresponding to the final position of the vessel received from the location information acquisition unit; and load and display the information of the vessel on a display when the image of the vessel is clicked. According to the present invention, the lighters wharf management system can automatically identify and manage vessels arriving and departing from the lighters wharf.

Description

Fish farm management system {MANAGEMENT SYSTEM OF LIGHTERS WHARF}

The present invention relates to a wharf management system, and more particularly, to a wharf management system capable of automatically identifying and managing vessels entering and leaving the wharf.

In general, a water wharf is a quay structure in which small ships dock and moor, and means a pier with a front water depth of less than 4.5 m in a port and less than 2.5 m in a fishing port.

In order to use the water wharf, a user fee must be paid, but there is a problem that it is impossible to grasp even if there is an unauthorized vessel entering or exiting due to the absence of a management system.

In the case of a fishery managed by the Busan Port Authority, an average of 50 to 100 ships enter and exit per day per fishery, and about 600 ships are anchored in disaster situations such as typhoons. Despite the fact that many vessels enter and exit, it is currently being managed manually based on the information reported by the owner, and real-time management is difficult for unauthorized use and unauthorized entry and departure.

As a result, management such as imposition and collection of usage fees is difficult, and it is also difficult to grasp the ship's arrival and departure and mooring conditions.

Accordingly, in order to quickly respond to an accident such as a collision between ships, it is necessary to grasp the information of the vessel and other anchored vessels, but there is a problem in that the risk of secondary accidents increases because it cannot be grasped.

In addition, environmental pollution and route threats may occur due to abandoned ships and long-term abandoned ships, and there is a problem in that it is difficult to prove abandoned ships due to the absence of a management system.

Republic of Korea Patent Registration No. 10-2137380 (2020.07.20) "Smart Port Management System" Republic of Korea Patent Registration No. 10-2206662 (2021.01.18) "Vision camera system and method for vehicle access management and object recognition at a port container terminal" Korean Registered Patent No. 10-2096746 (2020.03.27) "Blockchain-based access control system and method for port security"

An object of the present invention is to provide a wharf management system capable of automatically identifying and managing vessels entering and leaving the wharf.

The fish farm management system according to the present invention detects a ship through deep learning from a real-time video taken at the entrance of the fish farm by a CCTV camera, and matches the location information of the ship and the ship information obtained through the location information acquisition unit to the fish farm. Its purpose is to provide a waterfront management system that manages the arrival and departure of ships.

The tasks of the present invention are not limited to the technical tasks mentioned above, and other technical tasks not mentioned will be clearly understood by those skilled in the art from the following description.

A fishery management system according to an embodiment of the present invention includes: a ship identification unit for detecting a ship image by receiving and analyzing a real-time captured image from a photographing unit installed at the entrance of a fishery; A location information acquisition unit for receiving location information of a ship from an automatic vessel identification device or a fishing vessel location transmitter mounted on the vessel; a database for storing information in which the ship image and location information of the ship are matched and information of the detected ship, and storing the ship image; and a monitoring unit that checks and monitors the vessel's arrival and departure information and anchoring information from the database, loads and checks the vessel information stored in the database, and analyzes the vessel's event information. The unit receives the map data from the map server where the map data of the fishery is stored, displays the ship image at a point on the map data corresponding to the final position of the ship received from the location information acquisition unit, and displays the ship image When is clicked, information on the vessel may be loaded and displayed on the display.

Here, when the location information of the ship is not received from the automatic vessel identification device or the fishing vessel location transmitter, a character recognition unit for recognizing the ship name by analyzing the captured image and recognizing characters displayed on the ship; The drone may further include a drone that obtains coordinates of the photographing point of the photographing unit, and the monitoring unit may receive coordinate information of the photographing point from the drone.

Here, the ship identification unit detects a ship image through image analysis using a deep learning algorithm, and data generated through the deep learning algorithm may be stored in the database in the form of an image file.

Here, the character recognition unit enlarges and analyzes images captured by a plurality of the photographing units at different magnifications, and data labeled with characters displayed on the ship may be stored.

Here, the monitoring unit receives the map data and loads fish farm map data; displaying a camera icon in an area corresponding to the installation location of the photographing unit; generating a ship location update thread and an unregistered ship confirmation thread; and when the ship location update thread is executed, the ship image is displayed at a point on the map data corresponding to the final location of the ship received from the location information acquisition unit. When the unregistered vessel confirmation thread is executed, access information of the unregistered vessel is displayed, and when the vessel image is clicked, the vessel information can be loaded and displayed on the display at that time.

Here, the monitoring unit may overlay an image displaying a vessel and information of the vessel on the captured image based on the information received from the location information obtaining unit.

According to embodiments of the present invention, at least the following effects are provided.

The wharf management system according to the present invention can automatically identify and manage vessels entering and leaving the wharf.

In addition, it is possible to manage the arrival and departure of the ship by detecting whether or not the object in the captured image of the photographing unit is a ship through the ship identification unit and confirming the location of the ship from the location information acquisition unit.

In addition, the data generated through video analysis by the ship identification unit using a deep learning algorithm is stored in the database in the form of an image file, which is then used as a dataset for video analysis by CCTV, which is the shooting unit, so that the ship can be more accurately identified. there is.

In addition, the location of the ship is displayed as a ship image on the map data, and when the ship image is clicked, the ship information received from the location information acquisition unit is displayed, so monitoring can be easily performed.

In addition, the vessel information received from the location information acquisition unit and the image displaying the vessel are overlaid on the real-time image captured by the photographing unit, so that the vessel can be easily monitored in real time.

In addition, even in the case of a ship without a location information acquisition unit, information such as a ship name can be checked through a text recognition unit.

Effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the present specification.

1 is a device diagram of a fishery management system according to a first embodiment of the present invention
2 is a relationship diagram of a vessel identification unit of a waterfront management system according to a first embodiment of the present invention
3 to 5 are examples of monitoring screens of the monitoring unit of the watershed management system according to the first embodiment of the present invention.
6 is a monitoring flow chart of the monitoring unit of the fishery management system according to the first embodiment of the present invention
7 is a generating step algorithm of the monitoring unit of the fishery management system according to the first embodiment of the present invention.
8 is a device diagram of a fish farm management system according to a second embodiment of the present invention.

Since the present invention can apply various transformations and have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention, and methods for achieving them will become clear with reference to the embodiments described later in detail together with the drawings. However, it should be understood that the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms, and includes all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

Prior to the description, the terms described in the detailed description will be described. In the following embodiments, terms such as first and second are used for the purpose of distinguishing one component from another component without limiting meaning. Accordingly, it goes without saying that the first element mentioned below may also be the second element within the technical spirit of the present invention. Also, expressions in the singular number include plural expressions unless the context clearly dictates otherwise. In addition, terms such as 'include' or 'have' mean that features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, and that one or more other features or components are present. It does not preclude the possibility of being added.

In addition, in the drawings, the size of components may be exaggerated or reduced for convenience of explanation. For example, since the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of description, the present invention is not necessarily limited to the illustrated bar.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same or corresponding components are assigned the same reference numerals, and overlapping descriptions thereof will be omitted.

In the wharf management system according to an embodiment of the present invention, a ship is detected through deep learning from a real-time video taken at the entrance of the wharf with a CCTV camera, and the ship's location information and ship information obtained through the location information acquisition unit are used. It is about a wharf management system that manages ships entering and leaving the wharf by matching.

1 is a device diagram of a watershed management system according to a first embodiment of the present invention, and FIG. 2 is a relationship diagram of a vessel identification unit of the watershed management system according to a first embodiment of the present invention.

The watershed management system 1000 according to the first embodiment of the present invention includes a vessel identification unit 100, a location information acquisition unit 200, a database 300, and a monitoring unit 400.

The vessel identification unit 100 is a component that detects a vessel image by receiving and analyzing a real-time captured image from the photographing unit 10 installed at the entrance of the watershed. In this embodiment, the photographing unit 10 is provided as a CCTV camera to photograph ships arriving and departing.

The ship identification unit 100 receives a captured image from a CCTV camera through real-time streaming protocol (RTSP) communication, and detects a ship image from the real-time image. The vessel identification unit 100 analyzes CCTV image information in depth with deep learning learning-based image analysis technology and recognizes an object in the image as a vessel.

The vessel identification unit 100 includes CNN (Convolutional Neural Network), RNN (Recurrent Neural Networks), R-CNN (Recurrent Convolutional Neural Network), Fast RCNN, and Faster RCNN (Region based Convolutional Neural Network), YOLO (You Only Look) Once), the ship image in the camera image can be detected using any one of the SSD (Single Shot Detector) algorithm, and in this embodiment, an R-CNN-based deep learning algorithm is used.

In this embodiment, the deep learning algorithm for ship image detection uses a convolutional neural network (CNN) algorithm having a multilayer structure of input layer/hidden layer/output layer among deep learning algorithms of artificial intelligence.

CNN consists of one or several convolution layers and artificial neural network layers placed on top of them, and additionally utilizes weights and pooling layers. Thanks to this structure, CNN can sufficiently Utilize

Multi-layer convolutional neural network (CNN) algorithm including input layer/n hidden layers (Layer 1, Layer 2, Layer 3...)/output layer from camera input image detects ship images by extracting and classifying features of CCTV camera images.

Detects objects (containers, vehicles, people, loading and unloading equipment, etc.) .

A convolutional neural network (CNN) uses three layers including a convolutional layer, a pooling layer, and a fully connected layer (FC layer).

The CNN algorithm moves a mask (e.g., a 3x3 window) with weights in the input image by a certain distance and repeats convolution and subsampling by a convolutional layer and a pooling layer, respectively. Reduces the amount of data in the image, extracts the feature of the image, extracts the feature map by convolution, and classifies the object (human, ship, etc.) by neural network do.

In image processing, in the CNN algorithm, convolution is performed using a mask (eg, 3x3 window) with weights, and the mask (eg, 3x3 window) with weights in the input image. While moving , a mask having a weight is applied to the current input image, pixel values of the input image are multiplied by the weight of the mask, and the sum is determined as the pixel value of the output image.

Subsampling is a process of reducing the screen size, and max pooling is used to select the maximum value of the corresponding area.

The FC layer (Fully Connected Layer) classifies objects by learning by connecting them to the input terminal of the neural network.

Currently, a 5-layer convolution layer and a 3-layer fully_connected layer can be used. In order to learn object tracking, the object tracking database 300 can be used for learning.

The output of the Convolutional Layer is subsampled by the Max-Pooling Layer to reduce the size of the feature map of the image, and the output of the Max-Pooling is an object in the FC layer (Fully Connected Layer). classify the class of

As a result, to detect objects (humans, ships, etc.) in the image of the drone camera, and to extract and classify the features of each object, the object location area and type information are obtained from several convolutional layers in the middle of the CNN structure. The size of the feature map is reduced as it passes through the pooling layer, and object location area information is extracted from feature maps of different sizes to detect objects, and the FC layer (Fully Connected Layer , fully connected layer) to classify the ship image.

The basic structure of R-CNN extracts Region Proposals where objects are estimated to exist by using a Region Proposal generation algorithm called Selective Search from an input image. Each Region Proposal is in the form of an image in a rectangular Bounding Box, and after making all Region Proposals the same size, classification is performed through CNN.

The ship image detected by the ship identification unit 100 is matched with the location information of the ship received by the location information acquisition unit 200 .

The location information acquisition unit 200 is a component for acquiring ship information including ship location information. The location information acquisition unit 200 receives the location information of the ship from an automatic vessel identification device or a fishing vessel location transmitter mounted on the vessel.

AIS (Automatic Identification System) is an international standard system that exchanges information between ships or with land for ship operation information and port reports, and must be used on ships of 300 tons or more.

The fishing vessel location transmitter (V-pass) is a device that transmits the position of the fishing vessel and emergency rescue signal for prompt response in the event of a marine accident, and can automatically process the report of the fishing vessel's entry and exit in accordance with Article 15 of the Ship Safety Operation Regulations. . The terminal for fishing boats is a device in the form of a combination of GPS and wireless data modem. The 897Mhz receiver receives the position of the fishing boat received from the GPS and sends the data to the Korea Coast Guard. ), the location of the fishing boat is displayed on the electronic chart screen of the branch office, etc., enabling safety management such as real-time positioning.

It is possible to obtain the location information of the vessel by receiving a signal from the automatic vessel identification device and/or the fishing vessel location transmitter. At this time, the fishing vessel positioning device can receive signals from related organizations such as the bar that monitors vessel information, contact information and location in the 'fisheries integrated management system' of the Fishing Boat Safety Bureau and the 'fishing vessel safety operation management system' of the National Federation of Fisheries Cooperatives.

That is, the location information acquisition unit 200 may directly or indirectly receive the ship's location information from the ship's automatic identification device or fishing vessel location transmitting device mounted on the ship.

In addition to location information, various ship information is received from the location information acquisition unit 200 . This ship information may include MMSI (Maritime Mobile Service Identity), ship name, ship type, destination, navigation status, ground speed, and the like.

The database 300 stores information in which the ship image and location information of the ship are matched, the detected ship information, and the ship image.

Specifically, the database 300 receives and stores various vessel information from the location information acquisition unit 200 through TCP/IP communication, and transmits the vessel information to the vessel identification unit 100 through TCP/IP communication. The vessel identification unit 100 matches the detected vessel image with the received vessel information and stores them in the database 300 through TCP/IP communication.

In addition, data generated through image analysis by the vessel identification unit 100 using a deep learning algorithm is stored in the database 300 in the form of an image file. The stored image file is again used as a data set for video analysis of CCTV, which is the photographing unit 10.

The present vessel identification unit 100 detects an object (ship) through an R-CNN-based deep learning algorithm, and detects it by determining a feature of a video image. Because of these characteristics, there are many cases where an object is not recognized/misrecognized when the angle is slightly different. However, in the present embodiment, multiple units of the photographing unit 10 may capture images at various angles and build various data sets according to the various angles.

3 to 5 are examples of monitoring screens of the monitoring unit of the watershed management system according to the first embodiment of the present invention, and FIG. 6 is a monitoring flowchart of the monitoring unit of the watershed management system according to the first embodiment of the present invention. is an algorithm for the generation step of the monitoring unit of the watershed management system according to the first embodiment of the present invention.

The monitoring unit 400 is a component for monitoring ships entering and departing from the marina.

The monitoring unit 400 largely performs monitoring to display ship information on a map screen and real-time control monitoring.

Monitoring displaying ship information on the map screen receives map data from the map server where the map data of the wharf is stored, and locates the ship at a point on the map data corresponding to the final position of the ship received from the location information acquisition unit 200. The image is displayed, and when the ship image is clicked, the information of the ship is loaded and displayed on the display.

Specifically, the monitoring unit 400 executes a loading step, an icon display step, a creation step, and a display step when performing monitoring to display ship information on a map screen.

The loading step is a step of receiving map data and loading fish farm map data. That is, the map data of the fish farm is stored in the map server, and the map data is loaded.

Then, a camera icon is displayed on the map data in an area corresponding to the location where the photographing unit 10 is installed.

At this time, in addition to the camera unit 10, which is a CCTV installed at the entrance of the kennel, camera icons may be displayed at installation positions of other cameras that take pictures of the inside of the kennel.

The creation step is a step of creating a vessel location update thread and an unregistered vessel confirmation thread.

When the ship location update thread is executed, a ship image is displayed at a point on the map data corresponding to the final location of the ship received from the location information acquisition unit 200. If the thread end event is received, the thread is terminated. If not received, the latest location information of ships is received and the ship image is displayed at the final location.

Displays the access information of unregistered vessels when the unregistered vessel verification thread is executed.

In the display step, when the user clicks the ship image, ship information is loaded and displayed on the display.

A list of information received from the location information acquisition unit 200 and the like is displayed on the map image, and when the user clicks on any one ship image, the above information of the corresponding ship is displayed.

Real-time control monitoring overlays the image displaying the ship and ship information on the captured image taken by the photographing unit 10 to display ship information based on the real-time image and to display system or network errors that occur during system operation. Controls the captured image of the photographing unit 10 so that it can be checked in real time.

It receives the captured image from the photographing unit 10 through RTSP and displays the ship's arrival and departure and surrounding images in real time.

Then, ship information (transmitted from the location information acquisition unit 200 to the database 300) is received from the database 300, and the ship information is overlaid on the real-time image so that it is displayed together.

Next, the watershed management system 1000 according to the second embodiment of the present invention will be described.

8 is a device diagram of a fish farm management system according to a second embodiment of the present invention.

The watershed management system 1000 according to the second embodiment of the present invention includes a vessel identification unit 100, a positioning detection unit, a database 300, a monitoring unit 400, a character recognition unit 500, and A drone 600 is included. Since the configuration except for the character recognition unit 500 and the drone 600 is the same as that of the first embodiment, duplicate descriptions are omitted.

The character recognition unit 500 and the drone 600 are components for recognizing a ship when ship information cannot be grasped.

The text recognition unit 500 is a component for recognizing a ship name by analyzing a photographed image and recognizing text displayed on a ship when ship location information is not received from an automatic ship identification device or a fishing vessel location transmitter.

In addition to ship location information, various ship information is received from the location information acquisition unit 200 . For monitoring, information such as ship name in addition to ship location information is required. When information is not received from the location information acquisition unit 200, it is necessary to determine the ship's location and ship name.

Fishing vessels excluding the installation of a fishing vessel location transmitter include non-powered boats, fishing boats engaged in inland water fishing under the 「Inland Water Fisheries Act」, fishing boats engaged in testing, investigation, guidance and regulation related to the fishery industry, and licensed fishing under Article 8 of the 「Fishing Industry Act」 Fishing boats designated or approved as management vessels pursuant to Article 27 Paragraphs 1 and 3 of the same Act (however, fishing boats under the 「Fishing Management and Promotion Act」 are excluded), 「Ship Safety Fishing Regulations」 」 A fishing boat attached to a fishing vessel engaged in inshore gillnet fishing pursuant to Article 24 (1) 111 of the 「Enforcement Decree of the Fisheries Act」 in a specific area of the West Sea pursuant to Article 5 (2) 2, a fishing boat of less than 2 tons, without an upper deck, only on the side end. or a fishing boat without a structure on the upper deck. It is impossible to acquire location information and ship information through the location information acquisition unit 200 for such a fishing boat.

Accordingly, the character recognition unit 500 is a component that reads the ship name by receiving the captured image of the photographing unit 10 and recognizing the character.

Since the name of the ship is marked on the left, right, and rear of the stern, the ship name is read by receiving the captured image of the photographing unit 10 and recognizing the character.

And, the drone 600 is located on the photographing area of the photographing unit 10 . That is, the drone 600 flies above a point corresponding to the shooting area of the photographing unit 10 . The flight coordinates of the drone 600 are transmitted to the database 300, and location information of the ship can be known from the coordinates of the drone 600.

That is, when a ship enters the area photographed by the photographing unit 10, the text recognition unit 500 analyzes the ship name, and since the location information of the drone 600 becomes the location information of the ship, the flight coordinates of the drone 600 Through this, the position information of the ship can be known.

Therefore, according to the present invention, a ship is detected through deep learning from a real-time video taken at the entrance of the wharf by a CCTV camera, and the ship's location information obtained through the location information acquisition unit is matched with the ship information to enter and depart the wharf. A fish farm management system for managing is provided.

The use of all examples or exemplary terms (eg, etc.) in the present invention is simply to explain the present invention in detail, and the scope of the present invention is limited due to the examples or exemplary terms unless limited by the claims. it is not going to be In addition, those skilled in the art will know that various modifications, combinations and changes can be made according to design conditions and factors within the scope of the appended claims or equivalents thereof.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments and should not be determined, and not only the claims described later, but also all ranges equivalent to or equivalent to these claims are within the scope of the spirit of the present invention. would be said to belong.

1000, 2000 : Fish farm management system
100: vessel identification unit 200: location information acquisition unit
300: database 400: monitoring unit
500: text recognition unit 600: drone

Claims (6)

A vessel identification unit for detecting a vessel image by receiving and analyzing a real-time captured image from a recording unit installed at the entrance of a fish farm;
A location information acquisition unit for receiving location information of a ship from an automatic vessel identification device or a fishing vessel location transmitter mounted on the vessel;
a database for storing information in which the ship image and location information of the ship are matched and information of the detected ship, and storing the ship image; and
A monitoring unit that checks and monitors the vessel's arrival and departure information and anchorage information from the database, loads and checks the vessel information stored in the database, and analyzes the vessel's event information;
The monitoring unit,
The map data is received from the map server where the map data of the marina is stored, the ship image is displayed at a point on the map data corresponding to the final position of the ship received from the location information acquisition unit, and the ship image is clicked. A wharf management system that loads information on the vessel and displays it on a display.
According to claim 1,
a character recognition unit for recognizing a ship name by analyzing the captured image and recognizing characters displayed on the ship, when the vessel location information is not received from the automatic vessel identification device or the fishing vessel location transmitter;
It further includes; a drone that acquires coordinates on the photographing point of the photographing unit,
The monitoring unit receives the coordinate information of the shooting point from the drone.
According to claim 1,
The vessel identification unit,
Using a deep learning algorithm to detect ship images through image analysis,
Data generated through the deep learning algorithm is stored in the database in the form of an image file.
According to claim 2,
The character recognition unit,
The wharf management system in which the captured images of the plurality of photographing units are enlarged and analyzed at different magnifications, and the data labeled with the characters displayed on the ship are stored.
According to claim 1,
The monitoring unit,
receiving the map data and loading fish farm map data;
displaying a camera icon in an area corresponding to the installation location of the photographing unit;
Creating a vessel location update thread and an unregistered vessel confirmation thread;
When the ship location update thread is executed, the ship image is displayed at a point on the map data corresponding to the final location of the ship received from the location information acquisition unit;
When the unregistered vessel confirmation thread is executed, the access information of the unregistered vessel is displayed,
When the ship image is clicked, the ship's information is loaded and displayed on the display at the time.
According to claim 1,
The monitoring unit,
A marina management system for overlaying an image displaying a ship and information of the ship on the captured image based on the information received from the location information acquisition unit.

Images