KR102613592B1 - Method distant ship detection and detailed type identification using images - Google Patents

Abstract

A method for detecting distant vessels and identifying detailed types using images is provided. The ship detection method according to an embodiment of the present invention divides a maritime image into a plurality of regions, sequentially inputs the divided regions into an artificial intelligence model learned to receive the image, detects the ship, and outputs the regions. Detect ship areas and fuse the detected ship areas. As a result, it is possible to accurately detect distant vessels and identify their detailed types from images acquired using a camera in a maritime environment.

Description

{Method distant ship detection and detailed type identification using images}

The present invention relates to image detection/identification technology, and more specifically, to a method of detecting a distant vessel appearing in an image acquired from a vessel using an artificial intelligence model and identifying its detailed type.

As information for ship route planning and collision prevention, it is necessary to acquire images using cameras on ships and detect and identify distant ships from the acquired images.

However, if the distance to the target ship is very long, the size of the target ship appearing in the image is very small. Therefore, it is very difficult to properly detect ships by acquiring images in a maritime environment.

Even if a ship is detected, the detailed type of ship (e.g., warship, fishing vessel, passenger ship, etc.) must be identified, which is an even more difficult problem.

Recently, technology for detecting and identifying objects in images using deep learning has become active. In this method, the input image is changed to a specific standard (for example, 512x512) as shown in Figure 1 and input into the network. there is,

However, such resizing has the problem of further reducing the size of the target ship appearing in the maritime image.

The present invention was created to solve the above problems, and the purpose of the present invention is to provide a method and system for accurately detecting distant ships and identifying detailed types from images acquired using a camera in a maritime environment. there is.

In order to achieve the above object, a ship detection method according to an embodiment of the present invention includes dividing a maritime image into a plurality of regions; Step of sequentially inputting divided areas into a first artificial intelligence model trained to receive an image, detect and output a ship, and detecting ship areas in each area; and fusing the detected vessel areas.

In the segmentation step, the resolution image may be segmented so that some of the areas overlap each other. Additionally, adjacent areas due to division may overlap at least one of left and right and top and bottom.

In the fusion step, if the distance between detected vessel areas within an area where adjacent areas overlap is less than a predetermined distance, only the large vessel area may be left and the small vessel area may be removed.

A ship detection method according to an embodiment of the present invention includes selecting some of the fused ship areas; It may further include identifying the types of ships by sequentially inputting the ship areas selected in the selection step into a second artificial intelligence model that is trained to receive images, identify the types of ships, and output them.

In the selection step, only ship areas of a certain size or larger can be selected. In the identification step, the size of the vessel area can be adjusted based on the horizontal and vertical lengths of the selected vessel area and then input into the second artificial intelligence model.

The segmentation step may segment the maritime image, after removing areas where ships are unlikely to be present.

The area where ships are unlikely to exist may include at least one of an area higher than the horizon by a predetermined distance and a lower area.

Meanwhile, according to another embodiment of the present invention, a ship detection system includes a camera that generates a maritime image; and dividing the marine image generated by the camera into a plurality of areas, sequentially inputting the divided areas into the first artificial intelligence model learned to receive the image, detect the ship, and output the ship area in each area. It includes a processor that detects and fuses the detected vessel areas.

Meanwhile, according to another embodiment of the present invention, a vessel identification method includes dividing a maritime image into a plurality of regions; detecting vessel areas from the divided areas; It includes the step of identifying each type of vessel using the detected vessel areas.

Meanwhile, according to another embodiment of the present invention, a ship identification system includes a camera that generates a maritime image; and a processor that divides the marine image generated by the camera into a plurality of areas, detects ship areas from the divided areas, and identifies types of ships using the detected ship areas.

As described above, according to embodiments of the present invention, it is possible to accurately detect a distant vessel and identify its detailed type from an image acquired using a camera in a maritime environment.

Figure 1 is a diagram showing image resizing for deep learning network processing.
Figure 2 is a diagram provided to explain a method for detecting and identifying distant vessels in a maritime environment according to an embodiment of the present invention;
3 and 4 are diagrams illustrating region segmentation results for a high-resolution image;
Figure 5 is a diagram showing divided areas sequentially input to the ship detection network;
Figure 6 is a diagram showing the results of ship area fusion;
Figure 7 is a diagram illustrating the results of selecting the identification target vessel area;
8 is a conceptual diagram for identifying detailed types of ships;
Figures 9 and 10 are diagrams showing the results of detecting/identifying a distant vessel according to an embodiment of the present invention;
Figure 11 is a block diagram of a vessel detection/identification system according to another embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

In an embodiment of the present invention, a method for detecting distant ships and identifying detailed types (warships, fishing ships, passenger ships, yachts, etc.) from images acquired using a camera mounted on a ship in a maritime environment is presented.

To this end, in an embodiment of the present invention, vessel detection and type identification are performed step by step. Specifically, the vessel is first detected from the maritime image and then the detailed type of the vessel is identified for the detected vessel.

Additionally, in order to increase the detection success rate when using a deep learning network to detect ships, in an embodiment of the present invention, the input maritime image is divided and input into the deep learning network.

And, by limiting the identification target based on the size of the detected vessel area, the possibility of misidentifying the detailed type of vessel is reduced.

In addition, in order to accurately identify the detailed type of the detected vessel, in an embodiment of the present invention, the size of the detected vessel areas is adjusted to an appropriate size and input into the deep learning network.

Figure 2 is a diagram provided to explain a method for detecting and identifying distant vessels in a maritime environment according to an embodiment of the present invention.

To detect and identify a distant vessel, as shown, first, a maritime image obtained by shooting with a camera mounted on the vessel is input and divided into a number of areas (S110). The region division in step S110 is such that some of the regions overlap each other.

For example, as shown in FIG. 3, areas to be adjacent up and down during division are divided so that the upper and lower areas partially overlap, and areas to be adjacent left and right during division are divided so that the left and right regions partially overlap.

In FIG. 3, it is assumed that the resolution image is divided into four regions, but the number/method of division is illustrative, and it is possible to divide by a different number/method than shown.

For example, as illustrated in FIG. 4, it is possible to divide the divided areas so that only the left and right areas partially overlap. In addition, as shown, when dividing an area, it is possible to divide the area after removing the area that is higher than the horizontal line by a certain distance and the lower area.

Here, the area higher than the horizon by a certain distance and the lower area are areas where ships are unlikely to exist, and can be set manually by the user as well as automatically based on set standards.

Again, description will be made with reference to FIG. 2.

When region division is completed, the divided regions are sequentially input to the ship detection network, and ship regions are detected in each region (S120). FIG. 5 shows divided areas sequentially input to the ship detection network, specifically, the divided areas in the image of FIG. 4.

The ship detection network is a deep learning network that has been trained to receive video input, detect the ship area, and output it. Vessel area detection is performed independently for each area.

Next, the vessel areas detected in step S120 are fused (S130). In step S130, it is particularly important to fuse vessel areas where some of the detected vessel areas overlap.

For this purpose, for vessel areas detected within an area where adjacent areas overlap by division in step S110, if the distance between the detected vessel areas is less than a specified distance, only the large vessel area is left and the small vessel area is left. Regions can be merged by eliminating them.

That is, in the overlapping areas during division, it can be seen that only the large vessel areas are taken and the small vessel areas are removed for vessel areas detected at similar positions. Figure 6 shows the result of merging the vessel areas detected in the overlapping areas.

Thereafter, vessel areas to be identified are selected/set as ROI (Region Of Interest) from the vessel areas fused in step S130 (S140).

In step S140, the vessel area selected as the identification target vessel area may be an area larger than a certain size, for example, a vessel area larger than 100x50 pixels. However, the sizes presented are illustrative and may be modified.

Only the ship areas selected in step S140 are the ship areas for which the detailed type of ship will be identified. Figure 7 illustrates the results of selecting vessel areas to be identified based on area size.

Accordingly, the vessel areas selected/set in step S140 are sequentially input into the detailed vessel type identification network to identify the detailed types of vessels (S150). Figure 8 schematically shows the concept of a method of identifying detailed types of ships (warship, fishing ship, merchant ship, passenger ship, motorboat, yacht) using a ship detailed type identification network.

The detailed type identification network for ships is a deep learning network trained to receive image input, detect and output the detailed type of ship. Vessel type identification is also performed independently for each of the vessel areas selected in step S140.

Meanwhile, for the vessel areas selected in step S140, size adjustment is first performed before network input. Specifically, the longer of the horizontal and vertical lengths of the ship is found, the horizontal and vertical lengths are changed to twice the found length, and then input into the detailed ship type identification network.

For example, if the ship area is 120x70 pixels, twice the long length of 120 pixels is 240 pixels, so the ship area is adjusted to 240x240 pixels.

The remote vessel detection/identification method according to an embodiment of the present invention was actually implemented, and the results of detecting and identifying distant vessels from various images are presented in Figures 9 and 10.

As shown in Figures 9 and 10, it can be confirmed that accurate detection and detailed type identification were achieved even for distant vessels that appeared relatively small in maritime images.

Figure 11 is a block diagram of a vessel detection/identification system according to another embodiment of the present invention. As shown, the vessel detection/identification system according to an embodiment of the present invention includes a camera 210, an output unit 220, a processor 230, an input unit 240, and a storage unit 250. .

The camera 210 is mounted on a ship and is a means of photographing the sea, and generates a sea image.

The processor 230 divides the maritime image generated by the camera 210 into a plurality of regions, then detects and fuses ship regions in the divided regions using a ship detection network.

Additionally, the processor 230 selects vessel areas to be identified from the fused vessel areas, and sequentially inputs the selected vessel areas into the detailed ship type identification network to identify detailed types of ships.

The output unit 220 is a display for displaying the vessel detection results and vessel type identification results by the processor 230. The input unit 240 is an input means for receiving a user's command/setting and transmitting it to the processor 230.

The storage unit 250 provides storage space necessary for the processor 230 to function and operate.

So far, a method for detecting distant ships and identifying detailed types (warships, fishing ships, passenger ships, yachts, etc.) from images acquired using a camera mounted on a ship in a maritime environment has been described in detail using preferred embodiments.

Meanwhile, of course, the technical idea of the present invention can be applied to a computer-readable recording medium containing a computer program that performs the functions of the device and method according to this embodiment. Additionally, the technical ideas according to various embodiments of the present invention may be implemented in the form of computer-readable code recorded on a computer-readable recording medium. A computer-readable recording medium can be any data storage device that can be read by a computer and store data. For example, of course, computer-readable recording media can be ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, hard disk drive, etc. Additionally, computer-readable codes or programs stored on a computer-readable recording medium may be transmitted through a network connected between computers.

In addition, although preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the invention pertains without departing from the gist of the present invention as claimed in the claims. Of course, various modifications can be made by those of ordinary skill in the art, and these modifications should not be understood individually from the technical idea or perspective of the present invention.

S110: Video segmentation
S120: Ship detection network
S130: Ship detection fusion
S140: ROI settings
S150: Vessel Detailed Type Identification Network

Claims (12)

A vessel detection system comprising: dividing a maritime image into a plurality of image regions;
A ship detection system sequentially inputting divided image regions into a first artificial intelligence model trained to receive marine images, detect ships, and output them, and detect ship regions in each image region; and
The vessel detection system, if the distance between the two detected vessel areas is less than a predetermined distance, leaving only the larger vessel area of the two vessel areas and removing the smaller vessel area; Vessel detection method.
delete delete delete In claim 1,
A method for detecting a ship, further comprising the step of inputting the remaining ship area from the removal step into a second artificial intelligence model trained to receive marine images and identify and output the type of ship, thereby identifying the type of ship. .
delete A vessel detection system comprising: dividing a maritime image into a plurality of image regions;
A ship detection system sequentially inputting divided image regions into a first artificial intelligence model trained to receive marine images, detect ships, and output them, and detect ship regions in each image region;
If the distance between the two detected vessel areas is less than a predetermined distance, the vessel detection system removes the smaller vessel area among the two vessel areas, leaving only the larger vessel area;
The vessel detection system inputs the remaining vessel area from the removal step into a second artificial intelligence model that has been trained to receive marine images and identify and output the type of vessel, thereby identifying the type of vessel.
The steps to identify the type of vessel are:
A ship detection method characterized by adjusting the size of the ship area based on the horizontal and vertical lengths of the ship area remaining in the removal step and then inputting it into a second artificial intelligence model.
In claim 1,
The step of dividing a resolution image into multiple image regions is:
A ship detection method characterized by segmenting a maritime image after removing areas where ships are unlikely to exist.
In claim 8,
In areas where ships are unlikely to exist,
A vessel detection method comprising at least one of an area that is higher than the horizon by a predetermined distance and an area that is lower than the horizon by a predetermined distance.
A camera that produces marine images; and
The marine image generated by the camera is divided into multiple image areas, and the divided image areas are sequentially input to the first artificial intelligence model that has been trained to receive the marine image and detect and output the ship. A processor that detects ship areas and, if the distance between the two detected ship areas is less than a predetermined distance, removes the smaller ship area and leaves only the larger ship area among the two ship areas. vessel detection system.
A vessel detection system comprising: dividing a maritime image into multiple regions;
A vessel detection system comprising: detecting vessel areas from the divided areas;
If the distance between the two detected vessel areas is less than a predetermined distance, the vessel detection system removes the smaller vessel area among the two vessel areas, leaving only the larger vessel area;
A vessel identification method comprising: identifying, by the vessel detection system, each type of vessel using vessel areas.
A camera that produces marine images; and
The maritime image generated by the camera is divided into multiple image areas, vessel areas are detected from the divided image areas, and if the distance between the two detected vessel areas is less than a certain distance, the size of the two vessel areas is A ship identification system comprising: a processor that leaves only the larger ship area, removes the smaller ship area, and uses the ship areas to identify each type of ship.