KR101666466B1 - Marine risk management system and marine risk management method using marine object distance measuring system with monocular camera - Google Patents

Abstract

The present invention relates to a maritime risk management system using a marine object distance measuring system using a monocular camera, which includes a camera installed at a height from a predetermined sea level at an observation point and having a constant vertical angle of view and vertical resolution used for distance measurement A horizontal line detecting unit for detecting a horizontal line of the object and a water surface boundary line of a sea object and a water surface of the object, An object image processing unit for calculating a distance between the horizontal line and a surface boundary line of the object and calculating and outputting a distance from the observation point to the marine object from a vertical height of the camera, (10) using a monocular camera including a camera (120) 0), receives the marine object image taken from the marine object distance measuring system 100 using the monocular camera and the distance information from the observation point to the marine object as camera image information, Receives the AIS information from the AIS information processing unit 400, receives and reads the camera image information, the radar information, and the AIS information to detect the marine object, and detects the danger of the marine object And a maritime risk management system 200 for processing and outputting image information associated with a marine object, wherein the maritime risk management system 200 is configured to determine whether or not to read the received camera image information, radar information, and AIS information When the marine object is detected, the photographing position of the camera of the marine object distance measuring system 100 using the monocular camera is detected It generates a driving control signal for controlling the position for photographing the body by processing the object image of the object photographed in the maritime a control position wherein the output.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a marine risk management system and a marine risk management method using a marine object distance measuring system using a monocular camera,

The present invention relates to a maritime risk management system and a maritime risk management method, and more particularly, to a maritime risk management system and a maritime risk management method that measure distances of maritime objects through image processing of maritime objects photographed using a monocular camera, The present invention relates to a marine risk management system and a marine risk management method using a marine object distance measurement system using a monocular camera capable of real-time marine risk management in connection with a radar signal.

Currently, the maritime risk management system used in marine vessels and harbors performs recognition of marine objects, calculation of position / distance, danger warning, dangerous warning such as on-screen display and collision, and based on radar information and AIS information And the observation camera is used separately from radar information and AIS information for the main purpose of providing images.

However, radar is inevitably affected by the shaded area, and the detection ability depends largely on the distance and the size of the ship. The radar receiving sensitivity is deteriorated depending on the weather condition of the sea. Is impossible. In particular, it is possible to grasp only the presence and motion information of the ship, and other information such as the sharpness of the ship can not be automatically confirmed, and it is difficult to be combined with other image display devices such as an electronic chart.

As an alternative to these various problems, the automatic ship identification system (AIS) has been introduced and the mutual automatic recognition between vessels or between vessels and land monitoring authorities has become possible, contributing to safe ship operations especially in areas with high maritime traffic volume . In addition, effective ship control using AIS, navigation safety information, radar propagation shaded area problems are solved and various effects can be obtained.

However, according to the law of ship facilities, the obligation to install is limited to ships of a certain size or more. Therefore, small vessels such as small fishing vessels and marine vessels other than vessels that are not obliged to install are subject to the existence of vessels or land- There is a fundamental limitation that is impossible to recognize.

In recent years, distance measurement using photographed images of cameras has been used. For this purpose, stereovision systems using two or more cameras as in the following patent documents are widely used.

However, in the case of the stereo vision method, the calibration operation for correcting the values between the cameras is complicated, and since a certain distance is required between the plurality of cameras in order to increase the accuracy and the measurable distance, the size of the measuring device becomes larger, The cost is high.

For this reason, for this reason, various monocular distance measuring methods using a single camera, for example, a distance measuring method using two markers having a known distance as in the following paper 1, and a plurality of photographs A distance measurement method has been proposed.

However, in the case of the above-mentioned distance measurement method, there are various restriction conditions in the measurement object type, the measurement distance, and the measurement environment, or the calculation process is complicated, and there is an inadequate limit to the real-time image processing and the distance measurement based thereon. It is desired to develop a distance measurement method and a distance measurement system of a marine object which can perform real time distance measurement based on the simple single distance measurement method using a single camera in the environment.

Meanwhile, from the viewpoint of the maritime risk management system, the conventional radar system, the AIS system, and the image capturing system using the cameras are installed separately as independent systems, and the efficiency and completeness of the maritime risk management are inferior . Accordingly, the development of an image capturing system using an efficient camera that can be efficiently integrated into the conventional radar system and the AIS system, which can complement the maritime risk management of the upper radar system and the AIS system, It is urgently required to develop an integrated maritime risk management system that integrates the image capturing system using the image capturing system.

Korean Registered Patent Publication No. 10-1045323 (Real-time Port Image Control System and Method Using Multi-Point Camera System)

PAPER 1: Development of a low-cost monocular PSD motion capture system using two active markers at regular intervals (Proceedings of the Second Annual Conference of the Institute of Electronics Engineers of Japan SC No.4, 2009.3, 61-71) Thesis 2: Stereoscopic Image Generation Using Monocular Camera (The Journal of the Korea Broadcasting Engineers 17 권 1 호)

Accordingly, the present invention relates to a maritime risk management system and a maritime risk management system including an image capturing system using an efficient camera that can be efficiently integrated into a radar system and an AIS system and can complement a maritime risk management of a radar system and an AIS system And a method thereof.

In addition, the marine risk management system and the maritime risk management method integrating the marine object distance measurement system using the radar system, the AIS system, and the camera, the image processing using the simple method, and the one using the monocular camera And an object of the present invention is to provide a maritime risk management system and a maritime risk management method using a photographing system.

In order to achieve the above object, the present invention provides an marine risk management system using a marine object distance measuring system using a monocular camera in one aspect, and is installed at a height from a predetermined sea level at an observation point, (110) for receiving and outputting an image photographed by the camera, including a camera having a vertical resolution, and a control unit for detecting a horizontal line by receiving an image photographed by the photographing unit, The distance between the horizontal line and the water surface boundary line of the object is calculated, and the distance from the observation point to the sea surface object is calculated from the installation height of the camera, the vertical angle per pixel, And an object image processing unit 120 for calculating and outputting the distance The distance information from the marine object image taken from the marine object distance measuring system 100 using the monocular camera and the distance object from the observation point to the marine object is received as camera image information, Receives the radar information including the position information of the marine object from the information processing unit 300, receives the AIS information from the AIS information processing unit 400, and receives and reads the camera image information, the radar information and the AIS information, And a maritime risk management system (200) for detecting the risk of the detected maritime object, processing and outputting image information associated with the maritime object, and the maritime risk management system (200) , When a marine object is detected as a result of reading radar information and AIS information, a marine object distance measurement system using a monocular camera A driving control signal for controlling the photographing position of the camera of the image processing apparatus 100 to a position for photographing the detected marine object is generated and the object image of the marine object photographed at the controlled position is processed and output.

(수식1)에 의하여 연산되어 설정되며, 여기서, A는 카메라의 수직 화각, H는 카메라의 수직 해상도이고, 수평선과 수면 경계선 사이의 각도(θ)는

(수식2)에 의하여 연산되며, 여기서 Y₀는 수평선의 픽셀의 Y 좌표값, Y_target은 경계선의 픽셀의 Y 좌표값이고, 관측지점에서 해상 객체까지의 거리(D)는

(수식3)에 의하여 연산되며, 여기서 T는 카메라의 설치 높이이다.Preferably, the per-pixel vertical angle &_lt; _{RTI ID} = 0.0 &_gt;

(1), where A is the vertical angle of view of the camera, H is the vertical resolution of the camera, and the angle (?) Between the horizontal line and the water surface boundary is

(Formula 2), where Y ₀ is the Y coordinate value of the pixel on the horizontal line, Y _target is the Y coordinate value of the pixel on the boundary line, and the distance (D) from the observation point to the marine object is

(Equation 3), where T is the installation height of the camera.

Here, the object image processing unit 120 may detect a horizontal line from a captured image by a line detection method using Hough Transformation and calculate a Y coordinate value (Y ₀ ) of the pixel on the horizontal line.

Here, the object image processing unit 120 detects an object, which is a target having texture information other than the stored texture information, by comparing the texture information of the sea surface acquired and stored in advance with the texture of the image to be photographed, The lower end of the object is detected as the boundary line between the object and the water surface, and the Y coordinate value (Y _target ) of the pixel at the boundary line between the object and the water surface is calculated.

The object image processing unit 120 may calculate a distance only to an object when the Y coordinate value (Y _target ) of the pixel at the boundary line between the object and the water surface is equal to or less than the Y coordinate value (Y ₀ ) . ≪ / RTI >

Preferably, the driving control signal is generated to synthesize an object image of a marine object photographed at a controlled position on a radar image.

Further, it is preferable to generate the driving control signal to synthesize the object image of the marine object photographed at the controlled position on the AIS image, and output the synthesized object image. Preferably, the driving control signal is generated, The object image of the marine object can be synthesized and output to the camera image.

Preferably, the camera image information, the radar information, and the AIS information are received and read to detect a marine object, a risk of the detected marine object is determined, and an object determined as a danger is detected through the radar information, And outputs the synthesized object image of the marine object photographed at the controlled position to the radar image and outputs the synthesized object to the radar image when the object judged as a danger is found through the AIS information, If the object determined as a danger is found through the camera image information and is determined to be a dangerous object, the control unit may generate the drive control signal to output the combined object image of the maritime object photographed at the controlled position to the AIS image, Generates a drive control signal, and synthesizes an object image of a marine object photographed at a controlled position on a camera image, and outputs the synthesized image.

Here, it is preferable that the camera photographed image is selectively switched to a radar image or an AIS image according to a user's selection input.

In addition, it is preferable that distance information calculated by the object image processing unit 120 at the time of outputting the object image of the captured marine object is displayed together and displayed, more preferably, the risk of the detected marine object is determined, If it is judged to be dangerous, a danger alarm is generated and output.

According to another aspect of the present invention, there is provided a marine risk management method using a marine object distance measurement system using a monocular camera. The marine object distance measurement method includes a marine object image shot using a monocular camera output from a marine object distance measurement system, A camera image information receiving / processing step (S10) of receiving distance information as camera image information; A radar information / AIS information receiving / processing step (S20) of receiving radar information including position information of a marine object from the radar information processing unit 300 and receiving the AIS information from the AIS information processing unit 400; An object information processing / analysis step (S30) for detecting the marine object by reading the received camera image information, the radar information and the AIS information, and extracting the information of the detected marine object; An object risk determination step (S40) of analyzing object information of the detected object and information of the observation point and determining whether the detected object is a dangerous object; When the detected object is determined as a dangerous object, a driving control signal for controlling the shooting position of the camera of the marine object distance measuring system 100 using the monocular camera to a position for photographing the detected marine object is generated, A camera control step (S50) of controlling the camera to capture an object image of a marine object at a step S50; An image information processing step (S60) of receiving an object image of a marine object photographed at the controlled position and synthesizing the object image of the marine object photographed into at least one of a radar image, an AIS image, and a captured marine object image, ; And a camera image / integrated image display step (S70) for displaying the synthesized and outputted image on a predetermined display device.

(수식1)에 의하여 연산되어 설정되며, 여기서, θ_p는 픽셀 당 수직각, A는 카메라의 수직 화각, H는 카메라의 수직 해상도이다.Preferably, the camera image information reception / processing step (S10) is installed at a height from a predetermined sea level at an observation point performed in a marine object distance measurement system using a monocular camera, A fixed constant / information input step (S110) of receiving a view angle and a vertical resolution and calculating and storing a vertical angle per pixel; A horizontal line detecting step (S120) of detecting a horizontal line from an image taken by the camera at an observation point; A water surface boundary detecting step (S130) of detecting an object, which is a water surface of the sea object, and a boundary between the water surface, from the image including the sea object, which is a target to be imaged by the camera; A step (140) of calculating a distance between a horizontal line and a surface boundary line of an object to calculate a distance between the horizontal line and a surface boundary line of the object; And an object distance calculating step (S150) of calculating the distance from the observation point to the sea object from the installation height (H) of the camera, the vertical angle (? _P ) per pixel, and the distance between the horizontal line and the water surface boundary line of the object , The vertical angle (? _P ) per pixel

(Formula 1), where? _P is a vertical angle per pixel, A is the vertical angle of view of the camera, and H is the vertical resolution of the camera.

Also, preferably, in the image information processing step S60, if the object determined as a danger is an object that is detected through the radar information and determined to be dangerous, the object image of the marine object photographed at the controlled position is synthesized with the radar image The object image of the marine object photographed at the controlled position is synthesized and output to the AIS image, and if the object judged as the danger is the camera image If the object is found through information and judged to be dangerous, the object image of the marine object photographed at the controlled position is synthesized and output to the camera image.

The method may further include an alarm generating step (S80) of generating and outputting a danger alarm if it is determined in the object danger determination step (S40).

As described above, according to the present invention, image processing is performed in a simple manner using a single-eye distance measurement method using one camera in a marine environment, real-time distance measurement is performed on the basis of the image processing, , The calculation process is simple, the time and load required for image processing and distance calculation are reduced, and the image capturing system using the characteristic camera which minimizes the error is efficiently integrated into the radar system and the AIS system The risk management system and the maritime risk management method can be provided.

In addition, the marine risk management system and the maritime risk management method integrating the marine object distance measurement system using the radar system, the AIS system, and the camera, the image processing using the simple method, and the one using the monocular camera The maritime risk management system and the maritime risk management method using the photographing system can be provided.

Accordingly, it is possible to provide a maritime risk management system and a maritime risk management method with high accuracy and high risk detection capability at low cost.

1 is a block diagram of a maritime risk management system using a marine object distance measurement system using a monocular camera of the present invention.
2 is a flowchart of a maritime risk management method using a marine object distance measurement system using a monocular camera of the present invention
FIG. 3A is a block diagram of a marine object distance measurement system using a monocular camera of the present invention. FIG.
3B is a block diagram of the photographing unit of the marine object distance measuring system using the monocular camera of the present invention.
3C is a flowchart of a method for measuring the distance of a marine object of a marine object distance measuring system using a monocular camera of the present invention.
3D is a view for explaining a method of measuring the distance of a marine object using a vertical angle per pixel of a camera according to the present invention.
3E is a photographing image used in a method of measuring the distance of a marine object using a vertical angle per pixel of the camera.
FIG. 3F is a detailed flowchart of a camera image information generation process of a method of measuring the distance of a marine object using a vertical angle per pixel of the camera of the present invention.
4 is a detailed flowchart of an embodiment of a process of synthesizing and outputting captured marine objects of the present invention.
FIG. 5 is a detailed flowchart of another embodiment of a process of composing and outputting a captured marine object of the present invention. FIG.
6 is a detailed flowchart of another embodiment of a process of composing and outputting captured marine objects of the present invention.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which the claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

1 is a block diagram of a maritime risk management system using a marine object distance measurement system using a monocular camera of the present invention.

1, the marine risk management system using the marine object distance measurement system using the monocular camera of the present invention includes a marine object distance measurement system 100 using a monocular camera, a marine risk management system 200, And an AIS information processing unit 400. The above components may be implemented by one or more signal processing and / or application specific integrated circuits, hardware, software, or a combination of both hardware and software (Not shown) for controlling hardware, software, or hardware and software. When implemented as a single configuration module of a specific system, a control unit (not shown) Device. ≪ / RTI >

The marine object distance measurement system 100 using a monocular camera includes a camera having a predetermined vertical angle of view and a vertical resolution, which are installed at a height from a predetermined sea level in the observation point and used for distance measurement, A distance between the horizontal line and a surface boundary line of the object, and a distance between the horizontal line and a surface boundary line of the object, And an object image processing unit for calculating and outputting the distance from the observation point to the marine object from the installation height of the camera, the vertical angle per pixel, and the distance between the horizontal line and the water surface boundary line of the object, Contains information on the distance from the marine object image to the marine object calculated through it. And outputs it as the camera image information.

(수식1)에 의하여 연산되어 설정된다. 여기서, A는 카메라의 수직 화각, H는 카메라의 수직 해상도이고, 수평선과 수면 경계선 사이의 각도(θ)는

(수식2)에 의하여 연산되며, 여기서 Y₀는 수평선의 픽셀의 Y 좌표값, Y_target은 경계선의 픽셀의 Y 좌표값이고, 관측지점에서 해상 객체까지의 거리(D)는

(수식3)에 의하여 연산되며, 여기서 T는 카메라의 설치 높이이다.In monocular camera system 100 maritime object distance measurement by the distance in the operation to the marine object from the observation point of the object image processing unit 120, the vertical angle per pixel (θ _p) is

(Equation 1). Here, A is the vertical angle of view of the camera, H is the vertical resolution of the camera, and the angle (?) Between the horizontal line and the water surface boundary is

(Formula 2), where Y ₀ is the Y coordinate value of the pixel on the horizontal line, Y _target is the Y coordinate value of the pixel on the boundary line, and the distance (D) from the observation point to the marine object is

(Equation 3), where T is the installation height of the camera.

On the other hand, the object image processing unit 120 detects a horizontal line from a captured image by a line detection method using Hough Transformation, calculates a Y coordinate value (Y ₀ ) of the pixel on the horizontal line, The object is detected by comparing the texture information of the stored sea level surface with the texture of the image to be captured, and the target object having texture information different from the stored texture information is detected. The lower end of the detected object is detected as the boundary between the object and the water surface, The object image processing unit 120 calculates the Y coordinate value (Y _target ) of the pixel at the boundary between the object and the water surface, (Y _target ) is less than or equal to the Y coordinate value (Y ₀ ) of the pixel of the horizontal line.

A more detailed description of the construction of the marine object distance measuring system 100 using the monocular camera and the process of generating and outputting the camera image information will be described later with reference to FIGS. 3A to 3F.

The maritime risk management system 200 receives the marine object image taken from the marine object distance measurement system 100 using the monocular camera and the distance information from the observation point to the marine object as camera image information and the radar information processing unit 300 Receives the AIS information from the AIS information processing unit 400, reads the received camera image information, the radar information and the AIS information to detect the marine object, and detects And a processing unit for processing the image information associated with the marine object and outputting the processed image information. When a marine object is detected as a result of reading the received camera image information, the radar information, and the AIS information, The position of the camera of the marine object distance measuring system 100 used is controlled to a position for photographing the detected marine object It is configured to output to generates a driving control signal, process the object image of the object photographed in the maritime a controlled location.

For this, an object information processing unit 210, an image information processing unit 220, an integrated image processing unit 230, an alarm information processing unit 240, a camera control unit 250 and a marine guidance unit 260, And an alarm speaker and an integrated display unit in an additional configuration. The alarm generating unit generates a predetermined alarm such as an acoustic alarm, an image alarm, or acoustic information and image information when a dangerous object is found in the alarm information processing unit, and outputs the alarm to the alarm speaker and / or the display unit. At this time, the alarm speaker and the display unit may utilize the existing radar system or AIS system installed in the ship as it is.

The object information processing unit 210 receives the distance information from the sea object image taken from the sea object distance measuring system 100 and the distance information from the observation point to the sea object as camera image information, Receives the AIS information from the AIS information processing unit 400, reads the received camera image information, the radar information and the AIS information to detect a resolution object, and extracts the information of the detected resolution object do.

The image information processing unit 220 receives the received camera image information, the radar information, and the AIS information, converts the received image information into the predetermined display method, displays the image information on a predetermined display device, and may be configured as an independent device. Or may be configured as an aggregate of an already installed radar system or an image information processing unit of the AIS system.

The integrated image processing unit 230 synthesizes and outputs an object image of a marine object photographed at the received controlled position to at least one of a radar image, an AIS image, and a captured marine object image, The image is displayed on the predetermined display device.

The alarm information processing unit 240 analyzes the object information of the detected object and the information of the observation point to determine whether or not the detected object is a dangerous object. If the detected object is determined to be a dangerous object, And controls the camera controller 250 to generate a drive control signal for controlling the shooting position of the camera of the object distance measuring system 100 to a position for shooting the detected marine object.

The camera control unit 250 generates a drive control signal for controlling the position for photographing the marine object under the control of the alarm information processing unit 240 and transmits the generated drive control signal to the marine object distance measurement system 100.

The camera 111 of the photographing unit 110, which will be described later, is normally controlled to rotate 360 degrees in a predetermined cycle. When the detection of a marine object or the above marine object is determined as a dangerous object, Rotation and tilting of the camera 111 are controlled by the drive control signal. Depending on the embodiment, the shooting magnification of the camera 111, that is, zoom-in and zoom-out can be controlled together.

The maritime guidance unit 260 stores general electronic charts and is usually displayed in combination with the AIS information as part of the AIS system, and is displayed in combination with at least one of the camera image information, the radar information, and the AIS information according to the embodiment.

The radar information processing unit 300 compares a radio signal reflected or redrawn from a marine object at a specific position with a reference signal to calculate position information of the marine object, generates radar display information including the position information, and outputs the radar display information as radar information All or part of the radar system.

The AIS information processing unit 400 receives the AIS signal from the marine object at the specific position, reads the AIS information, generates the AIS display information including the AIS information, and outputs the AIS information as the AIS information.

In the case of the present invention, the processes of the radar information processing unit 300 and the AIS information processing unit 400 and the configurations and information associated therewith directly apply the configuration and information processing processes related to information processing and information display of the existing radar system and AIS system Accordingly, the marine object distance measuring system 100 using the monocular camera, which is a feature of the present invention, is integrated.

The control unit may generate the drive control signal according to a preset state of the existing radar system or the AIS system and may be configured to synthesize an object image of a marine object photographed at a controlled position on a radar image, The driving control signal may be generated to generate an object image of a marine object photographed at a controlled position and output to the AIS image. The object image of the marine object photographed at the controlled position may be generated And may be set to be output to the camera photographed image.

In particular, when a marine object extracted from information received from a marine object distance measuring system using a radar system, an AIS system, and a monocular camera is judged as a dangerous object, an object image of a marine object photographed at a controlled position is synthesized do.

The object image and each radar image or the AIS image may be synthesized in an overlay manner or may be synthesized in the form of a picture of a PIP function (Picture in Picture) or PBP (Picture By Picture) May be displayed on a dedicated display device (not shown) or an AIS image-dedicated display device, or may be provided with a separate integrated display device. In this case, the distance information calculated by the object image processing unit 120 at the time of outputting the object image of the taken marine object is outputted together and displayed.

In addition, it receives and reads camera image information, radar information, and AIS information, detects a marine object, determines whether the detected marine object is dangerous, and determines whether the object determined as a danger is detected through the radar information, If the object determined as a danger is found through the AIS information and is judged as a dangerous object, then the driving control signal is generated, and if it is determined that the object is a dangerous object, the object image of the sea object photographed at the controlled position is combined with the radar image, And outputs the driving control signal when the object determined as a danger is detected through the camera image information and judged as a dangerous object, And the object image of the marine object photographed at the controlled position is synthesized with the camera image and output, The image to be combined with the object image may be selected based on which of the camera image information, the radar information, and the AIS information is detected.

In particular, when a dangerous object is detected from a camera shot image having a relatively low numerical accuracy, the camera shot image is selectively switched to a radar image or an AIS image according to a user's selection input, It is possible to reconfirm the information of the radar image or the AIS image and the marine object displayed thereon.

In another aspect of the present invention, a maritime risk management system using a marine object distance measurement system using the basic monocular camera as described above is operated to perform a characteristic maritime risk management method.

FIG. 2 is a flowchart of a maritime risk management method using a marine object distance measurement system using a monocular camera of the present invention. Referring to FIG. 2, the marine risk management method of the present invention comprises: A camera image information receiving / processing step (S10) of receiving distance object information from a computed observation point to a marine object as camera image information using a marine object image photographed using the position information of the marine object from the radar information processing unit 300, AIS information reception / processing step (S20) of receiving radar information including AIS information, AIS information, and AIS information from the AIS information processing unit 400, reading the received camera image information, radar information and AIS information, An object information processing / analyzing step (S30) of detecting an object and extracting information of the detected marine object, An object risk determination step S40 for determining whether the detected object is a dangerous object by analyzing the information of the observation point, the sea object distance measurement system 100 using a monocular camera when the detected object is determined as a dangerous object, A camera control step (S50) of generating a drive control signal for controlling the shooting position of the camera of the detected marine object to a position for shooting the marine object and controlling the camera to capture an object image of the marine object at the controlled position, An image information processing step (S60) of receiving an object image of a marine object photographed at a predetermined position and synthesizing the object image of the captured marine object into at least one of a radar image, an AIS image, and a captured marine object image; And a camera image / integrated image display step (S70) for displaying the synthesized and outputted image on a predetermined display device.

The camera image information reception / processing step S10, the radar information / AIS information reception / processing step S20, and the object information processing / analysis step S30 are performed by the object information processing unit 210 and the image information processing unit 220, The object information processing unit 210 receives distance information from a marine object image taken from the marine object distance measuring system 100 and a marine object from the observation point as camera image information, Receives the AIS information from the AIS information processing unit 400, reads the received camera image information, the radar information and the AIS information to detect a marine object, and detects the detected marine object Extract information of an object.

(수식1)에 의하여 연산되어 설정되며, 여기서, θ_p는 픽셀 당 수직각, A는 카메라의 수직 화각, H는 카메라의 수직 해상도이다. 이에 대한 보다 상세한 설명은 도 3a 내지 도 3f를 통해 후술한다.Meanwhile, the camera image information receiving / processing step S10 is installed at a height from a predetermined sea level in the observation point in the marine object distance measuring system 100 using the monocular camera, (S110) for calculating and storing a vertical angle per pixel by receiving the angle of view and the vertical resolution, a horizontal line detecting step (S120) for detecting a horizontal line from the image taken by the camera at the observation point, (S130) of detecting a boundary between the object and the water surface, which is the water surface of the sea object and the position of the sea object, from the image including the sea object, (140) for calculating the distance between the horizon and the object to be calculated, and the installation height (H) of the camera, the vertical (θ _p) and the object distance calculating step (S150) for calculating the distance to the object from the observation point from the sea surface to-boundary distance of the horizon and objects; performs including, the vertical angle (θ _p) per pixel is the

(Formula 1), where? _P is a vertical angle per pixel, A is the vertical angle of view of the camera, and H is the vertical resolution of the camera. A more detailed description thereof will be described later with reference to FIGS. 3A to 3F.

In addition, the image information processing unit 220 receives the received camera image information, the radar information, and the AIS information, converts the received image information into the predetermined display method, and processes the image information to be displayed on a predetermined display device.

The object risk determination step S40 is performed by the alarm information processing unit 240. The object information determination unit 240 analyzes object information of the detected object and information of the observation point to determine whether the detected object is a dangerous object, The camera control unit 250 controls the camera control unit 250 to generate a drive control signal for controlling the shooting position of the camera of the marine object distance measuring system 100 using the monocular camera to the position for photographing the detected marine object .

The camera control step S50 is performed by the camera control unit 250. The camera control unit 250 generates a drive control signal for controlling the position for photographing the marine object under the control of the alarm information processing unit 240 And transmits it to the marine object distance measurement system 100.

The image information processing step S60 is performed by the integrated image processing unit 230 and the object image of the marine object photographed at the controlled position where the integrated image processing unit 230 is received is divided into a radar image, The synthesized and outputted image is outputted to the display device of the pre-set display device, i.e., the integrated display device or the existing display device of the AIS system or the existing display of the radar system in the camera image / integrated image display step S70 The synthesized image is displayed on the device.

In the image information processing step S60, if an object determined as a danger is detected as a dangerous object, the object image of a marine object photographed at a controlled position is synthesized and output to a radar image, If the judged object is found through the AIS information and is judged to be dangerous, the object image of the marine object photographed at the controlled position is synthesized and output to the AIS image, and the object judged as the danger is detected through the camera image information If the object is judged to be dangerous, the object image of the marine object photographed at the controlled position is synthesized and output to the camera image, and based on the information of the camera image information, the radar information and the AIS information, So that an image to be combined with the object image is selected.

 The alarm generation step S80 may further include an alarm information generation step S80 for generating and outputting a danger alarm when it is determined in step S40 that the object danger is a danger. The alarm generating unit generates alarms according to a preset method such as an acoustic alarm, an image alarm, or acoustic information and image information, and outputs the alarm information to the alarm speaker and / or the display unit.

Hereinafter, the configuration of the marine object distance measuring system 100 using the monocular camera and the process of generating and outputting the camera image information will be described in more detail.

The marine object distance measurement system 100 using the monocular camera according to the present invention is implemented as a constituent module of the maritime risk management system of the present invention described above but is not limited thereto and is constituted by an independent apparatus or system, Can be used for real-time positioning of vessels, docking, collision avoidance, and construction of a ship surveillance / control system together with various ship control related devices such as radar equipment and AIS equipment, The elements may be implemented in one or more signal processing and / or application specific integrated circuits, hardware, software or a combination of hardware and software, and may be implemented in hardware, software, or a separate control (Not shown), and one of the specific systems When implemented as a configuration module, a control unit (not shown) may be implemented by a general-purpose or dedicated central processing unit of the system.

FIG. 3A is a block diagram of a marine object distance measuring system 100 using a monocular camera according to the present invention. As shown in FIG. 3A, the distance between marine objects is measured using a vertical angle per pixel of a single camera.

3A, a marine object distance measuring system 100 using a monocular camera according to the present invention includes a photographing unit 110 including one camera, A distance between the horizontal line and a surface boundary line of the object is calculated, and a distance between the horizontal line and the surface boundary line of the object is calculated, And an object image processing unit 120 for calculating and outputting the distance from the observation point to the marine object from the horizontal line, the vertical angle per pixel, and the distance between the water surface boundaries of the object.

The object image processing unit 120 includes a horizontal line detecting unit 121 for detecting a horizontal line from an image to be photographed and an image processing unit for detecting a sea surface object and an object's water surface boundary line detecting unit 122 And a distance calculator 123 for calculating a distance from the observation point to an object detected on the basis of the distance and the distance to the object detected by the distance calculator 123. The fixed constants and information used in the object image processor 120 are input through the input interface, And a fixed constant / information setting unit 124.

The fixed constant / information setting unit 124 may include a memory (not shown) for storing fixed constants and information, and may include a user interface for controlling the input interface and receiving the constant and information from the user .

The constants and information set and stored in the fixed constants / information setting unit 124 are provided in real time upon request of the distance arithmetic unit 123, the horizontal line detecting unit 121, the object's surface boundary detection unit 122,

And an output interface for transmitting the camera image information including the photographed image and the calculated distance information to the maritime risk management system 200. [

The output interface may be configured to output auditory and visual information as a display and a speaker separately from the output interface. In this case, when the detected marine object approaches a certain distance or less, And outputting the alarm as audible and visual information to a display and a speaker based on the calculated distance information. The image photographed by the photographing unit 110 may be combined with the distance information provided by the distance calculator 123 at the output interface to visually provide the image or camera image information with the distance information integrated through a separate display .

3B is a block diagram of a photographing unit of a marine object distance measuring system using a monocular camera of the present invention. The photographing unit 110 includes a single camera having a predetermined vertical angle of view and a vertical resolution, which are installed at a predetermined height from the sea level and are used for distance measurement. The photographing unit 110 receives an image photographed by the camera, (Not shown) composed of an interface, a horizontal line detecting unit 121 and an object's water surface boundary detecting unit 122, and transmits the captured image to a maritime risk management system 200, Referring to FIG. 3B, in the present embodiment, one monocular camera 111 capable of rotating 360 degrees and capable of vertically tilting, and an electric motor for rotating and vertically tilting the monocular camera 111 And the camera driving control signal from the above-described maritime risk management system 200, It consists of a drive control unit 113 for controlling the East 112. The

 In general, the camera 111 is controlled to rotate 360 degrees at regular intervals. When the detection of a marine object or a marine object is determined to be a dangerous object, rotation and tilting of the camera are performed at a position and angle at which the object can be photographed, Signal, and according to the embodiment, the shooting magnification of the camera 111, that is, the zoom-in and the zoom-out are controlled together.

The observation point is an installation position of the camera 111 of the marine object distance measurement system 100 using a monocular camera. As shown in this embodiment, a specific point on the ship is generally, but not exclusively, When the system is installed, the observation point can be specified as the observation point, where the installation position of the camera on the ground is.

In the present embodiment, it is installed at a specific point on the ship's line, and is installed at a height from a predetermined sea level, and is used for distance measurement. According to aspects of the present invention, only one camera is included, and the angle of view (FOV) of the camera is determined by the angle of the camera, that is, the viewing angle through which the camera can capture an image, The vertical viewing angle A (°), that is, the maximum viewing angle of the camera in the vertical direction and the vertical resolution H (pixel) of the camera, that is, the maximum resolution in the vertical direction of the image captured by the camera, do.

(수식1)에 의하여 연산되어 설정되며, 여기서, A는 카메라의 수직 화각, H는 카메라의 수직 해상도이다. As a difference in angles per pixel (θ _p ), ie, one pixel difference on the Y axis of the image taken by the camera

(Equation 1), where A is the vertical angle of view of the camera and H is the vertical resolution of the camera.

For example, in the camera of the HM-324XP + model of the FLIR company, the viewing angle is 24 ° (H) X 18 ° (V) and the camera resolution is 320 , And the camera vertical resolution H = 240. Therefore, θ _p = A / H = 0.075 °. That is, when the difference of the Y coordinate value of the pixel is 1 pixel in the image taken by the HM-324XP + camera, the vertical angle per pixel is calculated as 0.075 ° It is used as a constant.

The installation height of the camera 111 is measured by a user using a height value T of the camera from the sea level and inputted through an input interface or by measuring a change in altitude and altitude of the sea level and outputting the altitude in real time Not shown) and can be input in real time. The height value (T) of the camera from the sea surface is used as a fixed constant value in the distance measurement. If the measured value is automatically measured and inputted in real time, the real time variation of the height value can be reflected in the distance calculation in real time.

In addition, when the camera is installed, it is installed at a position where the horizontal line can be seen. At this time, the camera is kept horizontal and the mounting position is adjusted so that the Y coordinate of the horizontal line is constant.

The horizontal line detecting unit 121 and the object's water surface boundary detecting unit 122 of the image processing unit (not shown) receive the image photographed from the photographing unit and detect a horizontal line. The horizontal object detecting unit 121 detects a horizontal object, And may be incorporated in a control unit (not shown) for controlling each configuration.

Edge detection and / or line detection methods of the digital image processing method may be used for this purpose.

The horizontal line detection unit 121 detects a horizontal line by a line detection method using Hough Transformation from a captured image and calculates a Y coordinate value (Y ₀ ) of the pixel on the horizontal line and outputs the calculated image And transmits the Y coordinate value (Y ₀ ) of the pixel to the distance measuring unit 30 as the output thereof.

Wherein when the photographed image is a grayscale image, an intermediate value between a maximum value and a minimum value of the gray scale level value of the grayscale image is set as a threshold value, and a pixel having a level value equal to or higher than the threshold value of the photographed image Is set to a maximum value of a level value, a pixel having a level value lower than the threshold value is set as a minimum value of a level value, binarization is performed, and a line detection method is performed on the binarized image through the Hough transform, It is possible to reduce the time and load required for the system.

The threshold value for binarization is set to the middle value between the maximum value and the minimum value of the gray scale level value of the gray scale image. This is because the horizontal line indicates the boundary between the sky (light color) and the sea (dark color) In the case of the image, the level value close to the maximum value of the level value in the case of the sky (255 (white) in the case of the 256 level image) is set to the minimum value of the level value in the case of the sea A characteristic having a large difference between the level values is taken into consideration, and 128 is set as a threshold value for an image having 256 levels as an intermediate value.

Threshold values for binarization are preset and changed according to the shooting mode of the camera and the camera through the input interface. However, according to the embodiment, , The time zone, and the region, it is possible to implement the binarization by setting the threshold value.

In order to detect a water surface boundary line of an object, which is a water surface of a target sea object and a sea object, the object's water surface boundary detection unit 122 compares the texture information of the sea surface acquired and stored in advance with the texture of the captured image, Detects a target object having texture information and other texture information, detects a lower end of the detected object as a boundary between an object and a water surface, and calculates and outputs a Y coordinate value (Y _target ) of a pixel at a boundary line between the object and the water surface The Y coordinate value (Y _target ) of the pixel is transmitted to the distance calculator 123 by detecting the water boundary line of the target sea object from the captured image. In order to achieve this, a preliminary step is to acquire the texture information of the sea surface in advance and perform a process of receiving the input. It is inputted by the user through the input interface or automatically extracts the texture of the sea surface texture) information may be obtained in advance and stored automatically so as to be used for the detection of the boundary line.

In the present invention, considering the specificity of a maritime object, since the background, which is a background for object detection, has almost constant texture information or texture on the sea surface due to the special characteristic of maritime vessels, And the texture information extracted from the image are compared with each other to detect an object having a different texture from the sea surface and the texture information, and at the bottom of the object, A method of acquiring the Y coordinate value of the pixel is adopted. As an object recognition or detection method using such a texture, MB-LBP (Multi-block Local Binary Patterns) method is preferably used in this embodiment.

At this time, since the height of the horizontal line is known, the objects having the Y coordinate value of the pixel in the image larger than Y ₀ (the position higher than the horizontal line) have no meaning of the distance measurement. Therefore, it is possible to reduce an entire operation by detecting an object only within a range where a Y coordinate value of a pixel in an image is smaller than Y ₀ .

The distance calculator 123 calculates the distance between the horizontal line and the water surface boundary of the object and calculates the distance from the observation point to the sea object from the installation height of the camera, the vertical angle per pixel, and the distance between the horizontal line and the object's water surface boundary And can be incorporated in a control unit (not shown) for controlling each configuration.

The distance calculation between the horizontal plane and the object's water surface boundary is performed using the Y coordinate value (Y ₀ ) of the horizontal line pixel transmitted from the horizontal line detecting unit 121 and the object's water surface boundary detecting unit 122 and the Y And is calculated as a difference of the coordinate value (Y _target ).

(수식1)에 의하여 연산되며, 여기서, A는 카메라의 수직 화각, H는 카메라의 수직 해상도이고, 수평선과 수면 경계선 사이의 각도(θ)는

(수식2)에 의하여 연산되며, 여기서 Y₀는 수평선의 픽셀의 Y 좌표값, Y_target은 경계선의 픽셀의 Y 좌표값이고, 관측지점에서 해상 객체까지의 거리(D)는

(수식3)에 의하여 연산되며, 여기서 T는 카메라의 설치 높이이다.The distance (D) from the observation point to the sea object

(1), where A is the vertical angle of view of the camera, H is the vertical resolution of the camera, and the angle (?) Between the horizontal line and the water surface boundary is

(Formula 2), where Y ₀ is the Y coordinate value of the pixel on the horizontal line, Y _target is the Y coordinate value of the pixel on the boundary line, and the distance (D) from the observation point to the marine object is

(Equation 3), where T is the installation height of the camera.

According to an embodiment of the present invention, the distance measuring unit 30 may calculate the distance between the object and the surface of the water surface only when the Y coordinate value (Y _target ) of the pixel is equal to or less than the Y coordinate value (Y ₀ ) By performing the distance calculation, it is possible to reduce the total calculation and the detection error.

The distance calculator 123 calculates the distance of the marine object detected at the detection site and outputs the distance to the marine risk management system together with the image of the marine object along with the image of the marine object through the output interface.

According to an embodiment of the present invention, a marine object distance measuring system using a monocular camera may independently perform a risk detection and alarm function to monitor the distance of the detected marine object in real time, and when approaching a certain distance or less, (Not shown) and a speaker (not shown) as audible and visual information based on the calculated distance information.

3C to 3F, a method for measuring the distance of a marine object using a vertical angle per pixel of the camera according to the present invention and a specific embodiment thereof will be described. The distance measurement method of the marine object using the vertical angle of the camera according to the present invention is basically performed by the marine object distance measurement system using the monocular camera as described above, .

FIG. 3C is a flowchart illustrating a method of measuring a distance of a marine object of a marine object distance measuring system using a monocular camera according to the present invention. FIG. 3D is a view for explaining a method of measuring a distance of a marine object using a vertical angle per pixel of the camera of the present invention. And FIG. 3E is an image taken in a method of measuring the distance of a marine object using a vertical angle per pixel of the camera.

3C and 3D, the method of measuring the distance of a marine object using a vertical angle per pixel of a camera according to the present invention is installed at a height from a predetermined sea level at an observation point, A horizontal line detecting step S120 of detecting a horizontal line from the image taken by the camera at the observation point, a fixed constant / information inputting step S110 for calculating and storing a vertical angle per pixel by receiving the height, vertical angle of view, (S130) of detecting a boundary object between a sea object and its position and a water surface from an image including a sea object, which is a target captured by the camera, (S140) of calculating a distance between a horizontal line and an object's water surface boundary line that calculates the distance of the camera, a height of the camera, (S150) for calculating the distance from the observation point to the marine object from the distance between the vertical and horizontal lines and the water surface boundary of the object, wherein the camera includes a distance between the captured image and the calculated observation point to the marine object And outputs image information.

Referring to FIG. 3D, in order to perform a method of measuring the distance D of a marine object using a vertical angle per pixel of a camera, the installation height (T) of one camera from the sea surface, the vertical angle _p is used as a basic fixed constant and is set and inputted in the fixed constant / information input step S110.

A single image photographed by one camera is used and the photographed image photographed by the camera has a horizontal resolution H _max (pixel) X vertical resolution V _max (pixel) as shown in FIG. 3D, the pixels of the bottom right pixel in the coordinates (0, 0) is set to the coordinates (H .V _{max max).} In the photographed image, a horizontal line is a straight line constituted by a point (pixel) from a coordinate value (OY ₀ ) to (H _max .Y ₀ ) of a pixel, a boundary line is a line object and the perimeter of the surface of the water, the coordinate value of the pixel _(target OY) in the straight line configuration to the point (H _max .Y _target), the coordinate values of the pixels of a horizontal line (Y ₀₎ and the Y coordinate value of the pixel of the boundary ( Y _target are detected in the horizontal line detecting step S120 and the boundary detecting step S130, respectively, and are basically used as variables in the calculation of the distance of the marine object.

In the fixed constant / information input step (S110), the installation height, the vertical angle of view, and the vertical resolution of the camera used for the distance measurement are inputted through the input interface, and the vertical angle per pixel is calculated and stored / set.

As mentioned earlier, the installation height (T) of the camera is automatically measured using a height elevation meter (not shown), which is pre-measured and input through the input interface or is output in real time by measuring the altitude and altitude change And is used as a fixed constant value in the distance measurement. If the automatic measurement is input in real time, the real time variation of the height value can be reflected in the distance calculation in real time.

(수식1)에 의하여 연산되어 설정되며, 여기서, θ_p는 픽셀 당 수직각, A는 카메라의 수직 화각, H는 카메라의 수직 해상도로서, 상술한 바와 같이, 카메라에 따라 사전 설정된 값을 가지며, 입력 인터페이스를 통하여 사용자에 의하여 입력되어 설정/저장된다.The vertical angle? _P per pixel

(Equation 1) where? _P is a vertical angle per pixel, A is a vertical angle of view of the camera, and H is a vertical resolution of the camera, and has a predetermined value according to the camera, as described above, Is input / set / saved by the user through the input interface.

In the fixed constant / information input step (S110), texture information of the sea surface acquired in advance is received and stored. The texture boundary information is stored in the water surface boundary detection step (S30) The object is detected by comparing the texture of the object with the texture information other than the texture information stored in advance and the lower end of the object is detected as the boundary between the object and the water surface to detect the Y coordinate value of the pixel at the boundary between the object and the water surface Is stored as preset information for calculating Y _target .

In addition, according to the embodiment, the previously acquired sea level and the color related information of the sky, for example, the gray scale level, and the texture information of the previously obtained sea level are obtained from the observation point by the horizontal line May be stored as preset information for detecting a horizontal line in the detecting step S120 or as preset information for detecting a boundary line in the boundary detecting step 130. [

The horizontal line detection step S120 detects a horizontal line from the captured image and computes a Y coordinate value (Y ₀ ) of the pixel of the horizontal line, and in this embodiment, it is performed by a line detection method using Hough Transformation . In the horizontal line detection, Hough Transformation is a process of binarizing a photographed image through an appropriate threshold for easy image processing and detecting an edge having a sharp change in color between pixels in a binarized image .

In the case of the present embodiment, the binarization process may be performed by setting an intermediate value between a maximum value and a minimum value of a gray scale level value of the gray scale image as a threshold value when the photographed image is a gray scale image, A pixel having a level value equal to or greater than the threshold value is set as a maximum value of a level value and a pixel having a level value less than the threshold value is set as a minimum value of a level value to binarize the binarized image, It is possible to reduce the time and load required for the image processing. The threshold value for binarization is set to the middle value between the maximum value and the minimum value of the gray scale level value of the gray scale image. This is because the horizontal line indicates the boundary between the sky (light color) and the sea (dark color) In the case of the image, the level value close to the maximum value of the level value in the case of the sky (255 (white) in the case of the 256 level image) is set to the minimum value of the level value in the case of the sea The intermediate value 128 is set as a threshold value for a 256 level image and a pixel having a level value equal to or higher than the intermediate value 128 is set to 255 ), And pixels having a level value lower than the intermediate value are processed as 0 (black).

Threshold values for binarization are preset and changed according to the shooting mode of the camera and the camera through the input interface. However, according to the embodiment, , The time zone, and the region, it is possible to implement the binarization by setting the threshold value.

Next, a binarized image is subjected to a process of detecting an edge using a mask such as a canny edge detector to detect a candidate pixel (a pixel) constituting a horizontal line, The Hough transform for the points detects a straight line containing the most points and when the straight line is near to the horizontal through the equation of the straight line, the straight line is detected as a horizontal line and the Y coordinate value Y ₀ ).

The water surface boundary detection step (S130) is performed by detecting a sea object from an image including a sea object, which is a target to be imaged by the camera, and then detecting a boundary between an object and a water surface, A method of detecting an object as a target having texture information different from the previously stored texture information by comparing the texture information with a texture of the image to be photographed using texture information of a previously obtained sea level, (Y _target ) of the pixel at the boundary between the object and the water surface by detecting the boundary between the object and the water surface.

Accordingly, the feature that the object to be detected is the background for the object detection due to the special characteristic of the maritime vessel has almost constant texture information or texture to the sea surface, and the detection of the water surface boundary of the sea object and the object The time and load required for the calculation can be reduced. In the present embodiment, MB-LBP (Multi-block Local Binary Patterns) is used as a method of object recognition or detection using a texture.

Further, in order to reduce the time and load required for detection and calculation of the surface boundaries of the marine objects, and the object, the surface of the water boundary detecting step the Y coordinate value of the Y coordinate value of the pixel of the image that is taken of the pixels in the horizontal (Y ₀ ) ≪ / RTI > In the case where the object to be detected is the object after the horizontal line, the object at the position higher than the horizontal line, that is, the object larger than the Y coordinate value of the pixel of the horizontal line, This is because measurement is meaningless.

In the horizontal line detecting step S120 and the water surface boundary detecting step S30, the distance between the horizontal line and the object's horizontal line is calculated as Y When the coordinate value (Y ₀ ) and the Y coordinate value (Y _target ) of the pixel at the boundary line between the object and the water surface are calculated, the difference is calculated as the distance between the horizontal line and the object boundary line.

(수식2)에 의하여 수평선과 수면 경계선 사이의 각도(θ)를 고정 상수로 저장된 픽셀 당 수직각(θ_p)과 검출된 Y₀는 수평선의 픽셀의 Y 좌표값(Y₀) 및 경계선의 픽셀의 Y 좌표값(Y_target)을 이용하여 연산하고, 이어

(수식3)에 의하여 고정 설정된 카메라의 설치 높이(T)와 연산된 수평선과 수면 경계선 사이의 각도(θ)를 이용하여 최종 관측지점에서 목표물인 해상 객체까지의 거리(D)를 연산한다.In the object distance calculation step S150, as shown in FIG. 3D,

The angle θ between the horizontal line and the water surface boundary by the formula (2) is defined as a constant angle (θ _p ) per pixel stored as a fixed constant and the detected Y ₀ is the Y coordinate value (Y ₀ ) of the pixel of the horizontal line and the pixel The Y coordinate value (Y _target )

(D) from the final observation point to the target sea object using the installation height (T) of the camera fixed by the equation (3) and the angle (?) Between the computed horizontal line and the water surface boundary.

FIG. 3E is an example of a photographing image used in a method of measuring the distance of a marine object using a vertical angle per pixel of a camera. The horizontal resolution 320 (pixel) x the vertical resolution 240 (pixel) taken by a thermal imaging camera It is a grayscale image. The pixel at the upper left corner is set to the coordinate value (0.0), and the pixel at the lower right corner is set to the coordinate value (320.240). Particularly, the present invention is applied to night imaging technology such as thermal imaging cameras to detect a marine object even when the ship is at night or after a bad weather, and in particular, can not be detected by radar or AIS The present invention detects obstacles such as a floating object, a reef, a bridge bridge, a neck line, and the like that may cause damage to the ship and calculates the distance in real time, and displays the distance from the imaged image or object image, It is possible to construct a maritime risk management system integrated with the radar system and the AIS system by determining the risk of the marine object.

FIG. 3F is a detailed flowchart of a method of generating camera image information of a method for measuring the distance of a marine object using a vertical angle of a camera according to the present invention. In the method of measuring distance of a marine object of the present invention, FIG. 5 is a flowchart illustrating a case where the present invention is performed in conjunction with a maritime risk management system using a measurement system.

Referring to FIG. 3F, the fixed constant / information setting unit 140 performs a fixed constant / information input process 310 to receive all the fixed constants / information necessary for performing the distance measuring method. When the setting of the fixed constant is completed (320), the photographing unit 110 is controlled (321) according to the drive control signal of the maritime risk management system 200 to start the image photographing. In the drive control process 321, the normal camera 111 is controlled in a 360-degree rotation state at regular intervals. When the detection of a marine object or the above-mentioned marine object is determined as a dangerous object, Rotation and tilting of the camera 111 are controlled by the drive control signal, and according to the embodiment, the shooting magnification of the camera 111, that is, zoom-in and zoom-out are controlled together.

The captured image is transmitted (336) to the maritime risk management system (200) in real time before the image processing, and is processed by the horizontal line detecting unit (121) of the image processing unit and the water surface boundary detecting unit (122) of the object. The horizontal line detecting unit 121 detects a horizontal line from an image to be photographed. When a horizontal line is detected, a horizontal line detecting unit 121 detects a horizontal line and a Y coordinate value Y ₀ ) and outputs 332. If a horizontal line is not detected, it is determined that the image is erroneous, and the process returns to the image capturing process 320 to receive a newly captured image.

Here, in order to detect the water surface boundary of the object and the object, the texture information acquired and stored in advance by comparing the texture information of the sea surface acquired and stored in advance with the texture of the captured image is called from the fixed constants / information setting unit 140 and used , The water surface boundary detection unit 122 attempts to detect a target sea object from the captured image (333), and when the target sea object is detected, the sea boundary detection unit (122) transmits the object image to the marine risk management system () (334) a Y coordinate value (Y _target ) of a pixel of the detected boundary line is detected and output (335). The water surface boundary detection process is performed so that the calculation amount and the load required for the water surface boundary detection are reduced by controlling the Y coordinate value of the pixel of the image to be photographed to be performed only for a portion that is equal to or less than the Y coordinate value Y ₀ of the pixel of the horizontal line .

Then, it is determined whether or not the Y coordinate value (Y _target ) of the pixel at the boundary line between the object and the water surface is less than or equal to the Y coordinate value (Y ₀ ) of the pixel of the horizontal line so as to reduce the calculation amount, 340) In step 350, the distance between the horizontal line and the boundary line is calculated only for the object when the Y coordinate value (Y _target ) of the pixel of the water surface boundary of the object is equal to or less than the Y coordinate value (Y ₀ ) of the pixel of the horizontal line (360) between the observation point and the object. When the Y coordinate value (Y _target ) of the pixel of the water boundary line is larger than the Y coordinate value (Y ₀ ) of the pixel of the horizontal line, it is judged as an error and the distance calculation process is terminated.

If a marine object is not detected or a boundary line is not detected, it is determined that there is no false image or marine object, and the process returns to the image capturing process 320 to receive a newly captured image.

(수식2)에 의하여 수평선과 수면 경계선 사이의 각도(θ)를 연산하고,

(수식3)에 의하여 관측지점에서 해상 객체까지의 거리(D)를 연산하여 객체거리 정보를 포함하는 카메라 영상정보를 해상 위험관리 시스템(200)으로 전송한다(370). 여기서 T는 카메라의 설치 높이로 고정상수/정보 설정부(124)로부터 호출된다.
Distance calculating section 123, Y-coordinate of the surface boundary of an object pixel (Y _target) and by receiving the Y coordinate value (Y ₀₎ of the pixel of the horizontal line between these primary ┃Y _{0 -} Y a _target at a distance ┃ (550), and then

([Theta]) between the horizontal line and the water surface boundary line by the equation (2)

(Equation 3) to calculate the distance D from the observation point to the marine object, and transmits the camera image information including the object distance information to the marine risk management system 200 (370). Here, T is called from the fixed constant / information setting unit 124 as the installation height of the camera.

4 to 5, a marine object distance measurement system and a marine object distance measurement system using a monocular camera based on the camera image information generated in the camera image information generation process using the marine object distance measurement system using the monocular camera, The functions of the integrated image processing unit 230 or the camera control unit 250 described above for integrating with the existing system through interlocking with the risk management system, the camera control step S50, the image information processing step S60, The description will be made within a range not overlapping with the description related to the integrated video display step S70.

FIG. 4 is a detailed flowchart of an embodiment of a process of composing and outputting captured maritime objects of the present invention, wherein a marine object distance measurement system using a monocular camera can be combined with an existing radar system.

Referring to FIG. 4, in the radar system, a process 410 of receiving radar information and displaying it on a display device is performed, and analysis 420 of radar information is performed to analyze radar information to determine whether a marine object exists or not (430). If it is determined that there is a marine object, it is determined whether or not the marine object is a dangerous object by analyzing radar information such as a traveling direction speed of the object. If it is determined that the object is a dangerous object, an alarm signal of sound or image is generated and output (480), and the driving control signal is transmitted to the marine object distance measuring system using the monocular camera to control the camera (450) (470) receiving the object image of the maritime object, synthesizing the object image of the marine object photographed at the controlled position on the radar image, and outputting it to the display device. The distance information calculated by the object image processing unit 120 may be displayed together with the radar image of the object image of the captured marine object.

According to the existing radar system configuration, all of the series of processes can be automatically processed by the system, and the radar information analysis process 420, the radar information analysis process 430, the risk object determination process 440, All or a part of the camera control process 450 may be performed in the process of analyzing and determining the radar information by the user and receiving the result and processing in the radar system.

According to an embodiment, in order to prepare for reconfirming the possibility that the accuracy of the radar information is low, a user's selection may be input, and the displayed image may be displayed as a camera photographed image or converted into an AIS image and output.

FIG. 5 is a detailed flow chart of another embodiment of the process of composing and outputting the captured marine objects of the present invention, wherein the marine object distance measurement system using a monocular camera can be combined with an existing AIS system.

Referring to FIG. 5, in the AIS system, the AIS information is received and displayed on the display device (510) is performed, and the analysis of the AIS information (520) is performed to analyze the radar information, (530). If it is determined that there is a marine object, the AIS information such as the position direction speed of the object is analyzed to determine whether or not the object is a dangerous object. If it is determined that the object is a dangerous object, an alarm signal of sound or image is generated and output (580), and the camera is controlled by transmitting the driving control signal to the marine object distance measurement system using the monocular camera (550) (570) receiving the object image of the maritime object and outputting the object image of the marine object photographed at the controlled position to the AIS image and outputting it to the display device (570). According to the configuration of the existing AIS system, all of the series of processes can be automatically processed by the system, and the analysis process 520 of the radar information, the analysis process 530 of the AIS information, the risk object determination process 540, All or a part of the camera control process 550 may be performed in the process of analyzing and determining AIS information by the user and receiving the result and processing in the AIS system. The distance information calculated by the object image processing unit 120 may be displayed together with the AIS image of the object image of the captured marine object.

According to an embodiment, in order to prepare for reconfirming the possibility that the accuracy of the AIS information is low, it may be implemented that the user's selection is input, the displayed image is converted into a camera image, or converted into a radar image and output.

 FIG. 6 is a detailed flowchart of another embodiment of the process of composing and outputting the captured marine objects of the present invention, in which a marine object distance measurement system using a monocular camera can be combined with an existing radar system and an AIS system to be.

6, in a system in which a marine object distance measurement system using a monocular camera, a radar system, and an AIS system are integrated, camera image information, AIS information, and radar information are received and processed (610) If it is used, the user's selection is inputted (610), and the process of displaying the selected image information on the display device is performed (620). The camera image information, the AIS information, and the radar information are analyzed to determine whether a marine object exists An object information processing / analysis process 630 is performed to determine whether a marine object exists. If it is determined that a marine object exists, the information of the object is analyzed to determine whether or not the object is a dangerous object. If it is determined as a dangerous object, an alarm signal of sound or image is generated and output (680). If the object determined as a danger is found through the radar information and determined to be dangerous (650), the image displayed on the display device (651), controls the camera by transmitting the driving control signal to the marine object distance measuring system using the monocular camera (652), receives the object image of the marine object photographed at the controlled position, The object image of the marine object photographed at the controlled position is synthesized with the radar image and output to the display device (654).

On the other hand, if the object judged as a danger is found through the radar information and is not an object judged to be dangerous but is found through the AIS information and determined to be dangerous (660), then the image displayed on the display device is converted / displayed as an AIS image (661), transmits the driving control signal to the marine object distance measuring system using the monocular camera to control the camera (662), receives the object image of the marine object photographed at the controlled position, The object image of the marine object is synthesized with the AIS image and output to the display device (664).

On the other hand, if the object judged as a danger is found through the radar information or the AIS information and is not determined as a dangerous object but is detected through the camera image information and determined to be dangerous (670), the image displayed on the display device (671), transmits the driving control signal to the marine object distance measuring system using the monocular camera to control the camera (672), and displays the object image of the marine object photographed at the controlled position And synthesizes the object image of the marine object photographed at the controlled position on the camera photographed image to be displayed on the display device (674). Here, if a dangerous object is detected from the camera image, the user's selection is inputted in consideration of the possibility that the accuracy is relatively numerically low (675), the camera image is switched to the radar image (676) (677), and the information of the radar image or the AIS image and the marine object displayed thereon can be reconfirmed.

As has been described above, specific combinations of configurations or steps of the present invention have been described in detail by way of the above embodiments, but other combinations of such configurations and steps are likewise possible. The present invention is not limited to the specific embodiments disclosed herein, and such other combinations are within the scope of the present invention. Therefore, the present invention is not limited to the above-described embodiments, and various modifications and changes may be made thereto by those skilled in the art to which the present invention belongs.

100: Maritime object distance measurement system using monocular camera
110: photographing unit 120: object image processing unit
200: Maritime risk management system

Claims (15)

A maritime risk management system using a marine object distance measurement system using a monocular camera,
A photographing unit 110 installed at the observation point at a height from a predetermined sea level to receive and output an image photographed by the camera including one camera having a constant vertical angle of view and a vertical resolution used for distance measurement, A distance between the horizontal line and a surface boundary line of the object is calculated, and a distance between the horizontal line and the surface boundary line of the object is calculated, And a object image processing unit (120) for calculating and outputting a distance from the observation point to the sea object from a vertical angle per pixel, a horizontal line and a distance between a water surface boundary line of the object and a sea object,
The distance information from the marine object image taken from the marine object distance measuring system 100 using the monocular camera and the distance information from the observation point to the marine object is received as camera image information and the position information of the marine object is received from the radar information processing unit 300 Receives the AIS information from the AIS information processing unit 400, receives and reads the camera image information, the radar information, and the AIS information to detect the sea object, and determines whether or not the detected sea object is dangerous And a maritime risk management system (200) for processing and outputting image information associated with a maritime object,
When the maritime object management system 200 detects a maritime object as a result of reading the received camera image information, radar information, and AIS information, the maritime risk management system 200 detects the photographing position of the camera of the maritime object distance measurement system 100 using the monocular camera, A driving control signal for controlling a position for photographing a marine object is generated, the object image of the marine object photographed at the controlled position is processed and output,
The vertical angle? _P per pixel

(1), where A is the vertical angle of view of the camera, H is the vertical resolution of the camera, and the angle (?) Between the horizontal line and the water surface boundary is

(Formula 2), where Y ₀ is the Y coordinate value of the pixel on the horizontal line, Y _target is the Y coordinate value of the pixel on the water surface boundary, and the distance (D) from the observation point to the marine object is

(Formula 3), where T is the installation height of the camera. The marine risk management system using the marine object distance measurement system using the monocular camera. delete The method according to claim 1,
Wherein the object image processing unit 120 detects a horizontal line from a photographed image by a line detection method through Hough Transformation and calculates a Y coordinate value Y ₀ of a pixel on the horizontal line. Maritime risk management system using marine object distance measurement system using. The method according to claim 1,
The object image processing unit 120 detects an object which is a target having texture information different from the stored texture information by comparing the texture information of the sea surface acquired and stored in advance with the texture of the image to be photographed, And a Y coordinate value (Y _target ) of a pixel of a water surface boundary line of the object is detected by detecting the water surface boundary line of the object. The method according to claim 1,
The object image processing unit 120 performs a distance operation only on an object when the Y coordinate value (Y _target ) of the pixel at the water surface boundary of the object is equal to or less than the Y coordinate value (Y ₀ ) of the pixel of the horizontal line Maritime risk management system using marine object distance measurement system using monocular camera. The method according to claim 1,
And generating the driving control signal to synthesize the object image of the marine object photographed at the controlled position on the radar image and output the synthesized object image to the radar image to output the combined marine object distance image to the marine object distance measuring system using the monocular camera. The method according to claim 1,
Wherein the control unit generates the driving control signal to synthesize the object image of the marine object photographed at the controlled position on the AIS image and outputs the synthesized object image to the AIS image to output the marine object distance measurement system using the marine object distance measurement system using the monocular camera. The method according to claim 1,
Wherein the control unit generates the driving control signal to synthesize an object image of a marine object photographed at a controlled position on a camera image and outputs the synthesized image to a camera image to output a marine object distance measurement system using the marine object distance measurement system. The method according to claim 1,
If the object determined as a danger is detected through the radar information and is judged as a dangerous object, it is determined whether or not the detected object is dangerous, If the object determined as a danger is found through the AIS information and is judged to be a dangerous object, the driving control signal is output to the radar image, And outputs the synthesized object image to the AIS image. If the object determined as a danger is found through the camera image information and is determined as a dangerous object, the drive control signal is generated And the object image of the marine object photographed at the controlled position is synthesized with the camera image and outputted. Maritime risk management system using marine object distance measurement system using camera. 9. The method of claim 8,
Wherein the image picked up by the camera is selectively switched to a radar image or an AIS image according to a user's selection input, and the converted image is output. 11. The method according to any one of claims 6 to 10,
And displays the distance information calculated by the object image processing unit (120) together when the object image of the captured marine object is displayed, and displays the displayed distance information. 11. The method according to any one of claims 6 to 10,
And a risk alarm is generated when the risk is determined to be a danger, and the risk alarm is generated and output, and the maritime risk management system using the marine object distance measurement system using the monocular camera. A marine risk management method using a marine object distance measurement system using a monocular camera,
A camera image information receiving / processing step (S10) of receiving a marine object image photographed using a monocular camera output from a marine object distance measuring system and distance information from a computed observation point to a marine object as camera image information;
A radar information / AIS information receiving / processing step (S20) of receiving radar information including position information of a marine object from the radar information processing unit 300 and receiving the AIS information from the AIS information processing unit 400;
An object information processing / analyzing step (S30) of detecting the marine object by reading the received camera image information, the radar information and the AIS information, and extracting the information of the marine object detected;
An object risk determination step (S40) of analyzing object information of the detected object and information of the observation point and determining whether the detected object is a dangerous object;
When the detected object is determined as a dangerous object, a driving control signal for controlling the shooting position of the camera of the marine object distance measuring system 100 using the monocular camera to a position for photographing the detected marine object is generated, A camera control step (S50) of controlling the camera to capture an object image of a marine object at a step S50;
An image information processing step (S60) of receiving an object image of a marine object photographed at the controlled position and synthesizing the object image of the marine object photographed into at least one of a radar image, an AIS image, and a captured marine object image, ; And
A camera image / integrated image display step (S70) of displaying the synthesized and outputted image on a predetermined display device; Lt; / RTI >
The camera image information reception / processing step S10
In a marine object distance measurement system using a monocular camera,
A fixed constant / information input step (S110) which is installed at the observation point at a height from a predetermined sea level and computes a vertical angle per pixel by receiving the installation height, vertical angle of view and vertical resolution of the camera used for distance measurement;
A horizontal line detecting step (S120) of detecting a horizontal line from an image taken by the camera at an observation point;
A water surface boundary detection step (S130) of detecting an object of water and a water surface boundary line of an object which is a water surface of the sea object from an image including a sea object which is a target to be imaged by the camera;
A step (140) of calculating a distance between a horizontal line and a surface boundary line of an object to calculate a distance between the horizontal line and a surface boundary line of the object; And
An object distance calculation step (S150) of calculating the distance from the observation point to the sea object from the installation height (H) of the camera, the vertical angle (? _P ) per pixel, and the distance between the horizontal line and the water surface boundary line of the object , And the vertical angle? _P per pixel

(Formula 1), where θ _p is a vertical angle per pixel, A is a vertical angle of view of the camera, and H is a vertical resolution of the camera. Maritime risk management method. delete 14. The method of claim 13,
In the video information processing step S60,
If an object judged as a danger is detected through the radar information and is judged as a dangerous object, the object image of the marine object photographed at the controlled position is synthesized and output to the radar image, and the object judged as the danger is detected through the AIS information If the object determined as a danger is an object determined to be dangerous and detected as a dangerous object, the object image of the marine object photographed at the controlled position is synthesized and output to the AIS image. Wherein the object image of the marine object photographed at the position is synthesized with the camera image and outputted. The method for managing the marine risk using the marine object distance measurement system using the monocular camera.

Images