KR20200050808A - System for predicting degree of collision risk and guiding safe voyage route through fusing navigation sensor inside ship and image information - Google Patents

Abstract

According to the present invention, a system for predicting a collision risk degree and guiding a safe voyage route by fusing an onboard navigation sensor and image information comprises: an automatic ship identification unit cooperating with a radar to transmit/receive voyage information including ship relation information of a surrounding sea area and sail relation information; a camera to provide image information of a surrounding sea area of a main ship; and a control unit for calculating a collision risk degree with the main ship to generate warning information according to the collision risk degree and to guide a safe voyage route through an electronic chart device by matching the voyage information provided from the automatic ship identification unit with the image information provided from the camera to analyze locations, lengths, azimuths, and sail speeds of other ships of the surrounding sea area of the main ship with respect to the main ship while tracking the other ships. Therefore, the present invention can quickly respond by providing warning information according to the collision risk degree and guide a safe voyage route capable of bypassing collision.

Description

Collision risk prediction and safety operation guide system through the fusion of the navigation sensor and image information.

The present invention relates to a safety operation related technology of a ship, and more specifically, by using camera image information, identifying information about an object in an area around the main ship, calculating a collision risk and guiding a safe operation route, It relates to the collision risk prediction and safety operation guide system through the fusion of the navigation sensor and image information.

Frequent marine accidents due to collisions between ships or collisions with maritime topographic features, which depend on the ship's own radar or the Automatic Identification System (AIS) to detect objects in the surrounding waters and determine their collision risk. It is often caused by the crew's subjective judgment.

In particular, it is impossible to accurately identify the type and image of an object within 200m of a small ship or a main ship in an area around the main ship by a radar or an automatic ship identification system.

In addition, even if an object is detected on the radar, it is displayed as a single point on the screen, and the type and size of the detected object cannot be accurately identified, leading to a collision due to inaccurate information.

On the other hand, a system is provided to provide an alarm when an object in the surrounding waters approaches the main ship, but it does not provide information on a safe route that can be bypassed by avoiding or avoiding the collision risk with the object.

For example, the direction, distance, and closest point of approach (CPA) of other ships approaching the main ship with radar plotting or apar radar (ARPA; Automatic Radar Plotting Aid) And by detecting the time of closest point of approach (TCA), warning information is provided as an alarm when approaching within a certain limit.

However, the sailor must grasp the risk of collision directly and decide the bypass route with reference to the radar. In bad weather, fog or at night, human error is inherent, which may lead to a collision due to wrong judgment. Can be.

Accordingly, there is a limitation in preventing collision accidents with the main ship at sea only by simple object detection by the radar and the automatic ship identification system.

Korean Registered Patent Publication No. 1048508 (Real-time port video control system and method using smart device. 2011. 07. 05) Korean Registered Patent Publication No. 1072397 (Augmented reality system for ships using a monitor and its implementation method. 2011. 10. 05)

The technical problem to be achieved by the spirit of the present invention is to accurately identify the type and image of a small ship in an area around the main ship and objects approaching the main ship, and calculate the collision risk by analyzing the orientation and distance of the object from the main ship. The purpose of the present invention is to provide a collision risk prediction and safety operation guide system through fusion of onboard navigation sensors and image information that can prevent collision accidents.

In order to achieve the above object, the present invention, in conjunction with a radar, a ship automatic identification unit for transmitting and receiving operation information including ship-related information and navigation-related information in the surrounding waters; A camera that provides video information of the surrounding waters of the main line; And by matching the operation information provided by the vessel automatic identification unit and the video information provided from the camera, while tracking the other vessels in the surrounding waters of the main ship, analyze the position, distance, bearing and constant speed of the other ship, A collision risk prediction and safety operation guide through fusion of onboard navigation sensor and image information, including a control unit that calculates the collision risk of the vehicle and generates warning information according to the collision risk, and guides a safe navigation route through an electronic navigation device. Provide a system.

Here, the control unit extracts ship information from the image information, tracks the other ship, and analyzes the distance and the operating direction and the direction and direction of the other ship, and compares the position of the main ship and the operating direction and the constant speed, and compares the specified time. The collision risk is calculated by determining the possibility of collision within a specific area, and ship information and collision are possible according to the collision risk, based on the chart information provided from the electronic chart device or the video information provided from the camera. Information can be provided.

In addition, the camera is an optical or thermal camera, and one or a plurality of cameras may be installed to monitor the entire orientation of the main ship.

In one embodiment, the camera is composed of a combination of single cameras, the control unit, the relative position information of the vessel and other ships analyzed from the image information of the single cameras from the radar or the ship automatic identification unit The collision risk can be calculated by matching the ship's flight information.

In one embodiment, the camera is composed of a combination of stereo cameras, the control unit, the radar or the ship automatic information of the distance between the main vessel and other vessels analyzed through the position difference by the stereo image information of the stereo cameras The collision risk may be calculated by matching the operation information of the other ship from the identification unit.

In one embodiment, the camera is composed of a panoramic camera, the control unit, the distance information between the main vessel and the other vessel analyzed through the position difference by the panoramic image information from the radar or other vessel from the ship automatic identification unit By matching with the flight information of, it is possible to calculate the collision risk.

In one embodiment, the control unit may calculate the collision risk by matching the distance information between the main vessel and the other vessel measured by the laser meter with the operation information of the other vessel from the radar or the vessel automatic identification unit. have.

In one embodiment, the control unit matches the distance information between the main ship and the other ship measured by scanning the other ship in the sea area around the main ship by the rider and matches the operation information of the other ship from the radar or the vessel automatic identification unit. Thus, the collision risk may be calculated.

In addition, the control unit, the video information provided from the camera or the chart information provided from the electronic chart device, by displaying the other ships by the collision risk step by step, and displays the safety route without the collision risk overlay display can do.

In addition, the control unit may simultaneously calculate the collision risks for a plurality of other ships, and simultaneously display the image information provided from the camera or the electronic chart provided from the electronic chart device.

In addition, the control unit further calculates a collision risk with terrain features in the surrounding waters of the main ship, generates warning information according to the collision risk, and provides the warning information from the video information provided by the camera or the electronic chart device It can be displayed by overlaying the chart information.

In addition, the control unit generates access warning information when the distance between the main ship and the other ship is within a preset distance, and may collide if there is a tendency to approach each other by the operating direction and constant speed of the main ship and the other ship within the preset distance. The warning information is generated, and if the trend is approached for a specific time or more within a predetermined distance, collision warning information may be generated to generate stepwise warning information according to the collision risk.

According to the present invention, by utilizing the camera image information, more detailed identification of the object in the waters around the main ship, and calculate the collision risk through the identified bearing and distance information to provide warning information so that you can respond quickly It has the effect.

In addition, it is possible to secure a safe flight by guiding a safe flight path that can be bypassed by avoiding a collision according to the collision risk.

Furthermore, it calculates the risk of collision with not only ships as moving objects in the waters around the main ship, but also terrain features, and provides step-by-step warning information according to the collision risk by overlaying it with chart information or image information to provide intuitive and visual collision risk. It has the effect.

1 illustrates a schematic configuration diagram of a collision risk prediction and safety operation guide system through fusion of an on-board navigation sensor and image information according to an embodiment of the present invention.
FIG. 2 illustrates a collision risk prediction through fusion of the navigation information sensor and image information of FIG. 1 and estimation of a collision prediction point according to the bearing and distance information of another ship by the safe navigation guide system.
3 and 4 illustrate a safety operation route according to the collision risk prediction by the in-flight navigation sensor and image information fusion of FIG. 2 and the collision risk by the safety operation guide system.
FIG. 5 illustrates the collision risk prediction through the onboard navigation sensor and image information fusion of FIG. 2 and display of other ship tracking and warning information by the imaging device of the safe navigation guide system.

Hereinafter, embodiments of the present invention having the above-described features will be described in more detail with reference to the accompanying drawings.

FIG. 1 illustrates a schematic configuration diagram of a collision risk prediction and safety operation guide system through fusion of an on-board navigation sensor and image information according to an embodiment of the present invention, and FIG. 2 shows an on-board navigation sensor and image information fusion of FIG. 1 Prediction of the collision risk through and the estimation of the predicted collision point according to the bearing and distance information of the other ship by the safe navigation guide system. FIGS. 3 and 4 illustrate the collision risk through the fusion of the navigation sensor and image information of FIG. 2 It is an example of the safe operation route according to the risk of collision by the prediction and safety operation guide system, and FIG. 5 is a prediction of collision risk through the fusion of the navigation information sensor and image information of FIG. 2 and other ships by the imaging device of the safety operation guide system. Following is an example of display of warning and warning information.

1 to 5, the collision risk prediction and safety operation guide system through the fusion of the on-board navigation sensor and the image information according to the present invention, as a whole, to the information and imaging equipment 130 by the on-board navigation sensor (110,120,150) It fuses the image information by predicting the collision risk of the ship and guides the safe route, that is, the ship automatic identification unit 120 interlocking with the radar 110, the camera 130 photographing the surrounding sea area, and calculating the collision risk It is composed of a control unit 140 that guides the safe driving route, and utilizes camera image information to more effectively identify information on objects in the surrounding waters of the main ship, calculate collision risk, and provide step-by-step warning information to respond quickly. Make it possible to do so.

First, the ship automatic identification unit 120, as an automatic ship identification system (AIS), in conjunction with the radar 110, transmits and receives operational information such as ship-related information and sailing-related information in the waters around the main ship, for example, ship automation The identification unit 120 may provide ship-related information, location information, navigation information of the navigation speed and direction of navigation, and security-related information or safety-related information between ships or between ships and land and sea control stations. Send and receive in real time.

Next, the camera 130 photographs the surrounding area of the main ship to provide image information, and may include an optical camera or a thermal camera that can be identified from the image information by capturing the surrounding area or other ship regardless of the weather or time of day. Can be.

Here, the camera 130 may be provided with one or a plurality of cameras, and may be arranged to monitor the periphery of the main ship, including the front of the ship's navigation direction (see FIG. 4). Preferably, considering that the angle of view of the camera is limited, a plurality of cameras may be configured as one set to be installed to perform omni-directional monitoring over the front and side as well as in the direction of navigation of the main ship.

Next, the control unit 140 matches the operation information (ship-related information, navigation-related information, safety-related information, etc.) provided by the ship automatic identification unit 120 with the image information provided by the camera 130, and the area around the main ship. While tracking the other ship's ship, it analyzes the position, distance, bearing, and cruising speed of the other ship, calculates the collision risk with the ship, generates warning information according to the collision risk, and generates an electronic chart (ECDIS: Electronic Chart Display and Information System (150) guides a safe route that can avoid collisions in correspondence with the collision risk.

That is, the control unit 140 extracts ship image or ship information of the ship type from the image information, and tracks the ship, that is, the other ship, and analyzes the distance, operating direction, orientation, cruise, and location of the ship. The collision risk is calculated by judging the possibility of a collision at a predicted collision point in a specific area within a specific time by comparing the operating direction and the cruise speed.

For example, as illustrated in FIG. 2, the control unit 140 generates access warning information when the distance between the main ship and the other ship is within a preset distance, and within this preset distance, the operating direction of the main ship and the other ship is If the trend is close to each other due to constant speed, it is possible to generate collision warning information, and if the trend is closer to a specific time within a predetermined distance, collision warning information can be generated to generate stepwise warning information according to the collision risk.

In addition, as shown in FIG. 3, the control unit 140 guides the first safety operation route SVR1 corresponding to the approach warning information and the second safety operation corresponding to the possible collision warning information, according to the collision risk level. The furnace (SVR2) may be guided, and the third safe driving route (SVR3) corresponding to the collision warning information may be guided.

On the other hand, the controller 140 may analyze the position of the other ship on the image from the image information to calculate the orientation and distance of the other ship from the main ship.

As a first example, the camera 130 is composed of a combination of single cameras 130a, and the control unit 140 identifies the relative positional information between the vessel and the other vessel analyzed from the image information by the radar 110 or the ship automatic identification. The collision risk may be calculated by matching the operation information of the other ship from the unit 120.

For example, since the lower ship on the image of the image information provided from the combination of the single cameras 130a is closer to the main ship than the above other ships, the combination of the single cameras 130a is the orientation of the other ship and the degree of access to the ship. Can provide relative location information.

Alternatively, as a second example, the camera 130 is composed of a combination of stereo cameras 130b, and the control unit 140 radar 110 displays distance information between the main ship and other ships analyzed through the location difference by the stereo image information. ) Or by matching the operation information of the other vessel from the vessel automatic identification unit 120, it is possible to calculate the collision risk.

For example, the control unit 140 may analyze a distance according to a position difference of the same ship on the image provided from the combination of the stereo cameras 130b.

Alternatively, as a third example, the camera 130 is composed of a panoramic camera 130c, and the controller 140 radar 110 displays distance information between the main ship and other ships analyzed through the location difference by the panoramic image information. Alternatively, the collision risk may be calculated by matching the operation information of the other ship from the ship automatic identification unit 120.

For example, the panoramic camera 130c may be arranged such that a plurality of cameras are arranged in a circular shape to photograph an out-of-board 360-degree image, but the same ship is simultaneously photographed by an adjacent camera.

That is, for example, the control unit 140 may analyze the distance according to the position difference of the same ship on the image by the panoramic camera 130c.

Alternatively, as a fourth example, the control unit 140 matches the distance information between the main ship and the other ship measured by the laser measuring instrument 130d with the operation information of the other ship from the radar 110 or the ship automatic identification unit 120. By doing so, the collision risk can be calculated.

For example, the control unit 140 analyzes the distance between the main ship and the other ship by rotating the laser meter 130d by a rotation module (not shown) to irradiate the laser with the orientation of the other ship analyzed from the image information. Can be.

Alternatively, as a fifth example, the control unit 140 scans the other vessels in the surrounding waters by the lidar 130e to detect the distance information between the main vessel and the other vessels measured by the radar 110 or the vessel automatic identification unit. The collision risk may be calculated by matching the operation information of the other ship from (120).

On the other hand, the control unit 140, based on the chart information provided from the electronic chart device 150, may display and provide electronic chart information by overlaying ship information and collision possibility information, which are warning information according to a collision risk level.

That is, the control unit 140 may display and display the other ships step by step according to the collision risk on the chart information image provided to the electronic chart device 150, and display the safety operation route without the collision risk.

For example, the warning information may be displayed step by step due to differences in color shades or text sizes.

Or, the control unit 140, as shown in (a), (b), (c) of FIG. 5, based on the image information provided from the camera 130, the warning information according to the collision risk to the image information It is possible to display and provide ship information (ship specifications, flight information) and possible collision information.

That is, the controller 140, on the image information provided from the camera 130, displays the other ships step by step according to the collision risk, overlays the safe route without collision risk and displays it through the imaging device 160 You may.

For example, the warning information may be displayed step by step due to differences in color shades or text sizes.

That is, since the radar 110 alone does not detect a small object or another ship closest to the main ship within 200 m, and does not identify the type of ship or an accurate image, it is more realistic from the video information of the camera 130. It is possible to intuitively identify the other ship, and match the operation information of the vessel automatic identification unit 120 to analyze the exact direction and distance of the other ship.

In addition, the controller 140 may simultaneously calculate collision risks for a plurality of other ships, and simultaneously display them on the electronic chart device 150 or the imaging device 160 to provide multiple information.

In addition, the control unit 140, in addition to the ship as a mobile object, further calculates the risk of collision with terrain features in the surrounding waters by the same analysis and calculation method, generates stepwise warning information according to the collision risk, and generates warning information. The electronic chart device 150 or the video device 160 may be displayed overlaid with chart information or video information.

Therefore, by configuring the collision risk prediction and safety operation guide system through the onboard navigation sensor and image information fusion as described above, the camera image information is utilized to more effectively identify detailed information about objects in the surrounding waters of the main ship, The collision risk is calculated through the identified bearing and distance information, and step-by-step warning information is provided to promptly respond. By avoiding the collision according to the collision risk, a safe route that can be bypassed can be guided to ensure safe operation. In addition, it calculates the risk of collision with geographical features as well as ships that are mobile objects in the surrounding waters of the main ship. And can visually provide a collision risk.

The configurations shown in the embodiments and drawings described in this specification are only the most preferred embodiments of the present invention, and do not represent all of the technical spirit of the present invention, and various equivalents can be substituted at the time of the application. It should be understood that there may be water and variations.

110: radar 120: ship automatic identification unit
130: camera 130a: single camera
130b: stereo camera 130c: panoramic camera
130d: laser meter 130e: lidar
140: control unit 150: electronic chart device
160: Video device SVR1,2,3: Safe route

Claims (12)

In conjunction with the radar, the vessel automatic identification unit for transmitting and receiving operation information including ship-related information and navigation-related information in the surrounding waters;
A camera that provides video information of the surrounding waters of the main line; And
By matching the operation information provided by the vessel automatic identification unit with the image information provided by the camera, tracking other ships in the surrounding waters of the main ship and analyzing the location, distance, bearing and cruising speed of the other ship, A collision risk prediction and safety operation guide system through fusion of onboard navigation sensors and image information, including a control unit that calculates the collision risk and generates warning information according to the collision risk, and guides a safe navigation route through an electronic navigation device. . According to claim 1,
The control unit,
Ship information is extracted from the above image information, while tracking other ships, comparing the distance, operating direction, bearing and cruising with the other ships analyzed, and comparing the location of the main ship with the operating direction and cruising, within a specific time within a specific time. The collision risk is determined to calculate the collision risk, and based on the chart information provided from the electronic chart device or the video information provided from the camera, ship information and collision possibility information are provided according to the collision risk, Anti-collision risk prediction and safety operation guide system through the fusion of navigational sensors and image information According to claim 2,
The camera is an optical or thermal imaging camera, and one or more of them are installed to monitor the azimuth of the main ship. The method of claim 3,
The camera is composed of a combination of single cameras,
The control unit may calculate the collision risk by matching the relative positional information between the main vessel and the other vessel analyzed from the image information of the single cameras with the operation information of the other vessel from the radar or the vessel automatic identification unit. The collision risk prediction and safety operation guide system through the onboard navigation sensor and image information fusion. The method of claim 3,
The camera is composed of a combination of stereo cameras,
The control unit matches the distance information between the main ship and the other ship analyzed through the position difference by the stereo image information of the stereo cameras to the operation information of the other ship from the radar or the vessel automatic identification unit, and the collision risk Characterized in that, the collision risk prediction and safety operation guide system through the onboard navigation sensor and image information fusion. The method of claim 3,
The camera is composed of a panoramic camera,
The control unit matches the distance information between the main ship and the other ship analyzed through the position difference by the panoramic image information of the panoramic camera to the operation information of the other ship from the radar or the vessel automatic identification unit, and the collision risk Characterized in that, the collision risk prediction and safety operation guide system through the onboard navigation sensor and image information fusion. The method of claim 3,
The control unit, by comparing the distance information between the main vessel and the other vessel measured by the laser measuring instrument with the operation information of the other vessel from the radar or the vessel automatic identification unit, characterized in that for calculating the collision risk, onboard Collision risk prediction and safety operation guide system through fusion of navigation sensor and image information. The method of claim 3,
The controller matches the distance information between the main ship and the other ship measured by scanning other ships in the sea area around the main ship by the rider to match the operation information of the other ship from the radar or the vessel automatic identification unit, and the collision risk Characterized in that, the collision risk prediction and safety operation guide system through the onboard navigation sensor and image information fusion. The method according to any one of claims 4 to 8,
The control unit, by stepping the other ships by the collision risk, the image information provided from the camera or the chart information provided from the electronic chart device, and displays the overly displayed safety route without the collision risk Characteristic, collision risk prediction and safety operation guide system through fusion of onboard navigation sensor and image information. The method of claim 9,
The control unit simultaneously calculates the collision risk for a plurality of other ships, and simultaneously displays the image information provided from the camera or the chart information provided from the electronic chart device, and the ship navigation sensor and the image information are fused. Anti-collision risk prediction and safety operation guide system. The method of claim 10,
The control unit further calculates a collision risk with terrain features in the surrounding waters of the main ship, generates warning information according to the collision risk, and generates the warning information from the video information provided from the camera or from the electronic chart device Collision risk prediction and safety operation guide system, which is displayed by overlaying on the information, through fusion of the navigation sensor and image information. The method of claim 11,
The control unit generates access warning information when the distance between the main ship and the other ship is within a preset distance, and if the trend is close to each other by the navigation direction and cruise of the main ship and the other ship within the preset distance, the collision warning warning information is possible. And generating collision warning information if it is within a predetermined time or more within a predetermined distance, generating stepwise warning information according to the collision risk, collision through onboard navigation sensor and image information fusion. Risk prediction and safe operation guide system.

Images