KR20220090301A - Ship safety management system using augmented reality - Google Patents

Abstract

The present invention relates to a ship safety operation management system using augmented reality. According to an embodiment of the present invention, a monitor for outputting an image; A camera that transmits an external image of the vessel; an object recognition unit for recognizing an object by analyzing the external image information received through the camera; an object analysis unit for extracting object data by analyzing the recognized object; an object database in which data about the object is stored; a matching unit for adjusting the external image information to match the object data and external topographic information according to the location of the ship in an external topographic database related to external topography; It may include an output unit for outputting the object data and the external topographic information adjusted through the matching unit.

Description

Ship safety management system using augmented reality

The present invention relates to a ship safety navigation management system using augmented reality, and more particularly, to a ship safety navigation management system that enables a sailor to easily recognize ship information using augmented reality.

Augmented Reality (增强現實: Augmented Reality) creates a new environment by combining the real world that users see with their eyes and the virtual world with additional information. Augmented reality systems that converge real and virtual environments have been researched and developed mainly in the United States and Japan since the late 1990s. Augmented reality, a concept that complements the real world with a virtual world, uses a virtual environment created with computer graphics, but the main character is the real environment. Computer graphics serve to provide additional information necessary for the real environment.

In the ship navigation system, the prior art accurately measures wave information through radar, as disclosed in "A method for safe navigation of a vessel by ESD having blue information" of Republic of Korea Patent Application No. 10-2007-0037443 No. 10-2007-0037443 is displayed and shared with each other at the same time through ESD, and provides a safe ship navigation method by ESD equipped with wave information that can improve the stability of vessel operation through wave information and ESD information.

However, in order to improve the stability of the vessel operation, there is a limit to the electronic chart display system (ECDIS) alone. And there is a need to help with female navigation.

The present invention was invented to improve the above problems, and the problem to be solved by the present invention is to provide information related to the operation by using augmented reality on the actual operation screen, and to compare the actual route with the planned route to provide an accurate route It is to guide the operation of the ship and to receive environmental information about the location of other ships so that the environment can be grasped more accurately.

The technical problem of the present invention is not limited to those mentioned above, and another technical problem not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, a ship safety navigation management system using augmented reality according to an embodiment of the present invention includes a monitor for outputting an image; A camera that transmits an external image of the vessel; an object recognition unit for recognizing an object by analyzing the external image information received through the camera; an object analysis unit for extracting object data by analyzing the recognized object; an object database in which data about the object is stored; a matching unit for adjusting the external image information to match the object data and external topographic information according to the location of the ship in an external topographic database related to external topography; It may include an output unit for outputting the object data and the external topographic information adjusted through the matching unit.

Air pressure sensor, temperature sensor, humidity sensor, illuminance sensor, position sensor (GPS), course sensor (Compass), speedometer, inclination angle sensor, automatic ship identification sensor (AIS), wind direction/wind speed sensor, wave direction for the corresponding position of the ship It may be provided with an environmental sensor unit including / wave height.

The environmental information collected by the environmental sensor unit may be transmitted to another vessel within a preset range at the corresponding position of the vessel.

The environmental information collected by the environmental sensor unit may be transmitted to a vessel scheduled to pass within 6 hours within a preset range from the corresponding position of the vessel.

It is possible to receive the environmental information of the environmental sensor unit of the nearest other vessel on the traveling path of the vessel, and output the environmental information to the monitor.

The other vessel may be limited to a vessel within 6 hours of the course of the vessel.

It is possible to predict the weather by analyzing the environmental information of the other vessel, and output the predicted weather to the monitor.

a route comparison unit comparing the planned route of the ship with the actual route of the ship; a correction route calculation unit for calculating a correction route for matching the planned route with the actual route; The method may further include a fuel calculation unit for calculating an amount of corrected fuel required for the correction route, and output the planned route, the actual route, and the corrected route to the monitor unit.

When the corrected route overlaps with the expected route of another ship, a warning signal may be sent to the other ship.

The details of other embodiments are included in the detailed description and drawings.

According to the ship safety navigation management system using augmented reality according to an embodiment of the present invention, the user can recognize an accurate route and operate the ship, thereby minimizing the risk due to route departure.

In addition, it is possible to predict the weather more accurately by receiving environmental information corresponding to the location of other ships, and to prepare for the operation accordingly, which has the effect of preventing accidents that may occur during operation in advance.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 is a block diagram showing the configuration of an information providing system according to an embodiment of the present invention.
2 is a block diagram showing a screen on which an information providing system according to an embodiment of the present invention is output.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art to which the present invention pertains can easily practice the present invention.

In describing the embodiments, descriptions of technical contents that are well known in the technical field to which the present invention pertains and are not directly related to the present invention will be omitted. This is to more clearly convey the gist of the present invention without obscuring the gist of the present invention by omitting unnecessary description.

For the same reason, some components are exaggerated, omitted, or schematically illustrated in the accompanying drawings. In addition, the size of each component does not fully reflect the actual size. In each figure, the same or corresponding elements are assigned the same reference numerals.

Also, device or element orientation (eg, “front”, “back”, “up”, “down”, “top”, “bottom”) The expressions and predicates used herein with respect to terms such as ", "left," "right," "lateral," etc. are used merely to simplify the description of the invention, and the associated apparatus Or it will be appreciated that it does not simply indicate or imply that an element must have a particular orientation.

Hereinafter, the present invention will be described with reference to the drawings for explaining a ship safety navigation management system using augmented reality according to an embodiment of the present invention.

1 is a configuration diagram showing the configuration of a ship safe navigation management system 10 according to an embodiment of the present invention.

According to the embodiment of the present invention, the ship safety operation management system 10 may be configured to include a control unit 100 , a database unit 200 , and an environmental sensor unit 300 .

The control unit 100 includes a camera 110 , a monitor 120 , a matching unit 140 , an output unit 160 , a receiving unit 150 , an object recognition unit 130 , a fuel amount calculation unit 170 , and an environment analysis unit. 180 , a hazardous area analysis unit 190 , and may include a flight monitoring unit 195 .

The environmental sensor unit 300 includes a barometric pressure sensor 310 , a temperature sensor 320 , a humidity sensor 330 , an illuminance sensor 340 , a GPS 350 , a course sensor (Compass) 355 , and a speedometer 360 . , an inclination angle sensor 365, an automatic ship identification sensor (AIS) 370, a wind direction / wind speed sensor 375, a wave direction / wave height sensor 380, and the database unit 200 is a danger zone database ( 210 ), an object database 220 , an external topographic database 230 , and an external image database 240 .

According to an embodiment of the present invention, the ship safety operation management system 10 includes a monitor 120 for outputting an image; A camera 110 for transmitting an external image of the vessel; an object recognition unit 130 for recognizing an object by analyzing external image information received through the camera 110; an object analysis unit 135 for extracting object data by analyzing the recognized object; an object database 220 in which data for the object is stored; a matching unit 140 for adjusting the external image information to match the object data and the external topographic information according to the location of the vessel in the external topographic database 230 related to the external topography; It may include an output unit 160 for outputting the object data and the external topographic information adjusted through the matching unit 140 .

The monitor 120 may be installed in the wheelhouse of the ship, and the camera 110 may be installed outside the ship to take an external image of the ship. The camera 110 may be one unit, and it may be possible to use one or more images.

The external image information collected through the camera 110 may be stored in an external image database 240 .

The external image information is transmitted to the object recognition unit 130 and analyzed. The object recognition unit 130 is for analyzing the target shown in the external image information, and includes a vessel (including a type of vessel, a state of a vessel (eg operating vessel, berth vessel, etc.)), a horizon line, and a pier. , buoys, airplanes, whales, clouds, islands, land, etc.

The object data of the target analyzed through the external image information may be stored in the object database 220 .

The matching unit 140 serves to match the object data and the external topographic information of the external topographic database 230 with the external image information. The object data and external topographic information matched by the matching unit 140 are output to the monitor 120 together with the external image information through the output unit 160 .

In order to be effectively applied to the process of matching with the external image information and to allow the navigator to easily see the actual navigation information, it is preferable that the external topographic information be composed of 3D data.

Referring to FIG. 2 , an embodiment in which an external image using augmented reality is displayed can be seen. The monitor 120 may display location information 121 , event information 122 , stability information 123 , weather information 124 , sensor information 125 , attention area information 126 , and the like.

The location information 121 is information on the current location of the ship, and may include latitude, longitude, and remaining sailing distance.

The event information 122 may include various situations that may occur inside the ship, for example, fire, projectile, collision, and the like.

The stability information is related to the operation monitoring unit 195, which will be described later, and will be described later.

The weather information 124 may be used by receiving weather information from the Meteorological Agency as weather information of the corresponding location of the ship, and it is also possible to output actual weather information based on environmental information to be described later.

The sensor information 125 indicates information of sensors located inside the vessel or vessel operation, and when there is a change in the corresponding information, a notification is given through a change in color or the like.

The area information 126 is related to the operation monitoring unit 195 and will be described later.

The external topographic database 230 may include an inland waterway, a seabed topography, and the like, and may also be displayed as a 3D image on the monitor 120 using augmented reality.

According to the present invention, the planned route (R2) of the ship and the actual route (R1) in which the ship is actually navigating can be displayed separately.

According to the present invention, the route comparison unit 165 for comparing the planned route (R2) of the ship and the actual route (R1) of the ship; a correction route (R3) calculation unit (167) for calculating a correction route (R3) for matching the planned route (R2) and the actual route (R1); Further comprising a fuel amount calculation unit 170 for calculating the amount of corrected fuel required for the corrected route (R3), the planned route (R2), the actual route (R1), and the corrected route (R3) the monitor (120) It can be printed in the department.

This means that the ship cannot cruise along the planned route (R2) and may deviate from the route under the influence of ocean currents and winds, etc. ) so that it can be easily identified that the vessel has deviated from the route.

In addition, by displaying the corrected route R3 for following the planned route R2, guide to locate the corrected route R3 along the planned route R2 again, so that easy navigation can be made.

When operating along the corrected route R3, it is possible to calculate a more necessary corrected fuel amount. This is to ensure effective operation, and to prevent fuel shortage by calculating the actual amount of fuel remaining, the amount of fuel required for future operation, and the amount of fuel expected when moving along the corrected route (R3).

This corrected fuel amount is stored as big data and may be used as an indicator to change the route when operating the same route later.

When calculating the correction route R3, it may be possible to correct the route by determining whether there is a danger on the route using the external topographic data and changing the route.

In addition, by using the object data to determine whether there is a danger on the corrected route (R3), and if another ship is located on the route, the expected route is determined using the AIS information of the ship to prevent collision with the ship It would also be possible to avoid.

The AIS is an Automatic Identification System, a state-of-the-art device that provides navigation information such as a ship's position, course, and speed in real time. This is a device to prevent collision of ships at sea, and it is a mandatory installation requirement for ships according to the International Maritime Organization (IMO). and progress can be assessed.

When the corrected route (R3) overlaps with the expected route of another ship, a warning signal may be sent to the other ship.

According to the embodiment of the present invention, the barometric pressure sensor 310, the temperature sensor 320, the humidity sensor 330, the illuminance sensor 340, the GPS 350, the course sensor (Compass) for the corresponding position of the ship ( 355), a speedometer 360, an inclination angle sensor 365, an automatic ship identification sensor (AIS) 370, a wind direction / wind speed sensor 375, a wave direction / environmental sensor unit 300 including a wave height sensor 380 can be provided

The environmental information collected by the environmental sensor unit 300 may be transmitted to another vessel within a preset range at the corresponding position of the vessel.

In the case of the weather information shown in FIG. 2 , only information about the past may be collected among information about the part where the ship is located and information about the surroundings. That is, the weather information transmitted from the on-land agency may have a time difference from the actual weather information, and the accuracy will decrease.

Therefore, in order to deliver accurate information, the information collected by the environmental sensor unit 300 is transmitted to other ships, so that the weather in which the current ship is located can be more accurately recognized.

The environmental information collected by the environmental sensor unit 300 may be transmitted to a vessel scheduled to pass within 6 hours within a preset range from the corresponding position of the vessel. This is to ensure that the weather information received from the agency can be delivered about 6 hours before, so that the relevant information can be delivered before that, so that more accurate information can be delivered to other ships.

In addition, according to the embodiment of the present invention, it is possible to receive the environmental information of the environmental sensor unit 300 of the nearest other vessel on the traveling path of the vessel, and output the environmental information to the monitor 120 .

The other vessel may be limited to a vessel within 6 hours of the course of the vessel. This is to solve the problem of the time difference of the weather described above, and a description thereof will be omitted.

According to an embodiment of the present invention, it is possible to predict the weather by analyzing the environmental information of the other vessel, and output the predicted weather to the monitor 120 .

This is to predict the environmental change according to the distance or time difference with other ships, and to predict future weather changes based on the received environmental information.

For example, in the case of atmospheric pressure, although the atmospheric pressure is not low at the location of the ship, if the pressure at the location of another ship is low, rain or typhoon can be predicted in the future even if it is not rainy weather.

The operation monitoring unit 195 may perform a stability analysis of the ship 10 by calculating whether or not static stability, dynamic stability, and resonance between the ship and the wave occurs.

For example, the operation monitoring unit 195 is configured to check the deterioration of the static stability of the ship 10 due to the change in cargo weight and inappropriateness of the cargo arrangement (cargo arrangement to increase the center of gravity) before departure, the ship ( 10) Calculate the GM value, which is the distance between the transverse meta sensor (M) and the center of gravity (G), and compare the calculated GM value with the preset static stability reference value. When the calculated GM value is less than a preset static stability reference value, it may be determined that the static stability is degraded to a dangerous state.

In addition, the operation monitoring unit 195 calculates whether resonance occurs between the vessel 10 and the wave during operation, in order to check the resonance risk of the vessel 10 due to the interaction between the vessel 10 and the wave. For example, the ship monitor 120 ring device 100 is synchronous rolling (Synchronous rolling), parametric rolling (Parametric rolling), high wave attack (High wave attack) and surf-riding and broaching (Surf-riding & broaching) It is possible to determine whether resonance occurs between the four types of ships 10 and waves. Here, the synchronous rolling occurs when the intrinsic oscillation period of the ship 10 and the encounter period of the waves coincide. And, parametric rolling is a phenomenon in which wave sway occurs despite the waves entering the bow or stern of the ship 10, and occurs when the encounter period of the waves is approximately half of the intrinsic oscillation period of the ship 10 do. And, high wave attack is a phenomenon in which the ship 10 is continuously placed at the highest point of the wave and thus the static restoring force decreases. Occurs when it is larger than a ship. And, surfriding and broaching, when the ship 10 receives the waves from the stern and operates, the center of the hull is located at the crest of the wave or the uphill wave front, so that a sudden sway of the bow occurs and the hull is placed in parallel with the wave occurs in

The operation information (including stability, GM, and resonance risk) calculated by the operation monitoring unit 195 may be transmitted to the danger zone analysis unit 190 .

The hazardous area analysis unit 190 determines whether the area is a hazardous area based on the operation information, the environment information, and the like. In more detail, if the flight information is in a dangerous state and the weather analyzed through the environmental information is bad (strong rain, strong wind, typhoon, high wave height, etc.) 210) to store the flight information and environment information.

When the number of times of storage of the corresponding zone in the dangerous zone database 210 is greater than the average, it may be determined as a dangerous zone.

Such a danger zone can be divided into stages (eg, stage 1, stage 2, etc. of the danger zone), and whether the stage of the danger zone is present can be displayed on the monitor 120 .

On the other hand, in the present specification and drawings, preferred embodiments of the present invention have been disclosed, and although specific terms are used, these are only used in a general sense to easily explain the technical content of the present invention and help the understanding of the present invention, It is not intended to limit the scope of the invention. It is apparent to those of ordinary skill in the art to which the present invention pertains that other modifications based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein.

<Explanation of symbols for main parts of the drawing>
100: control unit 110: camera
120: monitor 130: object recognition unit
140: matching unit 150: receiving unit
160: output unit 170: fuel amount calculation unit
180: Environmental Analysis Department 190: Risk Section Analysis Department
200: database unit 300: environmental sensor unit

Claims (9)

a monitor that outputs an image;
A camera that transmits an external image of the vessel;
an object recognition unit for recognizing an object by analyzing the external image information received through the camera;
an object analysis unit for extracting object data by analyzing the recognized object;
an object database in which data about the object is stored;
a matching unit for adjusting the external image information to match the object data and external topographic information according to the location of the ship in an external topographic database related to external topography;
Ship safe navigation management system using augmented reality, characterized in that it comprises an output unit for outputting the object data and the external topographic information adjusted through the matching unit.
The method of claim 1,
A ship safety navigation management system using augmented reality, characterized in that it comprises an environmental sensor unit including a barometric pressure sensor, a temperature sensor, a humidity sensor, and an illuminance sensor for the corresponding position of the ship.
3. The method of claim 2,
The environmental information collected by the environmental sensor unit is a ship safety operation management system using augmented reality, characterized in that the delivery to other ships in a preset range at the corresponding position of the ship.
3. The method of claim 2,
The environmental information collected by the environmental sensor unit is a ship safety navigation management system, characterized in that the delivery to a ship scheduled to pass a preset range within 6 hours from the corresponding position of the ship.
3. The method of claim 2,
A ship safety navigation management system using augmented reality, characterized in that receiving the environmental information of the environmental sensor unit of the nearest other vessel on the progress path of the vessel, and outputting the environmental information to the monitor.
6. The method of claim 5,
Said other vessel is a vessel safety operation management system using augmented reality, characterized in that limited to a vessel within 6 hours of the progress of the vessel.
6. The method of claim 5,
A ship safety navigation management system using augmented reality, characterized in that predicting the weather by analyzing the environmental information of the other ship, and outputting the predicted weather to the monitor.
The method of claim 1,
a route comparison unit for comparing the planned route of the ship with the actual route of the ship;
a correction route calculation unit for calculating a correction route for matching the planned route with the actual route;
Further comprising a fuel calculation unit for calculating the amount of corrected fuel required for the correction route,
Ship safety navigation management system using augmented reality, characterized in that the output of the planned route, the actual route, and the corrected route to the monitor unit.
9. The method of claim 8,
A ship safety navigation management system using augmented reality, characterized in that when the corrected route overlaps with the expected route of another ship, a warning signal is transmitted to the other ship.

Images