KR102115020B1 - Buoy monitoring terminal and method based on augmented reality - Google Patents

Abstract

An embodiment of the present invention comprises the steps of detecting location information of a terminal; Reading metadata previously stored in the terminal based on the location information of the terminal; Detecting position information of a buoy matching the buoy detection area on the read metadata; And displaying the buoy image and the status information of the buoy corresponding to the position information of the buoy together with the captured image from the terminal.

Description

Buoy monitoring method and terminal using augmented reality {BUOY MONITORING TERMINAL AND METHOD BASED ON AUGMENTED REALITY}

The present invention relates to a buoy monitoring method and a terminal, and relates to a method and a terminal capable of locating a buoy and monitoring its status using augmented reality.

The first lighthouse in Korea was installed in Palmido, Incheon in 1903, and the buoy was installed in Seosudo, Yalu River in 1908. Since then, the industry has been modernizing over the past 100 years, and numerous lighthouses and light trails, such as buoys, have been installed and operated in trade ports, and more than 4,000 locations have been operated according to recent statistics from the Ministry of Maritime Affairs and Fisheries. In particular, about 1,776 buoys (including state-owned and private sectors) are operated in the entire sea area. Various types of buoys have been diversified, and 18 standard types have been established and managed, and installation purposes have been diversified, and large induction lamps (LANBY) and spa buoys have been installed and operated in major ports.

Of course, in order to manage and operate these (back) buoys, 400 ~ 500 ton-class buoys are maintained and divided into east and west Namhae areas to systematically maintain (back) buoys and enact and revise the Route Marking Act in accordance with domestic and foreign route marking trends. By implementing, the management of route marking facilities is being done. However, real-time condition monitoring is not easy for marine signage facilities (route signs such as lighthouses, lights, and light buoys) that have many difficulties in site inspection due to the bad weather conditions in the sea. However, it is difficult to manage the marine signage facilities because the difference between the tide and the tide is severe and there are many narrow waterways. In addition, there is a problem in that marine signage facilities are not well-recognized due to frequent fogging, which is a significant obstacle to the safe operation of ships. Recently, a technology for displaying status information on a marine labeling facility on a terminal has been developed, but there is a limitation in that a marine labeling facility on the terminal cannot be accurately and conveniently detected due to waves, wind, and the like.

In addition, although various electronic devices and communication devices are installed in the marine labeling facilities to receive the status information of the marine labeling facilities from the marine labeling facilities, the number of marine labeling facilities is very large, and the device is applied to all marine labeling facilities. There is an economic problem in doing this, and the problem of the continuity of maintenance for marine signage facilities has been weakened steadily due to the frequent failures or defects of the devices due to environmental factors such as aging and maritime equipment in the offshore signage facilities. .

Republic of Korea Registered Patent Publication 10-1677039 B1

An object of the present invention is to provide a buoy monitoring method and a terminal using augmented reality that can monitor the status of the buoy in real time.

In addition, an object of the present invention is to provide a method and a terminal that can conveniently and quickly grasp the position of the buoy in augmented reality.

In addition, it is an object of the present invention to provide a method and a terminal capable of conveniently and accurately monitoring the buoy of the sea even in an environment where the flow is high due to waves or wind, such as a ship.

In addition, the object of the present invention is to register the location information and status information of the buoys in advance without using a separate electronic device to install the buoys, and utilize them to enhance the continuity of buoy maintenance and improve the economic efficiency of the buoys. It is to provide a terminal.

In addition, it is an object of the present invention to provide a method and a terminal capable of quickly grasping the position of a buoy and monitoring status information even at a long distance without the user having to inspect the buoy closely.

An embodiment of the present invention comprises the steps of detecting location information of a terminal; Reading metadata stored in advance in the terminal based on the location information of the terminal; Detecting position information of a buoy matching the buoy detection area on the read metadata; And displaying the buoy image and the status information of the buoy corresponding to the position information of the buoy together with the captured image from the terminal.

In another aspect, the step of detecting the location information of the terminal comprises: detecting the location coordinates of the terminal; Determining whether the terminal is located at a reference angle; Inducing the terminal to be positioned at a reference angle; And detecting the rotation angle and altitude information of the terminal. A buoy monitoring method using augmented reality may be provided.

In another aspect, the rotation angle includes a first rotation angle and a second rotation angle, and detecting the location information of the terminal further comprises: detecting altitude information of the terminal; and further comprising the reference The angle and the first rotation angle are the rotation angles of the terminal about a first axis perpendicular to the sea level, and the second rotation angle is the center of the terminal about a second axis horizontal to the sea level and perpendicular to the first axis. It is also possible to provide a method for monitoring a buoy using augmented reality, which is a rotation angle.

In another aspect, the status information of the buoy includes a buoy location information, an installation date of the buoy, a type of the buoy, painting information of the buoy, and a method of monitoring the buoy using augmented reality, including information about the flashing method of the lighting device of the buoy. It can also be provided.

In another aspect, the terminal may provide a buoy monitoring method using augmented reality, characterized in that it is located on the ship.

In another aspect, determining whether the amount of continuous change in the second rotation angle or altitude of the terminal is detected within a preset range; And fixing the buoy detection area when the amount of change is detected. It may also provide a buoy monitoring method using augmented reality.

In another aspect, at least one memory for storing instructions; And at least one processor; wherein the instructions are executable by the processor to cause the processor to perform operations, the operations comprising: detecting location information of the terminal; Reading metadata stored in advance in the terminal based on the location information of the terminal; Detecting the position information of the buoy matching the buoy detection area on the read metadata; Displaying a buoy image corresponding to the position information of the buoy and status information of the buoy together with a captured image from the terminal; It is also possible to provide a buoy monitoring terminal using augmented reality.

An embodiment of the present invention enables the user to monitor the marine signage facility within the image captured by the terminal, and can check the marine signage facility in real time just by changing the shooting area, making it more convenient and efficient Enable maintenance and management.

In addition, embodiments of the present invention can check in real time the location of the marine labeling facility in real time regardless of various environmental factors of the sea, such as the sea level or the height of the waves, or even when the marine labeling facility is located remotely from the user's current location. And to monitor the status information of marine labeling facilities.

In addition, an embodiment of the present invention allows the user to monitor the marine signage facilities in the region of interest by immediately reflecting the region of interest.

In addition, the embodiment of the present invention enables accurate definition of a region of interest requiring monitoring using the location coordinates of the terminal, not the location coordinates of the ship itself.

In addition, the embodiment of the present invention allows the user to quickly grasp whether or not the marine signage facility is lost.

In addition, the embodiment of the present invention is to remove the environmental factors, even if the photographing area constantly changes according to the environmental factors of the sea, so that the marine labeling facility can be accurately and quickly monitored.

1 is a schematic diagram of a buoy monitoring system using augmented reality according to an embodiment of the present invention.
2 is an exemplary block diagram of a terminal.
3 is a flowchart of a buoy monitoring method using augmented reality according to an embodiment of the present invention.
4 is a schematic diagram of a method of setting a reference angle by matching a current line to a reference line with a captured image on a display device.
5 is an exemplary view showing a shooting range according to a second rotation angle of the terminal.
6 is an exemplary view of an image in front displayed according to the shooting range of FIG. 5.
7 is an exemplary view of a buoy image displayed on a captured image and status information of the buoy.
8 is a flowchart of a buoy monitoring method using augmented reality according to another embodiment of the present invention.
9 is an exemplary view for a case in which a vessel continuously flows due to a wave in the sea.
10 is an exemplary view of a buoy detection area including coordinates of a plurality of buoys on metadata.
11 is an exemplary view of a buoy detection area including position coordinates of a plurality of buoys according to elevation elevation or change of a second rotation angle.
12 is an exemplary view of a buoy detection area including coordinates of a plurality of buoys according to a change in altitude or a second rotation angle.

The present invention can be applied to various transformations and may have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention and methods for achieving them will be clarified with reference to embodiments described below in detail with reference to the drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms. In the following examples, terms such as first and second are not used in a limited sense, but for the purpose of distinguishing one component from other components. In addition, a singular expression includes a plural expression unless the context clearly indicates otherwise. In addition, terms such as include or have means that a feature or component described in the specification exists, and does not preclude the possibility of adding one or more other features or components in advance. In addition, in the drawings, the size of components may be exaggerated or reduced for convenience of description. For example, since the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of description, the present invention is not necessarily limited to what is shown.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, and the same or corresponding components will be denoted by the same reference numerals, and redundant description thereof will be omitted. .

1 is a schematic diagram of a buoy monitoring system using augmented reality according to an embodiment of the present invention. And Figure 2 is an exemplary block diagram of a buoy monitoring terminal using augmented reality.

In FIG. 1, the sea 1 and the ship 2 operating on the sea and the user 3 boarding the ship 2 are shown, and the user 3 has the terminal 100. In addition, the terminal 100 may detect its own location information using a Global Navigation Satellite System (GNSS) system.

The GNSS system is a satellite positioning system and may provide location information of the terminal 100 using the GNSS satellite 4 orbiting the space orbit.

In addition, at least one buoy 200 may be installed on the sea 1. The buoy 200 is a route marking facility. In general, the ship (2) passing through the passage for the entry into the port or the ship (2) passing through the waterway in a special area with low water depth must sail along a narrow, winding path or waterway, so there is a risk of stranding. In particular, at night, it is more difficult to identify routes and waterways, which increases the risk. Moreover, due to the urbanization of the harbor, bright lights such as neon signs may disturb the crew's view, which may hinder the identification of routes or waterways. In order to prevent such a danger, it is possible to easily identify a route or a waterway even at night to help the safety navigation of the ship 2 and a route marking facility 200 arranged at regular intervals along the route or waterway is used. In addition, the buoy 200 is a typical example of a lighted buoy, but is not limited thereto.

The buoy 200 includes a lighting device, a location information sensing device that measures a position through GPS, an operating state sensing device that detects an operating state of the lighting device, a photovoltaic device and a battery that communicate with solar power generation, and communication. It may include, but is not limited to.

In addition, the operating state detection device of the buoy 200 periodically detects the state of the buoy 200 and / or periodically detects its location information to detect the detected information by the terminal 100 and / or a pre-registered marine sign. Can be sent to the facility management server.

As shown in FIG. 1, the buoy 200 is installed around the ship 2, but this is only an example, and the buoy 200 is the ship 2 according to the position where the ship 2 sails. It may be located at a considerable distance from, the buoy 200 may not be identified in the field of view of the user 3 aboard the ship 2, and the buoy 200 may or may not be recognized depending on the elevation of the wave. .

In addition, the virtual x-axis of FIG. 1 is an axis parallel to the sea surface of the sea 1, and the Y-axis is an axis perpendicular to the x-axis. And the Z axis is an axis perpendicular to the X and Y axes, which is another axis parallel to the sea level.

Referring to FIG. 2, the terminal 100 includes one or more memories and one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), It can be configured in various forms, such as individual logic circuits, software, hardware, firmware, or any combination thereof.

The memory includes instructions (eg, executable instructions) such as computer readable instructions or processor readable instructions. The instructions may include one or more instructions executable by a computer, such as by each of the one or more processors.

For example, one or more instructions may be sent to one or more processors to perform one or more processors to perform operations including analyzing location information of a terminal and processing augmented reality video processing such as a composite of a captured image and a buoy image. May be executable by

Further, the terminal 100 is one or more computers or other electronic devices used by the user 3 to execute applications that perform various tasks. For example, a computer, laptop computer, smart phone, mobile phone, PDA, tablet PC, or any other device operable to communicate with a web server. However, the present invention is not limited thereto, and the terminal 100 includes processing logic executed on various machines, interpreting and executing instructions stored in a plurality of memories, and for a graphical user interface (GUI) on an external input / output device. Various other elements may be included, such as processes for displaying graphical information. In addition, the terminal 100 may be connected to an input device (eg, mouse, keyboard, touch-sensitive surface, etc.) and an output device (eg, monitor, screen, etc.).

More specifically, the terminal 100 may include a processor 110, a memory 120, a display device 130, a photographing device 140, a communication device 150, and a sensing device 160.

The display device 130 includes a liquid crystal display (LCD) method, a field emission display (FED) method, a plasma display panel (PDP) method, and an electroluminescence device (EL). ) Method, an electrophoretic display method, an organic light emitting diode display method, but the image can be displayed, but is not limited thereto. Also, the display device 130 may include a touch screen.

The communication device 150 includes technical standards or communication methods for wireless mobile communication (for example, Global System for Mobile Communication (GSM), Code Division Multi Access (CDMA), High Speed Downlink Packet Access (HSDPA), HSUPA ( High speed uplink packet access (LTE), Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), etc. can transmit and receive wireless signals to and from each other on a mobile communication network.

In addition, the communication device 150 is a WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability) for Microwave Access),), and LoRa communication.

In addition, the communication device 150 may transmit and receive wireless signals in a short-range wireless communication method. For example, the communication unit may include Bluetooth ™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), Wireless (Wi-Fi) -Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technology can be used to support short-range communication.

In addition, the sensor device 160 may include at least one of an acceleration sensor, a gyro sensor, an altitude sensor, a pressure sensor, a temperature sensor, a compass sensor, a gravity sensor, and an angular velocity sensor.

3 is a flowchart of a buoy monitoring method using augmented reality according to an embodiment of the present invention. And FIG. 4 is a schematic diagram of a method of setting a reference angle by matching a current line to a reference line with a captured image on a display device. And Figure 5 is an exemplary view showing a shooting range according to the second rotation angle of the terminal. And Figure 6 is an exemplary view of the front image displayed according to the shooting range of Figure 5, Figure 7 is an exemplary view of the buoy image and the status information of the buoy displayed on the captured image.

Referring to FIG. 3, a method for monitoring a buoy using augmented reality according to an embodiment of the present invention includes detecting terminal location information (S110), determining whether the terminal is located at a reference angle (S120), and reference angle location Inducing step (S130), detecting the first rotation angle of the terminal (S140), detecting the altitude information and the second rotation angle of the terminal (S150), of the buoy matching the buoy detection area on the metadata It may include the step of detecting the location information (S160) and displaying the buoy image and the status information of the buoy in the captured image (S170).

In the terminal location information detection step (S110), the communication device 150 of the terminal 100 communicates with the GNSS satellite 4 to receive its location information, and the processor 110 based on the location information, the terminal 100 You can detect the current position coordinates of.

In step S120 of determining whether the terminal is located at a reference angle, the terminal 100 drives the photographing device 140, displays an image photographed from the photographing device 140 on the display device 130, and photographs The preset reference line 9 may be displayed in the image. And the terminal 100 detects the first rotation angle information of the terminal 100 based on the sensing information of the sensing device 160 and displays the first rotation angle on the display device 130 in the form of the current line 8 can do.

Here, the first rotation angle represents the rotation angle of the terminal around the Y axis of FIG. 1. In addition, the preset reference line is an angle formed by the virtual normal 7 on the front surface of the display device 130 of the terminal 100 around the Y axis on the X axis or the Z axis, and may be preset.

In the step (S130) of deriving the reference angle position, the terminal 100, when the position of the current line 8 and the position of the reference line 9 do not match based on the sensing information from the sensing device 160, the current An image in the form of an arrow that induces movement of the line 8 may be displayed on the display device 130.

The user can rotate the terminal 100 around the Y-axis so that the position of the current line 8 is located on the reference line 9, and the position of the current line 8 and the position of the reference line 9 are mutually If they match, the terminal 100 is positioned at a reference angle. In addition, when the positions of the current lines 8 and the reference lines 9 coincide with each other, the processor 110 may display a message that the reference angle setting of the terminal is completed on the display device 130.

In step S140 of detecting the first rotation angle of the terminal, the user confirms the message that the reference angle setting is completed displayed on the terminal 100 and then photographs the area where the buoy that needs to be monitored is expected to exist. ), The area photographed from the photographing device 140 may be changed.

In this case, the processor 110 may detect the first rotation angle information of the terminal 100 based on the continuous sensing information of the sensing device 160.

In step S150 of detecting the altitude information and the second rotation angle of the terminal, the processor 110 detects the altitude information and the second rotation angle information of the terminal 100 based on the continuous sensing information of the sensing device 160. can do.

Here, the second rotation angle is the rotation angle of the terminal around the X axis of FIG. 1, and refers to an angle that the virtual normal of the front surface of the display device of the terminal forms with the Y axis or the Z axis.

In step S160 of detecting the position information of the buoy matching the buoy detection area on the metadata, the processor 110 may read the metadata stored in advance in the memory 120.

The meta data may be map information having location coordinate information of buoys in a region of a preset range around a reference coordinate.

The processor 110 may read metadata matching the region photographed on the photographing apparatus 140 based on first and second rotation angle information and altitude information among the metadata stored in the memory 120.

However, the present invention is not limited thereto, and the processor 110 transmits location information of the terminal 100, first and second rotation angle information, and altitude information to the external server through the communication device 150, and is detected from the external server. Meta data matching information may be received.

In more detail, referring to FIG. 5, a photographing range of the photographing apparatus 140 varies according to a second rotation angle of the terminal 100. In addition, the shooting range of the photographing apparatus 140 is changed according to the first rotation angle and altitude information of the terminal 100.

In addition, the photographed image displayed on the display device 130 as shown in FIG. 6 may vary according to the photographed range displayed in FIG. 5.

Accordingly, the processor 110 detects the photographing area of the photographing apparatus 140 based on the first rotation angle, the second rotation angle, and the altitude information of the terminal 100. Then, the processor 110 reads the metadata including the map of the area corresponding to the photographing area of the photographing device 140 on the memory 120, and the position coordinate information of at least one preset buoy on the read metadata. Detects.

Referring to FIG. 7 in step S170 of displaying the buoy image and the status information of the buoy in the captured image, the processor 110 is a virtual buoy in the currently displayed captured image based on the position coordinate information detected on the metadata. An image 201 (two-dimensional or three-dimensional form) may be displayed. Also, the processor 110 may adjust and display the size of the buoy image 201 displayed according to the distance from the terminal 100. For example, a buoy image located at a long distance from the terminal 100 may be displayed relatively small and a buoy image located at a short distance may be displayed relatively large.

Further, the processor 110 may display status information 202 of the corresponding buoy together with the buoy image. However, the present invention is not limited thereto, and when the user selects the buoy image (via touch detection, etc.), the status information of the buoy corresponding to the selected buoy image may be displayed.

Here, the status information of the buoy may include the location information of the buoy, the installation date of the buoy, the type of the buoy, the painting information of the buoy, the flashing method information of the lighting device of the buoy.

In this way, the user can quickly change or check the region of interest by checking the photographed image displayed on the terminal 100, and the user simply changes the photographing region photographed through the photographing apparatus 140 to change the photographing region. It is possible to check the existing marine signage facilities in real time, making maintenance and management of more convenient and efficient marine signage facilities possible.

In addition, depending on various environmental factors of the sea, such as the height of the sea or the sea, it is difficult to visually recognize marine signage facilities such as the buoy 200, or even if the marine signage facility is located at a distance from the user's current location. As a result, the location of the offshore signage facility can be checked in real time and the status information of offshore signage facilities can be monitored.

In addition, the location coordinates of the terminal 100 are used to specify the status information of the maritime labeling facility in any of the marine labeling facilities on the omnidirectional area of the terminal 100 or all of the maritime labeling facilities in the omnidirectional area within a predetermined range. It is difficult to do this, and there is a problem in that it is complicated to display an offshore signage facility irrelevant to the direction of the ship 2 or the user's area of interest, and it is difficult for the user to immediately monitor the offshore signage facilities. However, according to an embodiment of the present invention, considering both the rotational angle information and the altitude information of the terminal 100 as well as the location coordinate information of the terminal 100, the image and status information of these are limited to the marine signage facilities in the shooting area. Since it can be displayed, it is possible to conveniently and instantly monitor the marine signage facilities located in the user's area of interest.

In addition, if the terminal 100 receives the position information of the ship 2 using the communication device of the ship 2 where the terminal 100 is located, the shape or size of the ship 2 and the communication device of the ship 2 According to the position of the terminal, the exact position of the terminal 100 is not detected, and the photographing area is not accurately defined. Therefore, there may be a problem that a marine marker facility is monitored for an area of non-interest rather than an area that a user wants to monitor. Accordingly, the terminal 100 according to the embodiment enables accurate definition of the photographing area by detecting its location information through communication with a GNSS satellite, thereby enabling monitoring of a marine marker facility for a user's region of interest.

In addition, the embodiment may detect the altitude information as well as the first and second rotation angles of the terminal 100 and immediately reflect that the photographing area according to the altitude of the terminal 100 is changed. In the case of a very large area such as the ocean, since the monitoring area varies greatly depending on where the imaging area is located, the embodiment enables accurate definition and detection of the monitoring area by quickly and accurately reflecting environmental factors.

In addition, the user can check the image of the buoy in the form of augmented reality and the image taken through the display device 130. In addition, when the image of the actual buoy is included in the captured image, the actual buoy image and the image of the buoy in augmented reality may be displayed as an overlapped image. Accordingly, according to an embodiment of the present invention, when the buoy is lost due to various reasons, the user immediately immediately confirms that the image of the buoy that does not overlap the actual buoy's captured image is displayed among the images of the buoy adjacent to the location of the terminal 100. Can be. This allows the user to quickly determine that the actual buoy corresponding to the image of the buoy is lost.

8 is a flowchart of a buoy monitoring method using augmented reality according to another embodiment of the present invention. And FIG. 9 is an exemplary view for a case where a ship continuously flows due to a wave of the sea. And FIG. 10 is an exemplary view of a buoy detection area including the position coordinates of a plurality of buoys on the metadata, and FIG. 11 is a buoy including the position coordinates of a plurality of buoys according to an elevation elevation or a second rotation angle change. It is an exemplary view of the detection area, and FIG. 12 is an exemplary view of the detection area of the buoy including the coordinates of the positions of the plurality of buoys according to the change of the altitude or the second rotation angle.

Referring to FIG. 8, a method for monitoring a buoy using augmented reality according to another embodiment of the present invention includes detecting a terminal location information (S210), determining whether the terminal is located at a reference angle (S220), and a reference angle Inducing the position (S230), detecting the first rotation angle of the terminal (S240), detecting the altitude information and the second rotation angle of the terminal (S250), the altitude or the second rotation angle is a preset range Determining whether a continuous change amount is detected within (S260), fixing the buoy detection area on the metadata (S270), detecting position information of the buoy matching the buoy detection area on the metadata (S280) and photographing A step of displaying the buoy image and the status information of the buoy in the image may be included (S290).

In more detail, in the terminal location information detection step (S210), the processor 110 may detect the location information of the terminal 100 using the sensing information of the sensing device 160 and the GNSS satellite 4.

In step S220 of determining whether the terminal is located at the reference angle, the processor 110 may determine whether the terminal 100 is located at the reference angle.

If the terminal is not located at the reference angle, in step S230 of deriving the reference angle position, the processor 110 determines the reference line according to the reference angle and the current line according to the second rotation angle of the current terminal 100. Each of these lines is displayed in the current photographed image displayed on the display device 130, and at the same time, an arrow image for moving the current line is displayed to induce the user to change the first rotation angle of the terminal 100. Can be.

In addition, when the first rotation angle of the terminal 100 coincides with the reference angle, the processor 110 may display a reference angle setting completion message indicating that the first rotation angle matches the reference angle on the display device 130. . In addition, the user may photograph the area with the photographing device 140 toward an area where augmented reality for the buoy, which is one of the marine sign facilities, is desired to appear.

In this case, the first rotation angle of the terminal 100 may vary according to an area to be monitored by the user. And in step S240 of detecting the first rotation angle of the terminal, the processor 110 may re-detect the first rotation angle of the terminal 110. In addition, in the step of detecting altitude information of the terminal and the second rotation angle (S250), the processor 110 may control altitude information of the terminal 110 based on sensing information of the sensing device 160 of the terminal 110. 2 Rotation angle information can be detected.

In addition, in the step (S260) of determining whether the altitude or the second rotation angle is continuously varied within a preset range, the processor 110 may determine the altitude of the terminal 100 based on the sensing information of the sensing device 160 or It may be determined whether the second rotation angle continuously changes within a preset range.

In more detail, referring to FIG. 9, when the ship 2 continuously leads due to waves or the like, for example, when the ship 2 repeatedly rises and falls in the direction of the arrow, that is, in the Y-axis direction, the ship (2) The terminal 100 carried by the user who has boarded may also repeat the rising and falling correspondingly. In addition, when the ship 2 flows due to waves or the like, these flows are not very irregular, but the rise and fall are repeated within a certain range and the periodic rise and fall motion is performed with a certain range of errors. Accordingly, the processor 110 may detect the rising and falling motion of the terminal 100 according to the movement of the ship 2 within a preset range based on the sensing information of the sensing device 160.

In addition, in the step of fixing the buoy detection area on the metadata (S270), the processor 110 is based on the continuous change amount of the altitude or the second rotation angle of the terminal 100 according to the rising and falling motion of the ship 2 The buoy detection area 6 on the meta data can be fixed.

For example, referring to FIG. 10, location coordinates of a plurality of buoys may be detected on a map of metadata corresponding to a photographing area. In addition, as illustrated in FIG. 11, when the photographing area is increased according to an elevation elevation or a change in the second rotation angle, a part 6a of the position coordinates of the buoy shown in the photographing area in FIG. 10 is not detected. In addition, as shown in FIG. 12, when the photographing area descends due to a high descent or a change in the second rotation angle, some of the coordinates 6a of the buoys appearing in the photographing area of FIG. The position coordinates of the buoy not shown may be detected (6b).

In this case, the processor 110 calculates an average value from the amount of variation in the Y-axis of the photographing area to fix the buoy detection area on the metadata corresponding to the photographing area (for example, the photographing area of FIG. 10) set as the average value. can do.

In step S280 of detecting the position information of the buoy matching the buoy detection area on the meta data, the processor 110 detects the position information of the buoy matching the fixed buoy detection area, and the buoy image and buoy in the captured image In step S290 of displaying status information of, the processor 110 may display the buoy image and the status information of the buoy in the current captured image.

According to another embodiment of the present invention, the terminal 100 detects the continuous change amount of the altitude or the second rotation angle, and fixes the photographing area with the average value of the change amount. This is a phenomenon in which the marine signage facility in the shooting area constantly changes as the shooting area continuously changes in response to a constant change in altitude or the second rotation angle. It is a phenomenon that users do not want due to flow. Therefore, the present invention allows the user to monitor the marine labeling facilities within the photographing area by fixing the photographing area to prevent such a phenomenon, thereby enabling the accurate and rapid monitoring of the maritime labeling facilities away from the environmental factors of the sea.

The buoy monitoring method using augmented reality according to another embodiment of the present invention may further include a message display step for a buoy located within a preset range from the current location.

In detail, the processor 110 corresponds to a photographing area among coordinate information of buoys within a preset distance from the current position coordinates of the terminal 100 in the metadata read from the memory 120 in step S160 described above. Coordinate information of the buoys may be detected, and proximity message information of the buoys for the buoys may be displayed on the display device 130.

Even if the buoy is close to the ship, the buoy adjacent to the ship may not be recognized depending on the height of the sea or waves, the shape of the ship, and the area around the area where the user is currently located. Therefore, the embodiment of the present invention enables the user to quickly grasp the information about the buoy close to the ship by detecting the location information of the buoy close to the ship and displaying the proximity message information even within the metadata read by the processor 110.

delete

The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and can be recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention or may be known and usable by those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. medium), and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes produced by a compiler, but also high-level language codes executable by a computer using an interpreter or the like. The hardware device can be changed to one or more software modules to perform the processing according to the present invention, and vice versa.

The specific implementations described in the present invention are exemplary embodiments, and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings are illustrative examples of functional connections and / or physical or circuit connections. It can be represented as a connection, or circuit connections. In addition, unless specifically mentioned, such as “essential”, “importantly”, etc., it may not be a necessary component for application of the present invention.

In addition, the detailed description of the present invention has been described with reference to preferred embodiments of the present invention. And it will be understood that various modifications and changes can be made to the present invention without departing from the technical scope. Therefore, the technical scope of the present invention is not limited to the contents described in the detailed description of the specification, but should be determined by the claims.

Claims (7)

Detecting a current position coordinate of the terminal;
Determining whether the terminal is located at a reference angle;
Inducing the terminal to be positioned at the reference angle by displaying an arrow-shaped image on the terminal when the terminal is not positioned at the reference angle;
The terminal positioned at the reference angle is rotated by a first rotation angle about a first axis perpendicular to the sea level, and rotated by a second rotation angle about a second axis horizontal to the sea level and perpendicular to the first axis. To do;
Detecting information about the first rotation angle, information about the second rotation angle, and altitude information of the terminal;
Detecting location information of the terminal including the current location coordinates, information about the first rotation angle, information about the second rotation angle, and altitude information of the terminal;
Determining whether the altitude of the terminal derived based on the second rotation angle or the altitude information of the terminal continuously changes within a preset altitude range;
If it is determined that the altitude of the terminal continuously changes within the preset altitude range, detecting a fixed area for buoy detection based on the second rotation angle or the average value of the altitude change;
Displaying an image photographed by the terminal on the terminal;
Reading metadata stored in advance in the terminal based on the current location coordinates of the terminal included in the location information of the terminal; And
Based on the read metadata, at least one augmented reality (AR) image representing at least one buoy located in the captured image and at least one buoy located in the captured image And superimposing the information on the image and displaying it on the terminal.
The location information of the terminal is detected based on the GNSS satellite,
The size of the displayed at least one augmented reality image is determined based on the distance between each of the at least one buoy located in the captured image and the terminal,
The status information includes at least one of the location information of the buoy, the date of installation of the buoy, the type of the buoy, the painting information of the buoy, and information about the flashing method of the lighting device of the buoy,
When a part of at least one buoy located in the captured image is located within a preset range from the terminal, a message for displaying that at least one buoy within the preset range is close is displayed on the terminal. The buoy monitoring method using augmented reality. delete delete delete According to claim 1,
The terminal is characterized in that located on the ship, the buoy monitoring method using augmented reality. delete At least one memory for storing instructions; And
In a terminal comprising at least one processor,
The instructions are executable by the processor to cause the processor to perform operations,
The above actions are:
Detects the coordinates of the current location of the terminal,
It is determined whether the terminal is located at a reference angle,
When the terminal is not located at the reference angle, the terminal is displayed so as to be positioned at the reference angle by displaying an arrow-shaped image on the terminal,
The terminal positioned at the reference angle is rotated by a first rotation angle about a first axis perpendicular to the sea level, and rotated by a second rotation angle about a second axis horizontal to the sea level and perpendicular to the first axis. and,
Detecting information about the first rotation angle, information about the second rotation angle, and altitude information of the terminal,
Detecting location information of the terminal including the current location coordinate, information about the first rotation angle, information about the second rotation angle, and altitude information of the terminal,
It is determined whether or not the altitude of the terminal derived based on the second rotation angle or the altitude information of the terminal continuously changes within a preset altitude range,
When it is determined that the altitude of the terminal continuously changes within the preset altitude range, a fixed area for detecting a buoy is detected based on the second rotation angle or the average value of the altitude change amount,
Displaying the image taken by the terminal on the terminal,
Based on the current position coordinates of the terminal included in the location information of the terminal, read metadata stored in advance in the terminal,
Based on the read metadata, at least one augmented reality image representing at least one buoy located in the captured image and at least one buoy positioned in the captured image are superimposed on the image. It includes the step of displaying on the terminal,
The location information of the terminal is detected based on the GNSS satellite,
The size of the displayed at least one augmented reality image is determined based on the distance between each of the at least one buoy located in the captured image and the terminal,
The status information includes at least one of the location information of the buoy, the date of installation of the buoy, the type of the buoy, painting information of the buoy, and information about the flashing method of the lighting device of the buoy,
When a part of at least one buoy located in the captured image is located within a predetermined range from the terminal, a message indicating that at least one buoy within the preset range is close is displayed on the terminal. The buoy monitoring terminal using augmented reality.

Images