KR102633001B1 - Method for implementing underground facilities as ar in an offline environment using combined data precessing of qr code and nfc - Google Patents

Abstract

The present invention includes the steps of recognizing the QR code of the Smart Pin with a mobile terminal, the step of reading GIS compressed encrypted data stored in the NFC chip of the Smart Pin by the application of the mobile terminal, and the step of reading the QR code data of the Smart Pin by the application. A QR code and NFC data complex processing method including converting the compressed encrypted data into reprocessed compressed encrypted data, and outputting underground structure AR data with added direction using the reprocessed compressed encrypted data by the application. A method of implementing offline AR using underground facilities is disclosed.

Description

Method for implementing offline AR in underground facilities using QR code and NFC data complex processing { METHOD FOR IMPLEMENTING UNDERGROUND FACILITIES AS AR IN AN OFFLINE ENVIRONMENT USING COMBINED DATA PRECESSING OF QR CODE AND NFC }

The present invention relates to a method of implementing offline AR of underground facilities. More specifically, the present invention relates to a method of implementing offline AR of underground facilities. More specifically, the QR code and NFC data included in the smart pin are processed complexly to provide augmented reality without receiving separate data about underground facilities through the Internet. It's about how it can be implemented.

Recently, various facilities such as telecommunication lines, high-voltage wires, gas pipes, and water pipes are being placed underground for the safety of pedestrians and the cityscape.

Representative facilities that are placed underground in this way include seven major buried pipes, such as water and sewage pipes, gas pipes, and communication pipes. However, although there are many types of buried pipes, it is difficult to determine their precise location because accurate burial data is not secured.

If excavation is performed based on incorrect underground facility information, it may lead to major accidents such as city gas explosions, electric communication pipe fires, ground subsidence due to damage to sewage and water pipes, and heat transmission pipe leaks. Therefore, it is important to accurately determine the location of underground facilities. This is very important.

However, as the parties in charge of management and maintenance of underground facilities are different, numerous markers are installed on the road, damaging the scenery (see Figure 1), and even buried information is inconsistent and inaccurate, so the need to unify it is constantly being raised.

To solve these problems, Republic of Korea Patent No. 10-2131513, 'Smart Safety Management System Using NFCQR' (hereinafter referred to as 'Patent Document 1'), uses a QR code or NFC tag attached to a facility or production facility using a smartphone. Once recognized, you can go to a web page linked to the web system to check the information, history, and maintenance details of the facility or production facility. We present a smart safety management system using NFCQR that can update information in real time on site.

However, information on water, electricity, communications, etc. is important information related to national security, but there is a problem that if all related information is integrated and transmitted and received over the Internet, it is exposed to the risk of hacking.

Republic of Korea Patent No. 10-2131513

The present invention was developed to solve the above problems, and is a method of implementing AR of underground facilities by processing QR code data and NFC data included in a smart pin without the need to receive separate GIS data through the Internet. The purpose is to provide.

In order to solve the above problems, the present invention includes the steps of recognizing the QR code of the Smart Pin with a mobile terminal, the step of reading the GIS compressed encrypted data stored in the NFC chip of the Smart Pin by an application of the mobile terminal, and QR comprising converting the compressed encrypted data into reprocessed compressed encrypted data using the QR code data of the smart pin, and outputting AR data of an underground structure with added direction using the reprocessed compressed encrypted data by the application. We disclose a method of implementing offline AR in underground facilities using a complex processing method of code and NFC data.

According to an embodiment of the present invention, an underground application using a QR code and NFC data complex processing method further includes the step of granting access to the compressed encrypted data to the application of the mobile terminal through a management server. A method of implementing offline AR for facilities is disclosed.

According to one embodiment of the present invention, the compressed encrypted data includes the steps of grouping the digits of the facility data numbers and replacing them with a single character, defining a data field that can unify the number of digits of the data, and Reducing the length of data by deleting duplicate data using a field, removing duplicate data values to specify a fixed digit, grouping the specified fixed digit into two digits, and replacing each grouped number with a single character. We disclose a method of implementing offline AR for underground facilities using a QR code and NFC data complex processing method, which includes the steps of:

According to an embodiment of the present invention, a method of implementing offline AR of underground facilities using a QR code and NFC data complex processing method is disclosed, wherein the single character includes 100 numbers or letters that do not overlap each other.

According to an embodiment of the present invention, the reprocessed compressed encrypted data converts the azimuth information included in the QR code into a compressed encrypted data format and then combines it with the compressed encrypted data transmitted from the NFC to reprocess it into data containing directionality. We disclose a method of implementing offline AR in underground facilities using QR code and NFC data complex processing method.

According to one embodiment of the present invention, the azimuth information is information that finds the true north direction using the spacing and arrangement relationship between three points included in the QR code. A QR code and NFC data complex processing method is used. A method of implementing offline AR using underground facilities is disclosed.

According to the present invention, the risk of hacking can be fundamentally prevented by enabling integrated management of underground facilities in an offline environment based on QR codes and NFC.

In addition, it makes it possible to implement AR at the location where the smart pin is placed without the need to use high-precision GPS, making it possible to check the accurate placement of underground facilities even in situations where GPS is not available, such as indoors.

In addition, information from each site can be utilized without separate equipment, and 3D underground facility information can be viewed intuitively through AR, improving the stability of construction and shortening the construction period.

In addition, it is possible to integrate and manage seven major underground facilities that were managed through different management agencies into one, making it easier to manage management costs and link between agencies.

In addition, by compressing information on multiple underground facilities into one smart pin, it is possible to prevent waste of markers and improve road scenery.

Figure 1 is a photograph of various types of markers installed on conventional roads.
Figure 2 is a conceptual diagram of a method for implementing offline AR in underground facilities using QR code and NFC data complex processing method according to an embodiment of the present invention.
Figure 3 is a diagram showing the structure of a mobile terminal used in an offline AR implementation method for underground facilities according to an embodiment of the present invention.
Figure 4 is a diagram showing the top surface of a smart pin according to an embodiment of the present invention.
Figure 5 is a flowchart of a method for implementing offline AR in underground facilities using QR code and NFC data complex processing method according to an embodiment of the present invention.
Figure 6a is a diagram to explain an example of implementing GIS data stored in NFC in AR without QR code information.
Figure 6b is a diagram to explain an example of reprocessing GIS data stored in NFC using QR code information and implementing it into AR.
Figure 7 is a diagram showing an example of application of offline AR to underground facilities using QR code and NFC data complex processing method.
Figure 8 is a diagram for explaining a method of reprocessing encrypted NFC data by adding direction using QR code data.

Hereinafter, the present invention will be described in more detail with reference to the drawings. In this specification, the same or similar reference numbers are assigned to the same or similar components even in different embodiments, and the description is replaced with the first description. As used herein, singular expressions include plural expressions unless the context clearly dictates otherwise.

In this specification, the same or similar reference numbers are assigned to the same or similar components even in different embodiments, and the description is replaced with the first description. As used herein, singular expressions include plural expressions unless the context clearly dictates otherwise. In addition, the suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of writing the specification, and do not have distinct meanings or roles in themselves.

Figure 2 is a conceptual diagram of a method for implementing offline AR in underground facilities using QR code and NFC data complex processing method according to an embodiment of the present invention.

Referring to Figure 2, the smart pin 200 applied to the present invention includes a head portion 210 and a body portion 220.

The head unit 210 may include a QR code 211, information about underground facilities, and information about the management entity of underground facilities. An NFC chip that stores GIS information may be embedded in the body portion 220.

In the past, each management entity of underground facilities managed separate GIS data, and either viewed documents during construction or used the GIS data by receiving it through a server operated by the management entity. However, there was a problem that each management entity had different data management methods and the data was inaccurate.

In the present invention, standardized GIS data is received from the management entity of underground facilities and stored in an NFC chip. At this time, since the amount of data that can be stored in the NFC chip is limited and data security needs to be strengthened, in the present invention, the data stored in the NFC chip is compressed and encrypted. This will be explained in detail with reference to FIG. 8 below.

The user can use the mobile terminal 100 to recognize the QR code 211 of the smart pin 200, receive underground facility information stored in the NFC chip, and implement the underground facility around the smart pin 200 in AR. .

Figure 3 is a diagram showing the structure of a mobile terminal 100 used in a method of implementing offline AR in underground facilities using a QR code and NFC data complex processing method.

The mobile terminal 100 includes a wireless communication unit 110, an input unit 120, a detection unit 140, an output unit 150, an interface unit 160, a memory 170, a control unit 180, and a power supply unit 190. ), etc. may be included.

The illustrated components are not essential for implementing the mobile terminal, so the mobile terminal described herein may have more or fewer components than those listed above.

More specifically, among the above components, the wireless communication unit 110 is used between the mobile terminal 100 and the server 10, between the mobile terminal 100 and the wireless communication system, and between the mobile terminal 100 and other mobile terminals 100. It may include one or more modules that enable wireless communication between the devices. Additionally, the wireless communication unit 110 may include one or more modules that connect the mobile terminal 100 to one or more networks.

This wireless communication unit 110 may include at least one of a broadcast reception module 111, a mobile communication module 112, a wireless Internet module 113, a short-range communication module 114, and a location information module 115. .

The input unit 120 includes a camera 121 or an image input unit for inputting an image signal, a microphone 122 or an audio input unit for inputting an audio signal, and a user input unit 123 for receiving information from a user, for example. , touch keys, push keys (mechanical keys, etc.).

The sensing unit 140 may include one or more sensors for sensing at least one of information within the mobile terminal, information on the surrounding environment surrounding the mobile terminal, and user information. For example, the sensing unit 140 includes a proximity sensor (141), an illumination sensor (142), a touch sensor, an acceleration sensor, a magnetic sensor, and a gravity sensor. Sensor (G-sensor), gyroscope sensor, motion sensor, RGB sensor, IR sensor (infrared sensor), fingerprint scan sensor, ultrasonic sensor , optical sensors (e.g., cameras (see 121)), microphones (see 122), battery gauges, environmental sensors (e.g., barometers, hygrometers, thermometers, radiation detection sensors, It may include at least one of a heat detection sensor, a gas detection sensor, etc.) and a chemical sensor (e.g., an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the mobile terminal disclosed in this specification can utilize information sensed by at least two of these sensors by combining them.

The output unit 150 is for generating output related to vision, hearing, or tactile sense, and includes at least one of a display unit 151, an audio output unit 152, a haptip module 153, and an optical output unit 154. can do. The display unit 151 can implement a touch screen by forming a layered structure or being integrated with the touch sensor. This touch screen functions as a user input unit 123 that provides an input interface between the mobile terminal 100 and the user, and can simultaneously provide an output interface between the mobile terminal 100 and the user.

The interface unit 160 serves as a passageway for various types of external devices connected to the mobile terminal 100. This interface unit 160 connects devices equipped with a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, and an identification module. It may include at least one of a port, an audio input/output (I/O) port, a video input/output (I/O) port, and an earphone port. In response to an external device being connected to the interface unit 160, the mobile terminal 100 may perform appropriate control related to the connected external device.

Additionally, the memory 170 stores data supporting various functions of the mobile terminal 100. The memory 170 may store a number of application programs (application programs or applications) running on the mobile terminal 100, data for operating the mobile terminal 100, and commands. At least some of these applications may be downloaded from an external server via wireless communication. Meanwhile, the application program may be stored in the memory 170, installed on the mobile terminal 100, and driven by the control unit 180 to perform operations (or functions) of the mobile terminal.

In addition to operations related to the application program, the control unit 180 typically controls the overall operation of the mobile terminal 100. The control unit 180 can provide or process appropriate information or functions to the user by processing signals, data, information, etc. input or output through the components discussed above, or by running an application program stored in the memory 170.

Additionally, the control unit 180 may control at least some of the components discussed above in order to run the application program stored in the memory 170. Furthermore, the control unit 180 may operate at least two of the components included in the mobile terminal 100 in combination with each other in order to run the application program.

The power supply unit 190 receives external power and internal power under the control of the control unit 180 and supplies power to each component included in the mobile terminal 100. This power supply unit 190 includes a battery, and the battery may be a built-in battery or a replaceable battery.

Figure 4 is a diagram showing the top surface of the smart pin 200 according to an embodiment of the present invention.

Referring to FIG. 4, the head portion 210 of the smart pin 200 may include a QR code 211 and underground facility information 212.

The QR code 211 may include data for calculating azimuth or depth of underground facilities when implementing AR. When a user runs the application using a mobile terminal, the azimuth is extracted by calculating the data recognized from the user's location and QR code, and this is used to reassemble the GIS data to implement the underground facility in AR in 3D space.

In the present invention, AR can be implemented in two ways. Depending on need, the two methods can be applied separately or simultaneously.

The first method collects user location information by connecting high-precision GPS equipment.

Analyze the user's direction after removing outliers from GPS data (using statistics and machine learning)

This is a way to express facilities in AR.

The second method is to use QR codes in areas where high-precision GPS equipment cannot be used.

This is a method that can implement AR in (shaded areas, indoors). According to an embodiment of the present invention, the user's direction can be analyzed more accurately by using one QR code outdoors and by using two or more QR codes indoors.

The process of calculating the azimuth angle using QR code data according to an embodiment of the present invention may be performed in the following manner.

First, the method of calculating the azimuth by recognizing a single QR code data uses the characteristic that the QR code consists of square dots at the top left, bottom left, and top right of the QR code to check the reference direction.

In order to check the reference direction using three points, a smart pin can be installed so that the three points of the QR code face a specific direction, or data about the direction can be inserted into the QR code.

When a user implements AR through an application, the application sets the direction that the camera of the user's mobile terminal is facing to the north direction in the program. All AR programs are initially launched in the same way.

The application can use data received through a QR code to check the direction of true north or calculate in which direction and at what angle the ground is tilted. The application can use QR code data to calculate the angle between the true north direction and the direction set as north on AR and use it as the direction angle to place the facility model on AR.

Figure 5 is a flowchart of a method for implementing offline AR in underground facilities using QR code and NFC data complex processing method according to an embodiment of the present invention.

According to an embodiment of the present invention, the method of implementing offline AR in underground facilities includes the steps of recognizing the QR code of the smart pin with a mobile terminal (S100), the step of authorizing the use of NFC data through the management server (S200), and the application A step of reading encrypted GIS data stored in Smart Pin's NFC (S300), a step of the application reprocessing the data read from NFC using data stored in Smart Pin's QR code (S400), and the application adding direction. It may include a step of implementing the GIS data into AR (S500).

In the step of recognizing the QR code of the smart pin with a mobile terminal (S100), the user can acquire the location and direction information for implementing AR by recognizing the QR code displayed on the smart pin using the mobile terminal.

In the step of authorizing the use of NFC data through the management server (S200), the administrator can authorize the use of the encrypted NFC data in the user application.

The management server and the user's mobile terminal can be connected through the Internet, but the data exchanged between the management server and the mobile terminal concerns the authority to use the data of a specific smart pin. In other words, GIS data of underground facilities is not transmitted over the Internet, but is received offline through NFC. Therefore, it is possible to completely block the risk of hacking that may occur while underground facility information is being transmitted over the Internet.

In the drawing, step S200 is shown as being performed after step S100, but step S200 may be performed before step S100. For example, if the administrator pre-authorizes a specific mobile terminal user to use a certain smart pin, the user can use NFC data after scanning the QR code of the smart pin.

In the step (S300) where the application reads the encrypted GIS data stored in the NFC of the smart pin, the data stored in the NFC chip is read.

In order to implement underground facilities with AR, a considerable amount of capacity is used. For example, when a 6m water pipe is laid in a straight section, a total of 4 pipes are included, and the data includes a total of 5 point data. In this case, data of approximately 300 bytes in size must be stored in NFC. Currently, the maximum capacity that can be stored in NFC is about 888 bytes, so a problem arises in that all data from seven major underground facilities cannot be stored in one NFC.

To solve this, data compression technology must be used, but because NFC stores data in text form rather than program files, it is difficult to apply conventional compression technologies such as 'repetition length coding' and 'Huffman coding'.

Specifically, repetition length encoding compresses data using the repetition number of specific characters excluding numbers. However, because underground facility data consists only of numbers, repetition length coding cannot be applied to compress underground facility data.

Huffman encoding compresses frequent characters into short binary codes and less frequent characters into long binary codes, depending on the frequency of character appearance. However, it is difficult to apply the Huffman coding method because it is impossible to specify characters with high frequency of occurrence in data about underground facilities.

Therefore, in one embodiment of the present invention, an underground facility data compression method different from the conventional method was applied.

Data for implementing underground facilities in AR are shown in Table 1 below.

data item explanation x(X) Facility location x (rectangular coordinate system) y(Y) Facility location y (rectangular coordinate system) depth(DEPTH) Facility burial depth (in the case of a pipeline, the depth to the center of the pipeline) elevation(EL) Height above sea level or height above sea level type Types of underground facilities (7 major types of buried pipes, types of obstacles) texture Facility material diameter pipe diameter width facility width height facility height number Facility management number

In the present invention, first, the digits of the facility data numbers are grouped and replaced with a single character (see Table 2).

Then, to compress the data, first define a data field that can unify the number of digits for each data (see Table 3).

Using the data field, duplicate data is deleted to reduce the length of the data, and duplicate primary data values are removed to specify a fixed number of digits (see Table 4).

Then, using a method of compressing data according to each data type, the data is sequentially grouped into two-digit numbers starting from the first digit, and compression is performed by replacing the two-digit number with a single character. For example, referring to Figure 8, it can be seen that the x-coordinate value of the facility (512866.987) has been compressed and encrypted and converted to (AR%$1).

At this time, the 2-digit number is replaced with a total of 100 data from 00 to 99, and the 100 replaced values must not overlap each other. According to an embodiment of the present invention, the substituted value can be defined by constructing a mapping table using extended ASCII code values.

When this compression method is applied to the information on the seven major underground facilities to be stored in NFC, the result is that the total data size is compressed by about 54%. Accordingly, all information on underground facilities can be stored within the NFC capacity of 888 bytes.

Figure 6a is a diagram to explain an example of implementing GIS data stored in NFC in AR without QR code information, and Figure 6b is a diagram illustrating a case in which GIS data stored in NFC is reprocessed using QR code information and implemented in AR. This is a diagram to explain an example, and Figure 7 is a diagram showing an example of application of offline AR to underground facilities using QR code and NFC data complex processing method.

Referring to Figure 6a, the application implements AR with the direction in which the camera of the user's mobile terminal is facing is set to the north direction in the program. Therefore, in environments where ultra-precision GPS cannot be applied, it is difficult to accurately set the direction of underground facilities implemented in AR. If construction is carried out without knowing the exact location and direction of underground facilities, there is a risk of serious accidents.

Referring to Figure 6b, the direction of the underground facility can be specified by reconstructing the AR data using QR data. The QR code includes data for calculating azimuth or depth of underground facilities when implementing AR.

When a user runs the application using a mobile terminal, the azimuth is extracted by calculating the data recognized from the user's location and QR code, and this is used to reassemble the GIS data to implement the underground facility in AR in 3D space.

Referring to Figure 7, you can see the AR screen of the underground facility implemented in this way. Since AR of underground facilities can be accurately implemented even without using ultra-precision GPS, the accurate placement of underground facilities can be confirmed even in areas where GPS use is not possible. Even in areas where GPS use is possible, GPS power can be saved and more accurate AR can be realized. It becomes possible.

Figure 8 is a diagram for explaining a method of reprocessing encrypted NFC data by adding direction using QR code data.

In Figure 8, the x-coordinate value among data items is used as an example. Other data items may be processed in the same way.

Facility managers each have information on the underground facilities they manage. Information held by the manager is stored in the form of GIS data (310). In the present invention, GIS data 310 held by the administrator is converted into compressed encrypted data 320 and stored in Smart Pin.

As shown, the method of generating compressed data is to sequentially group the numbers into two-digit numbers starting from the first digit and perform compression by replacing the two-digit numbers with a single letter.

Specifically, the x-coordinate of the GIS data is 512866.987, but the first two digits 56 are replaced with A, the next two digits 78 are replaced with R, and the next two digits 9 10 are replaced with %. Substitution can be done using a mapping table using extended ASCII code values.

Of the three digits after the decimal point, the first two digits 8 9 are replaced with $, and the next digit 10 is replaced with 1.

When compressed, excluding the '.' that separates the decimal point, it is expressed as a total of 5 digits, resulting in a compression rate of about 55%.

When a user implements AR through an application, the compressed encrypted data 320 stored in the smart pin is transmitted to the mobile terminal through NFC, and at this time, the QR data is synthesized to generate reprocessed compressed encrypted data 330.

The reprocessed compressed encrypted data 330 includes information about the location or direction included in the QR code. The application implements and displays AR using reprocessed compressed encrypted data 330 including directional information.

According to at least one embodiment of the present invention described above, the risk of hacking can be fundamentally prevented by enabling integrated management of underground facilities in an offline environment based on QR code and NFC, and information on each site can be stored separately. It can be used without equipment and allows intuitive viewing of 3D underground facility information through AR, thereby increasing the stability of construction and shortening the construction period. In addition, the seven major underground facilities that were managed through different management agencies can be integrated and managed as one. It facilitates management costs and inter-organizational linkage, and by condensing information on multiple underground facilities into one smart pin, improved effects can be expected compared to the prior technology, such as preventing waste of markers and improving road scenery. .

The method of implementing offline AR in underground facilities using the QR code and NFC data complex processing method described above is not limited to the configuration and method of the embodiments described above, and the embodiments are all of the embodiments so that various modifications can be made. Or, some of them may be selectively combined.

10: First buried pipe
20: Second buried pipe
100: mobile terminal
200: Smart pin
210: head part
211: QR code
212: Underground facility information
220: body part
310: GIS data
320: Compressed encrypted data
330: Reprocessed compressed encrypted data

Claims (6)

Recognizing the QR code of the smart pin with a mobile terminal;
Reading GIS compressed encrypted data stored in the NFC chip of the smart pin by the application of the mobile terminal;
The application converts the QR code data of the smart pin into the compressed encrypted data and converts it into reprocessed compressed encrypted data; and
It includes the step of the application outputting underground structure AR data with added direction using the reprocessed compressed encrypted data,
The method for reprocessing the GIS compressed encrypted data is,
It is processed offline without online access to the server, preventing the user's location information and GIS data from being leaked to the outside.
(a) grouping the digits of each data item of the underground facility and replacing them with a single digit;
(b) defining a data field that unifies the number of digits for each data;
(c) deleting duplicate data using the data field and creating a compressed data field specifying a fixed number of digits;
(d) Group each piece of data in the compressed data field sequentially into two numbers starting from the first digit, then replace the two digits with one character, delete the decimal point, and replace the third digit after the decimal point with one character. steps; and
(e) combining the directional data included in the QR code with the data of the compressed and encrypted NFC chip and replacing it with another character having the same number of digits as the compressed and encrypted data; A method of implementing offline AR in underground facilities using a complex processing method of QR code and NFC data, comprising: According to paragraph 1,
An underground facility offline AR implementation method using a QR code and NFC data complex processing method, further comprising granting access to the compressed encrypted data to the application of the mobile terminal through a management server. delete According to paragraph 1,
The single character above is,
A method of implementing offline AR in underground facilities using a complex processing method of QR code and NFC data, which is characterized by containing 100 numbers or letters that do not overlap each other. According to clause 4,
The reprocessed compressed encrypted data is,
An underground method using a complex processing method of QR code and NFC data, characterized in that the azimuth information contained in the QR code is converted into a compressed encrypted data format and then combined with the compressed encrypted data received from the NFC and reprocessed into data containing directionality. How to implement offline AR in facilities. According to clause 5,
The azimuth information is,
An offline AR implementation method for underground facilities using a complex processing method of QR code and NFC data, characterized in that it is information that finds the true north direction using the spacing and arrangement relationship between the three dots included in the QR code.

Images