KR102268798B1 - System and method for providing augmented reality user interface - Google Patents

Abstract

The present invention relates to a system and method for providing augmented reality. The augmented reality providing method includes: detecting a combination of a docking station and a user terminal; receiving ID information of the docking station coupled to the user terminal; location information of the docking station in a database based on the ID information extracting, receiving angle information of the docking station, selecting a display target object to be displayed on the display unit of the user terminal based on the location information and the angle information, and for the display target object and providing augmented reality related data to the user terminal.

Description

System and method for providing augmented reality user interface

The present invention relates to a system and method for providing augmented reality. Specifically, it relates to an augmented reality providing system and method for providing augmented reality information to a user terminal coupled to a docking station.

Augmented reality (AR) refers to a computer graphics technique in which a virtual object or information is synthesized in a real environment to appear as an object existing in the original environment.

Unlike conventional virtual reality, which targets only virtual space and objects, augmented reality has a feature of providing additional information that is difficult to obtain only in the real world by synthesizing virtual objects on the basis of the real world. Because of these characteristics, augmented reality can be applied to a variety of real environments, unlike existing virtual reality, which can be applied only to fields such as games. In particular, augmented reality is spotlighted as a next-generation display technology suitable for a ubiquitous environment.

Since research on augmented reality technology has been started, research to utilize augmented reality technology has continued in various fields. Recently, with the spread of smartphones, augmented reality technology has been widely used for location-based services.

In general, augmented reality technology displays information about a device currently being photographed on a preview screen in the form of a menu. In this case, the preview screen may display status information of the device being photographed in 2D or 3D form.

However, in the case of the conventional augmented reality providing device, since the user has to hold the user terminal in his hand to take a picture, the augmented reality image on the screen continues to shake due to the user's hand shaking, thereby causing dizziness.

In addition, in order to display the augmented reality image, it is necessary to measure the change in the position of the user terminal and continuously recognize the object in the captured image, so there is a problem in that the amount of computation required for operation increases, which slows down the implementation of augmented reality. there was.

An object of the present invention is to provide an augmented reality providing system and method capable of displaying an augmented reality interface for a viewing area viewed from a specific location on the user terminal by coupling the user terminal to a docking station installed at a specific location.

In addition, the present invention provides an augmented reality providing system and method for selecting a target object located within a camera field of view of a user terminal by using location information and angle information of a docking station, and displaying an augmented reality interface for the target object on the user terminal aims to provide

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention not mentioned may be understood by the following description, and will be more clearly understood by the examples of the present invention. Moreover, it will be readily apparent that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

Augmented reality providing method according to an embodiment of the present invention for achieving the above object, the step of detecting a combination of a docking station and a user terminal, receiving the ID information of the docking station coupled to the user terminal, Extracting the location information of the docking station from a database based on the ID information, receiving the angle information of the docking station, an indication to be displayed on the display unit of the user terminal based on the location information and the angle information selecting a target object; and providing augmented reality related data for the display target object to the user terminal.

In addition, the selecting of the display target object to be displayed on the display unit of the user terminal may include selecting an object adjacent to the docking station based on location information of the docking station, and based on angle information of the docking station , selecting a display target object located within an angle of view of a camera unit included in the user terminal from among the adjacent objects.

In addition, the selecting of the display target object to be displayed on the display unit of the user terminal, receiving the magnification information of the camera unit, and further comprising selecting a display target object to be displayed on the user terminal based on the magnification information can do.

In addition, when the angle information of the docking station is changed, reselecting a display target object to be displayed on the display unit of the user terminal based on the changed angle information; Augmented reality for the reselected display target object The method may further include updating information to the user terminal.

Also, the angle information may include information on an angle, an angle of view, and a magnification of a camera of the user terminal coupled to the docking station.

In addition, the position information and the angle information may be composed of three-dimensional coordinate information consisting of an X-axis, a Y-axis, and a Z-axis.

Augmented reality providing method according to another embodiment of the present invention for achieving the above object, receiving the ID information of the docking station coupled to a user terminal, transmitting the ID information to a server connected to the docking station Step, receiving data related to the docking station from the server and storing it in a memory, receiving location information and angle information of the docking station, based on the data stored in the memory, the location information and the angle information Selecting a display target object to be displayed on the display unit of the user terminal, generating an augmented reality interface for the display target object, and overlapping the augmented reality interface on the captured image and displaying on the screen include

In addition, the generating of the augmented reality interface for the display target object includes: receiving state information on the display target object; determining the state of the display target object by using the augmented reality related data and the state information It may include generating the augmented reality interface to display.

In addition, the receiving of the data related to the docking station from the server and storing the data in a memory may include storing the data in a memory of the docking station or a memory of the user terminal.

Augmented reality providing system according to an embodiment of the present invention for achieving the above object, a user terminal for displaying an augmented reality interface on a display unit, a docking station to which the user terminal is coupled, and the user terminal coupled to the and a server that selects a display target object to be displayed on the user terminal based on location information and angle information of the docking station, and provides augmented reality related data for the display target object to the user terminal.

In addition, the docking station may include a coupling table, an angle adjusting unit for adjusting the angle of the coupling table, an angle sensor measuring the position of the angle adjusting unit, and a memory unit for storing ID information.

Also, the server may receive the ID information of the docking station, and calculate the location information of the docking station based on the received ID information.

In addition, the server selects an object adjacent to the docking station based on the location information, and selects a display target object located within an angle of view of a camera unit included in the user terminal from among the adjacent objects based on the angle information. can

Also, the server may receive magnification information of a camera unit included in the user terminal, and select a display target object to be displayed on the user terminal based on the location information, the angle information, and the magnification information.

In addition, when the angle information of the docking station is changed, the server reselects a display target object to be displayed on the display unit of the user terminal based on the changed angle information, Augmented reality information may be updated in the user terminal.

Also, the angle information may include information on an angle, an angle of view, and a magnification of a camera of the user terminal coupled to the docking station.

Also, the user terminal may generate an augmented reality interface based on the augmented reality related data received from the server, and display the augmented reality interface on a display unit.

In addition, the user terminal receives the state information on the display target object, generates the augmented reality interface for displaying the state of the display target object by using the augmented reality related data and the state information, the display unit can be displayed

The augmented reality providing system and method of the present invention may provide a shake-free augmented reality image to the user by coupling the user terminal to a docking station installed at a specific location. Through this, it is possible to prevent the user from feeling dizzy while using the augmented reality.

In addition, the augmented reality providing system and method of the present invention may provide an augmented reality interface for a target object located within a camera field of view of a user terminal by using position information and angle information of a docking station. That is, the present invention can improve the augmented reality implementation speed by omitting the object recognition process required for augmented reality. Through this, the user terminal may provide a faster and smoother augmented reality image to the user, and the reaction speed to the user's input may be increased.

In addition to the above-described effects, the specific effects of the present invention will be described together while describing specific details for carrying out the invention below.

1 is a diagram illustrating an augmented reality providing system according to some embodiments of the present invention.
FIG. 2 is a diagram for explaining the user terminal of FIG. 1 .
FIG. 3 is a diagram for explaining a coupling structure of the user terminal and the docking station of FIG. 1 .
FIG. 4 is a diagram for explaining an operation method of the augmented reality providing system of FIG. 1 .
5 is a flowchart illustrating a method for providing augmented reality according to an embodiment of the present invention.
6 is a flowchart illustrating a method for providing augmented reality according to another embodiment of the present invention.
7 is a flowchart illustrating a method for providing augmented reality according to another embodiment of the present invention.
8 and 9 are diagrams for explaining an example of an augmented reality providing system according to some embodiments of the present invention.

The above-described objects, features and advantages will be described below in detail with reference to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the present invention pertains will be able to easily implement the technical idea of the present invention. In describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to indicate the same or similar components.

Hereinafter, an augmented reality providing system and method according to the present invention will be described with reference to FIGS. 1 to 9 .

1 is a diagram illustrating an augmented reality providing system according to an embodiment of the present invention.

Referring to FIG. 1 , the augmented reality providing system according to an embodiment of the present invention includes a user terminal 100 , a docking station 200 , a server 300 , and a target device 400 .

The user terminal 100 operates as an augmented reality display device. The user terminal 100 is coupled to the docking station 200 . The user terminal 100 may receive augmented reality related information from the server 300 through the docking station 200 and display it on the screen.

The user terminal 100 includes a tablet, a wireless phone, a mobile phone, a smart phone, an e-book, a portable multimedia player (PMP), a netbook ( net-book), PDA (Personal Digital Assistants), portable computer, etc. may be a device capable of transmitting and receiving information in a wireless environment. However, the present invention is not limited thereto.

The docking station 200 may be fixedly installed at a specific location and coupled to the user terminal 100 . A plurality of docking stations 200 may be installed, and unique ID information is assigned to each. The docking station 200 may exchange data with the server 300 through wireless or wired communication. Also, the docking station 200 may transmit the augmented reality related information received from the server 300 to the user terminal 100 .

The server 300 includes a database that stores various information about the target device 400 . The server 300 may store location information on the plurality of docking stations 200 and location information on the target device 400 adjacent to the docking station 200 . The server 300 may identify each docking station 200 based on ID information of the docking station 200 .

Also, the server 300 stores augmented reality related information on the target device 400 . If ID information is received from the docking station 200 coupled to the user terminal 100 , the server 300 attaches augmented reality related information about the target device 400 adjacent to the docking station 200 to the corresponding docking station. It may be provided to the user terminal 100 through the station 200 .

In this case, the server 300 selects a display target object to be displayed on the user terminal 100 in consideration of angle information of the docking station 200 . Subsequently, the server 300 may provide augmented reality related information on the selected display target object to the user terminal 100 .

Through this, the user terminal 100 takes a picture of the target device 400 and does not go through a process of identifying the target device 400 in the captured image, but is attached to the docking station 200 to which the user terminal 100 is coupled. Augmented reality related information on the adjacent target device 400 may be received from the server 300 and displayed on the screen.

Here, the augmented reality-related information is a concept including augmented reality information and an augmented reality interface. Augmented reality information is raw data used to model and render a photographed object. Augmented reality information may be modeled and rendered as a 2D or 3D image, and the rendered image is converted into an augmented reality interface and displayed in a preview form on the screen of the user terminal 100 .

The user terminal 100 receives augmented reality information on the target device 400 from the server 300 , generates an augmented reality interface based on the augmented reality information, and displays it on the screen.

Additionally, the user terminal 100 may directly receive the augmented reality interface for the target device 400 from the server 300 and display it on the screen as it is.

The target device 400 may be various electronic devices. For example, the target device 400 may be an automatic control device, an industrial device, or a switchboard. In this case, the target device 400 may include a circuit breaker, a relay, a communication link device, a watt-hour meter, a sensor unit, a human machine interface (HMI), etc. as components, and the operating state of the components included in the target device 400 . Accordingly, the overall operating state of the target device 400 may be determined.

The operating state of the target device 400 may be updated and stored in the server 300 in real time. The user terminal 100 may provide state information about the target device 400 to the user in the form of augmented reality.

In this case, the target device 400 may be a measurement target of a remote meter reading system (Automatic Meter Reading; hereinafter, AMR). In general, the remote meter reading system (AMR) refers to converting a measurement result into an electrical signal from a measurement target at a distant point, transmitting data through a communication network, and processing the data in a computer.

The server of the remote meter reading system (AMR) uses a data communication network (eg, backbone network, private network, Internet network included) provided to perform data communication based on TCP/IP (Transmission Control Protocol/Internet Protocol). interconnected with the collection device. The server performs data transmission and reception in real time with the data collection device through the data communication network, and it is possible to monitor the power state of each measurement target in real time.

In addition, the server receives the meter reading data of each measurement target from the data collection device in real time. Based on the meter reading data, the power status of each measurement target, for example, power failure, conduction, and leakage status is analyzed and provided to the manager.

Here, the meter reading data may be the sum total of the amount of power used by all measurement targets allocated to one data concentrator device, the amount of power used by each measurement target, the instantaneous power amount, etc. In addition, all data that can determine the power quality of each measurement target may be included.

The server 300 may operate as a data collection device or server of the remote meter reading system (AMR). That is, the server 300 may receive and store the meter reading data for the target device 400 in real time. Through this, the server 300 may monitor and remotely control the operating state of the target device 400 . However, the present invention is not limited thereto.

FIG. 2 is a diagram for explaining the user terminal of FIG. 1 .

Referring to FIG. 2 , the user terminal 100 includes a control unit 110 , a camera unit 120 , a sensor unit 130 , a communication unit 140 , an augmented reality processing unit 150 , a memory unit 160 , and a display unit. 170 and an interface unit 180 . However, since the components of the user terminal 100 shown in FIG. 2 are not essential, the user terminal 100 having more components or fewer components may be implemented.

The controller 110 may control the overall operation of the components included in the user terminal 100 . The controller 110 may execute an operating system (OS) and one or more software applications executed on the operating system. In addition, the controller 350 may access, store, manipulate, process, and generate data in response to the execution of the software.

The camera unit 120 is a device for capturing an image, and the captured image is transmitted to the controller 110 . The image captured by the camera unit 120 may be displayed on the display unit 170 in the form of a real-time preview.

The sensor unit 130 is a device for measuring the surrounding environment information of the user terminal 100 . The sensor unit 130 may include a temperature sensor, a humidity sensor, a position sensor, a motion sensor, and the like. Here, a GPS sensor that detects a GPS signal may be used as the location sensor, and a gyroscope sensor or an acceleration sensor may be used as the motion sensor. In addition, the sensor unit 130 may measure the current position, movement, tilt, direction, distance to the object to be photographed, ambient sound, brightness, and the like of the user terminal 100 .

The communication unit 140 exchanges data with the docking station 200 . In addition, the communication unit 140 exchanges data with the server 300 that stores and manages the augmented reality database or the target device 400 that is a photographing target. In this case, the augmented reality database is an augmented reality object and a database storing attribute information of the augmented reality object.

The communication unit 140 may include an electronic device, a module, or a program that transmits and receives data to and from another terminal device in a wired or wireless manner. The communication protocol used by the communication unit 140 includes, for example, CDMA (Code Division Multiple Access) communication, WCDMA (Wideband Code Division Multiple Access) communication, or broadband wireless communication. However, the present invention is not limited thereto.

The augmented reality processing unit 150 may generate an augmented reality interface provided to the user based on the augmented reality information. The augmented reality information may be stored in the database of the server 300 or stored in the memory 160 of the user terminal 100 and used.

The augmented reality processing unit 150 may use a modeling technique for accurately expressing a real object in virtual reality based on the augmented reality information, and a rendering technique for realistically expressing the real object. The modeling technique is a technique for synthesizing a 2D image input as a video by reflecting depth information. Modeling technology enables various applications that cannot be achieved in real space by performing object transformation and color change according to the user's intention. Rendering technology is a technology that allows a user to feel a sense of vitality in an augmented reality interface by realistically expressing the virtual model by adding lighting effects, shadow effects, and brightness changes to the generated virtual models.

The memory unit 160 stores an operating system for controlling the overall operation of the user terminal 100 , an application program, and data for storage (location information, compressed image file, video, etc.). In addition, the memory unit 160 may store the augmented reality object corresponding to the object to be photographed, location information and state information of the augmented reality object. That is, the memory unit 160 may store the display target object, state information of the display target object, and augmented reality information.

The memory unit 160 may include a random access memory (RAM) or read only memory (ROM) and a static storage device for storing static information and instructions. The memory unit 160 may be a volatile memory unit or a non-volatile memory unit. Memory unit 160 may also be another type of computer readable medium, such as a magnetic or optical disk.

The display unit 170 displays status information (or indicators) generated during the operation of the user terminal 100 , limited numbers and characters, a moving picture, a still picture, and the like. Also, the display unit 170 may display an image received through the camera unit 120 , and may display an augmented reality interface for a display target object located within a field of view of the camera unit 120 .

The interface unit 180 may receive an input from a user. The interface unit 180 may include an input device such as a button type input unit, a touch screen, a keyboard, or a mouse. For example, the touch screen may be formed on the upper surface of the display unit 170 and may receive an input through a user's touch. However, the present invention is not limited thereto.

FIG. 3 is a view for explaining a coupling structure of the user terminal and the docking station of FIG. 1 . FIG. 4 is a diagram for explaining an operation method of the augmented reality providing system of FIG. 1 .

Referring to FIG. 3 , the docking station 200 included in the augmented reality providing system of the present invention includes a coupling table 210 to which the user terminal 100 is coupled, a support 230 fixed to the bottom surface, and a coupling table ( and a connector 220 connecting the 210 and the support 230 .

In this case, the angle of the coupling table 210 is adjusted with respect to the connection part 220 , and the height of the connection part 220 may be adjusted.

One surface of the coupling table 210 includes a cradle 214 on which the user terminal 100 is mounted, and a connection terminal 212 physically or electrically connected to the user terminal 100 . A connection port 216 and a speaker unit 218 may be formed on the side of the coupling table 210 . However, this is only an example, and the present invention is not limited thereto.

An angle adjustment unit 225 is formed between the coupling table 210 and the connection unit 220 . The angle adjusting unit 225 may be rotated up, down, left and right, and a position sensor unit ( 230 in FIG. 4 ) is provided inside the angle adjusting unit 225 to measure the position information and angle information of the coupling table 210 . can do. In this case, the position information and the angle information may be composed of three-dimensional coordinate information consisting of an X-axis, a Y-axis, and a Z-axis, respectively.

For example, the location information may be composed of three-dimensional coordinate information based on the ground to indicate the location of the user terminal 100 coupled to the docking station 200 . The angle information may be used to calculate the target device 400 included in the range of the angle of view of the camera unit 120 included in the user terminal 100 by measuring the degree of inclination of the combination table 210 . Also, the angle information may include information on an angle, an angle of view, and a magnification of the camera unit 120 included in the user terminal 100 coupled to the docking station 200 .

Here, the angle means the shooting angle that occurs when the camera is positioned toward the subject to take a picture, the angle of view means the field of view of the scene captured by the camera, and the magnification is the ratio of the size of the image to the size of the object. it means.

Referring to FIG. 4 , in the augmented reality providing system according to an embodiment of the present invention, the user terminal 100 displays the augmented reality interface for the target device 400 located within the angle of view of the camera unit 120 on the display unit 170 . displayed on the

Here, even if the camera unit 120 does not substantially operate, the augmented reality providing system displays the augmented reality interface of the target device 400 located in the direction opposite to the display unit 170 of the user terminal 100 on the display unit ( 170) can be displayed.

The docking station 200 is coupled to the user terminal 100 . The docking station 200, the position angle sensor unit 235 for measuring the position information and angle information of the coupling table 210 to which the user terminal 100 is coupled, the user terminal 100 and the server 300 and data and a communication unit 240 for exchanging , and a memory unit 250 for storing ID information of the docking station 200 .

The server 300 works with the docking station 200 and the target device 400 to store data about the docking station 200 and the target device 400 . In addition, the server 300 provides augmented reality related information on the target device 400 to the docking station 200 or the user terminal 100 .

The server 300 includes an object information DB 310 , a state information DB 320 , a location information DB 330 , and an augmented reality DB 340 . Hereinafter, the target device 400 will be described by mixing the term 'object'.

The object information DB 310 includes various information about objects. For example, it may include information such as product information (product name, product number, product type, installation time, management method, etc.) of the object, location information where the object is installed, and a circuit diagram related to the object.

The state information DB 320 includes operation information of the object. For example, the state information DB 320 may include power amount information, temperature information, voltage information, or current information of each object. The state information of each object may be updated in real time, and may be used as a part of a remote meter reading system (AMR).

The location information DB 320 includes location information of the object. For example, the location information DB 320 may be composed of three-dimensional spatial coordinates of the object (eg, three-dimensional coordinate information including the X-axis, Y-axis, and Z-axis). Also, the location information DB 320 includes location information for each docking station 200 . In this case, location information on the docking station 200 may be stored together with ID information allocated to each docking station 200 . Similarly, the location information for the docking station 200 may be configured as three-dimensional spatial coordinates.

The augmented reality DB 340 stores augmented reality information that is raw data used to model and render an object. In addition, the augmented reality DB 340 stores the augmented reality interface provided to the user based on the augmented reality information. That is, the augmented reality DB 340 stores both the augmented reality information before modeling and rendering of the object, and the augmented reality interface after modeling and rendering, and may provide necessary data according to the request of the user terminal 100 . .

The server 300 uses the position information and angle information of the docking station 200 measured by the position and angle sensor unit 235 to display a target located within the angle of view of the camera unit 120 included in the user terminal 100 . object can be selected. Through this, the server 300 may select target devices 400 to be displayed on the display unit 170 of the user terminal 100 . Subsequently, the server 300 may provide augmented reality related information on the selected target device 400 to the user terminal 100 through the docking station 200 . The user terminal 100 may display the received augmented reality related information on the display unit 170 .

Specifically, in an embodiment of the present invention, the user terminal 100 receives the augmented reality information and location information for the selected target device 400 from the server 300, and augments the augmented reality information and the location information based on the location information. A reality interface can be created and displayed on the screen.

In another embodiment of the present invention, the user terminal 100 may directly receive the augmented reality interface for the target device 400 from the server 300 and display it on the display unit 170 as it is.

The augmented reality providing system of the present invention may omit the object recognition process of recognizing the target device 400 included in the image by analyzing the image captured by the camera unit 120 . That is, the augmented reality providing system of the present invention can improve the augmented reality implementation speed by omitting the object recognition process required for augmented reality. Through this, the user terminal may provide a faster and smoother augmented reality image to the user, and the reaction speed to the user's input may be increased.

In another embodiment of the present invention, the user terminal 100 may receive data related to the docking station 200 coupled to the user terminal 100 from a server. Here, data related to the docking station 200 includes location information of the docking station 200, product information of an object located in the vicinity of the docking station 200, location information of an object, state information of an object, augmented reality information of an object may include. That is, the data related to the docking station 200 may be data stored in the object information DB 310 , the state information DB 320 , the location information DB 330 , and the augmented reality DB 340 of the server 300 .

Data related to the docking station 200 may be stored and used in the memory of the user terminal 100 or the memory of the docking station 200 .

Subsequently, the user terminal 100 uses the received data related to the docking station 200 and the location information and angle information of the docking station 200 to display the object to be displayed on the display unit 170 of the user terminal 100 . can be selected

Next, the user terminal 100 may generate an augmented reality interface for the display target object, overlap the generated augmented reality interface with the captured image, and display it on the display unit 170 . However, this is only an example, and the present invention is not limited thereto.

Hereinafter, an augmented reality providing method according to some embodiments of the present invention performed in the augmented reality providing system of the present invention will be described in detail.

5 is a flowchart illustrating a method for providing augmented reality according to an embodiment of the present invention.

Referring to FIG. 5 , in the augmented reality providing method according to an embodiment of the present invention, a coupling between the docking station 200 and the user terminal 100 is first detected ( S110 ). For example, the docking station 200 may detect the coupling with the user terminal 100 through data exchange between the connection terminal 212 and the coupled user terminal 100 .

Then, the server 300 receives the ID information of the docking station 200 coupled to the user terminal 100 (S120). The docking station 200 coupled with the user terminal 100 transmits its ID information stored in the memory unit 250 to the server 300 .

Then, the server 300 extracts the location information of the docking station 200 based on the received ID information (S130). In this case, the location information of the docking station 200 may be 3D coordinate information stored in advance in the server 300 .

Then, the server 300 receives the angle information of the docking station 200 coupled to the user terminal 100 (S140). Here, the angle information may be the degree of inclination of the coupling table 210 measured by the position angle sensor unit 235 . Also, the angle information may include information on an angle, an angle of view, and a magnification of the camera unit 120 included in the user terminal 100 coupled to the docking station 200 .

Next, the server 300 selects a display target object to be displayed in the user terminal 100 based on the received location information and angle information of the docking station 200 ( S150 ). At this time, the server 300 compares the pre-input location information of the target device 400 with the location information of the docking station 200, and should be displayed on the display unit 170 of the user terminal 100 in consideration of the angle information. You can select the display target object to be displayed.

For example, the server 300 extracts the target devices 400 adjacent to the docking station 200 , and considers angle information of the docking station 200 , and the range of the angle of view of the camera unit 120 of the user terminal 100 . It is possible to select the target device 400 included in the .

Next, the server 300 or the user terminal 100 generates an augmented reality interface for the selected display target object (S160). In this case, the user terminal 100 may receive the augmented reality information on the display target object from the server 300 to generate the augmented reality interface. Also, the user terminal 100 may directly receive the augmented reality interface for the display target object from the server 300 .

Then, the user terminal 100 displays the augmented reality interface for the display target object on the display unit 170 (S170).

In this case, in the augmented reality providing method of the present invention, by coupling the user terminal 100 to the docking station 200 , it is possible to provide a shake-free augmented reality image to the user. In addition, the augmented reality providing method of the present invention may provide an augmented reality interface for a target object located within the camera field of view of the user terminal 100 by using the position information and angle information of the docking station 200 .

Accordingly, the augmented reality providing method of the present invention can omit the object recognition process required for augmented reality, so that the augmented reality implementation speed can be improved, and the reaction speed to the user's input can be increased.

6 is a flowchart illustrating a method for providing augmented reality according to another embodiment of the present invention. Hereinafter, differences from the augmented reality providing method described above with reference to FIG. 5 will be mainly described.

Referring to FIG. 6 , following step S140 described above, the server 300 receives magnification information of the camera unit 120 included in the user terminal 100 ( S245 ). Here, the magnification means a ratio of the size of the image to the size of the object, and the magnification information includes data on the current magnification of the camera unit 120 .

Next, the server 300 selects a display target object to be displayed in the user terminal 100 based on the received location information, angle information, and magnification information of the docking station 200 ( S250 ).

At this time, the server 300 compares the pre-input location information of the target device 400 with the location information of the docking station 200, and should be displayed on the display unit 170 of the user terminal 100 in consideration of the angle information. Select the object to be displayed. Subsequently, the server 300 may determine the size of the display target object to be displayed on the display unit 170 of the user terminal 100 based on the magnification information.

For example, the server 300 extracts the target devices 400 adjacent to the docking station 200 , and considers angle information of the docking station 200 , and the range of the angle of view of the camera unit 120 of the user terminal 100 . It is possible to select the target device 400 included in the . Then, when the magnification information is applied, the server 300 may determine the size of the displayed target object and the ratio between the target object and the background.

Next, the user terminal 100 receives the state information on the display target object from the server 300 (S255). The state information may include voltage, current, amount of power, temperature of the display target object, whether a component included in the object operates normally, and the like.

Next, the user terminal 100 receives augmented reality information on the display target object from the server 300 (S260). The augmented reality information corresponds to raw data for generating an augmented reality interface of the display target object.

Next, the user terminal 100 generates an augmented reality interface based on the received augmented reality information and state information (S265). The user terminal 100 may generate an augmented reality interface by receiving augmented reality information on the display target object from the server 300 . In this case, the generated augmented reality interface may include state information of the display target object (eg, an operating state, an amount of operating power, an amount of power change according to time, an operating time, temperature, voltage, and current measurements, etc.).

In this case, the augmented reality image of the display target object may be formed in different sizes, shapes, and colors according to state information of the display target object. Specifically, the augmented reality processing unit 150 of the user terminal 100 may additionally apply effects such as object transformation or color change to the augmented reality interface by reflecting a preset value set by the user.

Accordingly, the user terminal 100 may output the augmented reality interface changed to suit the user's request by reflecting the user's setting value. That is, the user terminal 100 may implement various expression effects desired by the user on the augmented reality interface as well as using the augmented reality interface provided by the server 300 .

Then, the user terminal 100 displays the augmented reality interface for the display target object on the display unit 170 (S270).

7 is a flowchart illustrating a method for providing augmented reality according to another embodiment of the present invention. Hereinafter, differences from the augmented reality providing method described above with reference to FIG. 5 will be mainly described.

Referring to FIG. 7 , following step S170 described above, the docking station 200 determines whether to change the angle ( S310 ). The position angle sensor unit 235 included in the docking station 200 detects whether the angle of the docking station 200 with respect to the coupling table 210 is changed.

Subsequently, when the docking station 200 changes the angle information, the server 300 receives the angle information for the changed angle (S320).

Next, the server 300 reselects the display target object based on the previously received location information of the docking station 200 and the changed angle information ( S330 ). Also, the server 300 may reselect the display target object by referring to the location information and the changed angle information as well as the magnification information. A description of this has been described with reference to FIG. 6 , and detailed description thereof will be omitted.

Next, the server 300 or the user terminal 100 generates an augmented reality interface for the reselected display target object ( S340 ). In this case, the user terminal 100 may receive the updated augmented reality information on the display target object from the server 300 to generate the augmented reality interface.

Also, the user terminal 100 may directly receive the changed augmented reality interface for the display target object from the server 300 . At this time, since the user terminal 100 outputs the augmented reality interface received from the server 300 to the display unit 170 as it is, the amount of computation of the control unit 110 is reduced, so that the operating speed of the augmented reality providing method can be increased. .

Next, the user terminal 100 displays the augmented reality interface for the display target object on the display unit 170 (S350).

8 and 9 are diagrams for explaining an example of an augmented reality providing system according to some embodiments of the present invention.

8 and 9 , the augmented reality providing system according to some embodiments of the present invention includes a plurality of docking stations D1 , D2 , and D3 that can be combined with the user terminal 100 . Although not clearly shown in the drawings, the augmented reality providing system of the present invention includes a server 300 for controlling a plurality of docking stations (D1, D2, D3).

The user terminal 100 coupled to any one of the plurality of docking stations D1, D2, and D3 displays, on the display unit 170, an augmented reality interface for a target device included in the angle of view of the camera among the plurality of target devices. can Here, the plurality of target devices (eg, A1, A2, A3, B1, B2, C1, C2) may be an automatic control device, an industrial device, a switchboard, or the like.

For example, in the case of the user terminal 100 coupled to the docking station D1, the augmented reality interfaces for the target devices A1, A2, and A3 included in the angle of view of the camera may be displayed on the screen.

In this case, the user terminal 100 does not perform a separate object recognition process.

Specifically, each docking station (D1, D2, D3) has unique ID information. The server 300 provides location information for a plurality of target devices (eg, A1, A2, A3, B1, B2, C1, C2) and location information for each docking station (D1, D2, D3). It can be extracted from a pre-stored database and used. The server 300 receives the ID information of the docking station D1 coupled with the user terminal 100 to extract location information of the docking station D1, and receives angle information from the docking station D1.

Next, the server 300 selects the target devices A1 , A2 , and A3 located in the direction toward which the user terminal 100 is facing based on the location information and the angle information of the docking station D1 as the display target object. Subsequently, the server 300 provides augmented reality related information on the target devices A1, A2, and A3, which are the selected display target objects, to the user terminal 100, and the user terminal 100 displays the received information on the display (170) is indicated.

If the angle information is changed by rotating the binding table of the docking station D1, the server 300 reselects the display target object based on the changed angle information. Then, the server 300 provides the augmented reality related information on the reselected display target object to the user terminal 100 , and displays the information received by the user terminal 100 on the display unit 170 .

At this time, the target object expressed in augmented reality is an object moving on a conveyor belt (R1, R2, R3), production equipment (G1, G2) that processes the object, inspection equipment that inspects an object, automatic control device, or industrial device ( For example, it may be A1, A2, A3, B1, B2, C1, C2) and the like. However, this is only an example, and the present invention is not limited thereto.

The user terminal 100 may be operated to display the augmented reality interface in the other docking stations D2 and D3 in the same manner as described above.

Additionally, the server 300 may extract status information on the target device selected as the display target object and provide the augmented reality interface including status information of the target device to the user terminal 100 . Through this, the user terminal 100 can grasp the operation status and normal operation of the corresponding area (or production line) at a glance, and control a specific target device.

The augmented reality providing system and method of the present invention may provide a shake-free augmented reality image to the user by coupling the user terminal to a docking station installed at a specific location. Through this, it is possible to prevent the user from feeling dizzy while using the augmented reality.

In addition, the augmented reality providing system and method of the present invention may provide an augmented reality interface for a target object located within a camera field of view of a user terminal by using position information and angle information of a docking station. That is, the present invention can improve the augmented reality implementation speed by omitting the object recognition process required for augmented reality. Through this, the user terminal may provide a faster and smoother augmented reality image to the user, and the reaction speed to the user's input may be increased.

Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those of ordinary skill in the art to which the present invention pertains can realize that the present invention can be implemented in other specific forms without changing the technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: user terminal
200: docking station
300: server
400: target device

Claims (18)

As an augmented reality providing method performed in a server connected to a docking station,
detecting a coupling between the docking station and the user terminal;
receiving ID information of the docking station coupled to the user terminal;
extracting location information of the docking station from a database based on the ID information;
receiving angle information of the docking station;
selecting a display target object to be displayed on a display unit of the user terminal based on the location information and the angle information; and
Comprising the step of providing augmented reality related data for the display target object to the user terminal,
The step of selecting a display target object to be displayed on the display unit of the user terminal comprises:
selecting an object adjacent to the docking station based on the location information of the docking station;
Selecting a display target object located within an angle of view of a camera unit included in the user terminal from among the adjacent objects based on angle information of the docking station
How to provide augmented reality.
delete According to claim 1,
The step of selecting a display target object to be displayed on the display unit of the user terminal comprises:
The augmented reality providing method further comprising receiving the magnification information of the camera unit, and selecting a display target object to be displayed on the user terminal based on the magnification information.
According to claim 1,
When the angle information of the docking station is changed, reselecting a display target object to be displayed on the display unit of the user terminal based on the changed angle information;
Augmented reality providing method further comprising the step of updating the augmented reality information on the reselected display target object to the user terminal.
According to claim 1,
The angle information is an augmented reality providing method including information on an angle, an angle of view, and a magnification of a camera of the user terminal coupled to the docking station.
According to claim 1,
The location information and the angle information is an augmented reality providing method consisting of three-dimensional coordinate information consisting of an X-axis, a Y-axis, and a Z-axis.
As an augmented reality providing method performed in a user terminal coupled to a docking station,
transmitting information of the docking station coupled to the user terminal to a server;
receiving data related to the docking station from the server and storing the data in a memory;
receiving position information and angle information of the docking station;
selecting a display target object to be displayed on a display unit of the user terminal based on the data stored in the memory, the location information, and the angle information;
generating an augmented reality interface for the display target object; and
Comprising the step of overlapping and displaying the augmented reality interface on the display unit,
The step of generating an augmented reality interface for the display target object includes:
Receiving state information about the display target object;
and generating the augmented reality interface for displaying the state of the display target object by using the augmented reality related data and the state information.
How to provide augmented reality.
delete 8. The method of claim 7,
The receiving of the data related to the docking station from the server and storing the data in a memory comprises storing the data in a memory of the docking station or a memory of the user terminal.
a user terminal displaying an augmented reality interface on a display unit;
a docking station to which the user terminal is coupled; and
A server that selects a display target object to be displayed on the user terminal based on location information and angle information of the docking station coupled to the user terminal, and provides augmented reality related data for the display target object to the user terminal including,
The server selects an object adjacent to the docking station based on the location information, and selects a display target object located within an angle of view of a camera unit included in the user terminal from among the adjacent objects based on the angle information
Augmented reality providing system.
11. The method of claim 10,
The docking station, an augmented reality providing system comprising a coupling table, an angle adjusting unit for adjusting the angle of the coupling table, an angle sensor for measuring the position of the angle adjusting unit, and a memory unit for storing ID information.
12. The method of claim 11,
The server receives the ID information of the docking station, and the augmented reality providing system for calculating the location information of the docking station based on the received ID information.
delete 11. The method of claim 10,
The server receives magnification information of a camera unit included in the user terminal, and selects a display target object to be displayed on the user terminal based on the location information, the angle information, and the magnification information.
11. The method of claim 10,
When the angle information of the docking station is changed, the server reselects a display target object to be displayed on the display unit of the user terminal based on the changed angle information, and augments reality for the reselected display target object Augmented reality providing system for updating information to the user terminal.
11. The method of claim 10,
The angle information is an augmented reality providing system including information on an angle, an angle of view, and a magnification of a camera of the user terminal coupled to the docking station.
11. The method of claim 10,
The user terminal generates an augmented reality interface based on the augmented reality-related data received from the server, and displays the augmented reality interface on a display unit.
11. The method of claim 10,
The user terminal receives the state information on the display target object, generates the augmented reality interface for displaying the state of the display target object using the augmented reality related data and the state information, and displays it on the display unit Augmented reality providing system.

Images