KR20190080244A - Device and method for providing augmented reality user interface - Google Patents

Abstract

The present invention relates to a system and a method for providing augmented reality. The method for providing augmented reality comprises the following steps: (a) photographing an image including an object; (b) recognizing the object included in the photographed image; (c) receiving information on the object; (d) calculating a remaining lifespan of the object based on the received information; (e) determining a necessity of replacing the object based on the remaining lifespan; (f) generating an augmented reality interface as to whether the object needs to be replaced when there is the necessity of replacing the object; and (g) displaying the image on a screen by overlapping the augmented reality interface with the photographed image.

Description

Technical Field [0001] The present invention relates to augmented reality providing system and method,

The present invention relates to augmented reality providing system and method. Specifically, the present invention relates to an augmented reality providing system and method for determining necessity of replacing a device being photographed, and providing information on a replaceable model of the device.

Augmented reality (AR) is a computer graphics technique that combines virtual objects or information into a real environment to make them look like objects in the original environment.

The augmented reality has a characteristic that it can reinforce and provide additional information that is difficult to obtain only in the real world by synthesizing a virtual object on the basis of the real world, unlike the existing virtual reality which only deals with a virtual space and objects. Because of this feature, the augmented reality can be applied to various real environments unlike existing virtual reality, which can be applied only to fields such as games. Especially, augmented reality is attracting attention as next generation display technology suitable for ubiquitous environment.

Research on augmented reality technology has been continued since the start of research on augmented reality technology. Recently, augmented reality technology has been widely used in location based services with the proliferation of smart phones.

Generally, the augmented reality technology displays information about the device currently being photographed on a preview screen in a menu form. At this time, the preview screen can display the status information of the device being photographed in 2D or 3D form.

However, in the case of the conventional method for providing an augmented reality, only the same augmented reality interface is provided without considering the remaining lifetime and operation state of the device being photographed. Therefore, the conventional method of providing an augmented reality has a problem that it is difficult for the user to grasp the information because the related information is not provided even when the device being taken is old and needs to be replaced or an operation error occurs and replacement is required .

An object of the present invention is to provide a system and method for providing an augmented reality that determines necessity of replacement of a device based on state information of a device being photographed and provides information on an alternative model to the augmented reality.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention which are not mentioned can be understood by the following description and more clearly understood by the embodiments of the present invention. It will also be readily apparent that the objects and advantages of the invention may be realized and attained by means of the instrumentalities and combinations particularly pointed out in the appended claims.

According to another aspect of the present invention, there is provided an augmented reality providing method including: (a) capturing an image including an object; (b) recognizing an object included in the captured image; (D) calculating the remaining lifetime of the object based on the received information, (e) determining the need to replace the object based on the remaining lifetime, (F) generating an augmented reality interface with respect to whether or not the object needs to be replaced if there is a necessity of replacing the augmented reality interface, and displaying (g) overlapping the augmented reality interface on the captured image and displaying on the screen.

In addition, the step (d) of calculating the remaining lifetime of the object may include calculating the remaining lifetime by comparing the lifetime of the object with the recommended lifetime of the object.

The step (d) of calculating the remaining lifetime of the object may include calculating the remaining lifetime by determining whether each component included in the object operates normally based on the received information of the object .

The information on the object may include a wear state, a fusing degree, a fuse state, an operation state of the switch, an operation frequency of the switch, an operation state of the breaker, or an operation frequency of the breaker, .

In addition, the step (e) of determining necessity of replacing the object may include determining that there is a necessity of replacing the object if the remaining lifetime is less than a preset reference value.

The method may further include displaying an augmented reality interface for the replaceable model of the object.

In addition, the step of displaying the augmented reality interface for the replaceable model of the object may include displaying the augmented reality image for the model of the same type as the object or the model compatible with the object, overlaid on the object .

In addition, the step of displaying the augmented reality interface for the replaceable model of the object may include the step of, if the plurality of replaceable models is a plurality, adding a list to the replaceable model based on the sorting criterion selected by the user And displaying it with a realistic menu.

Further, the sorting criterion may include compatibility with the object, a manufacturing date, an expected life span, a price, or a product review.

In addition, the step (b) of recognizing an object included in the photographed image may include the step of recognizing the object included in the image using the position information of the image, the marker of the object or the markerless information of the object ≪ / RTI >

The step (b) or (f) may be performed in the server, and the remaining steps may be performed in the user terminal.

According to an aspect of the present invention, there is provided an augmented reality providing system including a camera unit for capturing an image including an object, an object recognition unit for recognizing the object included in the image, information about the object, An augmented reality processing unit for generating an augmented reality interface based on the augmented reality information, a display unit for overlapping and displaying the augmented reality interface on the photographed image, And a controller for calculating a remaining lifetime of the object based on the information about the object and determining necessity of replacing the object based on the calculated remaining lifetime.

In addition, the controller may request information on an alternative model of the object to a server connected to the user terminal, and may receive the augmented reality interface and display the augmented reality interface on the display unit.

Also, the controller may calculate the remaining lifetime by comparing the year of manufacture of the object and the average lifetime of the object, or by determining whether each component of the object is normal or not based on the information about the object .

Also, the controller may display an augmented reality image for a model of the same type as the object or a model compatible with the object, so as to overlap the object.

The mobile terminal may further include a user terminal including the camera unit, the communication unit, the display unit, and the controller, and a server including the object recognition unit and the augmented reality processing unit.

The system and method for providing an augmented reality according to the present invention provide information on a replaceable model according to the remaining lifetime and operation state of the device being photographed so that the user can grasp the necessity of replacement of the device at a glance. This allows the user to intuitively and easily grasp the need for replacement of the device and information about the replaceable model.

Further, the system and method for providing an augmented reality according to the present invention can easily grasp the remaining lifetime of a device being photographed and whether an abnormal operation has occurred, thereby shortening the time required for managing a plurality of devices, have. In addition, since a quick work process can be performed within the same time, the working efficiency can be improved.

In addition, the system and method for providing an augmented reality of the present invention can provide the user with information on compatible devices with the device, so that the user can easily obtain information on the replaceable model without performing any search, It can be used as a publicity measure, so that the needs of the user and the needs of the seller can be satisfied at the same time.

The above and other objects, features and advantages of the present invention will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: FIG.

1 is a diagram illustrating an augmented reality providing system according to an embodiment of the present invention.
2 is a block diagram for explaining an augmented reality providing apparatus according to an embodiment of the present invention.
3 is a diagram illustrating an operation method of an augmented reality providing apparatus according to an embodiment of the present invention.
4 is a flowchart illustrating a method of providing an augmented reality according to an embodiment of the present invention.
5 is a flowchart for explaining an example of the operation of the life judgment algorithm of FIG.
6 and 7 are views for explaining an example of the operation of the augmented reality providing method according to an embodiment of the present invention.

The above and other objects, features, and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings, which are not intended to limit the scope of the present invention. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to denote the same or similar elements.

Hereinafter, a system and method for providing an augmented reality according to the present invention will be described with reference to Figs. 1 to 7.

1 is a diagram illustrating an augmented reality providing system according to an embodiment of the present invention.

Referring to FIG. 1, an augmented reality providing system according to an embodiment of the present invention includes a user terminal 100, a target device 200, and a server 300.

The user terminal 100 may operate as an augmented reality providing apparatus. The user terminal 100 photographs the target device 200 and receives object information and operation state information of the target device 200 from the server 300.

For example, the user terminal 100 receives object information including information on a manufacturing time, an installation time, a product name, a product number, a product type, an internal component, a recommended service life, etc. of the target device 200 being photographed can do. Also, the user terminal 100 may receive status information including information on the operation power amount, temperature, voltage, current, etc. of the target device 200 being photographed.

The user terminal 100 determines necessity of replacing the target device 200 based on object information and status information of the target device 200. [ If there is a need to replace the target device 200, the user terminal 100 may generate an augmented reality interface based on information on the replaceable model of the target device 200 and display the generated augmented reality interface on the screen.

At this time, the user terminal 100 can determine whether the target device 200 has an abnormal operation and determine the remaining service life of the target device 200 based on the status information of the target device 200. The user terminal 100 can receive the augmented reality information for the replaceable model of the target device 200 from the server 300 and generate an augmented reality interface based on the augmented reality information and display the augmented reality interface on the screen. Here, the augmented reality information includes the augmented reality-related data for the replaceable model of the target device 200. [

The augmented reality information can be modeled and rendered as a 2D or 3D image, and the rendered image is converted into an augmented reality interface and displayed in a preview form on the screen of the user terminal 100. [

In addition, the user terminal 100 may directly receive the augmented reality interface of the substitutable model for the target device 200 to be photographed from the server 300, and may display the augmented reality interface directly on the screen.

The user terminal 100 may be a mobile-based device, a wearable device (e.g., a head mounted device (HMD), a video see-through device, An optical see-through device), a screen-based device, and a projector-based device.

Also, the user terminal 100 may be a computer, a UMPC (Ultra Mobile PC), a workstation, a netbook, a PDA (Personal Digital Assistants), a portable computer, a web tablet, a wireless mobile phone, a mobile phone, a smart phone, an e-book, a portable multimedia player (PMP), and the like. However, the present invention is not limited thereto.

The target device 200 may be various electronic devices. For example, the target device 200 may be an automatic controller, industrial device, or switchboard. In this case, the target device 200 may include a circuit breaker, a relay, a communication link device, a watt-hour meter, a sensor unit, an HMI (Human Machine Interface) The overall operating state and remaining service life of the target device 200 can be determined.

As another example, the target device 200 may be an object of measurement of Automatic Meter Reading (AMR). Generally, the AMR refers to converting measurement results from an object at a remote location to an electrical signal, transmitting the data through the communication network, and processing the data in the computer.

The server of the AMR system can receive a plurality of data (e.g., data) through a data communication network (e.g., a backbone network, a private network, and an Internet network) provided to perform data communication based on a Transmission Control Protocol / Internet Protocol Collecting device. The server transmits and receives data in real time to and from the data collection device through the data communication network, and it is possible to monitor the power status of each measurement object in real time.

In addition, the server receives meter reading data of each measurement object from the data collection device in real time. Based on the meter reading data, the power state of each measurement object, for example, whether it is a power failure, a short circuit, or a discharge, whether it is fused, whether or not the bypass switch is operated, Here, the meter reading data may include all data capable of measuring the power quality of all the measurement objects allocated to one data centralizing apparatus.

The server 300 includes a database for storing various kinds of information about the target device 200. For example, the server 300 may store the object information about the target device 200, the status information of the target device 200, the information about the components included in the target device 200, Augmented reality related data can be stored.

The server 300 may operate as a data collection device or server of an AMR system. That is, the server 300 can receive and store the meter reading data for the target device 200 in real time. In this way, the server 300 can monitor and remotely control the operation status of the target device 200. However, the present invention is not limited thereto.

2 is a block diagram for explaining an augmented reality providing apparatus according to an embodiment of the present invention.

2, a user terminal 100 as an augmented reality providing apparatus according to an exemplary embodiment of the present invention includes a control unit 110, a camera unit 120, an object recognition unit 130, a position measurement unit 140, An augmented reality processing unit 150, a communication unit 160, a sensor unit 165, a memory unit 170, a display unit 180, and an interface unit 190. However, the components of the user terminal 100 shown in FIG. 2 are not essential, so that the user terminal 100 having more or fewer components can be implemented.

The control unit 110 may control the overall operation of the components included in the user terminal 100. The control unit 110 may execute an operating system (OS) and one or more software applications executed on the operating system. In addition, the control unit 350 can access, store, manipulate, process, and generate data in response to execution of the software. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media. The detailed operation of the control unit 110 will be described below.

The camera unit 120 is a device for capturing an image, and the captured image is transmitted to the control unit 110. The image photographed by the camera unit 120 may be displayed on the display unit 180 in a real-time preview format.

The object recognition unit 130 recognizes an object included in the image photographed by the camera unit 120. The object recognition method used in the object recognition unit 130 includes a marker based recognition method for recognizing an object using a marker included in an image and a markerless recognition method for recognizing an actual object registered in advance instead of a physical marker.

Specifically, the marker-based recognition method is a method of recognizing an object based on a marker such as a QR code, a data matrix (DataMatrix), or a 2D barcode. Since the marker-based recognition method utilizes a special type of marker that the system understands in advance, the operation speed and the recognition rate are high.

The markerless recognition method is divided into a natural image-based method and an environmental recognition-based method. The natural image-based method analyzes a large number of feature points existing in a general plane image, and uses the analyzed feature points as invisible markers. The natural image-based method uses a technique of extracting a feature point existing in a natural image and a technique of effectively converting the feature point into a descriptor for describing the feature point. The environment recognition based method is a technique of recognizing the environment itself as a three-dimensional map without a special type of marker or a pre-input image. The environment recognition based method can be a mapping technique for realizing a three-dimensional environment map by analyzing data from a camera or an RGBD sensor and a camera tracking technique for estimating the position of the current camera on the created map have.

The object recognition unit 130 can recognize the objects included in the photographed image by using the marker-based recognition method and the markerless recognition method described above. At this time, the object recognition unit 130 can recognize the object included in the image based on the position information on which the image was captured, the marker for the object, or the markerless information on the object. However, the method of recognizing the object by the object recognizing unit 130 is not limited to this, and various newly developed methods can be applied.

However, the operation of the object recognition unit 130 may be performed by a separate external device (e.g., the server 300), and only the result data on the execution result may be received and used by the user terminal 100 .

The position measuring unit 140 measures the current position of the user terminal 100 and the position of the object to be photographed. In more detail, the position measuring unit 140 may measure the current position of the user terminal 100 and the position of the object photographed by the user terminal 100 using the local position tracking technique. Local location tracking techniques may be used in accordance with the manner in which they are used, such as time of flight (TOF), spatial scan, inertial sensing, mechanical linkage, phase difference sensing, (direct field sensing), and the above techniques may be used in combination to increase the accuracy.

The location measurement unit 140 can perform location measurement using a location based service using near field communication (NFC). For example, the location based service may use Zigbee, iBeacon technology. The location based service can be used mainly in indoor environment where GPS can not be used and the relative position can be grasped through the signal strength of each node. In addition, the position measuring unit 140 can increase the accuracy of position measurement using various sensors (inertial measurement sensor, altimeter, compass) mounted on the user terminal 100. [

The augmented reality processing unit 150 may generate the augmented reality interface provided to the user based on the augmented reality information. The augmented reality information is raw data used for modeling and rendering objects to be photographed. The augmented reality information may be stored in the database of the server 300 or may be stored in the memory unit 170 of the user terminal 100 and used.

The augmented reality processing unit 150 can use a modeling technique for accurately expressing an actual object virtually on the basis of the augmented reality information and a rendering technique for realistically expressing it. The modeling technique is a technique of synthesizing the 2D image input by the video by reflecting the depth information. The modeling technique enables various applications that can not be achieved in the real space by performing object transformation, color change, and the like as the user intends. Rendering technology is a technology that enables the user to feel the vividness in the augmented reality interface by realistically expressing the virtual model by adding lighting effect, shadow effect, brightness change, etc. to the created virtual models.

However, the operation of the augmented reality processing unit 150 may be performed by a separate external device (for example, the server 300), and only the result data on the execution result may be received and used by the user terminal 100 .

The communication unit 160 exchanges data with the server 300 that stores and manages the augmented reality database or the target device 200 to be imaged. At this time, the augmented reality database is a database storing attribute information of augmented reality objects and augmented reality objects.

The communication unit 160 may include an electronic device, a module, or a program that exchanges data with other terminal devices by wire or wireless. The communication protocol used by the communication unit 160 includes, for example, CDMA (Code Division Multiple Access) communication, WCDMA (Wideband Code Division Multiple Access) communication, or broadband wireless communication. However, the present invention is not limited thereto.

The sensor unit 165 is a device for detecting information required for the augmented reality or context information that is further applied to the augmented reality. The sensor unit 165 may include a temperature sensor, a humidity sensor, a position sensor, a motion sensor, and the like. Here, a GPS sensor for sensing a GPS signal may be used as the position detection sensor, and a gyroscope sensor or an acceleration sensor may be used as the motion detection sensor. In addition, the sensor unit 165 may measure the current position, the movement, the tilt, the direction of the user terminal 100, the distance to the object to be photographed, the surrounding sound, brightness, and the like.

The memory unit 170 stores an operating system, an application program, and storage data (telephone number, SMS message, compressed image file, moving image, etc.) for controlling the overall operation of the user terminal 100. Also, the memory unit 170 may store the augmented reality object corresponding to the object to be imaged, the attribute information of the augmented reality object, and the state information. That is, the memory unit 170 stores the object to be imaged and the state information and the augmented reality information about the components included in the object.

The memory unit 170 may include a random access memory (RAM) or a read only memory (ROM) and a static storage device for storing static information and instructions. The memory unit 170 may be a volatile memory unit or a non-volatile memory unit. The memory portion 170 may also be another type of computer readable medium, such as a magnetic or optical disk.

The display unit 180 displays status information (or an indicator) generated during operation of the user terminal 100, limited numbers and characters, a moving picture, and a still picture. Also, the display unit 180 may display an image received through the camera unit 120 and display an augmented reality interface corresponding to the object to be photographed.

The interface unit 190 can receive input from the user. The interface unit 190 may include an input device such as a button-type input unit, a touch screen, a keyboard, a mouse, or the like. For example, the touch screen may be formed on the upper surface of the display unit 180 and receive the input through the user's touch. However, the present invention is not limited thereto.

3 is a diagram illustrating an operation method of an augmented reality providing apparatus according to an embodiment of the present invention.

3, the user terminal 100, which is an augmented reality providing apparatus according to an exemplary embodiment of the present invention, photographs the target device 200 and acquires an augmented reality interface On the display unit 180.

Here, the target device 200 may include a plurality of target devices 210 and 220. For example, the first and second target devices 210 and 220 may be any one of an automatic controller, an industrial device, or an electric switchboard. However, the present invention is not limited thereto.

The server 300 interworks with the target device 200 to store various data for the target device 200. [ The server 300 includes an object information DB 310, a state information DB 320, and an augmented reality DB 330. Hereinafter, the target device 200 will be described in combination with the term " object "

The object information DB 310 includes various information about the object. For example, the object information DB 310 stores the product information (product name, product number, product type, installation timing, management method, internal components, etc.) of the object, location information where the object is installed, And the like.

The state information DB 320 includes state information on the operation of the object. For example, the status information DB 320 may include information on the amount of power of each object, temperature information, voltage information, or current information. The status information of each object can be updated in real time and can be used as part of the AMR.

The augmented reality DB 330 stores augmented reality information, which is raw data used for modeling and rendering an object. Also, the augmented reality DB 330 stores the augmented reality interface provided to the user based on the augmented reality information. That is, the augmented reality DB 330 stores both the augmented reality information before modeling and rendering of the object, the augmented reality interface after modeling and rendering, and provides the necessary data according to a request of the user terminal 100 .

The user terminal 100 recognizes the target device 200 as an object included in the photographed image and receives the augmented reality information or the augmented reality interface for the target device 200 from the server 300 and displays it on the screen. At this time, the user terminal 100 displays different augmented reality interfaces on the display unit 180 depending on whether the target device 200 needs to be replaced or not. For example, when there is a need to replace the target device 200, the user terminal 100 displays the augmented reality interface for the replaceable model on the display unit 180. [

A detailed description of the augmented reality providing method performed by the user terminal 100 will be described later.

4 is a flowchart illustrating a method of providing an augmented reality according to an embodiment of the present invention.

Referring to FIG. 4, an augmented reality providing method according to an embodiment of the present invention first captures an image including an object in a camera unit 120 of a user terminal 100 (S110).

Then, the object recognition unit 130 recognizes an object included in the photographed image (S120).

Then, the control unit 110 receives object information and status information about the object through the communication unit 160 (S130). The control unit 110 may receive the object information and the status information about the recognized object from the server 300. Here, the object information may include information on the manufacturing time of the object, the installation timing, the product name, the product number, the product type, the internal components, the recommended service life, and the like. The state information may include an object's voltage, current, amount of power, temperature, normal operation of the components included in the object, and the like.

Then, the control unit 110 determines necessity of object replacement based on the received object information and status information.

First, the control unit 110 performs a life judgment algorithm for the object (S140).

At this time, the life estimation algorithm calculates the remaining life of the object based on the object information and the state information of the object.

For example, the life estimation algorithm can calculate the remaining lifetime by comparing the lifetime of the object with the recommended lifetime of the object.

As another example, the life estimation algorithm can calculate the remaining service life by determining whether each component included in the object is operating normally. At this time, the life judgment algorithm can determine whether each component is operating normally based on the state information of the object. Specifically, the life judgment algorithm determines the degree of wear (for example, the degree of wear on the surface where the components are brought into contact with each other by heat), the wear state of each component included in the object (for example, The remaining lifetime of the object can be calculated in consideration of the fuse state (for example, the degree of fusion of the fuse), the operation state and number of operations of the bypass switch, or the operation state and number of operations of the circuit breaker. However, the present invention is not limited thereto. .

Then, the control unit 110 determines whether the object needs to be replaced (S150). Specifically, if the remaining lifetime of the object is less than a preset reference value, the user terminal 100 determines that there is a need to replace the object. At this time, the predetermined reference value may be a period required for replacing the object (for example, construction period, component order period). For example, if the preset reference value is 6 months and the remaining lifetime of the object is 4 months, the user terminal 100 may determine that there is a need to replace the object.

Then, if it is determined that there is a need to replace the object, the user terminal 100 requests the server 300 for the augmented reality information for the new model to replace the object (S160). That is, the user terminal 100 requests the server 300 for information on the replaceable model of the object. Here, the replaceable model of the object means a model that is the same type as the object or a model that is compatible with the object.

Then, the controller 110 converts the received augmented reality information into an augmented reality interface and displays it on the display unit 180 (S170). Here, displaying the augmented reality interface for the replaceable model of the object may include overlapping and displaying an augmented reality image for the model that is compatible with the model or object of the object. However, the present invention is not limited to this, and the controller 110 may display the replaceable model of the object as a list-type augmented reality menu.

Specifically, when there are a plurality of replaceable models, the controller 110 may display a list of replaceable models sorted based on the compatibility with the object or the year of manufacture with an augmented reality menu. For example, the control unit 110 may convert the list of the most recently manufactured products into augmented reality interface in the order of high compatibility with the object, and display the converted list on the display unit 180. [

In addition, the augmented reality processing unit 150 of the user terminal 100 may additionally provide the augmented reality interface with effects such as object deformation or color change by reflecting preset values set by the user.

Accordingly, the user terminal 100 can output the changed augmented reality interface according to the user's request by reflecting the set value of the user. That is, the user terminal 100 can use various augmented reality interfaces provided by the server 300 as well as various expression effects desired by the user on the augmented reality interface.

If it is determined that there is a need to replace an object, the user terminal 100 directly requests the server 300 for an augmented reality interface for a new model to replace the object . In this case, the server 300 provides the augmented reality interface for the replaceable model of the object stored in the augmented reality DB 330 to the user terminal 100. [ Here, the augmented reality interface may include a 2D or 3D image for an object previously modeled and rendered.

Then, the control unit 110 displays the received augmented reality interface on the display unit 180. [ At this time, the user terminal 100 can directly output the augmented reality interface received from the server 300 to the display unit 180. [

For example, the server 300 may perform the operations of the object recognition unit 130 and the augmented reality processing unit 150 of the user terminal 100 described above. Then, the user terminal 100 can receive and use the resultant data of the operation performed from the server 300. [ Specifically, the user terminal 100 transmits the image captured by the camera unit 120 to the server 300, the server 300 recognizes the object included in the received image, and the augmented reality An interface can be created and provided to the user terminal 100.

In this case, the user terminal 100 directly outputs the data received from the server 300 (i.e., the augmented reality interface) to the display unit 180. [ Therefore, the operation amount of the control unit 110 is reduced, so that the operation speed can be increased.

5 is a flowchart for explaining an example of the operation of the life judgment algorithm of FIG.

5, the lifetime determining algorithm of the augmented reality providing method according to an exemplary embodiment of the present invention may be performed in the controller 110 or the server 300 of the user terminal 100. In the server 300, When the life judgment algorithm is performed, the user terminal 100 can use the result data of the life judgment algorithm received from the server 300. [

First, the life judgment algorithm determines whether the use period of the photographed object is within the recommended use life (S241). The life estimation algorithm can calculate the recommended life span based on statistical data on the average life span of objects and similar products by comparing the manufacture date of the object with the current date to calculate the usage period.

Then, the life determination algorithm determines whether the object is operating normally based on the state information of the object (S243). Specifically, the life estimation algorithm can determine whether the object is operating normally by considering the operation state of each component included in the object.

Then, the life estimation algorithm predicts the remaining life of the photographed object based on the object information and the state information (S245).

Specifically, the lifetime determining algorithm can calculate the remaining lifetime of the photographed object by comparing the lifetime of the object with the recommended lifetime of the object. In addition, the life estimation algorithm can calculate the remaining lifetime of the object by considering the wear state, fusion degree, fuse state, bypass switch operation state, or breaker operation state of each component included in the object.

Then, the life judgment algorithm judges the necessity of replacing the photographed object (S247).

For example, if the lifetime of the photographed object exceeds the recommended lifetime, the lifetime determination algorithm determines that the object needs to be replaced. Also, if the remaining lifetime of the object is smaller than a preset reference value, it can be determined that the object needs to be replaced. Here, the preset reference value may be a period required for replacing the object (for example, construction period, part order period).

Then, the augmented reality providing method according to an embodiment of the present invention performs the above-described steps S150 through S170. Hereinafter, the contents overlapping with those described above will be omitted.

6 and 7 are views for explaining an example of the operation of the augmented reality providing method according to an embodiment of the present invention.

6 and 7, in the method of providing an augmented reality according to an embodiment of the present invention, the user terminal 100 photographs the target device 200, and the target device 200 acquires the first and second objects (210, 220). The user terminal 100 may display an augmented reality interface (e.g., M11, M41) for the need for replacement of the first and second objects 210, 220 on the object.

Specifically, the user terminal 100 requests object information and status information about the first and second objects 210 and 220 to the server 300. Then, the control unit 110 of the user terminal 100 performs a life determination algorithm based on the received object information and state information. Since the lifetime determining algorithm has been described with reference to FIG. 5, detailed description thereof will be omitted.

The user terminal 100 can determine whether the first and second objects 210 and 220 are in normal operation, remaining life, and necessity of replacement through a life determination algorithm. The user terminal 100 then displays on the screen the augmented reality interface M11 for the need to replace the first object 210 and the augmented reality interface M41 for the need to replace the second object 220 .

At this time, in the case of an object requiring replacement, an augmented reality interface M21 indicating detailed information about the object is displayed on the screen. For example, if it is determined that there is a need to replace the first object 210, next to the first object 210, a manufacturing date, an expected remaining life, state information (e.g., high breaker contact wear) The augmented reality interface M21 including the model information M31 may be displayed. However, this is merely one example, and the present invention is not limited thereto.

7, when the user selects the menu of the replaceable model information M31, the user terminal 100 displays the augmented reality interface M32 including the information on the replaceable model on the screen .

At this time, when there are a plurality of replaceable models, the user terminal 100 transmits a list of replaceable models arranged based on compatibility with an object, manufacture date, life expectancy, price, As shown in FIG. For example, the user terminal 100 may convert the list of the most recently manufactured products into the augmented reality interface M32 in order of high compatibility with the objects, and display them on the display unit 180 . However, the sorting order is not limited to the above-described order, and the sorting order may be arranged to be provided to the user through a preset reference. Also, although not explicitly shown in the drawing, the user can select the sort order on the augmented reality interface M32, and the augmented reality interface M32 can rearrange the list according to the selected sort order.

For example, the models of S1 to S4 may be displayed in the augmented reality interface M32 in the order of high compatibility with the first object 210. [ If the user selects the S1 model, the augmented reality image (for example, 'Brand New Model') for the S1 model may overlap and be displayed on the first object 210. [ Through this, the user can intuitively perceive the state when the substitutable model of the object is actually installed through the augmented reality.

Therefore, the system and method for providing an augmented reality according to the present invention can provide the user with information necessary for the user by intuitively providing the user with information about the compatible device with the object to be photographed. In addition, the seller can market or promote his or her product through the augmented reality providing method. That is, the system and method for providing an augmented reality of the present invention can satisfy the needs of the user and the needs of the seller at the same time.

In addition, the system and method for providing an augmented reality of the present invention provides information on a replaceable model according to the remaining lifetime and operation state of the device being photographed, so that the user can grasp the necessity of replacement of the device at a glance. This allows the user to intuitively and easily grasp the need for replacement of the device and information about the replaceable model. Accordingly, the present invention can shorten the time required for managing a plurality of devices, thereby enhancing the convenience of the user, and enabling quick work processing within the same time, thereby improving work efficiency.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, You will understand. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

100: User terminal
200: Target device
300: server

Claims (16)

A method for providing an augmented reality performed by a user terminal or a server,
(A) photographing an image including an object;
(B) recognizing an object included in the photographed image;
(C) receiving information about the object;
(D) calculating a remaining lifetime of the object based on the received information;
(E) determining necessity of replacement of the object based on the remaining service life;
(F) generating an augmented reality interface for whether or not the object needs to be replaced if there is a necessity of replacing the object; And
(G) overlapping the augmented reality interface with the photographed image and displaying it on the screen
Method of providing augmented reality.
The method according to claim 1,
(D) calculating a remaining lifetime of the object,
And calculating the remaining lifetime by comparing a use period of the object with a recommended use lifetime of the object.
The method according to claim 1,
(D) calculating a remaining lifetime of the object,
Determining whether each component included in the object is operating normally based on the received information of the object, and calculating the remaining lifetime.
The method according to claim 1,
The information on the object may include information on a wear state of the respective components included in the object, a fusion state, a fuse state, an operation state of the switch, an operation count of the switch, an operation state of the breaker, Delivery method.
5. The method of claim 4,
(E) determining whether the object needs to be replaced,
And determining that there is a need to replace the object if the remaining lifetime is less than a preset reference value.
The method according to claim 1,
And displaying an augmented reality interface for a replaceable model of the object.
The method according to claim 6,
Wherein the step of displaying the augmented reality interface for the replaceable model of the object includes displaying an augmented reality image for a model of the same type as the object or a model compatible with the object, .
8. The method of claim 7,
Wherein displaying the augmented reality interface for the replaceable model of the object comprises:
And displaying a list of the replaceable models as an augmented reality menu based on an alignment criterion pre-entered or selected by the user when the replaceable model is a plurality.
9. The method of claim 8,
Wherein the sorting criterion includes compatibility with the object, a manufacturing date, an expected life span, a price, or a product review.
The method according to claim 1,
(B) recognizing an object included in the photographed image,
And recognizing the object included in the image by using the position information of the image, the marker of the object, or the markerless information of the object.
The method according to claim 1,
Wherein the step (b) or (f) is performed in the server, and the remaining steps are performed in the user terminal.
A camera unit for capturing an image including an object;
An object recognition unit for recognizing the object included in the image;
A communication unit for receiving the information about the object and the augmented reality information for the replaceable model of the object;
An augmented reality processing unit for generating an augmented reality interface based on the augmented reality information;
A display unit for overlapping and displaying the augmented reality interface on the photographed image; And
And a controller for calculating a remaining lifetime of the object based on the information about the object and determining necessity of replacing the object based on the calculated remaining lifetime
Augmented reality providing system.
13. The method of claim 12,
Wherein the control unit requests information on the substitutable model of the object from the server and receives the augmented reality interface and displays it on the display unit
Augmented reality providing system.
14. The method of claim 13,
Wherein the controller compares the year of manufacture of the object with the average lifetime of the object or determines whether the object is normal or not based on information about the object and calculates the remaining lifetime system.
13. The method of claim 12,
Wherein the control unit displays an augmented reality image for a model of the same type as the object or a model compatible with the object so as to overlap the object.
13. The method of claim 12,
A user terminal including the camera unit, the communication unit, the display unit, and the control unit;
And a server including the object recognition unit and the augmented reality processing unit.
Augmented reality providing system.

Images