KR20140136088A - Method for providing contents using Augmented Reality, system and apparatus thereof - Google Patents

Abstract

The present invention relates to a content providing method using an augmented reality (AR) and to a system and an apparatus for the same. Especially, the present invention can be used as a means capable of decoding an encoded item based on a technology for matching augmented reality feature points and can be used in both indoor and outdoor spaces based on the confirmation of a field visit and the decoding of information through a matching of augmented reality feature points by providing a compensation service depending on whether the encoded item is decoded or not. The present invention is able to provide a service ceaselessly even without a location measurement of a GPS positioning, a base station, Wi-Fi, an RFID, etc. The present invention is also able to perform a travel course guide by planning a coupon/information decoding order. The present invention can be also used as a marketing means for the management of a business district, the position of an opened store, etc. by grasping a motion line by collecting user logs.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a content providing method using Augmented Reality,

The present invention relates to a content providing method using an augmented reality, a system and an apparatus therefor, and more particularly, to a method and apparatus for using augmented reality (AR) feature point matching technology as a means for decoding a coded item, To a content providing method using an augmented reality providing compensation service according to the present invention, and a system and an apparatus therefor.

With the development of the mobile communication network and the development of the terminal specification, the mobile communication terminal has become a necessity of modern people and has evolved into a total entertainment device beyond the conventional category of simple communication apparatus or information providing apparatus.

Particularly, due to the development of mobile communication, interest in location based service (LBS) is increasing. A location-based service is a service that provides specific information to a user of a wireless Internet or a mobile terminal (e.g., a smartphone) according to the location of the user. Location-based services can generate a wide variety of information in terms of providing local information that is specific to users, and the possibility of utilization of the information can be considered to be endless. Some examples of location-based services include locating nearby locations such as gas stations or restaurants, locating information on facilities, and finding useful information, such as finding the location of the other.

In the case of a mobile terminal, which is currently in widespread use, a camera is mounted to capture an image, and an image can be displayed on the screen. Therefore, the local information provided through the location-based service is combined with the image captured by the camera, As a method of projecting various information becomes possible, a service in the form of Augmented Reality (AR) can be implemented. The augmented reality is a hybrid virtual reality that fuses a real environment with a virtual environment by utilizing a technique of displaying a three-dimensional virtual object overlaid on a camera image actually viewed by a user. This augmented reality can be applied to maximize the advantages of location based services to users in that it integrates the virtual world into the real world.

In this way, the augmented reality combines the real image with the virtual graphic, so that the virtual objects must be accurately positioned in the desired place in order to obtain more realistic and accurate images. In order to realize this, three-dimensional coordinates of the virtual object are required, and these coordinates should be coordinate values based on the camera.

On the other hand, map matching that can transmit coupon / marketing information as a method of decrypting encrypted information by using GPS is not applicable to a case where GPS is not supported.

Korean Registered Patent No. 10-1179969 B1, Registered on Aug. 30, 2012 (Name: Marker Detection Device)

In order to solve such a conventional problem, an object of the present invention is to download and display a plurality of encrypted items corresponding to a path condition through an application on a map, to display image data A content providing method using an augmented reality capable of providing decoded items and authentication information according to whether augmented reality feature point is matched with previously stored reference video data and providing a compensation service based on the item and authentication information, ≪ / RTI >

According to an aspect of the present invention, there is provided a terminal including a communication unit for communicating data with an application using an augmented reality in communication with a service device, a camera unit for capturing image data through an application, Downloading a plurality of encrypted items corresponding to path conditions from the service device, displaying the encrypted items on a map, acquiring image data according to predetermined shooting conditions at a position associated with a specific item of the items, Receiving the item and the authentication information decrypted from the service device according to whether the augmented reality feature point of the previously stored reference image data matches the previously stored reference image data, outputting the decrypted item and authentication information to the screen, Compensation service based on certification number The is characterized in that a control unit for control to provide.

In addition, in the terminal according to the present invention, the control unit registers the authentication image according to the augmented reality feature point matching result and the visit evaluation of the place related to the item in the social network service, and shares the authentication image with another terminal .

In addition, in the terminal according to the present invention, the path condition includes at least one of a region, a theme, a day, a weather, a season, or a time.

In addition, in the terminal according to the present invention, the photographing condition includes a condition for photographing the image data provided to the augmented reality related to the encrypted item and the specific image around the item.

A service apparatus according to an exemplary embodiment of the present invention includes a service communication unit that transmits and receives data related to an application using an augmented reality by communicating with at least one terminal and performs an authentication procedure according to a request of the terminal, When the captured image data is received from the terminal, it is checked whether the image data and pre-stored reference image data match the augmented reality feature point. If the augmented reality feature point matching is successful, Decrypting the decrypted item, and providing the decrypted item and the authentication information to the terminal.

In addition, in the service apparatus according to the present invention, the plurality of items corresponding to the path condition, the augmented reality feature point of the reference image data for comparison with the image data photographed from the terminal, the authentication information related to the ID of the terminal, And a service storage unit for storing at least one of compensation information provided when decrypting the encrypted item.

A content providing system using an augmented reality according to an embodiment of the present invention downloads a plurality of encrypted items corresponding to a path condition from a service device through an application, displays encrypted items on a map, And receives the item and the authentication information decrypted from the service device according to whether the acquired image data and the previously stored reference image data match the augmented reality feature point, A terminal for providing a compensation service based on the decrypted item and the authentication number, an authentication process according to a request of the terminal, and a plurality of terminals corresponding to a predetermined path condition And transmits the encrypted item from the terminal If the augmented reality feature point matching is successful, the encrypted item is decrypted, and the decrypted item and the authentication information are provided to the terminal And a service device for performing the service.

A method of providing contents using an augmented reality according to an embodiment of the present invention includes a step of downloading a plurality of encrypted items corresponding to a route condition from a service device through an application, A step in which the terminal acquires image data according to a preset photographing condition at a position associated with a specific item of the items and a step in which the terminal acquires image data according to whether the terminal acquires image data and pre- And outputting the decrypted item and the authentication information to the screen, and providing the compensation service based on the decrypted item and the authentication number. .

In addition, in the content providing method using the augmented reality according to the present invention, the downloading step may include a step in which the terminal transmits authentication information and location information to the service device, and a step in which the terminal receives the encrypted items And a control unit.

According to another aspect of the present invention, there is provided a method of providing a content using an augmented reality, the method comprising the steps of: receiving a matching failure message from a service device when the terminal fails to match the augmented reality feature point; And outputting the output signal.

According to another aspect of the present invention, there is provided a method of providing a content using an augmented reality, the method comprising: providing a compensation service corresponding to a discount or a coupon when the terminal succeeds in decrypting the encrypted item.

In addition, in the method for providing contents using the augmented reality according to the present invention, after the providing step, the terminal shares the success result of the augmented reality feature point matching or visiting information of the company with another terminal through the social network service And further comprising:

A method of providing content using an augmented reality according to an exemplary embodiment of the present invention includes the steps of performing an authentication procedure according to a request of a terminal, a service device transmitting a plurality of encrypted items corresponding to a predetermined path condition A step in which the service device confirms whether the image data and pre-stored reference image data are augmented reality feature point matching when image data photographed from the terminal is received, and if the augmented reality feature point matching is successful, And a step in which the service apparatus provides the decrypted item and authentication information to the terminal.

In the content providing method using the augmented reality according to the present invention, the confirming step may include a step of comparing the image data received from the terminal with the previously stored reference image data according to photographing conditions, And determining that success of matching of the augmented reality feature point of the image data is satisfied.

According to another aspect of the present invention, there is provided a method of providing content using an augmented reality, the method comprising: collecting compensation information based on decrypted items and authentication information of a service device; And a step of providing the data.

As another means for solving the problem of the present invention, there is provided a computer-readable recording medium on which a program for executing a content providing method using an augmented reality is recorded.

According to the present invention, it is possible to utilize both the indoor / outdoor space on the basis of the field visit confirmation and the decoding of the information through the augmented reality feature point matching.

In addition, it is possible to provide services without positioning, such as GPS (Global Positioning System) positioning, base station, WI-FI, and RFID (Radio Frequency Identification) positioning.

In addition, the travel route guide can be implemented by planning the coupon / information decoding order. For example, the user may select some items among the various items provided on the map based on the augmented reality as the user-centered route, set the travel route by setting the location where the selected items are located as the destination, Accordingly, the service can be provided while moving from the current location to the destination.

Also, by collecting the user log and identifying the mobile movement line, it can be utilized as a marketing means such as the management of the commercial area and the location of the opening of the shop.

1 is a diagram illustrating a content providing system using an augmented reality according to an embodiment of the present invention.
2 is a block diagram showing the configuration of a terminal according to the present invention.
3 is a block diagram showing a configuration of a service apparatus according to the present invention.
4 is a flowchart illustrating a content providing method using an augmented reality according to an embodiment of the present invention.
5 is a flowchart illustrating an operation method of a terminal according to the present invention.
6 is a flowchart illustrating an operation method of a service apparatus according to the present invention.
7A and 7B are diagrams for explaining a content providing method using an augmented reality according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and the accompanying drawings, detailed description of well-known functions or constructions that may obscure the subject matter of the present invention will be omitted. It should be noted that the same constituent elements are denoted by the same reference numerals as possible throughout the drawings.

The terms and words used in the present specification and claims should not be construed to be limited to ordinary or dictionary meanings and the inventor is not limited to the concept of terminology for describing his or her invention in the best way. It should be interpreted as meaning and concept consistent with the technical idea of the present invention. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention, and not all of the technical ideas of the present invention are described. Therefore, It is to be understood that equivalents and modifications are possible.

Hereinafter, a terminal according to an embodiment of the present invention will be described as a representative example of a mobile communication terminal connected to a communication network to transmit / receive data related to a content provided using an augmented reality, but the terminal is not limited to a mobile communication terminal, The present invention can be applied to various terminals such as all information communication devices, multimedia terminals, wired terminals, fixed type terminals and IP (Internet Protocol) terminals. Also, the terminal may be a mobile phone, a portable multimedia player (PMP), a mobile Internet device (MID), a smart phone, a desktop, a tablet PC, a notebook, And an information communication device, which can be advantageously used in a mobile terminal having various mobile communication specifications.

In addition, the present invention can provide a discount coupon service at the time of entry into a store through linkage of a movement pattern of a user and a consumption pattern, and can provide an indoor / outdoor positioning, an augmented reality, an intelligent search, a social commerce, , It can be used for marketing by collecting context aware information such as location information / image information / voice information / payment information in real time. In addition to commercial analysis when establishing a store, analysis of various movement patterns and consumption patterns of users in the store operation enables more sophisticated analysis of customer purchase patterns and enables customized marketing services to be operated in real time.

Hereinafter, a content providing system using an augmented reality according to an embodiment of the present invention will be described.

1 is a diagram illustrating a content providing system using an augmented reality according to an embodiment of the present invention.

Referring to FIG. 1, a content providing system 100 using an augmented reality according to the present invention includes a terminal 10, a service apparatus 20, and a communication network 30.

A processor mounted on the terminal 10 or the service device 20 according to the present invention can process program instructions for executing the method according to the present invention. In one implementation, the processor may be a single-threaded processor, and in other embodiments, the processor may be a multi-threaded processor. Further, the processor is capable of processing instructions stored on a memory or storage device.

The communication network 30 performs a series of data transmission and reception operations for data transmission and information exchange between the terminal 10 and the service apparatus 20. [ In particular, the communication network 30 may use various types of communication networks and may be, for example, a wireless LAN (WLAN), a Wi-Fi, a Wibro, a WiMAX, HSDPA, and High Speed Downlink Packet Access) or a wireless communication method such as Ethernet, xDSL (ADSL, VDSL), Hybrid Fiber Coax (HFC), Fiber to the Curb (FTTC) A wired communication method can be used. The communication network 30 is not limited to the above-described communication system, but may include any other widely known or later-developed communication systems as well as the communication systems described above.

The terminal 10 is connected to the service device 20 through the communication network 30 and transmits / receives all data for providing contents using the augmented reality. In particular, the terminal 10 according to the present invention downloads a plurality of encrypted items corresponding to the path condition from the service device 20 through the application. Then, the terminal 10 displays the encrypted items on the map.

The terminal 10 acquires image data according to a predetermined photographing condition at a position associated with a specific item among the items. The terminal 10 receives the decrypted item and the authentication information from the service device 20 according to whether the acquired image data and the stored reference image data match the augmented reality feature point. Then, the terminal 10 outputs the decrypted item and the authentication information to the screen.

The terminal 10 provides a compensation service based on the decrypted item and the authentication number. In addition, the terminal 10 registers the authentication image according to the augmented reality feature point matching result and the visit evaluation of the place related to the item in the social network service (SNS), and shares the authentication image with another terminal.

The service device 20 is connected to the terminal 10 through the communication network 30 and transmits / receives all data for providing contents using the augmented reality. In particular, the service apparatus 20 according to the present invention performs an authentication procedure at the request of the terminal 10. Then, the service apparatus 20 transmits a plurality of encrypted items corresponding to the predetermined path condition to the terminal 10. [

When the image data photographed from the terminal is received, the service device 20 confirms whether the image data and pre-stored reference image data are augmented reality feature point matching. Herein, the service apparatus 20 according to the present invention can apply a marker recognition-based detection method as a method for confirming whether or not the augmented reality feature point is matched. In this case, the feature point between the reference image data and the photographed image data is loaded as a tree data structure And stores image data having a predetermined resolution input through a camera in a buffer. Then, the service apparatus 20 generates a pyramid image for the reference image, applies a Gaussian filter to the pyramid images, and then selects points having a large difference between the surrounding pixels and the light and shade in the pyramid image Calculates a Laplacian measurement value between points around the selected point, and extracts the feature points by selecting only the upper values.

In this way, the service device 20 generates a rotated image around the extracted feature points, and assigns the generated rotated image to the tree structure to obtain a weight value. After that, the service apparatus 20 matches the minutiae having the high weight with the comparative minutiae, matches the extracted minutiae with the phases of the comparison minutiae, checks the positions thereof, and recognizes them as markers if they are the same. Accordingly, in order to recognize the feature point-based markers, the service device 20 extracts feature points for each frame of the real-time image, compares the feature points with the feature points of all the registered images, and determines whether the feature point matching is successful or failed based on the comparison result have.

Also, as a method of confirming whether or not the augmented reality feature point is matched, the service device 20 recognizes a feature point based marker by introducing a sequential index concept on the feature point based marker data. That is, the service device 20 can determine whether or not the feature point matching is performed according to the comparison of the marker data according to the index, without recognizing the feature point based markers through comparison with the entire marker data. Accordingly, the service apparatus 20 can reduce the total operation time per frame and maintain a constant recognition speed.

Also, as a method of confirming whether or not the augmented reality feature point is matched, the service device 20 matches the feature points of the reference image data and the feature points of the photographed image data based on the previously stored reference image data on the basis of the position information of the terminal 10 . Then, the service apparatus 20 sums the number of feature point matches for each group of the reference image data, and adds the number of feature points for each group of reference image data. Thereafter, the service apparatus 20 selects one group of reference image data having the largest matching similarity based on the sum of the feature point matching counts and the sum of the number of feature points, and determines an object corresponding to the selected group of reference image data .

 If the augmented reality feature point matching succeeds, the service device 20 decrypts the encrypted item and provides the decrypted item and the authentication information to the terminal.

Accordingly, the present invention can be applied to indoor / outdoor spaces on the basis of site visit confirmation and information decoding through augmented reality feature point matching. In addition, it is possible to provide services without any positioning without positioning of GPS positioning, base station, WI-FI, RFID, and the like. In addition, the travel route guide can be implemented by planning the coupon / information decoding order. For example, the user may select some items among the various items provided on the map based on the augmented reality as the user-centered route, set the travel route by setting the location where the selected items are located as the destination, Accordingly, the service can be provided while moving from the current location to the destination. Also, by collecting the user log and identifying the mobile movement line, it can be utilized as a marketing means such as the management of the commercial area and the location of the opening of the shop.

The contents providing process using the augmented reality according to the embodiment of the present invention will be described in more detail with reference to FIG. 2 to FIG. 7B.

2 is a block diagram showing the configuration of a terminal according to the present invention.

2, a terminal 10 according to the present invention includes a control unit 11, an input unit 12, a display unit 13, a storage unit 14, an audio processing unit 15, a communication unit 16, 17). In particular, the control unit 11 includes an application execution module 11a and an image data acquisition module 11b, and the storage unit 14 includes image data 14a and compensation information 14b.

The input unit 12 receives various information such as numbers and character information and transmits various signals to the control unit 11 in connection with setting various functions and controlling functions of the terminal 10. The input unit 12 may include at least one of a keypad and a touchpad that generates an input signal according to a user's touch or operation. At this time, the input unit 12 may be configured in the form of a touch panel (or a touch screen) together with the display unit 13 to simultaneously perform input and display functions. The input unit 12 may be any type of input device that can be developed in addition to an input device such as a keyboard, a keypad, a mouse, a joystick, and the like. In particular, the input unit 12 according to the present invention senses all input signals for providing contents using the augmented reality, and transmits the sensed input signals to the control unit 11.

The display unit 13 displays information on a series of operation states, operation results, and the like that occur while the functions of the terminal 10 are performed. In addition, the display unit 13 can display menus of the terminal 10 and user data input by the user. The display unit 13 may be a liquid crystal display (LCD), a thin film transistor LCD (TFT-LCD), a light emitting diode (LED), an organic light emitting diode (OLED) LEDs, active matrix organic light emitting diodes (AMOLED), active matrix OLEDs, retina displays, flexible displays, and three-dimensional displays. In this case, when the display unit 13 is configured as a touch screen, the display unit 13 may perform some or all of the functions of the input unit 12. [ In particular, the display unit 13 according to the present invention displays screen information related to content provision using an augmented reality, and displays an encrypted item or a decrypted item on a map. Also, the display unit 13 displays authentication information provided according to the augmented reality feature point matching and compensation information for providing a compensation service.

The storage unit 14 is a device for storing data, and includes a main storage device and an auxiliary storage device, and stores an application program required for functional operation of the terminal 10. [ The storage unit 14 may include a program area and a data area. Here, when the terminal 10 activates each function in response to a request from the user, the terminal 10 executes the corresponding application programs under the control of the controller 11 to provide each function. In particular, the storage unit 14 according to the present invention stores an operating system for booting the terminal 10, a program for executing an application, a program for acquiring image data, and a program for providing a compensation service. The storage unit 14 stores the image data 14a photographed according to the photographing conditions and the compensation information 14b for the compensation service provided from the service device 20 according to whether the augmented reality feature point is matched. Here, the compensation information 14b may include information such as discount coupons and points.

The audio processing unit 15 transmits the audio signal input from the speaker SPK or the microphone MIC to the control unit 11 for reproducing and outputting the audio signal. The audio processor 15 converts an analog audio signal input through a microphone into a digital format and transmits the audio signal to the controller 11. The audio processing unit 15 may convert the digital audio signal output from the control unit 11 into an analog signal and output it through a speaker. In particular, the audio processing unit 15 according to the present invention outputs an effect sound or an execution sound according to the execution of the application.

The communication unit 16 performs a function of transmitting and receiving data through the communication network 30 with the service apparatus 20. [ Here, the communication unit 16 includes RF transmitting means for up-converting and amplifying the frequency of the transmitted signal, RF receiving means for low-noise amplifying the received signal and down-converting the frequency. The communication unit 16 may include at least one of a wireless communication module (not shown) and a wired communication module (not shown). When the terminal 10 uses wireless communication, the wireless communication module uses any one of a wireless network communication module, a wireless LAN communication module, and a wireless fan communication module to transmit / receive data according to a wireless communication method. Data can be transmitted and received to the service apparatus 20. [ The wired communication module is for transmitting / receiving data by wire. The wired communication module can connect to the communication network 30 via a wired line and transmit / receive data to / from the service device 20. [ That is, the terminal 10 is connected to the communication network 30 using a wireless communication module or a wired communication module, and can exchange data with the service device 20 through the communication network 30. In particular, the communication unit 16 according to the present invention communicates with the service device 20 to transmit / receive data related to an application using an augmented reality.

The camera unit 17 acquires image data of a user's image. The camera unit 17 includes a camera sensor (not shown) that captures an image captured through a lens and converts the photographed optical signal into an electrical signal, and an analog-to-digital And a signal processing unit (not shown). Here, the camera sensor may be a charge coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) image sensor, and the signal processing unit may be implemented by a DSP (Digital Signal Processor). The camera unit 17 may be activated when an input signal for using the camera function is received. Then, the camera unit 17 transmits the collected image data to the control unit 11. In order to effectively perform such a function, the camera unit 17 may include an image processing unit (not shown). At this time, the image processor performs a function of generating screen data for displaying a video signal output from the camera sensor. At this time, the image processing unit processes the image signal output from the camera sensor in units of frames, and outputs the frame image data corresponding to the display characteristics and size of the display unit 13. [ In addition, the image processing unit includes an image codec (CODEC), compresses the frame image data displayed on the display unit 13 in a predetermined manner, or restores the compressed frame image data into original frame image data. Here, the image codec may be a Joint Photographic Coding Experts Group (JPEG) codec or a Moving Picture Experts Group (MPEG) codec. In particular, the camera unit 17 according to the embodiment of the present invention captures image data through an application and transmits the captured image data to the control unit 11.

The control unit 11 may be an operating system (OS) and a process unit for driving each configuration. In particular, the control unit 11 according to the embodiment of the present invention downloads a plurality of encrypted items corresponding to the path condition from the service device 20 through the application. Here, the terminal 10 transmits authentication information and location information to the service device 20, and receives the encrypted items according to the approval of the service device 20. [ At this time, the path condition includes conditions such as a region, a theme, a day, a weather, a season, or a time. Then, the control unit 11 displays the encrypted items on the map.

The control unit 11 acquires image data according to a preset photographing condition at a position associated with a specific item among the items. Here, the photographing condition includes a condition for photographing together the image data provided to the augmented reality with respect to the encrypted item and the specific image around the item.

The control unit 11 receives the item and the authentication information decrypting the encrypted item from the service device 20 according to whether the acquired image data and the previously stored reference image data are matched with the augmented reality feature point, Print on the screen. On the other hand, if the augmented reality feature point matching fails, the control unit 11 receives the matching failure message from the service device 20 and outputs a matching failure message in a pop-up form.

The control unit 11 receives the compensation service based on the decrypted item and the authentication number to provide. Here, if the decryption of the encrypted item is successful, the control unit 11 provides a discount service or a compensation service corresponding to the coupon. In addition, the control unit 11 shares the success result of the augmented reality feature point matching or visit information of the vendor with other terminals through the social network service.

In order to more effectively perform the functions of the terminal 10, the control unit 11 includes an application execution module 11a and an image data acquisition module 21b. In particular, the application execution module 11a performs all functions related to execution of an application capable of providing contents using an augmented reality. In addition, the image data acquisition module 21b photographs image data provided to the augmented reality according to the shooting conditions during execution of the application.

3 is a block diagram showing a configuration of a service apparatus according to the present invention.

Referring to FIG. 3, the service apparatus 20 according to the present invention includes a service control unit 21, a service storage unit 22, and a service communication unit 23. In particular, the service control unit 21 includes an authentication module 21a, an item management module 21b, an AR feature point matching module 21c, and a compensation service provision module 21d. The service storage unit 22 stores items 22a , AR feature point 22b of reference image data, authentication information 22c, and compensation information 22d.

The service communication unit 23 transmits and receives data for execution of the application with the terminal 10 through the communication network 30. [ In particular, the service communication unit 23 according to the present invention communicates with the terminal 10 to transmit and receive data related to the application using the augmented reality.

The service storage unit 22 stores all the programs according to the execution of the functions of the service apparatus 20. [ In particular, the service storage unit 22 according to the present invention may be a program for performing authentication related to the ID and position information of the terminal 10, a program for managing encrypted items or decrypted items, And a program for providing a compensation service according to whether the augmented reality feature point is matched or not. The service storage unit 22 also stores an item 22a including an encryption item and a decryption item provided in the terminal 10 according to a path condition and a check whether the image data received from the terminal 10 is augmented reality feature point matching The authentication feature information 22c provided according to whether the augmented reality feature point matches or not, and the compensation information 22d provided as compensation for augmented reality feature fixation success.

The service control unit 21 may be an operating system and a process unit for driving each configuration. In particular, the service control unit 21 according to the embodiment of the present invention performs an authentication procedure at the request of the terminal 10. Then, the service control unit 21 transmits a plurality of encrypted items corresponding to the predetermined path condition to the terminal 10.

When the image data photographed from the terminal 10 is received, the service control unit 21 confirms whether the received image data and pre-stored reference image data are augmented reality feature point matching. Here, the service control unit 21 compares the image data received from the terminal 10 with the previously stored reference image data according to the photographing conditions, and determines that the augmenting reality feature point matching of the image data is successful if the photographing condition is satisfied.

Here, the service control unit 21 according to the present invention can apply a marker recognition-based detection method as a method for checking whether augmented reality feature point is matched or not. In this case, the feature point between reference image data and photographed image data is loaded into a tree data structure And stores image data having a predetermined resolution input through a camera in a buffer. The service control unit 21 generates a pyramid image of the reference image, applies a Gaussian filter to the pyramid images, and then selects points having a large difference between the surrounding pixels and the light and dark in the pyramid image Calculates a Laplacian measurement value between points around the selected point, and extracts the feature points by selecting only the upper values.

In this way, the service control unit 21 generates a rotated image around the extracted feature points, and assigns the generated rotated image to the tree structure to obtain a weight value. Thereafter, the service control unit 21 matches the minutiae having the high weight with the comparative minutiae, matches the extracted minutiae with the phases of the comparison minutiae, checks the positions thereof, and recognizes them as markers if they are the same. Accordingly, in order to recognize the feature point-based markers, the service control unit 21 extracts feature points every frame of the real-time image, compares the feature points with the feature points of all the registered images, and determines whether the feature point matching is successful or failed based on the comparison result have.

Also, as a method for confirming whether or not the augmented reality feature point is matched, the service control unit 21 introduces the concept of a sequential index for the feature point-based marker data to recognize the feature point-based marker. That is, the service control unit 21 can determine whether or not the feature point matching is performed according to the comparison of the marker data according to the index, without recognizing the feature point based markers through comparison with the entire marker data. Accordingly, the service control unit 21 can reduce the total operation time per frame and maintain a constant recognition speed.

As a method of confirming whether the augmented reality feature point is matched, the service control unit 21 matches the feature points of the reference image data and the feature points of the photographed image data on the basis of the previously stored reference image data, based on the position information of the terminal 10 . The service control unit 21 sums the number of feature point matches for each group of reference image data, and adds the number of feature points for each group of reference image data. Thereafter, the service control unit 21 selects one group of reference image data having the largest matching similarity based on the sum of the number of feature points and the number of feature points, and determines an object corresponding to the selected group of reference image data .

When the augmented reality feature point matching is successful, the service control unit 21 decrypts the encrypted item and provides the decrypted item and the authentication information to the terminal 10. Then, the service control unit 21 collects the compensation information based on the decrypted item and the authentication information, and provides the collected compensation information to the terminal 10.

The service control unit 21 includes an authentication module 21a, an item management module 21b, an augmented reality feature point matching module 21c, and a compensation service providing module 21d ).

In particular, the authentication module 21a authenticates the identity and location information of the terminal 10. The item management module 21b provides an encrypted item matching the path information according to a request of the terminal 10 and provides the encoded item to the terminal 10 according to whether the augmented reality feature point matching is successful or not. In addition, the augmented reality feature point matching module 21c performs a function of determining whether the augmented reality feature point matching between the image data received from the terminal 10 and the reference image data is matched. In addition, the compensation service providing module 21d performs a function of providing compensation information to the terminal 10 in accordance with the success of the augmented reality feature point matching.

In addition, the service device 20 configured as described above may be implemented as one or more servers operating in a server-based computing-based manner or in a cloud-based manner. In particular, data related to content provision using an augmented reality can be provided through a cloud computing function that can be permanently stored in a cloud computing device on the Internet. Here, cloud computing utilizes Internet technology in digital terminals such as desktops, tablet computers, laptops, netbooks and smart phones to provide virtualized IT (Information Technology) resources such as hardware (server, storage, (Database, security, web server, etc.), service, data, etc. on demand.

On the other hand, a memory mounted on the terminal 10 or the service apparatus 20 stores information in the apparatus. In one implementation, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in other embodiments, the memory may be a non-volatile memory unit. In one implementation, the storage device is a computer-readable medium. In various different implementations, the storage device may include, for example, a hard disk device, an optical disk device, or any other mass storage device.

Although the present specification and drawings describe exemplary device configurations, the functional operations and subject matter implementations described herein may be embodied in other types of digital electronic circuitry, or alternatively, of the structures disclosed herein and their structural equivalents May be embodied in computer software, firmware, or hardware, including, or in combination with, one or more of the foregoing. Implementations of the subject matter described herein may be embodied in one or more computer program products, i. E. One for computer program instructions encoded on a program storage medium of the type for < RTI ID = 0.0 & And can be implemented as a module as described above. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that affects the machine readable propagation type signal, or a combination of one or more of the foregoing.

4 is a flowchart illustrating a content providing method using an augmented reality according to an embodiment of the present invention.

Referring to FIG. 4, in the content providing method using the augmented reality according to the present invention, the terminal 10 executes an application in step S11. In step S13, the terminal 10 communicates with the service device 20 to perform the authentication procedure. At this time, the authentication procedure verifies the authentication information about the ID and the location information of the terminal 10.

When the authentication is completed, the terminal 10 downloads a plurality of encrypted items corresponding to the path condition to the service device 20 in step S15. At this time, the path condition includes conditions such as a region, a theme, a day, a weather, a season, or a time. Then, the terminal 10 displays the encrypted items on the map in step S17.

In step S19, the terminal 10 acquires image data according to a preset photographing condition at a position associated with a specific item among the items. Here, the photographing condition includes a condition for photographing together the image data provided to the augmented reality with respect to the encrypted item and the specific image around the item. Then, the terminal 10 transmits the image data photographed in the step S21 to the service device 20. [

When the image data photographed from the terminal 10 is received, the service device 20 checks whether the image data received in step S23 and the previously stored reference image data match the augmented reality feature point. Here, the service device 20 compares the image data received from the terminal 10 with the previously stored reference image data according to the photographing condition, and determines that the augmenting reality feature point matching of the image data is successful if the photographing condition is satisfied.

If the augmented reality feature point matching is successful, the service device 20 decrypts the encrypted item and provides the decrypted item and the authentication information to the terminal 10 (S25 to S27).

When the terminal 10 receives the item and the authentication information decrypting the encrypted item from the service device 20 according to whether the image data and the pre-stored reference image data are matched with the augmented reality feature point, the terminal 10 transmits the decrypted item and the authentication information To the screen. On the other hand, when the augmented reality feature point matching fails in step S31, the terminal 10 receives the matching failure message from the service device 20 and outputs a matching failure message in a pop-up form.

The service device 20 collects the compensation information based on the decrypted item and the authentication information, and provides the collected compensation information to the terminal 10 (S33 to S35).

The terminal 10 provides a compensation service based on the decrypted item and the authentication number in step S37. Here, if the decryption of the encrypted item is successful, the terminal 10 provides a discount service or a compensation service corresponding to the coupon. In addition, the terminal 10 shares the success result of the augmented reality feature point matching or visit information of the vendor with other terminals through the social network service.

Accordingly, the present invention can be applied to indoor / outdoor spaces on the basis of site visit confirmation and information decoding through augmented reality feature point matching. In addition, it is possible to provide services without any positioning without positioning of GPS positioning, base station, WI-FI, RFID, and the like. In addition, the travel route guide can be implemented by planning the coupon / information decoding order. For example, the user may select some items among the various items provided on the map based on the augmented reality as the user-centered route, set the travel route by setting the location where the selected items are located as the destination, Accordingly, the service can be provided while moving from the current location to the destination. Also, by collecting the user log and identifying the mobile movement line, it can be utilized as a marketing means such as the management of the commercial area and the location of the opening of the shop.

5 is a flowchart illustrating an operation method of a terminal according to the present invention.

Referring to FIG. 5, the terminal 10 according to the present invention executes an application at step S41. At this time, the terminal 10 communicates with the service device 20 to perform an authentication procedure. At this time, the authentication procedure verifies the authentication information about the ID and the location information of the terminal 10.

When the authentication is completed, the terminal 10 downloads a plurality of encrypted items corresponding to the path condition to the service device 20 in step S43. At this time, the path condition includes conditions such as a region, a theme, a day, a weather, a season, or a time. Then, the terminal 10 displays the encrypted items on the map in step S45.

In step S47, the terminal 10 acquires image data according to a predetermined photographing condition at a position associated with a specific item among the items. Here, the photographing condition includes a condition for photographing together the image data provided to the augmented reality with respect to the encrypted item and the specific image around the item. Then, the terminal 10 transmits the photographed image data to the service device 20. [

The terminal 10 confirms whether the result of the augmented reality feature point matching success is received from the service device 20 in step S49. If the augmented reality feature point matching is successful, the terminal 10 receives the decrypted item and the authentication information from the service device 20 in step S51, and outputs the decrypted item and the authentication information on the map. On the other hand, if the augmented reality feature point matching fails in step S53, the terminal 10 receives the matching failure message from the service device 20 and outputs a matching failure message in a pop-up form.

The terminal 10 provides the compensation service based on the decrypted item and the authentication number in step S55. Here, if the decryption of the encrypted item is successful, the terminal 10 provides a discount service or a compensation service corresponding to the coupon. In addition, the terminal 10 shares the success result of the augmented reality feature point matching or visit information of the vendor with other terminals through the social network service.

6 is a flowchart illustrating an operation method of a service apparatus according to the present invention.

Referring to FIG. 6, the service device 20 according to the present invention performs an authentication procedure according to a request of the terminal 10 in step S61. At this time, the authentication procedure verifies the authentication information about the ID and the location information of the terminal 10.

When the authentication is completed, the service device 20 transmits a plurality of encrypted items corresponding to the path condition to the terminal 10 in step S63. At this time, the path condition includes conditions such as a region, a theme, a day, a weather, a season, or a time.

The service device 20 confirms whether image data is received from the terminal 10 in step S65. When the image data is received, the terminal 10 confirms whether the augmented reality feature point matching between the image data photographed according to the photographing condition and the reference image data is successful in step S67. That is, when the video data photographed from the terminal 10 is received, the service device 20 confirms whether the received video data and pre-stored reference video data match the augmented reality feature point. Here, the service device 20 compares the image data received from the terminal 10 with the previously stored reference image data according to the photographing condition, and determines that the augmenting reality feature point matching of the image data is successful if the photographing condition is satisfied.

If the augmented reality feature point matching succeeds, the service device 20 decrypts the encrypted item in step S69. Then, the service device 20 transmits the decrypted item and the authentication information to the terminal 10 in step S71.

The service device 20 collects the reward information based on the decrypted item and the authentication information in step S73. Then, the service device 20 provides the compensation information collected in step S77 to the terminal 10. [

On the other hand, if the augmented reality feature point matching fails, the service device 20 transmits a matching failure message to the terminal 10 in step S73.

7A and 7B are diagrams for explaining a content providing method using an augmented reality according to an embodiment of the present invention.

As shown in FIGS. 7A and 7B, in the content providing method using the augmented reality according to the present invention, the terminal 10 receives a plurality of encrypted items (for example, 701 to 712), and displays the downloaded items 701 to 7112 on the map.

Then, the terminal 10 displays decoded items 721 to 728 on the map according to the success or failure of the augmented reality feature point matching. At this time, the terminal 10 provides the decrypted items 721 to 728 together with the authentication information.

Computer-readable media suitable for storing computer program instructions and data include, for example, magnetic media such as hard disks, floppy disks, and magnetic tape, compact disk read only memory (CD-ROM) A magneto-optical medium such as a magneto-optical medium such as a magneto-optical disk such as a magneto-optical disk, a magneto-optical medium such as a floppy disk, and a read only memory (ROM) Access Memory), a flash memory, an erasable programmable ROM (EPROM), and a semiconductor memory such as an Electrically Erasable Programmable ROM (EEPROM). The processor and memory may be supplemented by, or incorporated in, special purpose logic circuits. Examples of program instructions may include machine language code such as those generated by a compiler, as well as high-level language code that may be executed by a computer using an interpreter or the like. Such a hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

While the specification contains a number of specific implementation details, it should be understood that they are not to be construed as limitations on the scope of any invention or claim, but rather on the description of features that may be specific to a particular embodiment of a particular invention Should be understood. Certain features described herein in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Further, although the features may operate in a particular combination and may be initially described as so claimed, one or more features from the claimed combination may in some cases be excluded from the combination, Or a variant of a subcombination.

Likewise, although the operations are depicted in the drawings in a particular order, it should be understood that such operations must be performed in that particular order or sequential order shown to achieve the desired result, or that all illustrated operations should be performed. In certain cases, multitasking and parallel processing may be advantageous. Also, the separation of the various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems will generally be integrated together into a single software product or packaged into multiple software products It should be understood.

It should be noted that the embodiments of the present invention disclosed in the present specification and drawings are only illustrative of specific examples for the purpose of understanding and are not intended to limit the scope of the present invention. It will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

The present invention utilizes the augmented reality feature point matching technology as a means to decode a ciphered item and provides a compensation service according to decryption. Through this, based on field visit confirmation and information decoding through augmented reality feature point matching, it can be utilized in both indoor and outdoor spaces. In addition, it is possible to provide services without any positioning without positioning of GPS positioning, base station, WI-FI, RFID, and the like. In addition, the travel route guide can be implemented by planning the coupon / information decoding order. Also, by collecting the user log and identifying the mobile line, it can be utilized as marketing means such as the management of the commercial area and the location of the opening of the shop. This is not only a possibility of commercialization or sales, but also a possibility of being industrially applicable since it is practically possible to carry out clearly.

10: terminal 20: service device 30: communication network
11: control unit 12: input unit 13:
14: storage unit 15: audio processing unit 16:
17: camera unit 21: service control unit 22: service storage unit
23: service communication unit 100: content providing system using augmented reality
11a: Application execution module 11b: Image data acquisition module
14a: video data 14b, 22d: compensation information
21a: Authentication module 21b: Item management module
21c: AR feature point matching module 21d: Compensation service providing module
22a: Item 22b: AR characteristic point of reference image data
22c: Authentication information

Claims (16)

A communication unit for communicating with the service device to transmit and receive data related to the application to which the augmented reality is applied;
A camera unit for photographing image data through the application; And
Downloading a plurality of encrypted items corresponding to a path condition from the service device through the application, displaying the encrypted items on a map, displaying images on a map in accordance with predetermined shooting conditions at a position associated with a specific item of the items And receiving the item and the authentication information decrypting the encrypted item from the service apparatus according to whether the acquired image data and the previously stored reference image data match the AR feature point, And a control unit for outputting the authentication information to the screen and controlling to provide a compensation service based on the decrypted item and the authentication number;
And a second terminal. The apparatus of claim 1, wherein the control unit
Registers the authentication image according to the augmented reality feature point matching result and the visit evaluation of a place related to the item in the social network service, and shares the authentication image with another terminal. 2. The method of claim 1,
Wherein the at least one condition includes at least one of a region, a theme, a day, a weather, a season, or a time. The image pickup apparatus according to claim 1,
And a condition for photographing together the image data provided to the augmented reality with respect to the encrypted item and a specific image around the item. A service communication unit for communicating with at least one terminal to transmit and receive data related to an application using an augmented reality; And
And transmitting a plurality of encrypted items corresponding to a predetermined path condition to the terminal. When the image data photographed from the terminal is received, the image data and the previously stored reference image A service control unit for checking whether data is matched with an augmented reality feature point, decrypting the encrypted item if the augmented reality feature point matching is successful, and controlling the decrypted item and authentication information to be provided to the terminal;
Wherein the service device comprises: 6. The method of claim 5,
A plurality of items corresponding to the path condition, augmented reality feature points of reference image data for comparison with the image data photographed from the terminal, authentication information related to the ID and location information of the terminal, A service storage unit for storing at least one of compensation information provided;
Further comprising: Downloading a plurality of encrypted items corresponding to a path condition from a service device through an application, displaying the encrypted items on a map, and displaying image data according to predetermined shooting conditions at a position associated with a specific item among the items And receiving the item and the authentication information decrypting the encrypted item from the service device according to whether the acquired image data and the previously stored reference image data match the AR feature point, A terminal for outputting information on a screen and providing a compensation service based on the decrypted item and an authentication number; And
And transmitting a plurality of encrypted items corresponding to a predetermined path condition to the terminal. When the image data photographed from the terminal is received, the image data and the previously stored reference image A service device for confirming whether or not the data is matched with an augmented reality feature point, decrypting the encrypted item if the augmented reality feature point matching is successful, and providing the decrypted item and authentication information to the terminal;
Wherein the augmented reality information includes at least one of: Downloading a plurality of encrypted items corresponding to path conditions from a service device through an application;
The terminal displaying the encrypted items on a map;
Wherein the terminal acquires image data according to a preset photographing condition at a position associated with a specific item of the items:
Wherein the terminal receives the item and the authentication information decrypting the encrypted item from the service device according to whether the acquired image data and the previously stored reference image data match the AR feature point, Outputting information to a screen; And
The terminal providing a compensation service based on the decrypted item and the authentication number;
The method of claim 1, further comprising: 9. The method of claim 8, wherein the downloading comprises:
The terminal transmitting authentication information and location information to the service device; And
The terminal receiving the encrypted items upon approval of the service device;
The method of claim 1, further comprising: 9. The method of claim 8, wherein the outputting comprises:
Receiving a matching failure message from the service device when the terminal fails to match the augmented reality feature point; And
The terminal outputting the matching failure message in a pop-up form;
The method of claim 1, further comprising the steps of: 9. The method of claim 8, wherein providing
And if the terminal succeeds in decrypting the encrypted item, providing a compensation service corresponding to a discount or a coupon. 9. The method of claim 8, wherein after the providing step,
Sharing a result of success of the augmented reality feature point matching or visit information of a vendor with another terminal through a social network service;
The method of claim 1, further comprising the steps of: The service device performing an authentication procedure at the request of the terminal;
The service device transmitting a plurality of encrypted items corresponding to a predetermined path condition to the terminal;
Confirming whether the service device matches the augmented reality feature point of the image data and the stored reference image data when the image data photographed from the terminal is received;
If the augmented reality feature point matching is successful, the service device decodes the encrypted item; And
The service device providing the decrypted item and authentication information to the terminal;
The method of claim 1, further comprising: 14. The method of claim 13,
Comparing the image data received from the terminal with the stored reference image data according to photographing conditions; And
Determining that the augmented reality feature point matching of the image data is successful if the service device satisfies the photographing condition;
The method of claim 1, further comprising: 14. The method of claim 13,
Collecting compensation information based on the decrypted item and the authentication information; And
The service device providing the collected compensation information to the terminal;
The method of claim 1, further comprising: A computer-readable recording medium on which a program for executing a content providing method using an augmented reality according to at least one of claims 8 to 15 is recorded.

Images