KR20170031962A - Method and apparatus of providing contents based on virtual reality - Google Patents

Abstract

The present invention relates to a method and a device for providing content based on virtual reality. A moving path for moving a specific place is traced based on a photographed image photographed while moving the specific place. A content image and the photographed image mapped on a specific location is virtualized and the generated virtual reality content is provided to a terminal device if the moving path reaches a specific location while tracing the moving path. The present invention can increase intuitive recognition of the provided content and content optimized to a visiting purpose of the specific place is provided by defining a user, who visits the specific place, as a target, thereby improving the satisfaction of a content providing service.

Description

TECHNICAL FIELD [0001] The present invention relates to an apparatus and a method for providing content based on virtual reality,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technology for providing virtualized contents targeted at a terminal device that has entered a specific place, and more particularly relates to a technology for providing contents virtualized based on a photographed image photographed by a terminal device entering a specific place (E. G., Advertising, information, etc.). ≪ / RTI >

2. Description of the Related Art Recently, as the use of mobile devices (hereinafter, terminal devices) such as smart phones and tablet pads has been rapidly increasing, interest in technologies for providing contents (e.g., advertisements, information, etc.)

As the interest in the technology for providing content using the terminal device increases, a large amount of contents are provided to the terminal device by a plurality of businesses (for example, advertisers).

However, in the case of the content provided to the unspecified majority, it is provided randomly without consideration of the user's interest, and thus it is hardly expected to be recognized as the spammy advertisement and expect the advertisement effect.

To solve this problem, a method of providing contents based on user information such as user's location information and exposed personal information is introduced. However, even if the contents are provided on the basis of the user information, since the contents are provided only as text or advertisement images, it is inconvenient for the user to directly check the store location or event place on the offline side and map the contents to the corresponding contents There is still a problem that the user is perceived as spammy.

Meanwhile, in order to solve the problem that the contents are recognized as spam-like advertisements as described above, instead of randomly providing a large amount of contents to a large number of unspecified persons, instead of providing a variety of contents and events targeting a user who has entered a specific place : Advertisers) are increasing.

Accordingly, there is a need for a technique for providing contents by targeting a user who visited a specific place, while improving intuitive awareness of the contents.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and it is an object of the present invention to provide a method and apparatus for virtualizing content related to a specific place based on a shot image photographed by a terminal device entering a specific place And to provide a device and a method that can provide the same.

Further, it is to be understood that the present invention is not limited to the above-mentioned embodiments, and other and further objects which are not mentioned can be clearly understood by those skilled in the art from the following description .

According to an aspect of the present invention, there is provided a content providing apparatus including: an operation management unit for managing a plurality of content images related to a specific place; And a control unit for controlling the terminal device based on a result of determining whether the terminal device has reached a specific position based on the photographed image of the terminal device when the captured image collected in association with the position of the terminal device in the specific place is confirmed, And a virtual reality contents providing unit for virtualizing and providing the content image.

The virtual reality contents providing unit may include a tracking unit for tracking a movement path for moving the specific location based on the captured image; A detecting unit detecting a content image mapped to the specific location when the specific location is reached during tracking of the movement route; And a generation unit for generating virtual reality contents by combining the sensed image related to the specific position and the detected content image based on a virtualization rendering technique.

The virtual reality contents providing unit may include a history management unit for accumulating and storing the virtual reality contents provided to the terminal device; And a re-visit management unit for preferentially providing the accumulated virtual reality contents before the captured image is received from the terminal apparatus when the terminal apparatus revisits the specific place.

The tracking unit tracks the moving route by using the photographed image changed by the moving direction and the moving line of the terminal apparatus in real time.

Wherein the operation management unit comprises: an entry confirmation unit for confirming whether the terminal device has entered the specific place; And a content manager for delivering the plurality of content images to the virtual reality content provider at a point of time when the terminal device agrees to receive the virtual reality service providing content related to the specific location, .

And the specific location is a location where service can be provided related to at least one content image among the plurality of content images.

According to a second aspect of the present invention, there is provided a method of operating a content providing apparatus, the method comprising: managing a plurality of content images related to a specific place; And

When a captured image photographed in real time based on the position of the terminal device is confirmed within the specific place, the terminal device determines whether the terminal device has reached a specific position based on the captured image of the terminal device, And a providing step of virtualizing and providing related contents.

The providing step may include: a tracking step of tracking a moving path for moving the specific location based on the photographed image received in real time; A detecting step of detecting a content image mapped to the specific position when the specific position is reached during tracking of the movement path; And a generation step of generating virtual reality contents by combining the photographed image at the specific position and the detected content image based on a virtualization rendering technique.

Wherein the providing step comprises: a storing step of accumulating and storing the virtual reality contents provided to the terminal device; And a re-visiting step of preferentially providing the accumulated virtual reality contents before the captured image is received from the terminal device when the terminal device revisits the specific place.

Wherein the tracking step tracks the moving route using a photographed image changed by a moving direction and a moving line of the terminal device in real time.

Wherein the managing step comprises: a position checking step of checking whether the terminal device has entered the specific place; And a delivery step of delivering the plurality of content images to the virtual reality contents provider when the terminal device agrees to receive the virtual reality service providing content related to the specific location, .

And the specific location is a location where service can be provided related to at least one content image among the plurality of content images.

According to the apparatus and method for providing contents based on the virtual reality of the present invention, it is possible to track a movement path for moving a specific place based on a photographed image taken while moving at a specific place, It is possible to increase the intuitive awareness of the provided content by providing the virtual reality contents generated by virtualizing the content image and the shot image mapped to the specific location to the terminal device when the user reaches the specific location, It is possible to provide content optimized for the purpose of visiting the place, thereby improving the satisfaction of the content providing service.

1 is a schematic configuration diagram of a content providing system according to an embodiment of the present invention;
2 is a schematic configuration diagram of a terminal device according to an embodiment of the present invention;
3 is a schematic configuration diagram of a hardware system for implementing a terminal device according to an embodiment of the present invention;
4 is a schematic configuration diagram of a content providing apparatus according to an embodiment of the present invention;
5 is a schematic configuration diagram of a hardware system for implementing a content providing apparatus according to an embodiment of the present invention;
6 is a diagram for explaining an operation flow in a terminal device according to an embodiment of the present invention;
FIG. 7 is a flowchart illustrating an operation flow in a hardware system for implementing a terminal device according to an embodiment of the present invention; FIG.
8 is a diagram for explaining an operational flow in a content providing apparatus according to an embodiment of the present invention;
FIG. 9 is a flowchart illustrating an operation flow in a hardware system for implementing a content providing apparatus according to an embodiment of the present invention; FIG.

It is noted that the technical terms used herein are used only to describe specific embodiments and are not intended to limit the invention. It is also to be understood that the technical terms used herein are to be interpreted in a sense generally understood by a person skilled in the art to which the present invention belongs, Should not be construed to mean, or be interpreted in an excessively reduced sense. Further, when a technical term used herein is an erroneous technical term that does not accurately express the spirit of the present invention, it should be understood that technical terms that can be understood by a person skilled in the art are replaced. In addition, the general terms used in the present invention should be interpreted according to a predefined or prior context, and should not be construed as being excessively reduced.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or similar elements throughout the several views, and redundant description thereof will be omitted. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. It is to be noted that the accompanying drawings are only for the purpose of facilitating understanding of the present invention, and should not be construed as limiting the scope of the present invention with reference to the accompanying drawings. The spirit of the present invention should be construed as extending to all modifications, equivalents, and alternatives in addition to the appended drawings.

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

1 is a diagram illustrating a content providing system according to an embodiment of the present invention.

As shown in FIG. 1, the content providing system according to an embodiment of the present invention may include a terminal device 100 and a content providing device 200.

The terminal device 100 receives virtualized content (hereinafter referred to as virtual reality content) related to a specific place from the content providing device 200. [

The specific place in the present invention may be a specific building in an off-line having a unique characteristic that the user of the terminal device 100 visits with a purpose. For example, a particular place may be a building that has unique characteristics, such as shopping in a shopping mall, handling tax-related tasks such as the Internal Revenue Service, or postal services like post offices.

For example, the terminal device 100 may be, for example, a smart phone, a tablet PC, a PDA, or the like, and may be a device capable of driving a camera for image shooting, May all be included.

Here, the content providing medium that allows the terminal device 100 to receive the virtual reality contents includes, for example, an application executed in the terminal device 100 itself or an online service Page. In an embodiment of the present invention, it is assumed that a content providing medium capable of receiving virtual reality contents in the terminal device 100 is an application.

In this regard, in the case of an application that operates as a content providing medium in the terminal device 100, the captured image taken while the terminal device 100 moves in a specific place so that the virtual reality contents can be generated is transmitted to the content providing device 200). Also, the virtual reality contents generated based on the captured image delivered in real time from the content providing apparatus 200 are received and provided to the terminal device 100.

For reference, in the terminal device 100, the current position is checked in order to determine whether or not the vehicle enters the specific place. The current position can be confirmed using, for example, a GPS (Global Positioning System) .

The content providing apparatus 200 is a device for providing virtual reality contents related to a specific place to a plurality of terminals including the terminal apparatus 100. In particular, a service to be provided by the present invention, that is, a terminal device 100, (Hereinafter, referred to as a virtual reality content (hereinafter, referred to as " virtual reality content ") that provides virtual reality content by determining whether the terminal device 100 has reached a specific location based on the captured image, Service).

Here, the specific location is a location for a plurality of shops that are located in a specific place (e.g., a shopping mall), and refers to a shop capable of providing services related to at least one of a plurality of contents related to a specific place. That is, at a specific location, at least one content capable of providing a service at a specific location among a plurality of contents related to a specific location is mapped.

Accordingly, in one embodiment of the present invention, a method for providing a virtualized content related to a specific place based on a captured image taken by a terminal device 100 that has entered a specific place is proposed. Hereinafter, The configuration of the apparatus 100 and the content providing apparatus 200 will be described in detail.

First, a terminal device 100 according to an embodiment of the present invention will be described with reference to FIG.

Here, FIG. 2 is a diagram showing a schematic configuration of a terminal device 100 according to an embodiment of the present invention.

2, the terminal device 100 according to an exemplary embodiment of the present invention provides a virtual reality content service from a location information management unit 110 and a content providing device 200, And a receiving service processing unit (120).

The service processing unit 120 corresponds to an application that is a content providing medium capable of processing virtual reality contents received from the content providing apparatus 200. [

All or at least a part of the configuration of the terminal device 100 including the location information management unit 110 and the service processing unit 120 may be implemented in the form of a software module or a hardware module executed by the processor, And can be implemented in a combined form.

As a result, when the terminal device 100 according to the embodiment of the present invention enters a specific place through the above configurations and agrees to use the virtual reality contents service, the virtual reality contents related to the specific place is received. Will be described concretely with respect to each configuration in the terminal apparatus 100

The location information management unit 110 communicates with the content providing apparatus 200 to confirm that the terminal apparatus 100 has entered a specific place.

More specifically, the location information management unit 110 recognizes a location location terminal (not shown) in a specific location and performs communication with the content providing apparatus 200 operating in conjunction with a location location terminal (not shown) And confirms that the terminal device 100 has entered a specific place.

In the present invention, it is described that communication with the content providing apparatus 200 confirms that the user has entered a specific place. However, the present invention is not limited to the specific content stored in the application operating as the content providing medium in the terminal apparatus 100, It is possible to confirm that the terminal device 100 has entered a specific place by using the location information value of the place.

The service processing unit 120 receives a usage agreement confirmation message for confirming whether to use the virtual reality contents service from the contents providing apparatus 200 after entering the specific place.

That is, the service processing unit 120 receives the usage agreement confirmation message from the content providing apparatus 200 as a text message or a popup message through the application. Then, the service processing unit 120 generates a consent message when the user agrees to use the virtual reality content service, and delivers the agreement message to the content providing apparatus 200. In the case where the user is intending to use the virtual reality content service, And transmits the generated message to the content providing apparatus 200.

Thereafter, when the service processor 120 agrees to use the virtual reality contents service, the service processing unit 120 controls the camera so that a captured image is generated along the direction and the moving direction of the terminal device 100 in real time.

That is, the service processing unit 120 delivers the photographed image generated in real time to the content providing apparatus 200 so that the virtual reality contents can be generated based on the photographed image photographed while moving at a specific place.

On the other hand, if the service processing unit 120 does not agree with the use of the virtual reality contents service, the service processing unit 120 controls the camera not to be driven. At this time, it is preferable that the camera operates according to the user's control regardless of using the virtual reality contents service.

Upon receipt of the virtual reality content from the content providing apparatus 200, the service processing unit 120 controls the received virtual reality content to be displayed through a display unit (not shown).

All or at least a part of the configuration of the terminal device 100 including the location information management unit 110 and the service processing unit 120 may be implemented in the form of a software module or a hardware module executed by the processor, And can be implemented in a combined form.

For reference, the virtual reality contents received from the content providing apparatus 200 track a movement route for moving a specific place based on a photographed image taken while moving at a specific place, and reach a specific position A content image mapped to a specific location and a shot image are virtualized, and a specific description related thereto will be described in a specific description of the content providing apparatus 200.

In each of the above-described configurations of the terminal device 100, it is mentioned that the present invention can be implemented in the form of a software module or a hardware module executed by a processor, or a combination of a software module and a hardware module.

As such, any combination of software modules, hardware modules, or software modules and hardware modules executed by a processor may be implemented in a hardware system (e.g., a computer system).

Therefore, a hardware system 1000 for implementing the terminal device 100 according to an embodiment of the present invention will be described with reference to FIG.

For the sake of reference, the following description is an example for implementing each configuration in the terminal device 100 described above in the hardware system 1000. It is to be noted that each configuration and operation thereof may be different from the actual system It should be done.

3 is a diagram illustrating a structure of a hardware system 1000 for implementing a terminal device 100 according to an embodiment of the present invention.

3, a hardware system 1000 in accordance with an embodiment of the present invention may have a configuration including a processor 1100, a memory interface 1200, and a peripheral device interface 1300. [

Each such configuration in hardware system 1000 may be a separate component or integrated into one or more integrated circuits, and each of these configurations may be coupled to a bus system (not shown).

Here, in the case of a bus system, any one or more individual physical buses, communication lines / interfaces, and / or multi-drop or point-to-point links connected by appropriate bridges, adapters, and / And abstraction that represents point-to-point connections.

The processor 1100 is responsible for executing various software modules stored in the memory 1210 by communicating with the memory 1210 via the memory interface 1200 to perform various functions in the hardware system.

2 may be stored in the memory 1210 in the form of a software module, and the other operating systems may be stored in the memory 1210. In the memory 1210, the location information management unit 110 and the service processing unit 120, Can be additionally stored.

In the case of the above operating systems (eg, embedded operating systems such as I-OS, Android, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS or VxWorks), typical system tasks (eg memory management, Instructions, software components, and / or drivers that control and manage the various hardware modules and software modules, and facilitate communication between the various hardware modules and software modules.

For reference, the memory 1210 may include a memory hierarchy including, but not limited to, a cache, a main memory, and a secondary memory, such as RAM (e.g., SRAM, DRAM , DDRAM), ROM, FLASH, magnetic and / or optical storage devices such as disk drives, magnetic tape, compact disc (CD) and digital video disc (DVD)

The peripheral device interface 1300 serves to enable communication between the processor 1100 and the peripheral device.

In one embodiment of the present invention, for example, a positioning device 1310, a communication device 1320, and an input / output device 1330 may be included in the above peripheral device to provide different functions to the computer system .

Here, the position determination device 1310 refers to a device that can confirm the current position of the terminal device 100, for example, such as a GPS (Global Positioning System).

The communication device 1320 performs a function of providing a communication function with another device. For example, an antenna system, an RF transceiver, at least one amplifier, a tuner, at least one oscillator, a digital signal processor, (CODEC) chipset, and memory, and the like, and may include known circuitry to perform this function.

Communication protocols supported by the communication device 1320 include long term evolution (LTE), time division multiple access (TDMA), code division multiple access (CDMA), global system for mobile communications (GSM) (Enhanced Data GSM Environment), Wideband Code Division Multiple Access (W-CDMA), Wi-Fi (IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n) voice over Internet Protocol (VoIP), e-mail, instant messaging, and short message service (SMS) protocols, and protocols for providing communication environments with other devices.

In addition, the input / output device 1330 functions as a controller for controlling an I / O device interlocked with other hardware systems. In an embodiment of the present invention, for example, a display 1321 capable of displaying various service screens Can be controlled.

As a result, each configuration in the content providing apparatus 200, which is stored in the form of a software module in the memory 1210, is executed by the processor 1100 via the memory interface 1100 and the peripheral device interface 1300 It is possible to receive the virtual reality contents related to the specific place from the content providing device 200 by performing communication with the position determination device 1310 and the communication device 1320. [

Hereinafter, each configuration in the hardware system 1000 related to the terminal device 100 will be described in detail with reference to FIG. 3. For convenience of explanation, the terminal 1210, which is stored in the memory 1210 in the form of a software module, It is assumed that each configuration in the apparatus 100 is in a state executed by the processor 1100.

The location information management unit 110 communicates with the content providing apparatus 200 to confirm that the terminal apparatus 100 has entered a specific place.

More specifically, the location information management unit 110 recognizes a location location terminal (not shown) in a specific location via the location confirmation apparatus 1310 and transmits the location location information to the location location terminal (not shown) through the communication apparatus 1320. [ And communicates with the content providing apparatus 200 operating in cooperation with the content providing apparatus 200 to confirm that the terminal apparatus 100 has entered a specific place.

In the present invention, it is described that communication with the content providing apparatus 200 confirms that the user has entered a specific place. However, the present invention is not limited to the specific content stored in the application operating as the content providing medium in the terminal apparatus 100, It is possible to confirm that the terminal device 100 has entered a specific place by using the location information value of the place.

The service processing unit 120 receives a usage agreement confirmation message for confirming whether to use the virtual reality contents service from the contents providing apparatus 200 after entering the specific place.

That is, the service processing unit 120 receives a usage agreement confirmation message from the content providing apparatus 200 via the communication device 1320 as a text message or a popup message through the application. Then, the service processing unit 120 generates a consent message when the user agrees to use the virtual reality content service, and then transmits the agreement message to the content providing apparatus 200 through the communication device 1320. When the user uses the virtual reality content service And then transmits the generated message to the content providing apparatus 200 through the communication device 1320. [

Thereafter, when the service processor 120 agrees to use the virtual reality contents service, the service processor 120 drives the camera through the input / output device 1330 so that the captured image is generated along the direction and the moving direction of the terminal device 100 in real- And controls the display 1331.

That is, the service processing unit 120 transmits the captured image, which is displayed through the display 1331 in real time, to the content server 1320 through the communication device 1320, in real time so that the virtual reality content can be generated based on the captured image captured while moving in a specific place To the providing apparatus 200.

On the other hand, the service processing unit 120 controls the input / output device 1330 so as not to drive the camera when the user does not agree to use the virtual reality contents service. At this time, it is preferable that the camera operates according to the user's control regardless of using the virtual reality contents service.

Upon receiving the virtual reality content from the content providing device 200 through the communication device 1320, the service processing unit 120 controls the display 1331 via the input / output device 1330 to display the virtual reality content do.

Hereinafter, the terminal device 100 and the hardware system for implementing the same will be described with reference to FIG. 4, and the configuration of the content providing apparatus 200 according to an embodiment of the present invention will be described.

FIG. 4 is a diagram showing a schematic configuration of a content providing apparatus 200 according to an embodiment of the present invention.

4, the content providing apparatus 200 according to an exemplary embodiment of the present invention includes an operation management unit 210 for managing a plurality of content images related to a specific place, The content image related to the specific location is virtualized based on the result of determining whether the terminal device 100 has reached the specific position based on the captured image of the terminal device 100, And a virtual reality contents providing unit 220 for providing virtual reality contents.

The content providing apparatus 200 according to an exemplary embodiment of the present invention may further include a history management unit 230 for accumulating and storing virtual reality contents provided to the terminal device 100, Visit management unit 240 for preferentially providing the virtual reality contents accumulated before the photographed image is received from the terminal device 100 when the location is revisited.

All or at least a part of the configuration of the content providing apparatus 200 including the operation management unit 210, the virtual reality content providing unit 220, the history management unit 230 and the revisit management unit 240 is executed by the processor May be implemented in the form of a software module or a hardware module, or a combination of a software module and a hardware module.

As a result, the content providing apparatus 200 according to an embodiment of the present invention tracks a movement route for moving a specific place on the basis of a photographed image while moving a specific place through the above configurations, and tracks a movement route The virtual reality content generated by virtualizing the content image and the shot image mapped to a specific location can be provided to the terminal device 100. In the following description, Will be described in detail.

The operation management unit 210 manages the overall operation status of a plurality of content images related to a specific place.

More specifically, the operation management unit 210 includes an entry confirmation unit 211 and a content management unit 212.

The entry confirmation unit 211 determines whether or not the terminal device 100 has entered a specific place by interlocking with a location measurement terminal (not shown) in a specific place.

That is, the entrance confirming unit 211 determines that the terminal device 100 has entered a specific place when a position signal of the terminal device 100 is detected through a location measurement terminal (not shown) in a specific place.

When the current location of the terminal device 100 in the specific location is received, the entry confirming unit 211 can confirm the location where the terminal device 100 has entered among various entry paths in the specific location.

Thereafter, the entry confirming unit 211 notifies the content managing unit 212 of the entry of the terminal device 100.

The content management unit 212 manages a plurality of content related to a specific place and an image related to the content (hereinafter referred to as a content image), and when the terminal device 100 agrees with the virtual reality content service, the virtual reality content is provided To the virtual reality contents providing unit 220 so that the plurality of contents images can be displayed.

More specifically, the content management unit 212 previously receives and stores content and content images that can be provided at a plurality of specific locations located in a specific location.

For example, if the specific location is a shopping mall and the specific location is a plurality of stores, the content management unit 212 receives and stores contents that can be provided from a plurality of stores in advance before the virtual reality contents service is provided. At this time, the content management unit 212 receives a plurality of contents related to a specific place, and simultaneously receives and stores relevant content images.

When the entry confirmation unit 211 confirms that the terminal device 100 has entered a specific place, the content management unit 212 generates a usage agreement confirmation message for confirming whether or not to use the virtual reality contents service, And transmits it to the terminal device 100.

Thereafter, the content management unit 212 receives a response message to the usage agreement confirmation message. That is, the content management unit 212 receives a consent message when the user agrees to use the virtual reality content service from the terminal device 100, and receives a fine-grained message in the case of using the virtual reality content service.

Upon receipt of the agreement message, the content management unit 212 distributes a plurality of content images related to the specific location to the virtual reality content provider 220 at a point of time when the terminal device 100 has agreed to receive the virtual reality content .

If the message is received, the content management unit 212 determines that the terminal apparatus 100 does not use the virtual reality contents service even if the terminal apparatus 100 enters a specific place, Lt; RTI ID = 0.0 > 220 < / RTI >

The virtual reality contents providing unit 220 provides virtual reality contents based on a shot image transmitted from the terminal device 100 entering a specific place.

More specifically, the virtual reality contents providing unit 220 includes a tracking unit 221, a detection unit 222, and a generation unit 223.

The tracking unit 221 tracks the movement path for moving the specific place based on the photographed image.

That is, the tracking unit 221 receives the photographed image photographed along the direction and the copper line moving in real time within a specific place from the terminal device 100. [ The tracking unit 221 tracks the movement path of the terminal device 100 within a specific place by using the captured image changed by the direction and the movement direction of the terminal device 100 in real time.

The tracking unit 221 then transmits the tracking result of the moving path that tracks the moving path of the terminal device 100 in real time to the detecting unit 222.

The detection unit 222 determines whether the terminal device 100 has reached a specific position based on the result of tracking the movement path.

That is, the detection unit 222 compares the captured image with an image (hereinafter referred to as a specific position image) related to a specific position that has been stored while tracking the movement path of the terminal device 100 based on the result of tracking the movement path, It is determined whether or not a specific position has been reached.

Here, the specific location image is a specific location, such as an interior of a particular location (e.g., an interior signboard, a sale product, etc.), an exterior interior (e.g., external signboards, initials, mutual feature information, etc.) You can include all the images you can see. In this regard, for example, if a particular location is a bakery, any image that can identify a particular bakery, such as a bakery business name, logo, merchandise, etc., can be a specific location image.

More specifically, the detecting unit 222 compares the photographed image included in the result of traversing the path and the previously stored specific position image.

That is, since the photographed image photographed while the terminal device 100 entering the specific place moves is related to the position of the terminal device 100, the photographed image included on the route is compared with the previously stored specific location image, A specific position image with a high value is detected and it is judged whether or not a specific position is reached.

As a result, the detection unit 222 determines that the terminal device 100 has arrived at a specific position when it is confirmed that the specific location image and the highest matching image having the highest matching value are traced on the basis of the captured image , And detects a content image mapped to a specific position. Then, the detection unit 222 delivers the content image and the movement path tracking result mapped to the specific position to the generation unit 223. [

The generation unit 223 generates the virtual reality content using the content image mapped to the specific location received from the detection unit 222 and the result of tracing the movement route.

That is, the generation unit 223 detects the photographed image of the specific location included in the movement route tracking result, and creates the virtual reality content by combining the photographed image of the specific location with the content image based on the virtualization rendering technique do.

Here, the virtualization rendering technique may be an Augmented Reality (AR) technique, which is an area of a virtual reality. The virtualization rendering technique may be applied to an actual photographed image photographed by the terminal device 100, So that a virtual content image appears to exist at an actual specific position.

The generating unit 223 generates virtual reality contents for at least one specific location on the movement path and provides the generated virtual reality contents to the terminal device 100. The generating unit 223 generates virtual reality contents Route guidance information for other specific locations providing services similar to the virtual reality contents of the selected specific location may be additionally synthesized and provided to the virtual reality contents.

For example, when the terminal device 100 selects the virtual reality contents mapped to the A company bakery among the virtual reality contents for at least one specific location existing on the movement route, the generation section 223 generates the virtual reality contents, Another company B providing bakery service similar to that of the bakery company C, and a bakery & coffee specialty shop C company.

The generating unit 223 then generates route guidance information for reaching the other B bakery and C bakery & coffee specialty shops with the current position of the terminal device 100, that is, the A company bakery as a starting point, May be further synthesized and provided.

Of course, the route guidance information provided additionally may be additionally synthesized to the virtual reality contents mapped to the A-company bakery selected by the terminal device 100, or may be separately synthesized and provided through a pop-up window.

The history management unit 230 accumulates and stores all the virtual reality contents provided to the terminal device 100 in the process of using the virtual reality contents service. That is, the history management unit 230 accumulates and stores the virtual reality contents by mapping the identification information of the terminal device 100. [ Here, the identification information of the terminal device 100 can be any information that can be confirmed as the terminal device 100 (e.g., user information, telephone number, etc.).

Then, when the terminal device 100 revisits a specific place, the history management unit 230 delivers the accumulated virtual reality contents to the re-visit management unit 240 to enable rapid virtual reality contents service.

The re-visit management unit 240 preferentially provides the accumulated virtual reality contents to the terminal device 100 based on the result of determining whether the terminal device 100 revisits the specific place.

More specifically, when it is confirmed that the terminal device 100 has entered a specific place, the rediscovery management unit 240 determines whether or not the terminal device 100 has visited again based on the identification information of the terminal device 100 .

That is, the re-visit management unit 240 determines whether the virtual reality content mapped to the identification information of the terminal device 100 exists. When the virtual reality content mapped to the identification information of the terminal device 100 exists, the rediscovery management unit 240 determines that the terminal device 100 enters a specific place and is provided with a virtual reality content service. If the virtual reality contents mapped to the identification information of the terminal device 100 do not exist, the terminal 100 determines that the virtual reality contents service is not used even if the terminal 100 visits a specific place for the first time or visits a specific place.

Thereafter, when the terminal device 100 revisits a specific place, the re-visit management unit 240 determines whether the virtual reality contents accumulated through the previous visit before the terminal device 100 moves to a specific place and collects the captured image .

At this time, if there are a plurality of purposes for visiting a specific place, the re-visiting management unit 240 can provide the virtual reality contents selected for the visited purpose separately for each visited purpose. Therefore, it is possible to provide the virtual reality contents more quickly when returning for the same purpose as the previous visit.

Accordingly, when a user who visited a specific place returns to the same destination for the same purpose, he / she receives the desired virtual reality contents at the same time as entering the specific place without performing the above-described virtual reality contents generation process, It can be achieved.

Meanwhile, in each of the configurations of the content providing apparatus 200 described above, it is noted that the present invention can be implemented in the form of a software module or a hardware module executed by a processor, or a combination of a software module and a hardware module have.

As such, any combination of software modules, hardware modules, or software modules and hardware modules executed by a processor may be implemented in a hardware system (e.g., a computer system).

Therefore, a hardware system for implementing the content providing apparatus 200 according to an embodiment of the present invention will be described below. Before describing the present invention, it is to be understood that the following description is merely an example for implementing each configuration in the content providing apparatus 200 described above in the computer system, and it is needless to say that the configuration and the operation thereof may be different from the actual system .

5 is a block diagram illustrating a hardware system for implementing a content providing apparatus 200 according to an embodiment of the present invention.

5, a hardware system 3000 according to an embodiment of the present invention may have a configuration including a processor 3100, a memory interface 3200, and a peripheral device interface 3300. [

Each such configuration in the hardware system 3000 can be a separate component or integrated into one or more integrated circuits, and each of these configurations can be coupled to a bus system (not shown).

Here, in the case of a bus system, any one or more individual physical buses, communication lines / interfaces, and / or multi-drop or point-to-point links connected by appropriate bridges, adapters, and / And abstraction that represents point-to-point connections.

Processor 3100 is responsible for executing various software modules stored in memory 3210 by communicating with memory 3210 via memory interface 3200 to perform various functions in the hardware system.

The memory 3210 includes an operation management unit 210, a virtual reality content providing unit 220, a history management unit 230, and a revisiting management unit 240, which are components of the content providing apparatus 200 described with reference to FIG. May be stored in the form of a software module, and other operating systems may be additionally stored.

For the above operating systems (eg, embedded operating systems such as I-OS, Android, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or VxWorks), common system tasks Instruction set, software component and / or driver for controlling and managing the various hardware modules, storage devices, power management, etc.) and facilitates communication between the various hardware modules and software modules.

For reference, the memory 3210 may include a memory hierarchy including, but not limited to, a cache, a main memory, and a secondary memory, such as RAM (e.g., SRAM, DRAM , DDRAM), ROM, FLASH, magnetic and / or optical storage devices such as disk drives, magnetic tape, compact disc (CD) and digital video disc (DVD)

The peripheral device interface 3300 serves to enable communication between the processor 3100 and the peripheral device.

In the case of the above peripheral device, for example, a communication device 3310 may be included in an embodiment of the present invention for providing different functions to a computer system.

Here, the communication device 3310 performs the function of providing communication with other devices, and may be implemented by, for example, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, (CODEC) chipset, and memory, and the like, and may include known circuitry to perform this function.

Communication protocols supported by the communication device 3310 include long term evolution (LTE), time division multiple access (TDMA), code division multiple access (CDMA), global system for mobile communications (GSM) (Enhanced Data GSM Environment), Wideband Code Division Multiple Access (W-CDMA), Wi-Fi (IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n) voice over Internet Protocol (VoIP), e-mail, instant messaging, and short message service (SMS) protocols, and protocols for providing communication environments with other devices.

As a result, each configuration in the content providing apparatus 200 stored in the form of a software module in the memory 3210 is transferred to the memory interface 3200 via the memory interface 3200 and the peripheral device interface 3300 when executed by the processor 3100 The communication with the communication device 3310 is performed so that the captured image related to the position of the terminal device 100 in a specific place is confirmed and the terminal device 100 The virtual reality contents related to the specific location can be provided to the terminal device 100 based on the result of determining whether the specific location has been reached.

Hereinafter, each configuration in the hardware system 3000 related to the content providing apparatus 200 will be described in detail with reference to FIG. 5. For convenience of explanation, the configuration is stored in the memory 3210 in the form of a software module It is assumed that each configuration in the content providing apparatus 200 is in a state executed by the processor 3100 via the memory interface 3200. [

The operation management unit 210 manages the overall operation status of a plurality of content images related to a specific place.

More specifically, the operation management unit 210 includes an entry confirmation unit 211 and a content management unit 212.

The entry confirmation unit 211 determines whether the terminal device 100 enters a specific place using a position signal of the terminal device 100 received from a position measurement terminal (not shown) in a specific place via the communication device 3310 Or not.

That is, the entrance confirming unit 211 determines that the terminal device 100 has entered a specific place when a position signal of the terminal device 100 is detected through a location measurement terminal (not shown) in a specific place.

When the current location of the terminal device 100 in the specific location is received, the entry confirming unit 211 can confirm the location where the terminal device 100 has entered among various entry paths in the specific location.

Thereafter, the entry confirming unit 211 notifies the content managing unit 212 of the entry of the terminal device 100.

The content management unit 212 manages a plurality of content related to a specific place and an image related to the content (hereinafter referred to as a content image), and based on the agreement message received through the communication device 3310, Detects a point of time when the terminal 100 agrees to the actual content service, and transmits a plurality of content images to the virtual reality content provider 220 so that the virtual reality contents can be provided at a time point when the terminal device 100 agrees with the virtual reality content service .

More specifically, the content management unit 212 previously receives and stores content and content images that can be provided at a plurality of specific locations located in a specific location.

For example, if the specific location is a shopping mall and the specific location is a plurality of stores, the content management unit 212 receives and stores contents that can be provided from a plurality of stores in advance before the virtual reality contents service is provided. At this time, the content management unit 212 receives a plurality of contents related to a specific place, and simultaneously receives and stores relevant content images.

When the entry confirmation unit 211 confirms that the terminal device 100 has entered a specific place, the content management unit 212 generates a usage agreement confirmation message for confirming whether or not to use the virtual reality contents service, And transmits it to the terminal device 100 through the communication device 3310.

Thereafter, the content management unit 212 receives the agreement message through the communication device 3310 when the terminal device 100 agrees to use the virtual reality content service, and transmits the agreement message to the communication device 3310). ≪ / RTI >

Upon receipt of the agreement message, the content management unit 212 distributes a plurality of content images related to the specific location to the virtual reality content provider 220 at a point of time when the terminal device 100 has agreed to receive the virtual reality content .

If the terminal device 100 receives a message with a false address, it is determined that the terminal device 100 does not use the virtual reality contents service even if the terminal device 100 enters the specific location, so that the virtual reality contents provider 220 .

The virtual reality contents providing unit 220 provides virtual reality contents based on a shot image transmitted from the terminal device 100 entering the specific place through the communication device 3310. [

More specifically, the virtual reality contents providing unit 220 includes a tracking unit 221, a detection unit 222, and a generation unit 223.

The tracking unit 221 tracks the movement path for moving the specific place based on the photographed image.

That is, the tracking unit 221 receives the photographed image photographed along the moving direction and the moving line in real time in the specific place from the terminal device 100 via the communication device 3310. [ The tracking unit 221 tracks the movement path of the terminal device 100 in a specific place by using the captured image changed by the moving direction and the moving direction of the terminal device 100 in a specific place.

The tracking unit 221 then transmits the tracking result of the moving path that tracks the moving path of the terminal device 100 in real time to the detecting unit 222.

The detection unit 222 determines whether the terminal device 100 has reached a specific position based on the result of tracking the movement path.

That is, the detection unit 222 compares the captured image with an image (hereinafter referred to as a specific position image) related to a specific position that has been stored while tracking the movement path of the terminal device 100 based on the result of tracking the movement path, It is determined whether or not a specific position has been reached.

Here, the specific location image is a specific location, such as an interior of a particular location (e.g., an interior signboard, a sale product, etc.), an exterior interior (e.g., external signboards, initials, mutual feature information, etc.) You can include all the images you can see. In this regard, for example, if a particular location is a bakery, any image that can identify a particular bakery, such as a bakery business name, logo, merchandise, etc., can be a specific location image.

More specifically, the detecting unit 222 compares the photographed image included in the result of traversing the path and the previously stored specific position image.

That is, since the photographed image photographed while the terminal device 100 entering the specific place moves is related to the position of the terminal device 100, the photographed image included on the route is compared with the previously stored specific location image, A specific position image with a high value is detected and it is judged whether or not a specific position is reached.

As a result, the detection unit 222 determines that the terminal device 100 has arrived at a specific position when it is confirmed that the specific location image and the highest matching image having the highest matching value are traced on the basis of the captured image , And detects a content image mapped to a specific position. Then, the detection unit 222 delivers the content image and the movement path tracking result mapped to the specific position to the generation unit 223. [

The generation unit 223 generates the virtual reality content using the content image mapped to the specific location received from the detection unit 222 and the result of tracing the movement route.

That is, the generation unit 223 detects the photographed image of the specific location included in the movement route tracking result, and creates the virtual reality content by combining the photographed image of the specific location with the content image based on the virtualization rendering technique do.

Here, the virtualization rendering technique may be an Augmented Reality (AR) technique, which is an area of a virtual reality. The virtualization rendering technique may be applied to an actual photographed image photographed by the terminal device 100, So that a virtual content image appears to exist at an actual specific position.

On the other hand, when the virtual reality contents for a specific location on the movement route are generated and provided to the terminal device 100, the generation unit 223 selects the virtual reality contents at a specific location from the terminal device 100 , Route guidance information for other specific locations providing services similar to the virtual reality contents of the selected specific location may be additionally synthesized and provided to the virtual reality contents.

For example, when the terminal device 100 selects the virtual reality contents mapped to the A company bakery among the virtual reality contents for at least one specific location existing on the movement route, the generation section 223 generates the virtual reality contents, Another company B providing bakery service similar to that of the bakery company C, and a bakery & coffee specialty shop C company.

The generating unit 223 then generates route guidance information for reaching the other B bakery and C bakery & coffee specialty shops with the current position of the terminal device 100, that is, the A company bakery as a starting point, May be further synthesized and provided.

Of course, the route guidance information provided additionally may be additionally synthesized to the virtual reality contents mapped to the A-company bakery selected by the terminal device 100, or may be separately synthesized and provided through a pop-up window.

The history management unit 230 accumulates and stores all the virtual reality contents provided to the terminal device 100 in the course of using the virtual reality contents service. That is, the history management unit 230 accumulates and stores the virtual reality contents by mapping the identification information of the terminal device 100. [ Here, the identification information of the terminal device 100 can be any information that can be confirmed as the terminal device 100 (e.g., user information, telephone number, etc.).

If the history management unit 230 determines that the terminal device 100 has visited the specific place again based on the position signal of the terminal device 100 received via the communication device 3310, And transmits the accumulated virtual reality contents to the revisit management unit 240 so that the virtual reality contents can be accumulated.

The re-visiting management unit 240 determines whether or not the terminal device 100 has been re-visited based on the position signal of the terminal device 100 received via the communication device 3310 And provides the virtual reality contents to the terminal device 100 preferentially.

More specifically, when it is confirmed that the terminal device 100 has entered a specific place based on the position signal of the terminal device 100 received via the communication device 3310, the re- 100 of the terminal device 100 based on the identification information of the terminal device 100.

That is, the re-visit management unit 240 determines whether the virtual reality content mapped to the identification information of the terminal device 100 exists. When the virtual reality content mapped to the identification information of the terminal device 100 exists, the rediscovery management unit 240 determines that the terminal device 100 enters a specific place and is provided with a virtual reality content service. If the virtual reality contents mapped to the identification information of the terminal device 100 do not exist, the terminal 100 determines that the virtual reality contents service is not used even if the terminal 100 visits a specific place for the first time or visits a specific place.

When the terminal device 100 returns to a specific place, the re-visit management section 240 determines that the virtual reality contents accumulated through the previous visit before the terminal device 100 moves to a specific place and collects the captured image .

At this time, if there are a plurality of purposes for visiting a specific place, the re-visiting management unit 240 can provide the virtual reality contents selected for the visited purpose separately for each visited purpose. Accordingly, the virtual reality contents can be provided more quickly when the user returns to the same destination for the same purpose.

In this way, when a user who visited a specific place returns to the same destination for the same purpose, he / she can receive the desired virtual reality contents at the same time as entering the specific place without performing the above-described virtual reality contents generation process, It can be achieved.

Hereinafter, the operation flow in the terminal device 100 according to the embodiment of the present invention and the hardware system 1000 for implementing the same will be described with reference to FIG. 6 and FIG. 7, 1 to 5 will be described with reference to corresponding reference numerals.

First, referring to FIG. 6, an operation flow in the terminal device 100 according to an embodiment of the present invention will be described.

First, the location information management unit 110 communicates with the content providing apparatus 200 to confirm that the terminal apparatus 100 has entered a specific place (S100).

More specifically, the location information management unit 110 recognizes a location location terminal (not shown) in a specific location and performs communication with the content providing apparatus 200 operating in conjunction with a location location terminal (not shown) And confirms that the terminal device 100 has entered a specific place.

In the present invention, it is described that communication with the content providing apparatus 200 confirms that the user has entered a specific place. However, the present invention is not limited to the specific content stored in the application operating as the content providing medium in the terminal apparatus 100, It is possible to confirm that the terminal device 100 has entered a specific place by using the location information value of the place.

The service processing unit 120 receives a usage agreement confirmation message for confirming whether or not to use the virtual reality contents service from the content providing apparatus 200 after entering the specific place (S110).

That is, the service processing unit 120 receives the usage agreement confirmation message from the content providing apparatus 200 as a text message or a popup message through the application.

Thereafter, the service processing unit 120 generates a response message for the use agreement message and transmits the response message to the content providing apparatus 200 (S120).

That is, when the user agrees to use the virtual reality content service, the service processing unit 120 generates a consent message and transfers the agreement message to the content providing apparatus 200. In the case where the user is intending to use the virtual reality content service, And transmits the generated message to the content providing apparatus 200.

Thereafter, when the service processor 120 agrees to use the virtual reality contents service, the service processing unit 120 controls the camera so that a captured image is generated along the direction and the moving direction of the terminal device 100 in real time.

That is, the service processing unit 120 delivers the photographed image generated in real time to the content providing apparatus 200 so that the virtual reality contents can be generated based on the photographed image photographed while moving at a specific place (S130).

On the other hand, if the service processing unit 120 does not agree with the use of the virtual reality contents service, the service processing unit 120 controls the camera not to be driven. At this time, it is preferable that the camera operates according to the user's control regardless of using the virtual reality contents service.

Upon receipt of the virtual reality content from the content providing apparatus 200, the service processing unit 120 controls the received virtual reality content to be displayed through a display unit (S140, S150).

Hereinafter, with reference to FIG. 7, the operation flow of each configuration in the hardware system 1000 for implementing the terminal device 100 according to the embodiment of the present invention will be described in detail.

For convenience of explanation, it is assumed that each configuration in the terminal device 100 stored in the memory 1210 in the form of a software module is executed by the processor 1100.

First, the location information management unit 110 communicates with the content providing apparatus 200 to confirm that the terminal apparatus 100 has entered a specific place (S100 ').

More specifically, the location information management unit 110 recognizes a location location terminal (not shown) in a specific location via the location confirmation apparatus 1310 and transmits the location location information to the location location terminal (not shown) through the communication apparatus 1320. [ And communicates with the content providing apparatus 200 operating in cooperation with the content providing apparatus 200 to confirm that the terminal apparatus 100 has entered a specific place.

In the present invention, it is described that communication with the content providing apparatus 200 confirms that the user has entered a specific place. However, the present invention is not limited to the specific content stored in the application operating as the content providing medium in the terminal apparatus 100, It is possible to confirm that the terminal device 100 has entered a specific place by using the location information value of the place.

After entering the specific place, the service processing unit 120 receives a usage agreement confirmation message for confirming whether to use the virtual reality contents service from the content providing device 200 (S110 ').

That is, the service processing unit 120 receives a usage agreement confirmation message from the content providing apparatus 200 via the communication device 1320 as a text message or a popup message through the application.

Thereafter, the service processing unit 120 generates a response message for the usage agreement message and transmits the response message to the content providing apparatus 200 (S120 ').

That is, the service processing unit 120 generates a consent message when the user agrees to use the virtual reality content service, and then transmits the agreement message to the content providing apparatus 200 through the communication device 1320. When the user uses the virtual reality content service And then transmits the generated message to the content providing apparatus 200 through the communication device 1320. [

Thereafter, when the service processor 120 agrees to use the virtual reality contents service, the service processor 120 drives the camera through the input / output device 1330 so that the captured image is generated along the direction and the moving direction of the terminal device 100 in real- And controls the display 1331.

That is, the service processing unit 120 transmits the captured image, which is displayed through the display 1331 in real time, to the content server 1320 through the communication device 1320, in real time so that the virtual reality content can be generated based on the captured image captured while moving in a specific place To the providing apparatus 200 (S130 ').

On the other hand, the service processing unit 120 controls the input / output device 1330 so as not to drive the camera when the user does not agree to use the virtual reality contents service. At this time, it is preferable that the camera operates according to the user's control regardless of using the virtual reality contents service.

Upon receiving the virtual reality content from the content providing apparatus 200 through the communication device 1320, the service processing unit 120 controls the display 1331 via the input / output device 1330 so that the virtual reality content is displayed (S140`, S150`).

The operation flow in the hardware system 1000 for implementing the terminal device 100 and the terminal device 100 will be described with reference to FIGS. 8 and 9, A description will be given of an operation flow in the system 200 and the hardware system 3000 for implementing the same.

First, an operation flow in the content providing apparatus 200 according to an embodiment of the present invention will be described with reference to FIG.

First, the operation management unit 210 manages the overall operation status of a plurality of content images related to a specific place (S200).

More specifically, the operation management unit 210 includes an entry confirmation unit 211 and a content management unit 212.

The entry confirmation unit 211 determines whether or not the terminal device 100 has entered a specific place by interlocking with a location measurement terminal (not shown) in a specific place.

That is, the entrance confirming unit 211 determines that the terminal device 100 has entered a specific place when a position signal of the terminal device 100 is detected through a location measurement terminal (not shown) in a specific place.

When the current location of the terminal device 100 in the specific location is received, the entry confirming unit 211 can confirm the location where the terminal device 100 has entered among various entry paths in the specific location.

Thereafter, the entry confirming unit 211 notifies the content managing unit 212 of the entry of the terminal device 100.

The content management unit 212 manages a plurality of contents related to a specific place and transmits a plurality of content images to the virtual reality contents service such that the virtual reality contents can be provided at the time when the terminal device 100 agrees with the virtual reality contents service. (220).

More specifically, the content management unit 212 previously receives and stores content and content images that can be provided at a plurality of specific locations located in a specific location.

For example, if the specific location is a shopping mall and the specific location is a plurality of stores, the content management unit 212 receives and stores contents that can be provided from a plurality of stores in advance before the virtual reality contents service is provided. At this time, the content management unit 212 receives a plurality of contents related to a specific place, and simultaneously receives and stores relevant content images.

When the entry confirmation unit 211 confirms that the terminal device 100 has entered a specific place, the content management unit 212 generates a usage agreement confirmation message for confirming whether or not to use the virtual reality contents service , And transmits it to the terminal device 100 (S200).

Thereafter, the content management unit 212 receives a response message to the use agreement message (S210). That is, the content management unit 212 receives a consent message when the user agrees to use the virtual reality content service from the terminal device 100, and receives a fine-grained message in the case of using the virtual reality content service.

Upon receipt of the agreement message, the content management unit 212 delivers the content image related to the specific location to the virtual reality content providing unit 220 at a point of time when the terminal device 100 has agreed to receive the virtual reality content .

If the terminal device 100 receives a message with a false address, it is determined that the terminal device 100 does not use the virtual reality contents service even if the terminal device 100 enters the specific location, so that the virtual reality contents provider 220 .

The virtual reality contents providing unit 220 provides virtual reality contents based on a shot image transmitted from the terminal device 100 entering a specific place.

More specifically, the virtual reality contents providing unit 220 includes a tracking unit 221, a detection unit 222, and a generation unit 223.

The tracking unit 221 tracks the movement path for moving the specific place based on the photographed image.

That is, the tracking unit 221 receives the photographed image photographed along the moving direction and the moving line in real time within the specific place from the terminal device 100 (S230). The tracking unit 221 tracks the movement path of the terminal device 100 in a specific place using the captured image changed by the direction and the movement direction of the terminal device 100 in real time (S240).

The tracking unit 221 then transmits the tracking result of the moving path that tracks the moving path of the terminal device 100 in real time to the detecting unit 222.

The detecting unit 222 determines whether the terminal device 100 has reached a specific position based on the result of tracking the movement path (S250).

That is, the detection unit 222 compares the captured image with an image (hereinafter referred to as a specific position image) related to a specific position that has been stored while tracking the movement path of the terminal device 100 based on the result of tracking the movement path, It is determined whether or not a specific position has been reached.

Here, the specific location image is a specific location, such as an interior of a particular location (e.g., an interior signboard, a sale product, etc.), an exterior interior (e.g., external signboards, initials, mutual feature information, etc.) You can include all the images you can see. In this regard, for example, if a particular location is a bakery, any image that can identify a particular bakery, such as a bakery business name, logo, merchandise, etc., can be a specific location image.

More specifically, the detecting unit 222 compares the photographed image included in the result of traversing the path and the previously stored specific position image.

That is, since the photographed image photographed while the terminal device 100 entering the specific place moves is related to the position of the terminal device 100, the photographed image included on the route is compared with the previously stored specific location image, A specific position image with a high value is detected and it is judged whether or not a specific position is reached.

As a result, the detection unit 222 determines that the terminal device 100 has arrived at a specific position when it is confirmed that the specific location image and the highest matching image having the highest matching value are traced on the basis of the captured image , And detects a content image mapped to a specific position. Then, the detection unit 222 delivers the content image and the movement path tracking result mapped to the specific position to the generation unit 223. [

The generating unit 223 generates the virtual reality contents using the content image mapped to the specific location received from the detecting unit 222 and the result of tracing the movement route (S270).

That is, the generation unit 223 detects the photographed image of the specific location included in the movement route tracking result, and creates the virtual reality content by combining the photographed image of the specific location with the content image based on the virtualization rendering technique do.

Then, the virtual reality contents providing unit 220 transmits the virtual reality contents to the terminal device 100 so that the virtual reality contents can be displayed on the terminal device 100 (S280).

The generating unit 223 generates virtual reality contents for at least one specific location on the movement path and provides the generated virtual reality contents to the terminal device 100. The generating unit 223 generates virtual reality contents A route guidance information for another specific location providing a service similar to the virtual reality contents of the selected specific location may be additionally synthesized and provided to the virtual reality contents.

For example, when the terminal device 100 selects the virtual reality contents mapped to the A company bakery among the virtual reality contents for at least one specific location existing on the movement route, the generation section 223 generates the virtual reality contents, Another company B providing bakery service similar to that of the bakery company C, and a bakery & coffee specialty shop C company.

The generating unit 223 then generates route guidance information for reaching the other B bakery and C bakery & coffee specialty shops with the current position of the terminal device 100, that is, the A company bakery as a starting point, May be further synthesized and provided.

Of course, the route guidance information provided additionally may be additionally synthesized to the virtual reality contents mapped to the A-company bakery selected by the terminal device 100, or may be separately synthesized and provided through a pop-up window.

If the terminal device 100 revisits a specific place, the history management unit 230 and the revisiting management unit 240 revisit the virtual reality contents before performing the steps of S210-S270, The processing step may be performed immediately.

More specifically, the history management unit 230 accumulates and stores all the virtual reality contents provided in the process of using the virtual reality contents service by the terminal device 100. That is, the history management unit 230 accumulates and stores the virtual reality contents by mapping the identification information of the terminal device 100. [ Then, when the terminal device 100 revisits a specific place, the history management unit 230 delivers the accumulated virtual reality contents to the re-visit management unit 240 to enable rapid virtual reality contents service.

The re-visit management unit 240 preferentially provides the accumulated virtual reality contents to the terminal device 100 based on the result of determining whether the terminal device 100 revisits the specific place.

More specifically, when it is confirmed that the terminal device 100 has entered a specific place, the rediscovery management unit 240 determines whether or not the terminal device 100 has visited again based on the identification information of the terminal device 100 .

That is, the re-visit management unit 240 determines whether the virtual reality content mapped to the identification information of the terminal device 100 exists. When the virtual reality content mapped to the identification information of the terminal device 100 exists, the rediscovery management unit 240 determines that the terminal device 100 enters a specific place and is provided with a virtual reality content service. If the virtual reality contents mapped to the identification information of the terminal device 100 do not exist, the terminal 100 determines that the virtual reality contents service is not used even if the terminal 100 visits a specific place for the first time or visits a specific place.

Thereafter, when the terminal device 100 returns to a specific place, the re-visit management section 240 determines whether the virtual reality contents accumulated through the previous visit before the terminal device 100 moves to a specific place and collects the captured image .

At this time, if there are a plurality of purposes for visiting a specific place, the re-visiting management unit 240 can provide the virtual reality contents selected for the visited purpose separately for each visited purpose. Therefore, it is possible to provide the virtual reality contents more quickly when returning for the same purpose as the previous visit.

In this way, when a user who visited a specific place returns to the same destination for the same purpose, he / she can receive the desired virtual reality contents at the same time as entering the specific place without performing the above-described virtual reality contents generation process, It can be achieved.

Hereinafter, with reference to FIG. 9, the operation flow of each component in the hardware system 3000 for implementing the content providing apparatus 200 according to the embodiment of the present invention will be described in detail.

For convenience of explanation, it is assumed that each configuration in the content providing apparatus 200 stored in the memory 3210 in the form of a software module is executed by the processor 3100.

First, the operation management unit 210 manages the overall operation status of a plurality of content images related to a specific place (S200 ').

More specifically, the operation management unit 210 uses the position signal of the terminal apparatus 100 received from a position measurement terminal (not shown) in a specific place via the communication apparatus 3310 to determine whether the terminal apparatus 100 It is determined whether or not the user has entered the place.

That is, the operation management unit 210 determines that the terminal device 100 has entered a specific place when a position signal of the terminal device 100 is detected through a location measurement terminal (not shown) in a specific place.

The operation management unit 210 manages a plurality of contents related to a specific place and an image related to the contents (hereinafter referred to as a content image), and based on the agreement message received through the communication device 3310, Detects a point of time when the terminal device 10 agrees with the virtual reality content service, and delivers a plurality of contents to the virtual reality content provider 220 so that the virtual reality content is provided when the terminal device 10 agrees with the virtual reality content service.

More specifically, the operation management unit 210 receives and stores contents and content images that can be provided at a plurality of specific locations located in a specific place.

For example, if the specific location is a shopping mall and the specific location is a plurality of stores, the content management unit 212 receives and stores contents that can be provided from a plurality of stores in advance before the virtual reality contents service is provided. At this time, the content management unit 212 receives a plurality of contents related to a specific place, and simultaneously receives and stores relevant content images.

When the operation management unit 210 confirms that the terminal device 100 has entered a specific place, the operation management unit 210 generates a usage agreement confirmation message for confirming whether or not to use the virtual reality contents service, and transmits the usage agreement confirmation message to the communication device 3310 To the terminal device 100 (S210 ').

Thereafter, the operation management unit 210 receives a response message to the use agreement message (S220 '). That is, when the operation management unit 210 agrees to use the virtual reality contents service from the terminal device 100, the operation management unit 210 receives the agreement message via the communication device 3310, 3310). ≪ / RTI >

When the agreement message is received, the operation management unit 210 transmits a plurality of content images related to the specific place to the virtual reality contents providing unit 220 at a point of time when the terminal device 100 has agreed to receive the virtual reality contents .

If the terminal device 100 receives a message with a false address, it is determined that the terminal device 100 does not use the virtual reality contents service even if the terminal device 100 enters the specific location, so that the virtual reality contents provider 220 .

The virtual reality contents providing unit 220 provides virtual reality contents based on a shot image transmitted from the terminal device 100 entering the specific place through the communication device 3310. [

More specifically, the virtual reality contents providing unit 220 tracks a moving route for moving the specific location based on the photographed image.

That is, the virtual reality contents providing unit 220 receives the photographed image photographed along the moving direction and the moving direction in real time within the specific place from the terminal device 100 through the communication device 3310 (S230 '). The virtual reality contents providing unit 220 tracks the movement path of the terminal device 100 in a specific place using the shot image changed by the direction and the moving direction of the terminal device 100 in real time ).

The virtual reality contents providing unit 220 determines whether the terminal device 100 has reached a specific position based on the result of tracing the movement path (S250 ').

That is, the virtual reality contents providing unit 220 compares the captured image with an image (hereinafter referred to as a specific location image) related to the specific location, while tracking the movement path of the terminal device 100 based on the result of the movement path tracking And it is determined whether or not a specific position is reached based on the calculated matching value.

Here, the specific location image is a specific location, such as an interior of a particular location (e.g., an interior signboard, a sale product, etc.), an exterior interior (e.g., external signboards, initials, mutual feature information, etc.) You can include all the images you can see. In this regard, for example, if a particular location is a bakery, any image that can identify a particular bakery, such as a bakery business name, logo, merchandise, etc., can be a specific location image.

More specifically, the virtual reality contents providing unit 220 compares the photographed image included in the movement route tracking result with the previously stored specific location image.

That is, if the specific location image stored in the virtual reality content tracking unit 220 tracking the movement route based on the captured image and the captured image having the highest matching value are identified, the terminal device 100 reaches the specific location , And detects a content image mapped to a specific position (S260 '). Thereafter, the virtual reality contents providing unit 220 generates virtual reality contents using the content image mapped to the specific location and the travel path tracking result (S270 ').

That is, the virtual reality content providing unit 220 detects a shot image at a specific position included in the movement path tracking result, and combines the shot image and the content image at a specific position on the basis of a virtualization rendering technique, Create content.

Thereafter, the virtual reality contents providing unit 220 transmits the virtual reality contents to the terminal device 100 through the communication device 3310 so that the virtual reality contents can be displayed on the terminal device 100 (S280 ').

When the virtual reality contents providing unit 220 generates virtual reality contents for at least one specific location on the movement route and provides the virtual reality contents to the terminal device 100, When the virtual reality contents are selected, route guidance information for other specific locations providing services similar to the virtual reality contents of the selected specific location may be additionally synthesized and provided to the virtual reality contents.

For example, when the terminal device 100 selects the virtual reality contents mapped to the A company bakery among the virtual reality contents for at least one specific location existing on the movement route, the generation section 223 generates the virtual reality contents, Another company B providing bakery service similar to that of the first bakery, a bakery & coffee specialty shop of the company C, etc., generates route guidance information to another specific location from the bakery A of the company A and adds it to the virtual reality contents .

Of course, the route guidance information provided additionally may be additionally synthesized to the virtual reality contents mapped to the A-company bakery selected by the terminal device 100, or may be separately synthesized and provided through a pop-up window.

If the terminal device 100 returns to a specific place, the visit-visiting management unit 240 performs control so that the return-visit processing step is performed immediately before the step S210-S270 of generating the virtual reality contents described above is performed You may.

The re-visit management unit 240 preferentially provides the accumulated virtual reality contents to the terminal device 100 based on the result of determining whether the terminal device 100 revisits the specific place.

To this end, the history management unit 230 accumulates and stores all the virtual reality contents provided to the terminal device 100 in the process of using the virtual reality contents service.

That is, the re-visit management unit 240 determines whether the virtual reality content mapped to the identification information of the terminal device 100 exists. When the virtual reality content mapped to the identification information of the terminal device 100 exists, the rediscovery management unit 240 determines that the terminal device 100 enters a specific place and is provided with a virtual reality content service. If the virtual reality contents mapped to the identification information of the terminal device 100 do not exist, the terminal 100 determines that the virtual reality contents service is not used even if the terminal 100 visits a specific place for the first time or visits a specific place.

When the terminal device 100 returns to a specific place, the re-visit management section 240 determines that the virtual reality contents accumulated through the previous visit before the terminal device 100 moves to a specific place and collects the captured image .

In this way, when a user who visited a specific place returns to the same destination for the same purpose, he / she can receive the desired virtual reality contents at the same time as entering the specific place without performing the above-described virtual reality contents generation process, It can be achieved.

As described above, the terminal device 100 and the content providing device 200 according to the embodiment of the present invention, and the respective hardware systems 1000 (FIG. 1) for implementing the terminal device 100 and the content providing device 200, , 3000), a movement path for moving a specific place on the basis of a shot image taken while moving a specific place is tracked, and when a specific position is reached during the movement path tracking, By providing the virtual reality contents generated by virtualizing the content image and the shot image to the terminal device 100, intuitive recognition of the provided content can be enhanced, and the user who has visited the specific place can be targeted The contents can be provided and the satisfaction of the contents providing service can be improved.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. It is therefore to be understood that the embodiments described above are to be considered in all respects only as illustrative and not restrictive.

For example, the terminal apparatus 100 and the content providing apparatus 200 according to the present invention can store information by a memory, in one embodiment, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in other embodiments the memory may be a non-volatile memory unit. The memory may also include, for example, a hard disk device, an optical disk device, or any other mass storage device.

The terminal device 100 and the content providing device 200 can also be connected to one or more network interface devices such as, for example, an Ethernet card, a serial communication device such as an RS-232 port and / or a wireless interface device such as an 802.11 card, Device. In other implementations, such input / output devices may include driver devices configured to transmit output data to other input / output devices and receive input data, such as keyboards, printers, display devices, and the like.

The terminal apparatus 100 and the content providing apparatus 200 can be realized by an instruction that causes one or more processing apparatuses to perform the above-described functions and processes at the time of execution. Such instructions may include, for example, interpreted instructions such as script commands, such as JavaScript or ECMAScript commands, or other instructions stored in executable code or computer readable media.

The terminal apparatus 100 and the content providing apparatus 200 according to the present invention may be implemented in a distributed manner over a network such as a server farm or a single computer apparatus.

Implementations of the functional operations and the subject matter described herein may be implemented in digital electronic circuitry, or may be implemented in computer software, firmware, or hardware, including the structures disclosed herein, and structural equivalents thereof, It can be implemented. Implementations of the subject matter described herein may be implemented as one or more computer program products, i. E. One or more modules relating to computer program instructions encoded on a type of program storage medium for execution by, or control of, the operation of the processing system Can be implemented.

The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that affects the machine readable propagation type signal, or a combination of one or more of the foregoing.

The term "system" or "device" as used herein encompasses all apparatuses, apparatus, and machines for processing data, including, for example, a programmable processor, a computer, or a multiprocessor or computer. The processing system may, in addition to the hardware, comprise code that forms an execution environment for a computer program upon request, such as, for example, code comprising a processor firmware, a protocol stack, a database management system, an operating system, .

A computer program (also known as a program, software, software application, script or code) may be written in any form of programming language, including compiled or interpreted language, a priori or procedural language, Components, subroutines, or other units suitable for use in a computer environment. A computer program does not necessarily correspond to a file in the file system. The program may be stored in a single file provided to the requested program, or in multiple interactive files (e.g., a file storing one or more modules, subprograms, or portions of code) (E.g., one or more scripts stored in a markup language document). A computer program may be deployed to run on multiple computers or on one computer, located on a single site or distributed across multiple sites and interconnected by a communications network.

On the other hand, computer readable media suitable for storing computer program instructions and data include semiconductor memory devices such as, for example, EPROM, EEPROM and flash memory devices, such as magnetic disks such as internal hard disks or external disks, Non-volatile memory, media and memory devices, including ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuits.

Implementations of the subject matter described herein may include, for example, a back-end component such as a data server, or may include a middleware component, such as an application server, or may be a web browser or a graphical user, for example a user, who may interact with an implementation of the subject- Front-end components such as client computers with interfaces, or any combination of one or more of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, such as, for example, a communications network.

While the specification contains a number of specific implementation details, it should be understood that they are not to be construed as limitations on the scope of any invention or claim, but rather on the description of features that may be specific to a particular embodiment of a particular invention Should be understood. Likewise, the specific features described herein in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Further, although the features may operate in a particular combination and may be initially described as so claimed, one or more features from the claimed combination may in some cases be excluded from the combination, Or a variant of a subcombination.

It is also to be understood that although the present invention is described herein with particular sequence of operations in the drawings, it is to be understood that it is to be understood that it is to be understood that all such illustrated acts have to be performed or that such acts must be performed in their particular order or sequential order, Can not be done. In certain cases, multitasking and parallel processing may be advantageous. Also, the separation of the various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems will generally be integrated together into a single software product or packaged into multiple software products It should be understood that

As such, the present specification is not intended to limit the invention to the specific terminology presented. Thus, while the present invention has been described in detail with reference to the above examples, those skilled in the art will be able to make adaptations, modifications, and variations on these examples without departing from the scope of the present invention. The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

According to the apparatus and method for providing contents based on the virtual reality of the present invention, contents (e.g., advertisement, information, etc.) related to a specific place based on a shot image taken at a terminal device entering a specific place ), It can not be used not only for the related technology but also for the market or operation of the applicable device, as it is beyond the limit of the existing technology, It is an invention that can be used for commercial use.

100: terminal device
110: location information management unit 120: service processing unit
200: Service device
200: Content providing device
210:
211: entry confirmation unit 212: content management unit
220: Offering virtual reality contents
221: tracking unit 222:
223: Generation unit 230: History management unit
240:

Claims (12)

An operation management unit for managing a plurality of content images related to a specific place; And
Wherein when the captured image related to the location of the terminal device is confirmed within the specific location, the content related to the specific location is determined based on a result of determining whether the terminal device has reached a specific location based on the captured image of the terminal device Virtual reality contents provided by virtualizing images
The content providing apparatus comprising: The method according to claim 1,
The virtual reality contents providing unit,
A tracking unit for tracking a movement path for moving the specific place based on the photographed image;
A detecting unit detecting a content image mapped to the specific location when the specific location is reached during tracking of the movement route; And
And generating a virtual reality content by combining the captured image related to the specific location and the detected content image based on a virtualization rendering technique. 3. The method of claim 2,
The virtual reality contents providing unit,
A history management unit for accumulating and storing the virtual reality contents provided to the terminal device; And
And a re-visit management unit for preferentially providing the accumulated virtual reality contents before the captured image is received from the terminal apparatus when the terminal apparatus revisits the specific place. 3. The method of claim 2,
The tracking unit includes:
Wherein the terminal tracks the moving route by using a photographed image changed by a moving direction and a copper line of the terminal device in real time. The method according to claim 1,
The operation management unit,
An entry confirming unit for confirming whether or not the terminal device has entered the specific place; And
And a content manager for delivering the plurality of content images to the virtual reality content provider at a point of time when the terminal device agrees to receive the virtual reality service providing content related to the specific location Wherein the content providing apparatus comprises: The method according to claim 1,
The specific position may be,
Wherein the content providing device is a location capable of providing a service related to at least one content image among the plurality of content images. A management step of managing a plurality of content images related to a specific place; And
When a captured image photographed in real time based on the position of the terminal device is confirmed within the specific place, the terminal device determines whether the terminal device has reached a specific position based on the captured image of the terminal device, Providing a virtualization of related content
The content providing apparatus comprising: 8. The method of claim 7,
Wherein the providing step comprises:
A tracking step of tracking a moving path for moving the specific location based on the photographed image received in real time;
A detecting step of detecting a content image mapped to the specific position when the specific position is reached during tracking of the movement path; And
And generating a virtual reality content by combining the sensed image at the specific position and the detected content image based on a virtualization rendering technique. 9. The method of claim 8,
Wherein the providing step comprises:
A storage step of accumulating and storing the virtual reality contents provided to the terminal device; And
And a re-visiting step of preferentially providing the accumulated virtual reality contents before the captured image is received from the terminal device when the terminal device revisits the specific place. Lt; / RTI > 9. The method of claim 8,
Wherein the tracking step comprises:
Wherein the terminal tracks the moving route by using a photographed image changed by a moving direction and a moving line of the terminal device in real time. 8. The method of claim 7,
Wherein,
A location confirmation step of confirming whether or not the terminal device has entered the specific place; And
And a delivery step of delivering the plurality of content images to the virtual reality contents provider at a point of time when the terminal device agrees to receive the virtual reality service providing content related to the specific location Wherein the content providing apparatus comprises: 8. The method of claim 7,
The specific position may be,
Wherein the location information is a location where service can be provided related to at least one content image among the plurality of content images.

Images