KR102648834B1 - Method and apparatus of providing contents based on virtual reality - Google Patents

Abstract

The present invention discloses an apparatus and method for providing content based on virtual reality. In other words, the movement path moving to a specific place is tracked based on the captured image while moving to a specific place, and when a specific location is reached while tracking the movement path, the content image and captured image mapped to the specific location are virtualized. By providing the created virtual reality content to a terminal device, intuitive awareness of the provided content can be increased, and content optimized for the purpose of visiting a specific place can be provided targeting users who have visited a specific place, thereby increasing satisfaction with the content provision service. It can be improved.

Description

Device and method for providing content based on virtual reality {METHOD AND APPARATUS OF PROVIDING CONTENTS BASED ON VIRTUAL REALITY}

The present invention relates to a technology for providing virtualized content targeting a terminal device that has entered a specific location, and content related to a specific location based on a captured image taken by a terminal device that has entered a specific location (e.g., a shopping mall). This relates to a device and method that can virtualize and provide (e.g. advertisements, information, etc.).

Recently, as the use of mobile devices (hereinafter referred to as terminal devices) such as smartphones and tablet pads has rapidly increased, interest in technology that provides content (e.g., advertisements, information, etc.) using terminal devices is rapidly increasing.

As interest in technology for providing content using terminal devices increases, a large amount of content is being provided to terminal devices by a number of businesses (eg, advertisers).

However, in the case of content provided to an unspecified number of people, it is provided randomly without considering the user's interests, so it is mostly perceived as spam advertising, making it difficult to expect advertising effectiveness.

To solve this problem, a method of providing content based on user information such as user location information and exposed personal information was introduced. However, even if content is provided based on user information, the content is provided simply as text or advertising images, which causes the inconvenience of having to directly check the offline store location or event location for each content and map it to the corresponding content. There is still a problem that users perceive it as spam advertising.

Meanwhile, in order to solve the problem of content being perceived as spam advertising as mentioned above, instead of randomly providing a large amount of content to an unspecified number of people, businesses that want to provide various content and events targeting users who enter a specific location (e.g. : advertisers) is increasing.

Accordingly, there is a need for technology to target users who visit a specific location and provide content while improving intuitive awareness of the content.

The present invention was created in consideration of the above-mentioned circumstances, and the goal to be achieved by the present invention is to virtualize content related to a specific location based on a captured image captured by a terminal device entering a specific location (e.g., a shopping mall). The purpose is to provide devices and methods that can provide.

In addition, the problems to be solved by the present invention are not limited to those mentioned above, and other problems to be solved that are not mentioned will be clearly understood by those skilled in the art to which the present invention pertains from the following description. .

A content providing device according to a first aspect of the present invention for achieving the above object includes an operation management unit that manages a plurality of content images related to a specific location; And when the captured image collected in relation to the location of the terminal device within the specific location is confirmed, the captured image related to the specific location is based on the result of determining whether the terminal device has reached the specific location based on the captured image of the terminal device. It is characterized by including a virtual reality content providing unit that virtualizes and provides content images.

The virtual reality content providing unit includes a tracking unit that tracks a movement path to the specific location based on the captured image; a detection unit that detects a content image mapped to the specific location when the specific location is reached while tracking the movement path; and a generating unit that generates virtual reality content by combining the captured image related to the specific location and the detected content image based on a virtual rendering technique.

The virtual reality content providing unit includes a history management unit that accumulates and stores the virtual reality content provided to the terminal device; and a revisit management unit that preferentially provides the accumulated virtual reality content before the captured image is received from the terminal device when the terminal device revisits the specific location.

The tracking unit is characterized in that it tracks the movement path using a captured image changed according to the direction and movement line in which the terminal device moves in real time.

The operation management unit includes an entry confirmation unit that checks whether the terminal device has entered the specific location; And if the terminal device agrees to receive a virtual reality service that provides content related to the specific location, a content management unit that delivers the plurality of content images to the virtual reality content provider at the time the terminal device agrees to receive the service. It is characterized by

The specific location is characterized as a location where services related to at least one content image among the plurality of content images can be provided.

A method of operating a content providing device according to a second aspect of the present invention for achieving the above object includes a management step of managing a plurality of content images related to a specific location; and

When a captured image is confirmed in real time based on the location of the terminal device within the specific location, the specific location and It is characterized by including a provision step of virtualizing and providing related content.

The providing step includes a tracking step of tracking a movement path moving to the specific location based on the captured image received in real time; A detection step of detecting a content image mapped to the specific location when the specific location is reached while tracking the movement path; And a generation step of generating virtual reality content by combining the captured image at the specific location and the detected content image based on a virtualization rendering technique.

The provision step includes a storage step of accumulating and storing the virtual reality content provided to the terminal device; and, when the terminal device revisits the specific location, a revisit service step of preferentially providing the accumulated virtual reality content before the captured image is received from the terminal device.

The tracking step is characterized by tracking the movement path using captured images changed according to the direction and movement line in which the terminal device moves in real time.

The management step includes a location confirmation step of checking whether the terminal device has entered the specific location; And if the terminal device agrees to receive a virtual reality service that provides content related to the specific location, a delivery step of transmitting the plurality of content images to the virtual reality content provider at the time the terminal device agrees to receive the service. It is characterized by

The specific location is characterized as a location where services related to at least one content image among the plurality of content images can be provided.

Accordingly, according to the device and method for providing content based on virtual reality of the present invention, the movement path moving to a specific place is tracked based on the captured image while moving to the specific place, and while tracking the movement path, a specific location is tracked. When a location is reached, virtual reality content created by virtualizing content images and captured images mapped to a specific location is provided to the terminal device, thereby increasing intuitive awareness of the provided content and targeting users who have visited a specific location. By providing content optimized for the purpose of visiting the location, satisfaction with the content provision service can be improved.

1 is a schematic configuration diagram of a content provision system according to an embodiment of the present invention.
Figure 2 is a schematic configuration diagram of a terminal device according to an embodiment of the present invention.
Figure 3 is a schematic configuration diagram of a hardware system for implementing a terminal device according to an embodiment of the present invention.
Figure 4 is a schematic configuration diagram of a content providing device according to an embodiment of the present invention.
Figure 5 is a schematic configuration diagram of a hardware system for implementing a content providing device according to an embodiment of the present invention.
Figure 6 is a diagram for explaining the operation flow in a terminal device according to an embodiment of the present invention.
7 is a flowchart illustrating the operation flow in a hardware system for implementing a terminal device according to an embodiment of the present invention.
Figure 8 is a diagram for explaining the operation flow in the content providing device according to an embodiment of the present invention.
9 is a flowchart illustrating the operation flow in a hardware system for implementing a content providing device according to an embodiment of the present invention.

It should be noted that the technical terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention. In addition, the technical terms used in this specification, unless specifically defined in a different way in this specification, should be interpreted as meanings commonly understood by those skilled in the art in the technical field to which the present invention pertains, and are not overly comprehensive. It should not be interpreted in a literal or excessively reduced sense. Additionally, if the technical terms used in this specification are incorrect technical terms that do not accurately express the spirit of the present invention, they should be replaced with technical terms that can be correctly understood by those skilled in the art. In addition, general terms used in the present invention should be interpreted according to the definition in the dictionary or according to the context, and should not be interpreted in an excessively reduced sense.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings. However, identical or similar components will be assigned the same reference numbers regardless of the reference numerals, and duplicate descriptions thereof will be omitted. Additionally, when describing the present invention, if it is determined that a detailed description of related known technologies may obscure the gist of the present invention, the detailed description will be omitted. In addition, it should be noted that the attached drawings are only intended to facilitate easy understanding of the spirit of the present invention, and should not be construed as limiting the spirit of the present invention by the attached drawings. The spirit of the present invention should be construed as extending to all changes, equivalents, or substitutes other than the attached drawings.

Hereinafter, an embodiment of the present invention will be described with reference to the attached drawings.

1 is a diagram illustrating a content provision system according to an embodiment of the present invention.

As shown in FIG. 1, the content providing system according to an embodiment of the present invention may include a terminal device 100 and a content providing device 200.

The terminal device 100 receives virtualized content (hereinafter referred to as virtual reality content) related to a specific location from the content providing device 200.

The specific location in the present invention may be a specific offline building with unique characteristics that the user of the terminal device 100 visits for a purpose. For example, a specific location may be a building with unique characteristics, such as a place for shopping, such as a shopping mall, a place for tax-related work, such as the National Tax Service, or a postal service, such as a post office.

For example, the above terminal device 100 may include, for example, a smart phone, tablet PC, and PDA, but is not limited thereto, and is a device capable of driving a camera for image capture and receiving virtual reality content. can all be included.

Here, the content providing medium that enables reception of virtual reality content on the terminal device 100 includes, for example, an application running on the terminal device 100 itself or an online service that can be accessed by calling a browser on the terminal device 100. This may be a page, but in one embodiment of the present invention, it is assumed that the content providing medium that enables reception of virtual reality content in the terminal device 100 is an application.

In this regard, in the case of an application that operates as a content providing medium in the terminal device 100, the captured images taken while the terminal device 100 moves to a specific location are provided in real time to the content providing device ( 200). In addition, virtual reality content generated based on captured images transmitted in real time from the content providing device 200 is received and provided to the terminal device 100.

For reference, the terminal device 100 checks the current location to determine whether to enter a specific location. This current location may be checked using, for example, a Global Positioning System (GPS). .

The content providing device 200 is a device for providing virtual reality content related to a specific location to multiple terminals, including the terminal device 100. In particular, the service to be provided by the present invention, that is, the terminal device 100 is a device for providing virtual reality content related to a specific location. A service that determines whether the terminal device 100 has reached a specific location based on the captured image taken while moving and provides virtual reality content by virtualizing content images and captured images related to the specific location (hereinafter referred to as virtual reality content) It is a device to perform a service.

Here, a specific location refers to the location of multiple stores located in a specific location (e.g., a shopping mall), and refers to a store that can provide services related to at least one content among multiple contents related to a specific location. That is, at least one content that can provide a service at a specific location is mapped to a specific location among a plurality of contents related to the specific location.

Accordingly, in one embodiment of the present invention, we would like to propose a method of virtualizing and providing content related to a specific location based on a captured image captured by a terminal device 100 entering the specific location. Hereinafter, a terminal for implementing this will be proposed. The configuration of the device 100 and the content providing device 200 will be described in detail.

First, the terminal device 100 according to an embodiment of the present invention will be described with reference to FIG. 2.

Here, FIG. 2 is a diagram showing the schematic configuration of the terminal device 100 according to an embodiment of the present invention.

As shown in FIG. 2, the terminal device 100 according to an embodiment of the present invention provides a virtual reality content service from the location information management unit 110 and the content providing device 200 that confirm that a specific location has been entered. It has a configuration including a receiving service processing unit 120.

Here, the service processing unit 120 corresponds to an application that is a content providing medium capable of processing virtual reality content received from the content providing device 200.

All or at least part of the configuration of the terminal device 100, including the location information management unit 110 and the service processing unit 120, is implemented in the form of a software module or hardware module executed by a processor, or is implemented in the form of a software module and a hardware module. It can be implemented in a combined form.

Ultimately, the terminal device 100 according to an embodiment of the present invention enters a specific location through the above configurations and receives virtual reality content related to the specific location when the user agrees to use the virtual reality content service. Hereinafter, this is implemented. Each configuration within the terminal device 100 will be described in detail.

The location information management unit 110 communicates with the content providing device 200 to confirm that the terminal device 100 has entered a specific location.

More specifically, the location information management unit 110 recognizes a location determination terminal (not shown) within a specific location and performs communication with the content providing device 200 that operates in conjunction with the location determination terminal (not shown). This confirms that the terminal device 100 has entered a specific location.

In the present invention, it is mentioned that entering a specific location is confirmed by communicating with the content providing device 200. However, without communication with the content providing device 200, a specific location is stored in the application that operates as a content providing medium in the terminal device 100. It is also possible to confirm that the terminal device 100 has entered a specific location using the location information value of the location.

The service processing unit 120 receives a usage consent confirmation message confirming whether to use the virtual reality content service from the content providing device 200 after entering a specific location.

That is, the service processing unit 120 receives the use consent confirmation message from the content providing device 200 as a text message or a pop-up message through an application. Next, if the user agrees to use the virtual reality content service, the service processing unit 120 generates a consent message and delivers it to the content providing device 200. If the user does not agree to the use of the virtual reality content service, the service processing unit 120 generates a consent message and delivers it to the content providing device 200. A message is generated and delivered to the content providing device 200.

Afterwards, if the service processing unit 120 agrees to use the virtual reality content service, it controls the operation of the camera so that a captured image is generated along the direction and line in which the terminal device 100 moves in real time.

That is, the service processing unit 120 transmits captured images generated in real time to the content providing device 200 so that virtual reality content can be created based on captured images while moving to a specific location.

Meanwhile, the service processing unit 120 controls the camera not to operate if the user does not agree to use the virtual reality content service. At this time, it is desirable for the camera to operate under user control regardless of whether the virtual reality content service is used.

When the service processing unit 120 receives virtual reality content from the content providing device 200, it controls the received virtual reality content to be displayed through a display unit (not shown).

All or at least part of the configuration of the terminal device 100, including the location information management unit 110 and the service processing unit 120, is implemented in the form of a software module or hardware module executed by a processor, or is comprised of a software module and a hardware module. It can be implemented in a combined form.

For reference, the virtual reality content received from the content providing device 200 tracks the movement path moving to a specific place based on the captured image while moving to the specific place, and reaches a specific location while tracking the movement path. When this is done, the content image and the captured image mapped to a specific location are virtualized and created, and a detailed description related to this will be discussed in the detailed description of the content providing device 200.

It has been mentioned that each component in the terminal device 100 described above may be implemented in the form of a software module or hardware module executed by a processor, or may be implemented in a form of a combination of a software module and a hardware module.

In this way, a software module, a hardware module, or a combination of a software module and a hardware module executed by a processor may be implemented as a hardware system (eg, a computer system).

Therefore, hereinafter, the hardware system 1000 for implementing the terminal device 100 according to an embodiment of the present invention will be described with reference to FIG. 3.

For reference, the content described below is an example for implementing each configuration within the terminal device 100 described above within the hardware system 1000, and keep in mind that each configuration and its resulting operation may be different from the actual system. You will have to do it.

FIG. 3 is a diagram illustrating the structure of a hardware system 1000 for implementing the terminal device 100 according to an embodiment of the present invention.

As shown in FIG. 3, the hardware system 1000 according to an embodiment of the present invention may have a configuration including a processor 1100, a memory interface 1200, and a peripheral device interface 1300.

Each component within the hardware system 1000 may be a separate component or integrated into one or more integrated circuits, and each of these components may be coupled to a bus system (not shown).

Here, in the case of a bus system, any one or more individual physical buses, communication lines/interfaces, and/or multi-drop or point connected by appropriate bridges, adapters, and/or controllers. It is an abstraction that represents point-to-point connections.

The processor 1100 communicates with the memory 1210 through the memory interface 1200 to perform various functions in the hardware system, thereby executing various software modules stored in the memory 1210.

Here, the location information management unit 110 and the service processing unit 120, which are each component of the terminal device 100 described with reference to FIG. 2, may be stored in the memory 1210 in the form of software modules, and other operating systems may be stored in the memory 1210. Additional information can be stored.

For the above operating systems (e.g. I-OS, Android, Darwin, RTXC, LINUX, UNIX, OS It contains various procedures, instruction sets, software components and/or drivers that control and manage (control, power management, etc.) and plays a role in facilitating communication between various hardware modules and software modules.

For reference, the memory 1210 may include a memory hierarchy including, but not limited to, cache, main memory, and secondary memory. In the case of the above memory hierarchy, for example, RAM (e.g., SRAM, DRAM) , DDRAM), ROM, FLASH, magnetic and/or optical storage devices (e.g., disk drives, magnetic tapes, compact disks (CDs), and digital video discs (DVDs)).

The peripheral device interface 1300 serves to enable communication between the processor 1100 and peripheral devices.

In the case of the above peripheral devices, they are intended to provide different functions to the computer system, and in one embodiment of the present invention, for example, a location confirmation device 1310, a communication device 1320, and an input/output device 1330 may be included. .

Here, the location confirmation device 1310 refers to a device that can confirm the current location of the terminal device 100, and may include, for example, a Global Positioning System (GPS).

In addition, the communication device 1320 performs the role of providing a communication function with other devices, for example, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, and a codec. (CODEC) includes, but is not limited to, a chipset, memory, etc., and may include known circuits that perform this function.

Communication protocols supported by the communication device 1320 include, for example, Long Term Evolution (LTE), time division multiple access (TDMA), code division multiple access (CDMA), global system for mobile communications (GSM), and EDGE. (Enhanced Data GSM Environment), W-CDMA (wideband code division multiple access), Wi-Fi (IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n, etc.), Bluetooth, Wi-MAX, VoIP ( Protocols for voice over Internet Protocol (Voice over Internet Protocol), email, instant messaging, and SMS (short message service) may be applicable, but are not limited thereto, and all protocols that provide a communication environment with other devices may be included.

In addition, the input/output device 1330 serves as a controller to control I/O devices linked with other hardware systems. In one embodiment of the present invention, for example, the input/output device 1330 is used to display various service screens. can be in charge of control.

Ultimately, each component within the content providing device 200 stored in the form of a software module in the memory 1210 is executed through the memory interface 1100 and the peripheral device interface 1300 when executed by the processor 1100. By performing communication with the location confirmation device 1310 and the communication device 1320, virtual reality content related to a specific location can be received from the content providing device 200.

Hereinafter, each component in the hardware system 1000 related to the terminal device 100 will be described in detail with reference to FIG. 3. For convenience of explanation, the terminal stored in the form of a software module in the memory 1210 It is assumed that each component in the device 100 is executed by the processor 1100.

The location information management unit 110 communicates with the content providing device 200 to confirm that the terminal device 100 has entered a specific location.

More specifically, the location information management unit 110 recognizes a location determination terminal (not shown) in a specific location through the location confirmation device 1310, and recognizes the location determination terminal (not shown) through the communication device 1320. It is confirmed that the terminal device 100 has entered a specific location by performing communication with the content providing device 200 that operates in conjunction with.

In the present invention, it is mentioned that entering a specific location is confirmed by communicating with the content providing device 200. However, without communication with the content providing device 200, a specific location is stored in the application that operates as a content providing medium in the terminal device 100. It is also possible to confirm that the terminal device 100 has entered a specific location using the location information value of the location.

The service processing unit 120 receives a usage consent confirmation message confirming whether to use the virtual reality content service from the content providing device 200 after entering a specific location.

That is, the service processing unit 120 receives the usage consent confirmation message from the content providing device 200 through the communication device 1320 as a text message or a pop-up message through an application. Next, the service processing unit 120 generates a consent message when the user agrees to use the virtual reality content service and then transmits it to the content providing device 200 through the communication device 1320, and the user uses the virtual reality content service. If there is no agreement, a non-agree message is generated and then delivered to the content providing device 200 through the communication device 1320.

Afterwards, if the service processing unit 120 agrees to use the virtual reality content service, the service processing unit 120 drives the camera through the input/output device 1330 to generate a captured image along the direction and line in which the terminal device 100 moves in real time. This controls the display 1331.

That is, the service processing unit 120 transmits the captured image displayed through the display 1331 in real time to the content through the communication device 1320 so that virtual reality content can be created based on the captured image while moving to a specific location. It is transmitted to the provision device 200.

Meanwhile, the service processing unit 120 controls the input/output device 1330 so that the camera is not driven if the user does not agree to use the virtual reality content service. At this time, it is desirable for the camera to operate under user control regardless of whether the virtual reality content service is used.

When the service processing unit 120 receives virtual reality content from the content providing device 200 through the communication device 1320, the service processing unit 120 controls the display 1331 through the input/output device 1330 to display the virtual reality content. do.

Now that the description of the terminal device 100 and the hardware system for implementing it has been completed, the configuration of the content providing device 200 according to an embodiment of the present invention will be described with reference to FIG. 4 .

Figure 4 is a diagram showing the schematic configuration of a content providing device 200 according to an embodiment of the present invention.

As shown in FIG. 4, the content providing device 200 according to an embodiment of the present invention includes an operation management unit 210 that manages a plurality of content images related to a specific location and a terminal device 100 within the specific location. When the captured image collected in relation to the location is confirmed, the content image related to the specific location is virtualized based on the result of determining whether the terminal device 100 has reached a specific location based on the captured image of the terminal device 100 (i.e. It has a configuration that includes a virtual reality content providing unit 220 that provides virtual reality content.

In addition, in addition to the above-described configuration, the content providing device 200 according to an embodiment of the present invention includes a history management unit 230 that accumulates and stores virtual reality content provided to the terminal device 100 and a specific terminal device 100. When revisiting a location, it may be configured to further include a revisit management unit 240 that preferentially provides accumulated virtual reality content before the captured image is received from the terminal device 100.

All or at least part of the configuration of the content provision device 200, including the operation management unit 210, the virtual reality content provision unit 220, the history management unit 230, and the revisit management unit 240, is executed by a processor. It may be implemented in the form of a software module, a hardware module, or a combination of a software module and a hardware module.

In the end, the content providing device 200 according to an embodiment of the present invention tracks the movement path of moving to a specific place based on the captured image while moving to the specific place through the above configurations, and tracks the movement path. When a specific location is reached, virtual reality content created by virtualizing the content image and captured image mapped to the specific location can be provided to the terminal device 100. Hereinafter, each configuration of the content providing device 200 for this is described. Let's explain this in detail.

The operation management unit 210 manages the overall operation status of multiple content images related to a specific location.

More specifically, the operation management unit 210 includes an entry confirmation unit 211 and a content management unit 212.

The entry confirmation unit 211 determines whether the terminal device 100 has entered a specific location in conjunction with a location measurement terminal (not shown) within the specific location.

That is, when the entry confirmation unit 211 detects the location signal of the terminal device 100 through a location measurement terminal (not shown) within a specific location, it determines that the terminal device 100 has entered the specific location.

When the current location of the terminal device 100 within a specific location is received in this way, the entry confirmation unit 211 can confirm the location where the terminal device 100 entered among various entry routes within the specific location.

Afterwards, the entry confirmation unit 211 notifies the content management unit 212 that the terminal device 100 has entered.

The content management unit 212 manages a plurality of content related to a specific location and images related to the content (hereinafter referred to as content images), and determines whether the virtual reality content will be provided at the time the terminal device 100 agrees to the virtual reality content service. A plurality of content images are transmitted to the virtual reality content providing unit 220 so that

More specifically, the content management unit 212 receives and stores in advance content and content images that can be serviced at a number of specific locations located in a specific location.

For example, when the specific location is a shopping mall and there are multiple stores, the content management unit 212 receives and stores content that can provide services from multiple stores in advance before the virtual reality content service is provided. At this time, the content management unit 212 receives a plurality of content related to a specific location and simultaneously receives and stores related content images.

In addition, when the content management unit 212 confirms from the entry confirmation unit 211 that the terminal device 100 has entered a specific location, it generates a usage consent confirmation message confirming whether to use the virtual reality content service, This is transmitted to the terminal device 100.

Afterwards, the content management unit 212 receives a response message to the use consent confirmation message. That is, the content management unit 212 receives a consent message from the terminal device 100 when the user agrees to use the virtual reality content service, and receives a non-consent message when the user does not agree to use the virtual reality content service.

When the consent message is received, the content management unit 212 sends a number of content images related to a specific location to the virtual reality content provider 220 at the time the terminal device 100 agrees to receive them so that virtual reality content can be created. Deliver.

If a non-agreement message is received, the content management unit 212 determines that the terminal device 100 is not using the virtual reality content service even if it has entered a specific location, and the virtual reality content provision unit determines that the virtual reality content is not provided. Send a non-agreement message to (220).

The virtual reality content providing unit 220 provides virtual reality content based on captured images transmitted from the terminal device 100 entering a specific location.

More specifically, the virtual reality content providing unit 220 includes a tracking unit 221, a detection unit 222, and a generating unit 223.

The tracking unit 221 tracks the movement path to the specific location based on the captured image.

That is, the tracking unit 221 receives captured images taken along the moving direction and line of movement within a specific location in real time from the terminal device 100. The tracking unit 221 tracks the movement path of the terminal device 100 within a specific location using the captured image changed by the moving direction and movement line of the terminal device 100 in real time.

Thereafter, the tracking unit 221 transmits the movement path tracking result of tracking the movement path of the terminal device 100 in real time to the detection unit 222.

The detection unit 222 determines whether the terminal device 100 has reached a specific location based on the movement path tracking result.

That is, the detection unit 222 tracks the movement path of the terminal device 100 based on the movement path tracking result and compares the captured image with a pre-stored image related to a specific location (hereinafter, a specific location image) to calculate a matching value. Based on this, it is determined whether a specific location has been reached.

Here, a specific location image refers to a specific location, such as the internal interior (e.g., internal signage, products for sale, etc.), external interior (e.g., external signage, initials, business characteristic information, etc.), and services provided at a specific location within a specific location. Any image that can be viewed can be included. In this regard, for example, if a specific location is a bakery, all images that can identify the specific bakery, such as the bakery's business name, logo, and products for sale, can become specific location images.

More specifically, the detection unit 222 compares the captured image included in the movement path tracking result with a pre-stored specific location image.

In other words, the captured image captured while the terminal device 100 entering a specific location is moving is related to the location of the terminal device 100, so the captured image included in the movement path is compared with the previously stored specific location image to determine the best match. By detecting a specific location image with a high value, it is determined whether a specific location has been reached.

Ultimately, the detection unit 222 determines that the terminal device 100 has reached a specific location when the captured image with the highest matching value is confirmed while tracking the movement path based on the captured image. , Detects content images mapped to specific locations. Afterwards, the detection unit 222 transmits the content image mapped to a specific location and the movement path tracking result to the generation unit 223.

The creation unit 223 generates virtual reality content using the content image mapped to a specific location and the movement path tracking result received from the detection unit 222.

That is, the generator 223 detects a captured image of a specific location included in the movement path tracking result and generates virtual reality content by combining the captured image of the specific location and the content image based on a virtual rendering technique. do.

Here, the virtual rendering technique may be an Augmented Reality (AR) technique, which is a field of Virtual Reality, and the actual captured image and virtual content image captured by the terminal device 100 It refers to a graphic technique that synthesizes to make it appear as if a virtual content image exists in a specific location.

Meanwhile, when the creation unit 223 generates virtual reality content for at least one specific location existing on the movement path and provides it to the terminal device 100, the virtual reality content for the specific location is generated from the terminal device 100. If is selected, route guidance information for another specific location that provides services similar to the virtual reality content of the selected specific location may be additionally synthesized with the virtual reality content and provided.

For example, when the terminal device 100 selects the virtual reality content mapped to Company A's bakery among the virtual reality content for at least one specific location existing on the movement path, the creation unit 223 selects the VR content mapped to Company A's bakery. Detects specific locations such as other company B bakeries and company C bakeries & coffee shops that provide similar services.

Afterwards, the generator 223 generates route information starting from the current location of the terminal device 100, that is, Company A's bakery, and reaches other Company B bakeries and Company C bakeries & coffee shops, and uses this as virtual reality content. It can also be provided by additional synthesis.

Of course, the additionally provided route guidance information may be additionally synthesized and provided to the virtual reality content mapped to the bakery of Company A selected by the terminal device 100, or may be separately synthesized and provided through a pop-up window.

The history management unit 230 accumulates and stores all virtual reality content provided to the terminal device 100 in the process of using the virtual reality content service. That is, the history management unit 230 accumulates and stores virtual reality content by mapping it to the identification information of the terminal device 100. Here, any information that can identify the terminal device 100 (e.g., user information, phone number, etc.) can be used as the identification information of the terminal device 100.

Thereafter, when the terminal device 100 revisits a specific location, the history management unit 230 transfers the accumulated virtual reality content to the revisit management unit 240 to enable rapid virtual reality content service.

The revisit management unit 240 preferentially provides accumulated virtual reality content to the terminal device 100 based on the result of determining whether the terminal device 100 has revisited a specific location.

More specifically, when it is confirmed that the terminal device 100 has entered a specific location, the revisit management unit 240 determines whether the terminal device 100 will revisit based on the identification information of the terminal device 100. .

That is, the revisit management unit 240 determines whether virtual reality content mapped to the identification information of the terminal device 100 exists. If there is virtual reality content mapped to the identification information of the terminal device 100, the revisit management unit 240 determines that the terminal device 100 has entered a specific location and has been provided with a virtual reality content service. If there is no virtual reality content mapped to the identification information of the terminal device 100, it is determined that the terminal device 100 visited a specific place for the first time or visited a specific place but did not use the virtual reality content service.

Thereafter, when the terminal device 100 revisits a specific location, the revisit management unit 240 controls the virtual reality content accumulated through previous visits before the terminal device 100 moves to the specific location and collects the captured images. is given priority.

At this time, if there are multiple purposes for previously visiting a specific place, the revisit management unit 240 may provide virtual reality content selected according to the purpose of the previous visit, divided by the purpose of the visit. Accordingly, virtual reality content can be provided more quickly when returning for the same purpose as the previous visit.

Through this, if a user who visited a specific place returns for the same purpose as the previous visit, he or she receives the desired virtual reality content as soon as he or she enters the specific place without performing the virtual reality content creation process described above, and quickly and accurately determines the purpose of the visit. It becomes achievable.

Meanwhile, it has been mentioned that each configuration within the content providing device 200 described above may be implemented in the form of a software module or hardware module executed by a processor, or may also be implemented in the form of a combination of software modules and hardware modules. there is.

In this way, a software module, a hardware module, or a combination of a software module and a hardware module executed by a processor may be implemented as a hardware system (eg, a computer system).

Therefore, hereinafter, a hardware system for implementing the content providing device 200 according to an embodiment of the present invention will be described. Prior to explanation, the content described below is an example for implementing each configuration within the content providing device 200 described above within a computer system, and of course, each configuration and its resulting operation may be different from the actual system. .

Figure 5 is a diagram showing the structure of a hardware system for implementing the content providing device 200 according to an embodiment of the present invention.

As shown in FIG. 5, the hardware system 3000 according to an embodiment of the present invention may have a configuration including a processor 3100, a memory interface 3200, and a peripheral device interface 3300.

Each component within the hardware system 3000 may be a separate component or integrated into one or more integrated circuits, and each of these components may be coupled to a bus system (not shown).

Here, in the case of a bus system, any one or more individual physical buses, communication lines/interfaces, and/or multi-drop or point connected by appropriate bridges, adapters, and/or controllers. It is an abstraction that represents point-to-point connections.

The processor 3100 communicates with the memory 3210 through the memory interface 3200 to perform various functions in the hardware system, thereby executing various software modules stored in the memory 3210.

Here, the memory 3210 includes the operation management unit 210, the virtual reality content provision unit 220, the history management unit 230, and the revisit management unit 240, which are each component in the content providing device 200 described with reference to FIG. 4. can be stored in the form of software modules, and other operating systems can be additionally stored.

For the above operating systems (e.g., I-OS, Android, Darwin, RTXC, LINUX, UNIX, OS It contains various procedures, instruction sets, software components and/or drivers that control and manage (storage device control, power management, etc.) and plays a role in facilitating communication between various hardware modules and software modules.

For reference, the memory 3210 may include a memory hierarchy including, but not limited to, cache, main memory, and secondary memory. In the case of the above memory hierarchy, for example, RAM (e.g., SRAM, DRAM) , DDRAM), ROM, FLASH, magnetic and/or optical storage devices (e.g., disk drives, magnetic tapes, compact disks (CDs), and digital video discs (DVDs)).

The peripheral device interface 3300 serves to enable communication between the processor 3100 and peripheral devices.

In the case of the above peripheral device, it is intended to provide different functions to the computer system, and in one embodiment of the present invention, for example, a communication device 3310 may be included.

Here, the communication device 3310 performs the role of providing a communication function with other devices, and for this purpose, for example, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, and a codec. (CODEC) includes, but is not limited to, a chipset, memory, etc., and may include known circuits that perform this function.

Communication protocols supported by the communication device 3310 include, for example, Long Term Evolution (LTE), time division multiple access (TDMA), code division multiple access (CDMA), global system for mobile communications (GSM), and EDGE. (Enhanced Data GSM Environment), W-CDMA (wideband code division multiple access), Wi-Fi (IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n, etc.), Bluetooth, Wi-MAX, VoIP ( Protocols for voice over Internet Protocol (Voice over Internet Protocol), email, instant messaging, and SMS (short message service) may be applicable, but are not limited thereto, and all protocols that provide a communication environment with other devices may be included.

Ultimately, each component within the content providing device 200 stored in the form of a software module in the memory 3210 is executed by the processor 3100 through the memory interface 3200 and the peripheral device interface 3300. By performing communication with the communication device 3310, if the captured image collected in relation to the location of the terminal device 100 within a specific location is confirmed, the terminal device 100 is activated based on the captured image of the terminal device 100. Based on the result of determining whether a specific location has been reached, virtual reality content related to a specific location can be provided to the terminal device 100.

Hereinafter, each configuration in the hardware system 3000 related to the content providing device 200 will be described in detail with reference to FIG. 5. For convenience of explanation, the components stored in the memory 3210 in the form of software modules will be described in detail. It is assumed that each component in the content providing device 200 is executed by the processor 3100 through the memory interface 3200.

The operation management unit 210 manages the overall operation status of multiple content images related to a specific location.

More specifically, the operation management unit 210 includes an entry confirmation unit 211 and a content management unit 212.

The entry confirmation unit 211 allows the terminal device 100 to enter a specific location by using the location signal of the terminal device 100 received from a location measurement terminal (not shown) within a specific location through the communication device 3310. Determine whether it was done or not.

That is, when the entry confirmation unit 211 detects the location signal of the terminal device 100 through a location measurement terminal (not shown) within a specific location, it determines that the terminal device 100 has entered the specific location.

When the current location of the terminal device 100 within a specific location is received in this way, the entry confirmation unit 211 can confirm the location where the terminal device 100 entered among various entry routes within the specific location.

Afterwards, the entry confirmation unit 211 notifies the content management unit 212 that the terminal device 100 has entered.

The content management unit 212 manages a plurality of contents related to a specific location and images related to the contents (hereinafter referred to as content images), and based on the consent message received through the communication device 3310, the terminal device 100 creates a virtual Detects the point in time when consent is given to the real content service, and transmits a plurality of content images to the virtual reality content provider 220 so that the virtual reality content can be provided at the point in time when the terminal device 100 agrees to the virtual reality content service. .

More specifically, the content management unit 212 receives and stores in advance content and content images that can be serviced at a number of specific locations located in a specific location.

For example, when the specific location is a shopping mall and there are multiple stores, the content management unit 212 receives and stores content that can provide services from multiple stores in advance before the virtual reality content service is provided. At this time, the content management unit 212 receives a plurality of content related to a specific location and simultaneously receives and stores related content images.

In addition, when the content management unit 212 confirms from the entry confirmation unit 211 that the terminal device 100 has entered a specific location, it generates a usage consent confirmation message confirming whether to use the virtual reality content service, This is transmitted to the terminal device 100 through the communication device 3310.

Afterwards, the content management unit 212 receives a consent message through the communication device 3310 if the terminal device 100 agrees to use the virtual reality content service, and if the terminal device 100 does not agree to use the virtual reality content service, the communication device ( A non-agreement message is received through 3310).

When the consent message is received, the content management unit 212 sends a number of content images related to a specific location to the virtual reality content provider 220 at the time the terminal device 100 agrees to receive them so that virtual reality content can be created. Deliver.

If the non-agree message is received, the non-agree message is sent to the virtual reality content provider 220 so that the terminal device 100 determines that the virtual reality content service is not used even if it has entered a specific location and the virtual reality content is not provided. conveys.

The virtual reality content provider 220 provides virtual reality content based on captured images transmitted from the terminal device 100 that has entered a specific location through the communication device 3310.

More specifically, the virtual reality content providing unit 220 includes a tracking unit 221, a detection unit 222, and a generating unit 223.

The tracking unit 221 tracks the movement path to the specific location based on the captured image.

That is, the tracking unit 221 receives captured images along the direction and line of movement within a specific location in real time from the terminal device 100 through the communication device 3310. The tracking unit 221 tracks the movement path of the terminal device 100 within a specific location using captured images changed by the direction and movement line of the terminal device 100 within the specific location in real time.

Thereafter, the tracking unit 221 transmits the movement path tracking result of tracking the movement path of the terminal device 100 in real time to the detection unit 222.

The detection unit 222 determines whether the terminal device 100 has reached a specific location based on the movement path tracking result.

That is, the detection unit 222 tracks the movement path of the terminal device 100 based on the movement path tracking result and compares the captured image with a previously stored image related to a specific location (hereinafter, a specific location image) to calculate a matching value. Based on this, it is determined whether a specific location has been reached.

Here, a specific location image refers to a specific location, such as the internal interior (e.g., internal signage, products for sale, etc.), external interior (e.g., external signage, initials, business characteristic information, etc.), and services provided at a specific location within a specific location. Any image that can be viewed can be included. In this regard, for example, if the specific location is a bakery, all images that can identify the specific bakery, such as the bakery's business name, logo, and products for sale, can become specific location images.

More specifically, the detection unit 222 compares the captured image included in the movement path tracking result with a pre-stored specific location image.

In other words, the captured image captured while the terminal device 100 entering a specific location is moving is related to the location of the terminal device 100, so the captured image included in the movement path is compared with the previously stored specific location image to determine the best match. By detecting a specific location image with a high value, it is determined whether a specific location has been reached.

Ultimately, the detection unit 222 determines that the terminal device 100 has reached a specific location when the captured image with the highest matching value is confirmed while tracking the movement path based on the captured image. , Detects content images mapped to specific locations. Afterwards, the detection unit 222 transmits the content image mapped to a specific location and the movement path tracking result to the generation unit 223.

The creation unit 223 generates virtual reality content using the content image mapped to a specific location and the movement path tracking result received from the detection unit 222.

That is, the generator 223 detects a captured image of a specific location included in the movement path tracking result and generates virtual reality content by combining the captured image of the specific location and the content image based on a virtual rendering technique. do.

Here, the virtual rendering technique may be an Augmented Reality (AR) technique, which is a field of Virtual Reality, and the actual captured image and virtual content image captured by the terminal device 100 It refers to a graphic technique that synthesizes to make it appear as if a virtual content image exists in a specific location.

Meanwhile, when the creation unit 223 generates virtual reality content for a specific location on the movement path and provides it to the terminal device 100, if the virtual reality content for the specific location is selected from the terminal device 100, , route guidance information for other specific locations that provide services similar to the virtual reality content for the selected specific location may be additionally synthesized with the virtual reality content and provided.

For example, when the terminal device 100 selects the virtual reality content mapped to Company A's bakery among the virtual reality content for at least one specific location existing on the movement path, the creation unit 223 selects the VR content mapped to Company A's bakery. Detects specific locations such as other company B bakeries and company C bakeries & coffee shops that provide similar services.

Afterwards, the generator 223 generates route information starting from the current location of the terminal device 100, that is, Company A's bakery, and reaches other Company B bakeries and Company C bakeries & coffee shops, and uses this as virtual reality content. It can also be provided by additional synthesis.

Of course, the additionally provided route guidance information may be additionally synthesized and provided to the virtual reality content mapped to the bakery of Company A selected by the terminal device 100, or may be separately synthesized and provided through a pop-up window.

The history management unit 230 accumulates and stores all virtual reality content provided to the terminal device 100 in the process of using the virtual reality content service. That is, the history management unit 230 accumulates and stores virtual reality content by mapping it to the identification information of the terminal device 100. Here, any information that can identify the terminal device 100 (e.g., user information, phone number, etc.) can be used as the identification information of the terminal device 100.

Thereafter, when the history management unit 230 confirms that the terminal device 100 has revisited a specific location based on the location signal of the terminal device 100 received through the communication device 3310, a rapid virtual reality content service is provided. As much as possible, the accumulated virtual reality content is delivered to the revisit management unit 240.

The revisit management unit 240 accumulates information based on the result of determining whether the terminal device 100 has revisited a specific location based on the location signal of the terminal device 100 received through the communication device 3310. Virtual reality content is provided to the terminal device 100 first.

More specifically, when the return visit management unit 240 confirms that the terminal device 100 has entered a specific location based on the location signal of the terminal device 100 received through the communication device 3310, the terminal device ( It is determined whether the terminal device 100 is to be revisited based on the identification information of the terminal device 100.

That is, the revisit management unit 240 determines whether virtual reality content mapped to the identification information of the terminal device 100 exists. If there is virtual reality content mapped to the identification information of the terminal device 100, the revisit management unit 240 determines that the terminal device 100 has entered a specific location and has been provided with a virtual reality content service. If there is no virtual reality content mapped to the identification information of the terminal device 100, it is determined that the terminal device 100 visited a specific place for the first time or visited a specific place but did not use the virtual reality content service.

Accordingly, when the terminal device 100 revisits a specific location, the revisit management unit 240 provides virtual reality content accumulated through previous visits before the terminal device 100 moves to the specific location and collects the captured images. is given priority.

At this time, if there are multiple purposes for previously visiting a specific place, the revisit management unit 240 may provide virtual reality content selected according to the purpose of the previous visit, divided by the purpose of the visit. Accordingly, virtual reality content can be provided more quickly when returning for the same purpose as the previous visit.

Through this, when a user who has visited a specific place returns for the same purpose as the previous visit, he or she receives the desired virtual reality content upon entering the specific place without performing the virtual reality content creation process described above and quickly and accurately determines the purpose of the visit. It becomes achievable.

Hereinafter, the operation flow in the terminal device 100 and the hardware system 1000 for implementing the same according to an embodiment of the present invention will be described with reference to FIGS. 6 and 7, and for convenience of explanation, the above The configurations shown in Figures 1 to 5 will be described by referring to the corresponding reference numbers.

First, with reference to FIG. 6, the operation flow in the terminal device 100 according to an embodiment of the present invention is as follows.

First, the location information management unit 110 communicates with the content providing device 200 to confirm that the terminal device 100 has entered a specific location (S100).

More specifically, the location information management unit 110 recognizes a location determination terminal (not shown) within a specific location and performs communication with the content providing device 200 that operates in conjunction with the location determination terminal (not shown). This confirms that the terminal device 100 has entered a specific location.

In the present invention, it is mentioned that entering a specific location is confirmed by communicating with the content providing device 200. However, without communication with the content providing device 200, a specific location is stored in the application that operates as a content providing medium in the terminal device 100. It is also possible to confirm that the terminal device 100 has entered a specific location using the location information value of the location.

The service processing unit 120 receives a usage consent confirmation message confirming whether to use the virtual reality content service from the content providing device 200 after entering a specific location (S110).

That is, the service processing unit 120 receives the use consent confirmation message from the content providing device 200 as a text message or a pop-up message through an application.

Thereafter, the service processing unit 120 generates a response message to the use consent confirmation message and delivers it to the content providing device 200 (S120).

That is, if the user agrees to use the virtual reality content service, the service processing unit 120 generates a consent message and delivers it to the content providing device 200. If the user does not agree to the use of the virtual reality content service, the service processing unit 120 generates a consent message and delivers it to the content providing device 200. A message is generated and delivered to the content providing device 200.

Afterwards, if the service processing unit 120 agrees to use the virtual reality content service, it controls the operation of the camera so that a captured image is generated along the direction and line in which the terminal device 100 moves in real time.

That is, the service processing unit 120 transmits the captured image generated in real time to the content providing device 200 so that virtual reality content can be created based on the captured image while moving to a specific location (S130).

Meanwhile, the service processing unit 120 controls the camera not to operate if the user does not agree to use the virtual reality content service. At this time, it is desirable for the camera to operate under user control regardless of whether the virtual reality content service is used.

When the service processing unit 120 receives virtual reality content from the content providing device 200, it controls the received virtual reality content to be displayed through a display unit (not shown) (S140, S150).

Hereinafter, the operation flow of each component in the hardware system 1000 for implementing the terminal device 100 according to an embodiment of the present invention will be described in detail with reference to FIG. 7.

For convenience of explanation, it is assumed that each component in the terminal device 100 stored in the form of a software module in the memory 1210 is executed by the processor 1100.

First, the location information management unit 110 communicates with the content providing device 200 to confirm that the terminal device 100 has entered a specific location (S100`).

More specifically, the location information management unit 110 recognizes a location determination terminal (not shown) in a specific location through the location confirmation device 1310, and recognizes the location determination terminal (not shown) through the communication device 1320. It is confirmed that the terminal device 100 has entered a specific location by performing communication with the content providing device 200 that operates in conjunction with.

In the present invention, it is mentioned that entering a specific location is confirmed by communicating with the content providing device 200. However, without communication with the content providing device 200, a specific location is stored in the application that operates as a content providing medium in the terminal device 100. It is also possible to confirm that the terminal device 100 has entered a specific location using the location information value of the location.

The service processing unit 120 receives a usage consent confirmation message confirming whether to use the virtual reality content service from the content providing device 200 after entering a specific location (S110`).

That is, the service processing unit 120 receives the usage consent confirmation message from the content providing device 200 through the communication device 1320 as a text message or a pop-up message through an application.

Thereafter, the service processing unit 120 generates a response message to the use consent confirmation message and delivers it to the content providing device 200 (S120`).

That is, when the user agrees to use the virtual reality content service, the service processing unit 120 generates a consent message and then transmits it to the content providing device 200 through the communication device 1320, and the user uses the virtual reality content service. If there is no agreement, a non-agree message is generated and then delivered to the content providing device 200 through the communication device 1320.

Afterwards, if the service processing unit 120 agrees to use the virtual reality content service, the service processing unit 120 drives the camera through the input/output device 1330 to generate a captured image along the direction and line in which the terminal device 100 moves in real time. This controls the display 1331.

That is, the service processing unit 120 transmits the captured image displayed through the display 1331 in real time to the content through the communication device 1320 so that virtual reality content can be created based on the captured image while moving to a specific location. It is transmitted to the provision device 200 (S130`).

Meanwhile, the service processing unit 120 controls the input/output device 1330 so that the camera is not driven if the user does not agree to use the virtual reality content service. At this time, it is desirable for the camera to operate under user control regardless of whether the virtual reality content service is used.

When the service processing unit 120 receives virtual reality content from the content providing device 200 through the communication device 1320, the service processing unit 120 controls the display 1331 through the input/output device 1330 to display the virtual reality content. Do (S140`, S150`).

After completing the description of the terminal device 100 and the operational flow in the hardware system 1000 for implementing the terminal device 100, the content providing device according to an embodiment of the present invention will be described with reference to FIGS. 8 and 9. The operation flow in (200) and the hardware system (3000) for implementing it will be described.

First, the operation flow in the content providing device 200 according to an embodiment of the present invention will be described with reference to FIG. 8.

First, the operation management unit 210 manages the overall operation status of multiple content images related to a specific location (S200).

More specifically, the operation management unit 210 includes an entry confirmation unit 211 and a content management unit 212.

The entry confirmation unit 211 determines whether the terminal device 100 has entered a specific location in conjunction with a location measurement terminal (not shown) within the specific location.

That is, when the entry confirmation unit 211 detects the location signal of the terminal device 100 through a location measurement terminal (not shown) within a specific location, it determines that the terminal device 100 has entered the specific location.

When the current location of the terminal device 100 within a specific location is received in this way, the entry confirmation unit 211 can confirm the location where the terminal device 100 entered among various entry routes within the specific location.

Afterwards, the entry confirmation unit 211 notifies the content management unit 212 that the terminal device 100 has entered.

The content management unit 212 manages a number of contents related to a specific location, and provides a number of content images as virtual reality content so that virtual reality content can be provided at the time the terminal device 100 agrees to the virtual reality content service. Delivered to study (220).

More specifically, the content management unit 212 receives and stores in advance content and content images that can be serviced at a number of specific locations located in a specific location.

For example, when the specific location is a shopping mall and there are multiple stores, the content management unit 212 receives and stores content that can provide services from multiple stores in advance before the virtual reality content service is provided. At this time, the content management unit 212 receives a plurality of content related to a specific location and simultaneously receives and stores related content images.

In addition, when the content management unit 212 confirms from the entry confirmation unit 211 that the terminal device 100 has entered a specific location, it generates a usage consent confirmation message confirming whether to use the virtual reality content service. , and transmits it to the terminal device 100 (S200).

Afterwards, the content management unit 212 receives a response message to the use consent confirmation message (S210). That is, the content management unit 212 receives a consent message from the terminal device 100 when the user agrees to use the virtual reality content service, and receives a non-consent message when the user does not agree to use the virtual reality content service.

When the consent message is received, the content management unit 212 transmits the content image related to the specific location to the virtual reality content providing unit 220 at the time the terminal device 100 agrees to receive it so that virtual reality content can be created. .

If the non-agree message is received, the non-agree message is sent to the virtual reality content provider 220 so that the terminal device 100 determines that the virtual reality content service is not used even if it has entered a specific location and the virtual reality content is not provided. conveys.

The virtual reality content providing unit 220 provides virtual reality content based on captured images transmitted from the terminal device 100 entering a specific location.

More specifically, the virtual reality content providing unit 220 includes a tracking unit 221, a detection unit 222, and a generating unit 223.

The tracking unit 221 tracks the movement path to the specific location based on the captured image.

That is, the tracking unit 221 receives captured images along the moving direction and line of movement within a specific location in real time from the terminal device 100 (S230). The tracking unit 221 tracks the movement path of the terminal device 100 within a specific location using the captured image changed by the moving direction and movement line of the terminal device 100 in real time (S240).

Thereafter, the tracking unit 221 transmits the movement path tracking result of tracking the movement path of the terminal device 100 in real time to the detection unit 222.

The detection unit 222 determines whether the terminal device 100 has reached a specific location based on the movement path tracking result (S250).

That is, the detection unit 222 tracks the movement path of the terminal device 100 based on the movement path tracking result and compares the captured image with a previously stored image related to a specific location (hereinafter, a specific location image) to calculate a matching value. Based on this, it is determined whether a specific location has been reached.

Here, a specific location image refers to a specific location, such as the internal interior (e.g., internal signage, products for sale, etc.), external interior (e.g., external signage, initials, business characteristic information, etc.), and services provided at a specific location within a specific location. Any image that can be viewed can be included. In this regard, for example, if the specific location is a bakery, all images that can identify the specific bakery, such as the bakery's business name, logo, and products for sale, can become specific location images.

More specifically, the detection unit 222 compares the captured image included in the movement path tracking result with a pre-stored specific location image.

In other words, the captured image captured while the terminal device 100 entering a specific location is moving is related to the location of the terminal device 100, so the captured image included in the movement path is compared with the previously stored specific location image to determine the best match. By detecting a specific location image with a high value, it is determined whether a specific location has been reached.

Ultimately, the detection unit 222 determines that the terminal device 100 has reached a specific location when the captured image with the highest matching value is confirmed while tracking the movement path based on the captured image. , Detects content images mapped to specific locations. Afterwards, the detection unit 222 transmits the content image mapped to a specific location and the movement path tracking result to the generation unit 223.

The generation unit 223 generates virtual reality content using the content image mapped to a specific location and the movement path tracking result received from the detection unit 222 (S270).

That is, the generator 223 detects a captured image of a specific location included in the movement path tracking result and generates virtual reality content by combining the captured image of the specific location and the content image based on a virtual rendering technique. do.

Thereafter, the virtual reality content providing unit 220 transmits the virtual reality content to the terminal device 100 so that the virtual reality content can be displayed on the terminal device 100 (S280).

Meanwhile, when the creation unit 223 generates virtual reality content for at least one specific location existing on the movement path and provides it to the terminal device 100, the virtual reality content for the specific location is generated from the terminal device 100. If is selected, a step of additionally synthesizing and providing route guidance information for another specific location that provides services similar to the virtual reality content of the selected specific location to the virtual reality content may be further performed.

For example, when the terminal device 100 selects the virtual reality content mapped to Company A's bakery among the virtual reality content for at least one specific location existing on the movement path, the creation unit 223 selects the VR content mapped to Company A's bakery. Detects specific locations such as other company B bakeries and company C bakeries & coffee shops that provide similar services.

Afterwards, the generator 223 generates route information starting from the current location of the terminal device 100, that is, Company A's bakery, and reaches other Company B bakeries and Company C bakeries & coffee shops, and uses this as virtual reality content. It can also be provided by additional synthesis.

Of course, the additionally provided route guidance information may be additionally synthesized and provided to the virtual reality content mapped to the bakery of Company A selected by the terminal device 100, or may be separately synthesized and provided through a pop-up window.

If the terminal device 100 revisits a specific location, the revisit is performed by the history management unit 230 and the revisit management unit 240 before performing the above-described virtual reality content generating steps (S210-S270). Processing steps may be performed directly.

More specifically, the history management unit 230 accumulates and stores all virtual reality content provided while the terminal device 100 uses the virtual reality content service. That is, the history management unit 230 accumulates and stores virtual reality content by mapping it to the identification information of the terminal device 100. Thereafter, when the terminal device 100 revisits a specific location, the history management unit 230 transfers the accumulated virtual reality content to the revisit management unit 240 to enable rapid virtual reality content service.

The revisit management unit 240 preferentially provides accumulated virtual reality content to the terminal device 100 based on the result of determining whether the terminal device 100 has revisited a specific location.

More specifically, when it is confirmed that the terminal device 100 has entered a specific location, the revisit management unit 240 determines whether the terminal device 100 will revisit based on the identification information of the terminal device 100. .

That is, the revisit management unit 240 determines whether virtual reality content mapped to the identification information of the terminal device 100 exists. If there is virtual reality content mapped to the identification information of the terminal device 100, the revisit management unit 240 determines that the terminal device 100 has entered a specific location and has been provided with a virtual reality content service. If there is no virtual reality content mapped to the identification information of the terminal device 100, it is determined that the terminal device 100 visited a specific place for the first time or visited a specific place but did not use the virtual reality content service.

Afterwards, when the terminal device 100 revisits a specific location, the revisit management unit 240 controls the virtual reality content accumulated through previous visits before the terminal device 100 moves to the specific location and collects the captured images. is given priority.

At this time, if there are multiple purposes for previously visiting a specific place, the revisit management unit 240 may provide virtual reality content selected according to the purpose of the previous visit, divided by the purpose of the visit. Accordingly, virtual reality content can be provided more quickly when returning for the same purpose as the previous visit.

Through this, when a user who has visited a specific place returns for the same purpose as the previous visit, he or she receives the desired virtual reality content upon entering the specific place without performing the virtual reality content creation process described above and quickly and accurately determines the purpose of the visit. It becomes achievable.

Hereinafter, the operation flow of each component in the hardware system 3000 for implementing the content providing device 200 according to an embodiment of the present invention will be described in detail with reference to FIG. 9.

For convenience of explanation, it is assumed that each component in the content providing device 200 stored in the form of a software module in the memory 3210 is executed by the processor 3100.

First, the operation management unit 210 manages the overall operation status of multiple content images related to a specific location (S200`).

More specifically, the operation management unit 210 uses the location signal of the terminal device 100 received from a location measurement terminal (not shown) in a specific location through the communication device 3310 to specify the terminal device 100. Determine whether you have entered the location.

That is, when the operation management unit 210 detects the location signal of the terminal device 100 through a location measurement terminal (not shown) in a specific location, it determines that the terminal device 100 has entered the specific location.

The operation management unit 210 manages a plurality of contents related to a specific location and images related to the contents (hereinafter referred to as content images), and based on the consent message received through the communication device 3310, the terminal device 100 creates a virtual The point in time at which the real content service is agreed upon is detected, and a plurality of contents are delivered to the virtual reality content providing unit 220 so that the virtual reality content is provided at the point at which the terminal device 10 agrees to the virtual reality content service.

More specifically, the operation management unit 210 receives and stores content and content images that can be serviced at a number of specific locations located in a specific location in advance.

For example, when the specific location is a shopping mall and there are multiple stores, the content management unit 212 receives and stores content that can provide services from multiple stores in advance before the virtual reality content service is provided. At this time, the content management unit 212 receives a plurality of content related to a specific location and simultaneously receives and stores related content images.

In addition, when the operation management unit 210 confirms that the terminal device 100 has entered a specific location, it generates a use consent confirmation message to confirm whether to use the virtual reality content service, and sends it to the communication device 3310. It is transmitted to the terminal device 100 through (S210`).

Afterwards, the operation management unit 210 receives a response message to the use consent confirmation message (S220`). In other words, the operation management unit 210 receives a consent message through the communication device 3310 if the terminal device 100 agrees to use the virtual reality content service, and if it does not agree to the use of the virtual reality content service, the communication device ( A non-agreement message is received through 3310).

When the consent message is received, the operation management unit 210 sends a plurality of content images related to a specific location to the virtual reality content provider 220 at the time the terminal device 100 agrees to receive them so that virtual reality content can be created. Deliver.

If the non-agree message is received, the non-agree message is sent to the virtual reality content provider 220 so that the terminal device 100 determines that the virtual reality content service is not used even if it has entered a specific location and the virtual reality content is not provided. conveys.

The virtual reality content providing unit 220 provides virtual reality content based on captured images transmitted from the terminal device 100 that has entered a specific location through the communication device 3310.

More specifically, the virtual reality content provider 220 tracks the movement path to the specific location based on the captured image.

That is, the virtual reality content provider 220 receives captured images along the direction and line of movement in real time within a specific location from the terminal device 100 through the communication device 3310 (S230`). The virtual reality content provider 220 tracks the movement path of the terminal device 100 within a specific location using the captured image changed by the direction and movement line in which the terminal device 100 moves in real time (S240`) ).

The virtual reality content provider 220 determines whether the terminal device 100 has reached a specific location based on the movement path tracking result (S250`).

That is, the virtual reality content provider 220 tracks the movement path of the terminal device 100 based on the movement path tracking result and compares the captured image with a pre-stored image related to a specific location (hereinafter, a specific location image). Based on the calculated matching value, it is determined whether a specific location has been reached.

Here, a specific location image refers to a specific location, such as the internal interior (e.g., internal signage, products for sale, etc.), external interior (e.g., external signage, initials, business characteristic information, etc.), and services provided at a specific location within a specific location. Any image that can be viewed can be included. In this regard, for example, if the specific location is a bakery, all images that can identify the specific bakery, such as the bakery's business name, logo, and products for sale, can become specific location images.

More specifically, the virtual reality content provider 220 compares the captured image included in the movement path tracking result with a pre-stored specific location image.

In other words, the virtual reality content provider 220 tracks the movement path based on the captured image, and when the captured image with the highest matching value to the pre-stored specific location image is identified, the terminal device 100 reaches the specific location. It is determined that this has been done, and the content image mapped to a specific location is detected (S260`). Afterwards, the virtual reality content providing unit 220 creates virtual reality content using the content image mapped to a specific location and the movement path tracking result (S270`).

In other words, the virtual reality content provider 220 detects a captured image of a specific location included in the movement path tracking result, and combines the captured image of the specific location and the content image based on a virtualization rendering technique to provide virtual reality. Create content.

Thereafter, the virtual reality content providing unit 220 transmits the virtual reality content to the terminal device 100 through the communication device 3310 so that the virtual reality content can be displayed on the terminal device 100 (S280`).

Meanwhile, when the virtual reality content providing unit 220 generates virtual reality content for at least one specific location existing on the movement path and provides it to the terminal device 100, When virtual reality content is selected, a step may be further performed to provide route guidance information for another specific location that provides services similar to the virtual reality content of the selected specific location by additionally combining it with the virtual reality content.

For example, when the terminal device 100 selects the virtual reality content mapped to Company A's bakery among the virtual reality content for at least one specific location existing on the movement path, the creation unit 223 selects the VR content mapped to Company A's bakery. Detects specific locations such as other company B bakeries and company C bakeries & coffee shops that provide similar services, generates route information from company A bakery to other specific locations, and additionally synthesizes this into virtual reality content. It can also be provided.

Of course, the additionally provided route guidance information may be additionally synthesized and provided to the virtual reality content mapped to the bakery of Company A selected by the terminal device 100, or may be separately synthesized and provided through a pop-up window.

If the terminal device 100 revisits a specific location, the revisit management unit 240 controls the revisit processing step to be performed immediately before performing the above-described virtual reality content generating steps (S210-S270). You may.

The revisit management unit 240 preferentially provides accumulated virtual reality content to the terminal device 100 based on the result of determining whether the terminal device 100 has revisited a specific location.

To this end, the history management unit 230 accumulates and stores all virtual reality content provided to the terminal device 100 in the process of using the virtual reality content service.

That is, the revisit management unit 240 determines whether virtual reality content mapped to the identification information of the terminal device 100 exists. If there is virtual reality content mapped to the identification information of the terminal device 100, the revisit management unit 240 determines that the terminal device 100 has entered a specific location and has been provided with a virtual reality content service. If there is no virtual reality content mapped to the identification information of the terminal device 100, it is determined that the terminal device 100 visited a specific place for the first time or visited a specific place but did not use the virtual reality content service.

Accordingly, when the terminal device 100 revisits a specific location, the revisit management unit 240 provides virtual reality content accumulated through previous visits before the terminal device 100 moves to the specific location and collects the captured images. is given priority.

Through this, when a user who has visited a specific place returns for the same purpose as the previous visit, he or she receives the desired virtual reality content upon entering the specific place without performing the virtual reality content creation process described above and quickly and accurately determines the purpose of the visit. It becomes achievable.

As discussed above, the terminal device 100 and the content providing device 200 according to an embodiment of the present invention, and each hardware system 1000 for implementing the terminal device 100 and the content providing device 200. According to the operation flow in , 3000), the movement path moving to a specific place is tracked based on the captured image while moving to the specific place, and when a specific location is reached while tracking the movement path, the mapped By providing virtual reality content created by virtualizing content images and captured images to the terminal device 100, intuitive awareness of the provided content can be increased, and targeting users who have visited a specific location can provide a site optimized for the purpose of visiting a specific location. By being able to provide content, satisfaction with content provision services can be improved.

As mentioned above, those skilled in the art to which the present invention pertains will understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive.

For example, the terminal device 100 and the content providing device 200 according to the present invention can store information through memory. In one implementation, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in another implementation, the memory may be a non-volatile memory unit. Additionally, memory may include, for example, a hard disk device, an optical disk device, or any other mass storage device.

The terminal device 100 and the content providing device 200 may also use one or more network interface devices such as an Ethernet card, a serial communication device such as an RS-232 port, and/or a wireless interface device such as an 802.11 card for external input and output. It can be included as a device. In other implementations, these input/output devices may include driver devices configured to transmit output data to and receive input data from other input/output devices, such as keyboards, printers, and display devices.

The terminal device 100 and the content providing device 200 can be realized by commands that, when executed, cause one or more processing devices to perform the above-described functions and processes. For example, such instructions may include interpreted instructions, such as script instructions such as JavaScript or ECMAScript instructions, executable code, or other instructions stored on a computer-readable medium.

The terminal device 100 and the content providing device 200 according to the present specification may be implemented in a distributed manner over a network, such as a server farm, or may be implemented as a single computer device.

Implementations of the functional operations and subject matter described in this specification may be implemented as digital electronic circuits, computer software, firmware or hardware including the structures disclosed herein and their structural equivalents, or a combination of one or more of these. Implementation is possible. Implementations of the subject matter described herein may comprise one or more computer program products, that is, one or more modules of computer program instructions encoded on a tangible program storage medium for execution by or for controlling the operation of a processing system. It can be implemented.

The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that affects a machine-readable radio wave signal, or a combination of one or more of these.

In this specification, 'system' or 'device' includes all instruments, devices, and machines for processing data, including, for example, programmable processors, computers, or multiple processors or computers. In addition to the hardware, the processing system may include code that forms an execution environment for computer programs on demand, such as code making up processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these. .

A computer program (also known as a program, software, software application, script, or code) may be written in any form of a programming language, including compiled, interpreted, a priori, or procedural languages, as a stand-alone program, or as a module. It can be deployed in any form, including components, subroutines, or other units suitable for use in a computer environment. Computer programs do not necessarily correspond to files in a file system. A program may be stored within a single file that serves the requested program, or within multiple interacting files (e.g., files storing one or more modules, subprograms, or portions of code), or as part of a file that holds other programs or data. (e.g., one or more scripts stored within a markup language document). The computer program may be deployed to run on a single computer or multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

Meanwhile, computer-readable media suitable for storing computer program instructions and data include semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic disks such as internal hard disks and external disks, magneto-optical disks, and CDs. -Can include all forms of non-volatile memory, media, and memory devices, including ROM and DVD-ROM disks. The processor and memory may be supplemented by, or integrated into, special-purpose logic circuitry.

Implementations of the subject matter described herein may include backend components, such as a data server, middleware components, such as an application server, or, for example, a web browser or graphical user through which a user may interact with an implementation of the subject matter described herein. It may be implemented in a front-end component, such as a client computer with an interface, or in a computing system that includes any combination of one or more of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, such as a telecommunications network.

Although this specification contains details of numerous specific implementations, these should not be construed as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features that may be unique to particular embodiments of particular inventions. It must be understood. Likewise, certain features described herein in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination. Furthermore, although features may be described as operating in a particular combination and initially claimed as such, one or more features from a claimed combination may in some cases be excluded from that combination, and the claimed combination may be a sub-combination. It can be changed to a variant of a sub-combination.

Additionally, although operations are depicted in the drawings in a specific order herein, this should not be understood to mean that such operations must be performed in the specific order or sequential order shown or that all illustrated operations must be performed to obtain desirable results. Can not be done. In certain cases, multitasking and parallel processing may be advantageous. Additionally, the separation of various system components in the above-described embodiments should not be construed as requiring such separation in all embodiments, and the described program components and systems may generally be integrated together into a single software product or packaged into multiple software products. You must understand that you can

As such, this specification is not intended to limit the invention to the specific terms presented. Accordingly, although the present invention has been described in detail with reference to the above-described examples, those skilled in the art may make modifications, changes, and variations to the examples without departing from the scope of the present invention. The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

According to the device and method for providing content based on virtual reality of the present invention, content (e.g., advertisements, information, etc.) related to a specific place is based on a captured image taken by a terminal device entering a specific place (e.g., a shopping mall). ) in that it can be provided in a virtualized manner, by overcoming the limitations of existing technology, the possibility of not only using the related technology but also marketing or selling the applied device is not only sufficient, but it is also realistically and clearly feasible, making it feasible for the industry. It is an invention with great potential for use.

100: terminal device
110: Location information management department 120: Service processing department
200: Service device
200: Content provision device
210: Operations Management Department
211: Entry confirmation unit 212: Content management unit
220: Virtual reality content provision department
221: tracking unit 222: detection unit
223: Creation unit 230: History management unit
240: Return visit management department

Claims (12)

An operations management department that manages multiple content images related to a specific location; and
When the captured image collected in relation to the location of the terminal device within the specific location is confirmed, the content image related to the specific location is based on the result of determining whether the terminal device has reached the specific location based on the captured image of the terminal device. It includes a virtual reality content provision department that provides virtualized virtual reality content,
The virtual reality content provision department,
a history management unit that accumulates and stores the virtual reality content provided to the terminal device; and
When the terminal device revisits the specific location, content provision comprising a revisit management unit that classifies and provides the accumulated virtual reality content according to the purpose of the visit before the captured image is received from the terminal device. Device. ◈Claim 2 was abandoned upon payment of the setup registration fee.◈ According to claim 1,
The virtual reality content provision department,
a tracking unit that tracks a movement path to the specific location based on the captured image;
a detection unit that detects a content image mapped to the specific location when the specific location is reached while tracking the movement path; and
A content providing device comprising a generator that generates virtual reality content by combining the captured image related to the specific location and the detected content image based on a virtual rendering technique. delete ◈Claim 4 was abandoned upon payment of the setup registration fee.◈ According to claim 2,
The tracking unit,
A content providing device, characterized in that the movement path is tracked using captured images changed according to the direction and movement line in which the terminal device moves in real time. ◈Claim 5 was abandoned upon payment of the setup registration fee.◈ According to claim 1,
The operation management department,
an entry confirmation unit that checks whether the terminal device has entered the specific location; and
When the terminal device agrees to receive a virtual reality service that provides content related to the specific location, a content management unit that transmits the plurality of content images to the virtual reality content provider at the point in time when the terminal device agrees to receive the content. A content providing device characterized by: ◈Claim 6 was abandoned upon payment of the setup registration fee.◈ According to claim 1,
The specific location is,
A content providing device, characterized in that the location is capable of providing services related to at least one content image among the plurality of content images. A management step of managing multiple content images related to a specific location; and
When an image captured in real time based on the location of the terminal device within the specific location is confirmed, information related to the specific location is determined based on the result of determining whether the terminal device has reached a specific location based on the captured image of the terminal device. It includes a provision step of providing virtual reality content that virtualizes the content,
The provision step is,
A storage step of accumulating and storing the virtual reality content provided to the terminal device; and
When the terminal device revisits the specific location, the content characterized in that it includes a return visit provision step of providing the accumulated virtual reality content by classifying it according to the purpose of the visit before the captured image is received from the terminal device. How the provision device operates. ◈Claim 8 was abandoned upon payment of the setup registration fee.◈ According to claim 7,
The provision step is,
A tracking step of tracking a movement path moving to the specific location based on the captured image received in real time;
A detection step of detecting a content image mapped to the specific location when the specific location is reached while tracking the movement path; and
A method of operating a content providing device, comprising a generation step of generating virtual reality content by combining the captured image at the specific location and the detected content image based on a virtualization rendering technique. delete ◈Claim 10 was abandoned upon payment of the setup registration fee.◈ According to claim 8,
The tracking step is,
A method of operating a content providing device, characterized in that the movement path is tracked using a captured image changed according to the direction and movement line in which the terminal device moves in real time. ◈Claim 11 was abandoned upon payment of the setup registration fee.◈ According to claim 7,
The management step is,
A location confirmation step of checking whether the terminal device has entered the specific location; and
If the terminal device agrees to receive a virtual reality service that provides content related to the specific location, a delivery step of delivering the plurality of content images to virtualize the content related to the specific location at the time the terminal device agrees to receive it. A method of operating a content providing device comprising: ◈Claim 12 was abandoned upon payment of the setup registration fee.◈ According to claim 7,
The specific location is,
A method of operating a content providing device, characterized in that the location is capable of providing services related to at least one content image among the plurality of content images.

Images