KR102204027B1 - System for providing augmented reality contents through data stored by location based classification and method thereof - Google Patents

Abstract

The present invention relates to a system for providing smart augmented reality content, and more particularly, a predetermined area in which each object (for example, a store registered in an augmented reality content providing service) is distributed according to an area or building with reference to GPS information. After classification, a learning model for each location is created by learning the mapping relationship between the image of each object included in the classified area or building and the store information related to the goods and services provided by each object, and a specific object from the user terminal When an image is provided along with GPS information, a system for extracting store information for a specific object image by applying a specific object image to a learning model for each location, and outputting the extracted store information in augmented reality through a user terminal; and It's about how.

Description

System for providing augmented reality contents through data classified and stored on a regional basis and its method {SYSTEM FOR PROVIDING AUGMENTED REALITY CONTENTS THROUGH DATA STORED BY LOCATION BASED CLASSIFICATION AND METHOD THEREOF}

The present invention relates to a system and method for providing an augmented reality content through data classified and stored on a regional basis, and in more detail, each object collected from each object (for example, a store registered in an augmented reality content providing service) Store information related to images and goods and services provided by each object is classified and stored on a regional basis, and when a specific object image is provided with GPS information from the user terminal, each generated based on the stored data is classified on a regional basis. The present invention relates to a system and method for extracting store information for a specific object image by applying it to a learning model for each location, and outputting the extracted store information to augmented reality through a user terminal.

Recently, as the performance of user terminals such as smartphones and tablet PCs has been greatly improved, various services and additional functions have been developed.

Among these various services and additional functions, a technology for providing location information of a user terminal is one of the fields that are being studied most actively because the technical utility that can be combined with other services is very high.

For example, when a user wants to know his/her current location, he/she can easily check information about his/her location using his/her user terminal, and if the user searches for a specific location he/she wants, the movement route to that location is displayed in real time. You can move while being guided.

As one of the technologies for providing location information, there is an Augmented Reality technology that superimposes a 3D virtual image on a real image or background and displays it as a single image. In the augmented reality, the real-world information built in advance based on real-time processing is superimposed on an image of the real world that is actually observed through a special interface, and provides an interaction with the real world. It is a technology that allows you to judge the situation.

Meanwhile, the technology for providing location information using the augmented reality can be expanded and used to provide advertisements or information. It is the most widely known. In the process of displaying a camera shot image through an application program, a virtual avatar is displayed as augmented reality. , There is a service that guides the store's advertisement or local information about the user's location or gaze through an avatar displayed in augmented reality.

In addition, in recent years, by applying this, when a user photographs a specific object with a camera and requests additional information, the server side recognizes the object and develops a technology that displays information related to the object in augmented reality. Is losing. For example, when a user projects a specific product with a camera, detailed information about the product is displayed in augmented reality. That is, by using data learned through machine learning, an object located in the user's gaze or being photographed is guessed, and information related to the object is displayed on the user terminal through augmented reality.

However, the content providing service linked to the conventional augmented reality as described above takes unnecessary work time because all information registered on the server side must be referenced, and thus data noise increases, as well as the Resource usage has increased, and there is a problem in that augmented reality content cannot be quickly provided to users.

Therefore, in the present invention, each object image collected from each object and store information related to goods and services provided by each object are classified and stored on a regional basis, and each location generated based on the stored data classified on the regional basis. As a learning model, we propose a method for extracting store information related to a specific object image requested from a user terminal, and providing the extracted store information to a user terminal to output in augmented reality.

Next, the prior art existing in the technical field of the present invention will be briefly described, and then the technical matters that the present invention intends to achieve differently from the prior art will be described.

First, Korean Patent No. 0651508 (2006.11.29.) relates to a method for providing local information using augmented reality and a local information service system for the same, a GPS receiver and a gyro sensor capable of measuring the position and posture in a mobile communication terminal. The present invention relates to a method and system for providing regional information on the captured terrain and facilities by matching images by finding the location and posture of the camera.

In other words, the prior art captures the surrounding area desired by the user and at the same time receives information on the terrain and facilities of the surrounding area captured from the server, and allows the user to view it in a form synthesized with the photographed real-world image. It describes methods and systems that can be used in various fields such as shopping and advertising.

In addition, Korean Patent Registration No. 0983912 (2010.09.27.) relates to an information processing device and information input and search method using an augmented reality technique, by fusing an image signal input from a portable terminal and location information obtained from a GPS terminal. It relates to a method of acquiring information about an object or environment visible through a camera input through a web service and displaying it to a user through an augmented reality technique.

That is, the prior art describes a method for easily receiving information through a mobile device possessed by the user, even if there is no appropriate explanatory data or a device for providing additional information when a user goes to an unfamiliar place or discovers a new object. have.

As a result of reviewing the prior art above, most of the prior art is a configuration in which the information requested by the user is superimposed on the live-action image, and when the user goes to an unfamiliar place or discovers a new object, the user has a mobile device. It only describes the technical concept of a configuration that can receive information through augmented reality through a device.

On the contrary, the present invention classifies and stores each object image collected from each object and store information related to goods and services provided by each object on a regional basis, and learns for each location based on the stored data classified on the regional basis. By creating a model, extracting store information related to a specific object image requested from the user terminal through the learning model for each location and outputting it to augmented reality through the corresponding user terminal, the time, speed, and efficiency required for data extraction can be improved. It can increase and increase server-side resource utilization. Therefore, the prior art does not describe or suggest such a technical feature of the present invention.

The present invention was created in order to solve the above problems, and it is possible to classify and store each object image collected from each object in an augmented reality content providing server and store information related to goods and services provided by each object on a regional basis. It is an object to provide a system and a method thereof.

In addition, the present invention divides and classifies a predetermined area in which each object is distributed in an augmented reality content providing server according to an area or building based on GPS information, and based on the classified information, the image of each object and the store information provided by each object are Another object is to provide a system and method for easily storing and managing augmented reality contents for each object by learning the mapping relationship and generating a learning model for each location.

In addition, when the augmented reality content providing server provides augmented reality content for a specific object according to the request of the user terminal, the present invention uses a learning model generated according to the corresponding area or building based on the GPS information of the user terminal. Another object is to provide a system and method capable of increasing the time, speed and efficiency required for data extraction by extracting augmented reality content for an object.

In addition, the present invention extracts augmented reality content for a specific object using a learning model for each location from the augmented reality content providing server and provides it to a user terminal, thereby reducing unnecessary work requiring reference to all objects registered in a predetermined area. Another object is to provide a system and method capable of providing augmented reality content to a user terminal through efficient machine learning because data noise can be reduced.

An augmented reality content providing system according to an embodiment of the present invention comprises a classification unit for classifying and storing object images collected from each object and store information related to goods and services provided by each object based on a region, and the classification and storage A learning unit that creates a learning model for each location to output store information for a specific object image by learning the mapping relationship between the image of each object and the store information related to the products sold or services provided by each object, And a content provider configured to extract store information for a specific object image by applying the specific object image to the generated learning model when an augmented reality content including a specific object image and GPS information is requested from the user terminal. It is characterized.

In addition, the content providing unit, by referring to the GPS information provided from the user terminal, checks which learning model to extract store information from among the generated learning models for each location, and selects one specific learning model. Then, by inputting the specific object image to the selected specific learning model, it characterized in that it further comprises extracting store information related to a product sold or a service provided by the object.

In addition, the object is a store registered in the augmented reality content providing service, the object image is an image including a signboard, a name, a store, or a combination of the store, and the store information is a product sold by the object or a service provided. It characterized in that it includes images, text, graphics, or a combination thereof related to.

In addition, the augmented reality content providing system may further include a streaming output unit that converts the extracted store information into streaming data and then transmits it to the user terminal for display as augmented reality data.

In addition, the augmented reality content providing system further includes a data collection unit for collecting object images and store information from each object registered in the augmented reality content providing service, the learning unit, from the object previously registered through the data collection unit When the changed object image and store information are collected, or when the object image and store information are collected from a new object, the generated learning model for each location is updated.

In addition, the augmented reality content providing method according to an embodiment of the present invention, in the augmented reality content providing server, classifies object images collected from each object and store information related to goods and services provided by each object based on a region Classification step of storing, in the augmented reality content providing server, by learning the mapping relationship between the image of each object included in the classified area or building and the store information related to the goods sold or services provided by each object, A learning step of generating a learning model for each location for outputting store information for an object image, and when an augmented reality content including a specific object image and GPS information is requested from a user terminal in the augmented reality content providing server, the And a content providing step of extracting store information for the specific object image by applying the specific object image to the generated learning model.

In addition, in the content providing step, in the augmented reality content providing server, by referring to the GPS information provided from the user terminal, which of the generated learning models for each location is applied to extract store information. Determining and selecting any one specific learning model, and by inputting the specific object image to the selected specific learning model, further comprising the step of extracting store information related to the product or service provided by the object It features.

In addition, the augmented reality content providing method further comprises a streaming output step of converting the extracted store information into streaming data in the augmented reality content providing server and transmitting it to the user terminal to display the augmented reality data. It is characterized.

In addition, the augmented reality content providing method further comprises a data collection step of collecting object images and store information from each object registered in the augmented reality content providing service, in the augmented reality content providing server, the learning step, When the changed object image and store information are collected from the previously registered object through the data collection step, or when the object image and store information are collected from a new object, updating the generated learning model for each location It is characterized.

As described above, according to the augmented reality content providing system and method of the present invention, each object image collected from each object in the augmented reality content providing server and store information related to goods and services provided by each object are classified on a regional basis. It is stored and classified based on the region, and since the data for each location is processed based on the stored data, information for each object can be easily stored and managed in a database, and accordingly, the utilization of server-side resources is increased. Of course, there is an effect that can reduce the construction cost.

In addition, when the augmented reality content providing server provides augmented reality content for a specific object at the request of a user terminal, the present invention uses only a learning model generated according to a corresponding area or building based on the GPS information of the user terminal. Since the augmented reality content for a specific object is extracted, the time, speed, and efficiency required for content extraction are greatly increased, and accordingly, there is an effect of providing a user with an experience in which augmented reality content can be used at a high speed.

In addition, in the present invention, since the augmented reality content providing server extracts augmented reality content for a specific object using a learning model for each location and provides it to a user terminal, there is no need to refer to all objects registered in a predetermined area. Data noise can be greatly reduced, and as a result, there is an effect of providing augmented reality content to a user terminal through efficient machine learning.

1 is a diagram showing the configuration of an augmented reality content providing system according to an embodiment of the present invention.
2 is a conceptual diagram illustrating a process of providing an augmented reality content according to an embodiment of the present invention.
3 is a diagram showing in detail the configuration of an augmented reality content providing server according to an embodiment of the present invention.
4 is a diagram for describing an example of classification based on GPS information applied to the present invention.
5 is a diagram showing in detail the configuration of a user terminal according to an embodiment of the present invention.
6 is a flow chart showing in detail an operation process of a method for providing an augmented reality content according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the augmented reality content providing system and method of the present invention will be described in detail with reference to the accompanying drawings. The same reference numerals in each drawing indicate the same member. In addition, specific structural or functional descriptions of the embodiments of the present invention are exemplified only for the purpose of describing the embodiments according to the present invention, and unless otherwise defined, all terms used herein including technical or scientific terms They have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this specification. It is desirable not to.

Here, the object referred to in the present invention means a store registered in an augmented reality content service provided by the augmented reality content providing server 100. In addition, the store is a place, a store, etc. that sells various types of goods, provides services, or sells various foods.

1 is a diagram showing the configuration of an augmented reality content providing system according to an embodiment of the present invention.

As shown in Fig. 1, the system of the present invention includes an augmented reality content providing server 100, a plurality of user terminals 200, a database 300, a plurality of objects 400, and the like.

In addition, although the system of the present invention is not shown in the drawing, the server-side operator or manager performs various function settings, data input, content provision result confirmation, etc. for augmented reality provision service, and each store-side owner It may additionally include a store owner terminal for performing data input and checking results of content use by general users.

The augmented reality content providing server 100 classifies and classifies a predetermined area in which a plurality of objects (ie stores) are scattered according to an area or building based on GPS information, and classifies each area based on the classified information. A learning model for each location is created by learning the mapping relationship between the image for each object and the store information provided by each object, and it is stored and managed in the database 300.

That is, the augmented reality content providing server 100 divides a predetermined area into a plurality of compartments with a radius of several meters to tens of meters like a checkerboard (see FIG. 4), and the image of each object included in each compartment and store information It learns the mapping relationship and creates a learning model for each location. More specifically, the augmented reality content providing server 100 classifies and stores each object image collected from each object and store information related to goods and services provided by each object based on a region, and classifies it based on the region. It is to create a learning model for each location based on the saved data.

In this case, the object image is an image including a signboard, a name, a store entrance, a store interior, or a combination thereof, and the store information is an image, text, graphic, or a combination thereof related to the product or service provided by the object. It is information including.

In addition, the image for each object collected and managed by the augmented reality content providing server 100 and the store information provided by each object include GPS information, and accordingly, the augmented reality content providing server 100 When creating a learning model, the GPS information is referred to to learn the mapping relationship between the image of each object and the store information provided by each object.

In addition, the augmented reality content providing server 100 extracts and provides the augmented reality content for a specific object at the request of the user terminal 200 located in a corresponding region. At this time, since the augmented reality content providing server 100 extracts augmented reality content for a specific object using only a learning model generated according to a corresponding area or building based on the GPS information provided from the user terminal 200, By categorizing objects according to GPS information, the method of storing and processing data can be improved, and based on this, the time or speed required for extracting content can be greatly improved.

For example, it is assumed that a learning model for each location is created by dividing a predetermined area into two parts, A and B, and then learning the mapping relationship between the image of each object included in the A and B areas and the store information provided by each object. At this time, when a user located in area A requests augmented reality content by photographing a specific object image in area A, the augmented reality content providing server 100 proceeds to extract the content using the learning model created for area A. Is to do. Then, the search time required for the augmented reality content extraction as well as the time and efficiency are greatly increased.

This is one of the characteristic configurations of the present invention. When the augmented reality content providing server 100 extracts and provides an augmented reality content for a specific object at the request of the user terminal 200, the user terminal 200 Since the necessary augmented reality content is extracted using only the learning model generated according to the area or building in which the user is located, unnecessary work of referencing all objects registered in the predetermined area where the user is located can be reduced. You will get an experience of using augmented reality content at a high speed.

Meanwhile, when the augmented reality content providing server 100 generates a learning model for each location, it is preferable to use a convolution neural network (CNN), which is a type of supervised learning, but is not limited thereto. Artificial Neural Network) or RNN (Recurrent Neural Network) can be applied.

The user terminal 200 is a mobile communication device such as a smartphone or a tablet PC possessed by the user, and an application program capable of using an augmented reality content is installed.

In addition, the user terminal 200 is augmented after a user moving in a predetermined area photographs an image of a specific object located in the direction of his/her gaze (ie, a signboard of a store, a business name, a store entrance, etc.) through the manipulation of the application program. When a real content request is performed, the current GPS information of the user terminal 200 is checked, and the GPS information is transmitted to the augmented reality content providing server 100 through a network along with the object image.

In addition, the user terminal 200 receives from the augmented reality content providing server 100 various products sold by a specific object photographed by the user or detailed content for a service provided and displays it on the screen in augmented reality.

Accordingly, the user does not need to enter the store directly, by simply photographing a sign or business name through an application program installed on the user terminal 200 in an external space, and converts various products or services provided by the store into augmented reality content. Can be confirmed.

At this time, the augmented reality content providing server 100 selects a specific learning model with reference to the GPS information provided from the user terminal 200, and then the augmented reality content for store information related to the object through the selected specific learning model. Since is extracted and provided to the user terminal 200, the user can use the augmented reality content at a very high speed.

The database 300 includes object images including images of signs, trade names, and stores collected from all objects (ie stores) registered in the augmented reality content providing service, as well as goods or food sold by each object, and services provided. Or, it manages by storing detailed information including a combination of these.

In addition, the database 300 stores and manages the learning model for each location created or updated in the augmented reality content providing server 100, and various operation programs and the user used in the augmented reality content providing server 100 It stores and manages update information of an application program used in the terminal 200.

2 is a conceptual diagram illustrating a process of providing an augmented reality content according to an embodiment of the present invention.

As shown in FIG. 2, the augmented reality content providing server 100 collects images and store information of each object 400 registered in the augmented reality content providing service, and stores and manages it in the database (①).

That is, the augmented reality content providing server 100 collects store information, which is detailed information on various goods, food, and services sold by the object (ie, store), as well as object images in which signboards, trade names, and store appearances are imaged. Then, GPS information is assigned to each object image and store information, and stored in the database 300.

At this time, each object image and store information can be directly presented by the corresponding business operator or collected by the server-side administrator through a store visit, and can be collected whenever store information is upgraded or a new store is registered. .

In addition, the augmented reality content providing server 100 classifies a predetermined area into a radius of several meters to tens of meters through GPS information after the object image and store information for each object are collected, and each included in the classified area By learning the mapping relationship between the image of the object and store information, a learning model for each location is created (②).

Meanwhile, the user terminal 200 checks the operation of the user's application program to obtain a specific object image in a predetermined area in which the user is located (③), and transmits an augmented reality content request signal to the augmented reality content providing server 100 (④). At this time, the augmented reality content request signal includes a specific object image captured by the user terminal 200 and current GPS information of the user terminal 200.

Then, the augmented reality content providing server 100 selects a specific learning model from among a plurality of previously generated learning models for each location based on the current GPS information of the user terminal 200, and inputs the object image to the selected learning model. Thus, augmented reality content for a specific object image is extracted (⑤). That is, various content information related to a specific object image is extracted using a learning model generated based on the region where the user is located.

When content related to a specific object image provided from the user terminal 200 is extracted, the augmented reality content providing server 100 transmits the corresponding content to the user terminal 200 by streaming (⑥), and the user terminal ( 200) converts the corresponding content transmitted by streaming into augmented reality data and displays it on the application program (⑦).

That is, conventionally, when an augmented reality content is requested from a user terminal as shown in FIG. 2A, the server side attempts to search all objects built into a database to extract data and provide it to the user terminal. As shown in, each object image collected from each object and store information related to goods and services provided by each object are classified and stored on a regional basis, and when augmented reality data is requested from the user terminal 200, it is divided according to the location. This is a method that tries to search using only the appropriate category among the categories. Therefore, the conventional method has a limitation in that the efficiency of the search is low and the load on the augmented reality content providing server 100 is increased in the actual commercial stage, but the present invention can significantly reduce the efficiency of the search.

3 is a diagram showing in detail the configuration of an augmented reality content providing server according to an embodiment of the present invention, and FIG. 4 is a view for explaining an example of classification based on GPS information applied to the present invention.

As shown in FIG. 3, the augmented reality content providing server 100 includes a data collection unit 110, a classification unit 120, a learning unit 130, a communication unit 140, and a content request confirmation unit ( 150), a content providing unit 160, a streaming output unit 170, an update management unit 180, and the like.

In addition, although the augmented reality content providing server 100 is not shown in the drawing, a power supply for supplying operating power to each component, an input for inputting data related to setting of various functions, a control unit for collectively controlling each component, etc. It may further include.

The data collection unit 110 collects object images and store information from each object registered in the augmented reality content providing service, and stores them in the database 300. At this time, the data collection unit 110 stores the image of each object and detailed store information sold or provided by the object in the database 300, including GPS information.

In addition, the data collection unit 110 may collect changed information whenever store information sold or provided by each object is changed, and may collect an image and store information of the object each time an object is newly registered. have.

The classification unit 120 performs a function of classifying each area to provide an augmented reality content providing service into a radius of several meters to tens of meters based on GPS information. That is, as shown in FIG. 4, the classification unit 120 may classify a predetermined area in which each object is distributed according to an area or building in a checkerboard shape.

That is, the classification unit 120 classifies object images for each object collected from the data collection unit 110 and store information related to goods and services provided by each object on a regional basis and stores them in the database 300. will be.

The learning unit 130 learns a mapping relationship between an image of each object included in an area or building that is classified and stored through the classification unit 120 and store information related to products sold or services provided by each object. Thus, a learning model for each location is generated to output store information for a specific object image.

In this case, in the case of a building divided into a plurality of floors such as a department store, the learning unit 130 may generate a learning model for each floor based on information classified by each floor by the classification unit 120.

In addition, when the learning unit 130 collects changed object images and store information from objects previously registered in the data collection unit 110, or when object images and store information are collected from a new object, each of the generated locations Each learning model can be updated.

The communication unit 140 receives an augmented reality content request signal from the user terminal 200 located in a specific area through a network, transmits it to the content request confirmation unit 150, and is output from the streaming output unit 170. Contents to be displayed in augmented reality are transmitted to the user terminal 200.

The content request confirmation unit 150 checks whether an augmented reality content request signal including a specific object image and GPS information is received from the user terminal 200. That is, a request for a user located in a predetermined area to take an image of a specific object through an application program installed in the user terminal 200 owned by the user, and to check detailed store information sold or provided by the object as augmented reality content To see if it's done.

When the augmented reality content is requested from the user terminal 200 as a result of the confirmation of the content request confirmation unit 150, the content providing unit 160 provides the GPS of the user terminal 200 included in the augmented reality content request signal. The information is checked, and one specific learning model is selected from among a plurality of location-specific learning models previously generated by the learning unit 130 with reference to the checked GPS information.

In addition, the content providing unit 160 inputs a specific object image included in the augmented reality content request signal to the selected specific learning model, and provides images, texts, graphics, or images related to products sold or services provided by the object. The store information including the combination of is extracted.

The streaming output unit 170 converts store information related to a product sold or a service provided by the object extracted by the content providing unit 160 into streaming data, and then, through the communication unit 140, a corresponding user terminal ( 200) to display as augmented reality data.

The update management unit 180 performs upgrade management of various operation programs used in the augmented reality content providing server 100 and an application program used in the user terminal 200.

For example, when the user terminal 200 executes the application program, the update management unit 180 checks the current version, and if there is an upgrade version, the update management unit 180 automatically upgrades the application program.

5 is a diagram showing in detail the configuration of a user terminal according to an embodiment of the present invention.

As shown in FIG. 5, the user terminal 200 includes a camera 210, a content request unit 220, a Global Navigation Satellite System (GNSS) module 230, a communication unit 240, a content display unit 250, etc. Consists of including.

In addition, although not shown in the drawing, the user terminal 200 is a power supply unit that supplies operating power to each component, an input unit for data input for various functions, and when the user terminal is located inside a building where satellite navigation positioning is difficult, It may further include an indoor positioning unit for checking floor information through data transmission/reception with a communication device such as an installed AP.

The content request unit 220 transmits an augmented reality content request signal to the augmented reality content providing server 100 through the communication unit 240 according to the manipulation of the user who has executed the application program.

At this time, the augmented reality content request signal includes an image of a specific object located in the gaze direction of the user photographed by the camera 210 and current location information (ie, GPS information) checked by the GNSS module 230.

The GNSS module 230 calculates current location information by receiving navigation signals from a plurality of satellites, and outputs the calculated current location information to the content request unit 220.

The communication unit 240 transmits the augmented reality content request signal received from the content request unit 220 to the augmented reality content providing server 100 through a network, and according to the augmented reality content request signal, the augmented reality content Streaming data is received from the providing server 100 and transmitted to the content display unit 250.

The content display unit 250 converts the streaming data received from the communication unit 240 into augmented reality data and outputs it on a screen. That is, detailed store information related to a product or service sold by a specific object is displayed in augmented reality on an actual image displayed on the screen through an application program.

Next, an embodiment of a method for providing an augmented reality content according to the present invention configured as described above will be described in detail with reference to FIG. 6. At this time, the order of each step according to the method of the present invention may be changed by a use environment or a person skilled in the art.

6 is a flow chart showing in detail an operation process of a method for providing an augmented reality content according to an embodiment of the present invention.

First, the augmented reality content providing server 100 collects object images such as signboards, trade names, and store photos from each object 400 registered in the augmented reality content providing service, and at the same time, products or services sold by each object Collect detailed store information for (S100).

After collecting the object image and product information for each object through the step S100, the augmented reality content providing server 100 divides a predetermined area in which each object is distributed into a plurality of areas or buildings with reference to GPS information, Object images collected from each object belonging to a divided area or building and store information related to goods and services provided by each object are classified on a regional basis and stored in the database 300 (S200).

In addition, the augmented reality content providing server 100 is classified and stored through the step S200 for each area or building, and the image of each object collected in the step S100 and store information related to the product or service provided by each object. By learning the mapping relationship to each other, a learning model for each location is generated for outputting store information for a specific object image (S300).

Now, the augmented reality content providing server 100 determines whether an augmented reality content including a specific object image and GPS information is requested from the user terminal 200 (S400). That is, a user located in a predetermined area does not directly enter the store, but by simply illuminating the sign or name of a specific store through his or her user terminal 200 from the outside, various products or services provided by the store are converted into augmented reality content. It is to check if you are performing the request to confirm.

As a result of the determination in step S400, when an augmented reality content is requested from the user terminal 200, the augmented reality content providing server 100 checks the GPS information provided from the user terminal 200 requesting the augmented reality content ( S500).

In addition, the augmented reality content providing server 100 selects any one specific learning model matching the GPS information checked in step S500 among the learning models for each location generated in step S300, and then the selected specific learning model. By inputting a specific object image, augmented reality content for store information related to the object is extracted (S600).

Thereafter, the augmented reality content providing server 100 converts the store information extracted in step S600 into streaming data, and then transmits it to the user terminal 200 through a network to display the augmented reality data (S700).

For example, assuming that there are 5 stores registered with augmented reality content service in area A in the city center and 3 stores in area B, the augmented reality content providing server 100 includes object images of 5 stores in area A and 5 By learning the mapping relationship to store information related to the products sold or services provided by the store, a learning model for area A is created. Likewise, object images of 3 stores in area B and products sold by 3 stores are provided. By learning the mapping relationship for service-related store information, a learning model for area B is created. At this time, if the user views the name of a specific store in area B, the augmented reality content providing server 100 inputs the mutual image to the learning model created for area B and extracts store information related to the mutual image. And, the result is displayed in augmented reality through the user terminal 200.

In addition, in the case of a department store consisting of one building, information is different for each floor. For example, assuming that there are 10 jewelry trade names on the first floor and 15 clothes trade names on the second floor, the augmented reality content providing server 100 A learning model is created on the first floor by learning the object images of 10 jewelery trade names and the mapping relationship to store information related to products sold or services provided by 10 stores, and similarly, the object images of 15 suits on the second floor And by learning the mapping relationship for store information related to products sold or services provided by 15 stores, a second-level learning model is created. At this time, if the user views a specific suit on the second floor, the augmented reality content providing server 100 inputs the mutual image to the learning model created for the second floor to extract store information related to the mutual image, and , The result is displayed in augmented reality through the user terminal 200. Here, the augmented reality content providing server 100 may directly receive location information of the second floor from the user, or may receive location information of the second floor through indoor positioning of the user terminal 200.

As described above, the present invention classifies and classifies a predetermined area in which each object is distributed according to an area or building based on GPS information, and generates a learning model for each location based on the classified information. It can be easily stored and managed, and it is possible to increase the utilization of server-side resources and reduce construction costs.

In addition, the present invention greatly increases the time, speed, and efficiency required to extract augmented reality content for a specific object in an augmented reality content providing server, and provides a user with an experience that can use augmented reality content at a high speed. have.

In addition, according to the present invention, it is not necessary to refer to all objects registered in a predetermined area, data noise can be greatly reduced, and as a result, augmented reality content can be provided to a user terminal through efficient machine learning.

As described above, the present invention has been described with reference to the embodiments shown in the drawings, but this is only exemplary, and various modifications and equivalent other embodiments are available from those of ordinary skill in the field to which the technology belongs. You will understand that it is possible. Therefore, the technical protection scope of the present invention should be determined by the following claims.

100: augmented reality content providing server 110: data collection unit
120: classification unit 130: learning unit
140: communication unit 150: content request confirmation unit
160: content providing unit 170: streaming output unit
180: update management unit 200: user terminal
210: camera 220: content request unit
230: GNSS module 240: communication unit
250: content display unit 300: database
400: object

Claims (10)

A classification unit for classifying and storing object images collected from each object and store information related to goods and services provided by each object on a regional basis;
By learning the mapping relationship between the classified and stored image of each object and store information related to the products sold or services provided by each object, a learning model for each location to output store information for a specific object image is created. Learning department; And
When an augmented reality content including a specific object image and GPS information is requested from a user terminal, a content providing unit for extracting store information for a specific object image by applying the specific object image to the generated learning model; and ,
The content providing unit,
By referring to the GPS information provided from the user terminal, selecting one specific learning model among the generated learning models for each location, inputting the specific object image to the selected specific learning model, and selling from the object An augmented reality content providing system, further comprising: extracting store information related to a product or a service to be provided, thereby increasing a time and speed required for extracting store information for the specific object image. delete The method according to claim 1,
The object is a store registered in the augmented reality content providing service provided by the augmented reality content providing system,
The object image is an image including a signboard, a business name, a store, or a combination of the store,
The store information is an augmented reality content providing system, characterized in that information including images, texts, graphics, or a combination thereof related to a product sold or a service provided by the object. The method according to claim 1,
The augmented reality content providing system,
And a streaming output unit configured to convert the store information extracted from the content providing unit into streaming data and then transmit it to the user terminal to display as augmented reality data. The method according to claim 1,
The augmented reality content providing system,
A data collection unit for collecting object images and store information from each object registered in the augmented reality content providing service; further includes,
The learning unit,
When the changed object image and store information are collected from the previously registered object through the data collection unit, or when the object image and store information are collected from a new object, updating the generated learning model for each location Augmented reality content providing system characterized by. A classification step of classifying and storing, in the augmented reality content providing server, object images collected from each object and store information related to goods and services provided by each object based on a region;
In the augmented reality content providing server, learning the mapping relationship between the classified and stored image of each object and store information related to the product or service provided by each object, and outputting store information for a specific object image A learning step of generating a learning model for each location for each location; And
In the augmented reality content providing server, when an augmented reality content including a specific object image and GPS information is requested from a user terminal, the specific object image is applied to the generated learning model to extract store information for a specific object image Includes; providing content to
The content providing step,
In the augmented reality content providing server, by referring to the GPS information provided from the user terminal, selecting any one specific learning model among the generated learning models for each location, and the specific object image in the selected specific learning model By inputting, by extracting store information related to a product sold or a service provided by a corresponding object, the time and speed required for extracting store information for the specific object image can be increased. How to provide augmented reality content. delete The method of claim 6,
The object is a store registered in the augmented reality content providing service provided by the augmented reality content providing server,
The object image is an image including a signboard, a business name, a store, or a combination of the store,
The store information is an augmented reality content providing method, characterized in that information including images, texts, graphics, or a combination thereof related to products sold or services provided by the object. The method of claim 6,
The augmented reality content providing method,
In the augmented reality content providing server, a streaming output step of converting the store information extracted in the content providing step into streaming data and then transmitting it to the user terminal to display the augmented reality data; How to provide real content. The method of claim 6,
The augmented reality content providing method,
In the augmented reality content providing server, a data collection step of collecting object images and store information from each object registered in the augmented reality content providing service; further includes,
The learning step,
When the changed object image and store information are collected from the previously registered object through the data collection step, or when the object image and store information are collected from a new object, updating the generated learning model for each location Augmented reality content providing method, characterized in that.

Images