KR20230081278A - Method of providing product information or store information based on augmended reality and device implementing thereof - Google Patents

Abstract

According to an embodiment of the present invention, provided is a method of providing product or store information based on augmented reality, which includes the steps of: receiving, by a communication unit of a server, customer-generated information from a customer device; creating, by a server's control unit, AR/VR content information using the customer-generated information; and transmitting, by the communication unit, the AR/VR content information to the customer device.

Description

Method and apparatus for providing product or store information based on augmented reality

The present invention relates to a method and apparatus for providing product information based on augmented reality.

Supermarkets, convenience stores, and large shopping malls display and sell many types of products, and consumers spend a lot of time searching for products in the space.

Of course, guide signs and standardized product display methods are provided for consumers, but it may be difficult for consumers to easily find products they want depending on the display methods of each retailer or each store. In addition, if the consumer cannot easily check the information or price of the product, the consumer must take a separate action to check it.

Accordingly, a method of applying augmented reality technology to help consumers easily find products and quickly check product information will be looked at.

This specification proposes a way to find a product or quickly check information on a product through augmented reality or virtual reality content.

This specification proposes a way for customers to find products or quickly check information on products without visiting the store.

This specification proposes a method of promoting store products or informing customers of benefits that customers can receive through augmented reality or virtual reality content to customers inside or outside the store.

The problem to be solved by the present invention is not limited to the above-mentioned problems, and other problems not mentioned herein will be clearly understood by those skilled in the art from the following description.

A method for providing product or store information based on augmented reality according to an embodiment of the present invention is a method in which a communication unit of a customer device transmits any one or more of store identification information or location information or condition information or image information of a customer device to a server. The step of transmitting to and the communication unit receiving AR/VR content information corresponding to the identification information of the store corresponding to the identification information of the store or corresponding to location information or condition information or image information from the server, and the control unit of the customer device AR / Outputting the VR content information to the display unit of the customer device.

A method for providing product or shop information based on augmented reality according to another embodiment of the present invention is any one of shelf identification information in a store, location information in a store, condition information, or image information of a customer device through a communication unit of a customer device. The step of transmitting the above to the server and the step of receiving AR/VR content information from the server by the communication unit that corresponds to identification information of the shelf, corresponds to location information, satisfies condition information, or can be combined with image information, and The controller includes outputting AR/VR content information to a display unit of a customer device.

A method for providing product or shop information based on augmented reality according to another embodiment of the present invention includes the steps of receiving customer-generated information from a customer device by a communication unit of a server, and using the customer-generated information by a control unit of the server. and generating AR/VR content information and transmitting the AR/VR content information to a customer device by a communication unit.

In one embodiment of the present invention, the customer device includes a display unit that outputs information and receives an input corresponding to selection of information, a communication unit that transmits customer-generated information to a server and receives AR/VR content information from the server, and an AR/VR content information. and a control unit controlling the display unit to output VR content information.

In one embodiment of the present invention, the server includes a communication unit that receives customer-side generated information from a customer device and transmits AR/VR content information to the customer device, and a control unit that generates AR/VR content information using the customer-side generated information. do.

In the case of implementing an embodiment of the present invention, it is possible to find a product or quickly check product information using augmented reality or virtual reality content.

In the case of implementing an embodiment of the present invention, a customer can find a product or quickly check information on a product.

In the case of implementing an embodiment of the present invention, products of the store may be promoted through augmented reality or virtual reality contents to customers inside or outside the store, or benefits that customers may receive may be notified.

The effects provided by the present invention are not limited to the effects mentioned above, and other effects not mentioned herein will be clearly understood by those skilled in the art from the description below.

1 is a diagram showing the configuration of a customer device according to an embodiment of the present invention.
2 is a diagram showing the configuration of a guide device according to an embodiment of the present invention.
3 is a diagram showing the configuration of a server according to an embodiment of the present invention.
4 is a diagram illustrating a process of receiving store information as AR/VR content information when a user passes by a specific store or inputs information of a specific store according to an embodiment of the present invention.
5 is a diagram illustrating a process of receiving AR content information from outside a store when a user sets a particular store as a store of interest or automatically or manually inputs identification information on a particular store according to an embodiment of the present invention. .
6 is a diagram in which a customer device according to an embodiment of the present invention displays AR/VR content related to a store outside the store.
7 is a diagram illustrating a process in which a user inside a store or outside the store uses a customer device to use a store extension service according to an embodiment of the present invention.
8 is a diagram illustrating a process in which a customer device outputs user benefit information as AR/VR content information when a user visits a store according to an embodiment of the present invention.
FIG. 9 is a diagram illustrating a process of displaying content related to a product specialized for a corresponding space on the customer device when the customer device moves to a space where a product is displayed according to an embodiment of the present invention.
FIG. 10 is a diagram illustrating a process in which a store or product manufacturer generates information on a product or discount and provides the information to a server according to an embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and will be implemented in various forms different from each other, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention pertains. It is provided to completely inform the person who has the scope of the invention, and the present invention is only defined by the scope of the claims.

In order to clearly describe the present invention, parts irrelevant to the description are omitted, and the same reference numerals are assigned to the same or similar components throughout the specification. In addition, some embodiments of the present invention are described in detail with reference to exemplary drawings. In adding reference numerals to components of each drawing, the same components may have the same numerals as much as possible even if they are displayed on different drawings. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description may be omitted.

Also, terms such as first, second, A, B, (a), and (b) may be used in describing the components of the present invention. These terms are only used to distinguish the component from other components, and the nature, sequence, order, or number of the corresponding component is not limited by the term. When an element is described as being “connected,” “coupled to,” or “connected” to another element, that element is or may be directly connected to that other element, but intervenes between each element. It will be understood that may be "interposed", or each component may be "connected", "coupled" or "connected" through other components.

In this specification, devices such as mobile phones, smartphones, tablets, Head Mounted Displays (HMDs), and Google Glass that are possessed by users and support AR (Augmented Reality) or VR (Virtual Reality) functions in conjunction with offline stores are collectively referred to as customer devices. (Customer Device). In addition, the customer device may include a transparent display.

In addition, a digital watch worn on a wrist may also be a customer device in an embodiment of the present invention.

In addition, a device that communicates with a customer device and provides various information so that the customer device can display information about a store is referred to as a server.

Meanwhile, the devices disposed in the store include a store device and a guide device. In a store, a store manager may use a store device to manage products in the store, or a store manager may use the store device to check product information and place an order.

The guide device is disposed on a shelf of a store or at a specific location of the store, and may provide information related to the shelf, information on products placed on the shelf, or information about the corresponding store to a customer device or a store device.

1 is a diagram showing the configuration of a customer device according to an embodiment of the present invention. The customer device 100 includes a communication unit 110, a storage unit 120, a camera 130, a display unit 140, and a control unit 150. A customer of the store, ie, a user, may possess the customer device 100 .

The communication unit 110 communicates with the server 500 and the guide device 300 . The communication unit 110 may transmit customer-side generated information to the server 500 and receive augmented reality/virtual reality (AR/VR) content information from the server 500 .

The display unit 140 outputs information and receives an input corresponding to selection of the information. A touch screen and the like correspond to this.

The controller 150 controls the communication unit 110, the camera 130, the display unit 140, and the storage unit 120. The storage unit 120 stores temporary or permanent information transmitted by the server 500 and the guide device 300 . The display unit 140 outputs information and may be configured as a touch screen. In the case of a touch screen, a touch input may be received.

An application that receives and processes AR/VR content information may be installed in the controller 150 . The display unit 140 is controlled to output AR/VR content information.

AR/VR content information is augmented reality or virtual reality content, and includes detailed store information, product information, or user-related payment/discount information.

When the camera 130 of the customer device 100 generates image information and transmits it to the server, the communication unit 110 may receive augmented reality content displayed on the corresponding image from the server 500 .

Whether the AR/VR content information is augmented reality or virtual reality content may be determined according to the type of customer generated information transmitted from the customer device 100 to the server 500 and the type of information provided by the server 500. .

The customer-side generated information includes any one or more of store identification information, location information of the customer device 100, identification information of the counter in the store, location information of the customer device 100 in the store, condition information, or image information.

2 is a diagram showing the configuration of a guide device according to an embodiment of the present invention. The guide device 300 may be installed for each shelf, or may be installed on the floor below the shelf or the ceiling above the shelf.

The guide device 300 may transmit the information stored in the storage unit 330 after the sensor 320 collects records of moving or rearranging objects within the assigned area according to the request of the customer device 100 .

In addition, the guide device 300 is disposed at the entrance of the store or outside the store to provide customer devices 100 around the store with information necessary to receive store identification information or store-specific AR/VR content. can

In addition, the guide device 300 may be disposed in a specific area within the store and provide advertisement or promotional information to customer devices 100 entering the corresponding area. To this end, as shown in FIG. 9 to be described later, the guide device 300 may transmit store-related information or information about an area to the customer device 100.

3 is a diagram showing the configuration of a server according to an embodiment of the present invention. The server communicates with a plurality of customer devices and transmits AR/VR content information to the customer devices. The controller 550 controls the communication unit 510 and the database 520, which are each component.

The communication unit 510 of the server 500 communicates with the customer device 100 . Depending on the embodiment, the communication unit 510 of the server 500 may communicate with a POS system, a store device, and the like. The communication unit 510 receives customer-side generated information and transmits AR/VR content information to the customer device 100 .

The database 520 includes information about a product in a store or product sale information, promotion information, or the like transmitted by the customer device 100, a POS system or store device of a store, or a promotion device (PR device) of a product manufacturer or public relations company, or the like. Store purchase information, etc. The stored information is then used to generate AR/VR content information.

The control unit 550 generates AR/VR content information by using customer-side generated information.

4 is a diagram illustrating a process of receiving store information as AR/VR content information when a user passes by a specific store or inputs information of a specific store according to an embodiment of the present invention.

The control unit 150 of the customer device 100 can check the current location of the customer device 100 at regular intervals to determine if there is a store providing AR/VR content in the vicinity (S1a).

Alternatively, the customer device 100 may transmit store identification information to the customer device 100 so that the customer device 100 may confirm that there is a store nearby (S1b).

After confirmation, the customer device 100 transmits store identification information to the server 500 (S2). The store identification information is an example of customer-side generated information. The server 500 receives store identification information and transmits AR/VR content information for the corresponding store (S3).

AR/VR content information includes a variety of store information. For example, a billboard advertising a store special sale or a product subject to a special sale can be expressed in augmented reality or virtual reality on the customer device 100. .

Further, when the user touches or selects AR/VR content information output on the display unit 140 of the customer device 100, detailed information is output on the display unit 140 (S5).

After S5, the user may purchase a product sold in the store using the customer device 100. Alternatively, when the user plans to pick up or receive a predetermined product from the corresponding store, the current location or state of the corresponding product can be checked using AR/VR content information.

That is, the communication unit 510 of the server 500 may receive a reservation purchase request for a product or information on a destination or delivery location of a product from the customer device 100 . Also, the controller 550 of the server 500 may select a specific store according to the received information and deliver the received information to a store device of the corresponding store.

The embodiment of FIG. 4 is summarized as follows. The communication unit 110 of the customer device 100 transmits the customer side generated information to the server (S2). One embodiment of customer-side generated information includes any one or more of store identification information, location information of the customer device 100, condition information, or image information.

Then, the server generates AR/VR content information using the customer-side generated information received in S2 and transmits it as in S3. That is, the communication unit 110 of the customer device 100 receives AR/VR content information corresponding to store identification information or store identification information corresponding to location information, condition information, or image information from the server 500. Do (S3).

When the customer device 100 inputs and transmits the store name, location, store code number, etc., the server 500 generates AR/VR content information in response to the store's identification information if there is a corresponding store. can do.

Meanwhile, as a result of searching the database 520 for a store corresponding to the name or location of the store, the code number of the store, or condition information (a store offering a discount on a specific product), the control unit 550 of the server 500 searches for two or more stores. In this case, the control unit 550 of the server 500 may select one store based on the distance from the customer device 100 or the number of times the customer device 100 frequently uses each store.

In addition, when the customer device 100 transmits a signboard of a store, a visual medium advertising the store, or an image captured around the customer device, the control unit 550 of the server 500 transmits location information of the customer device 100. , or by identifying a character in an image, a corresponding store may be searched in the database 520 . Again, when multiple stores are searched in this process, the controller 550 of the server 500 selects one store based on the distance to the customer device 100 or the number of times the customer device 100 frequently uses each store. You can choose.

Then, the control unit 150 of the customer device 100 outputs AR/VR content information to the display unit 140 of the customer device 100 (S4).

Here, as an example, the image information is an image captured by the camera 130 of the customer device 100 of a signboard of a store or a visual medium (including a QR code or billboard, etc.) advertising the store, or the surroundings of the customer device.

And, the AR / VR content information received in response to this includes any one or more of information on products sold at the store, promotion information of the store, interactive game contents for which rewards are provided, or metaverse contents provided by the server 500. do.

The condition information includes any one or more of a product, a discount condition of the product, a location of a store that sells the product, or identification information of a user set in a customer device. For example, the user may input "Ramen". Alternatively, the user may input "1 + 1 gift".

5 is a diagram illustrating a process of receiving AR content information from outside a store when a user sets a particular store as a store of interest or automatically or manually inputs identification information on a particular store according to an embodiment of the present invention. . This allows the user to input identification information about a specific store by inputting the identification information of the store he/she plans to visit or is interested in, or the logo or QR code on the billboard (S1c). The input store identification information is transmitted to the server 500 (S2), and subsequent processes of S3 to S5 are the same as those of FIG. 4 described above.

When the embodiment of FIGS. 4 and 5 is applied, the user can search for products placed in the store through an app or the web from the outside without entering the store, and purchase, pick-up, and delivery are possible remotely. The convenience of shopping is improved.

In addition, even if the user does not enter the store, the store's logo or QR code can be photographed from outside (bus stop, subway, etc. advertisement promotion material, library, etc.). In this case, when the customer device 100 recognizes a specific design or code and transmits the identification information of the store to the server 500, the server 500 converts the store's promotion and handling product information into AR/VR content information. It can be transmitted to the customer device 100. Accordingly, the user can check store-related information without visiting the store, and can additionally receive purchase/pickup/delivery.

When the embodiment of FIGS. 4 and 5 is applied, the user runs the app at home without going to the store (consent to use personal information), and the nearest stores are located in order of distance or visit by entering location-based or direct information. may be output to the customer device 100 as

In addition, when the user searches for a product of interest, the user can check whether the corresponding product is handled and information about the product augmented with AR/VR content in conjunction with real-time inventory for each store. Thereafter, the user can check AR/VR content information using a mobile device that is the customer device 100 or a device that is an AR platform, and purchases (reservation/pickup/ gifts, etc.)

In addition, the user may input the name of a product in a product search process or input an image of a product searched for on a search portal or the like.

6 is a diagram in which a customer device according to an embodiment of the present invention displays AR/VR content related to a store outside the store. The customer device 100 may photograph a signboard of a store, photograph a visual medium advertising the store, or request store information from a specific location to the server.

The visual medium includes advertisements, CIs, logos, and the like displayed outside the customer device 100 . Advertising videos and billboards at bus stops are visual media. In addition, QR codes and barcodes are also visual media.

The customer device 100a photographs the signboard 210 of the store outside the store (S8). As a result, the AR/VR application installed in the customer device 100a is an image of the inside of the store 200, products displayed on the shelf of the store 200, or today's special products or products of interest to the user as shown in 100b. The registered product 10 is displayed on the display unit 140 .

Special products or products of interest are hereinafter referred to as pop-up products. This means a product that is enlarged and displayed in the image of the shelf, as shown in 10b in the AR/VR content information. In addition, the location of the pop-up product is displayed as shown in 10a, so that the user can check the location of the corresponding product.

Therefore, the user can check the current status of products placed in the store or whether a product of interest is on sale without directly visiting the store 200 .

In S8, in addition to the embodiment in which the user uses the customer device 100a to photograph visual media placed outside, the customer device 100a may automatically request AR/VR content information of the store 200 from the server 500. there is.

In one embodiment, when the customer device 100a is located within a predetermined distance (for example, 30 meters) from the store, the control unit 150 of the customer device 100a executes an AR/VR application so that the adjacent store 200 ) of AR/VR content information may be received from the server 500. In addition, the user can check the AR/VR content displayed on the customer device 100b through vibration or alarm sound.

In a state displayed as shown in 100b of FIG. 6 , the user may select a detailed view displayed as shown in 10b on the customer device 100b to check product information or place an order as shown in 100c (S11).

As an embodiment, the user may order a corresponding product from outside the store using the customer device 100c and receive delivery by inputting a delivery address. Alternatively, the user may use the customer device 100c to select takeout and visit the store 200 to receive the packaged product when the product is ready.

When the embodiment of FIG. 6 is applied, it is possible for a customer to check whether a desired product is available without visiting a store, and to receive purchase/pickup delivery. In addition, customers can purchase products on the go without visiting the store.

On the other hand, when the embodiment of FIG. 6 is applied, when a user passes by a store, a discount event is held at the store 200, a product set to be of interest to the user is on sale, or the store 200 In the case of promotional activities, the customer device 100 requests user confirmation through a predetermined alarm or vibration.

When the user checks the customer device 100, the AR/VR application installed in the customer device 100 outputs the AR/VR content information received from the server 500.

In one embodiment, when a user passes in front of a store and receives an alarm and opens an app to take a picture of the store, or when the customer device 100 automatically recognizes the store, the AR/AR/AR/new product of the store runs around the store. VR content information may be output.

When a user grabs a strategic product or new product displayed in AR/VR by touching the content displayed on the screen, the user's customer device 100 may receive a coupon exchangeable with the product from the server 500 .

When the embodiment of FIG. 6 is applied, the controller 150 of the customer device 100 displays on the display unit 140 an input interface for requesting a reservation purchase of a product or inputting a destination or delivery address of a product.

Also, the communication unit 110 may transmit information input through the input interface to the server 500 . As a result, the server 500 may instruct a store device of a specific store to pre-purchase a product or transmit information about product receipt or delivery to the store device of the store.

The store device is a device that is installed in a store and allows a store operator to place an order or manage products in the store. When a user requests the purchase of a specific product using the customer device 100, the server 500 places an order based on the user's location or preferred store information, or information on a nearby store based on the receipt or delivery location of the corresponding product. Determine the store to receive the In addition, the server 500 transmits information on reservation purchase or delivery/receipt transmitted by the customer device 100 to the store device of the determined store.

7 is a diagram illustrating a process in which a user inside a store or outside a store uses a customer device to use a store expansion service according to an embodiment of the present invention.

Stores in all stores or in some areas may provide a predetermined expansion service for inducing customers to purchase products. An example of the extended service is a game, metaverse-linked SNS, or ESG (Environment, Social, Governance) activity. The server 500 may store information on extended services provided by stores and transmit AR/VR content information providing extended services according to a user's location to the customer device 100 .

The customer device 100 transmits activating information including current location information, information on photographing visual media placed inside/outside the store, or predetermined code information directly input by the user to the server 500 (S13). ).

The code information may be a code composed of numbers or letters, a barcode, or a QR code for a store. The activating information may further include a surrounding picture taken by the user or a coupon received by the customer device 100 .

The surrounding photos taken by the user may inform the server 500 of surrounding conditions (traffic conditions, weather, congestion, nearby objects, etc.) of the area where the user is located. Then, the server 500 analyzes the received activating information and searches for stores capable of providing extended services to the corresponding user (S14).

At this time, provision of the extended service may be determined according to the location or current time of the customer device 100, user information included in the customer device 100, and the like. And the server 500 transmits the AR/VR content information implementing the extended service to the customer device 100 (S15).

An embodiment of the extended service is to provide an image or video of an interactive game or a product that the corresponding user can purchase at a discount in an AR/VR manner.

In addition, an embodiment of the extended service is to output advertisement information provided by the store to the customer device 100 in an AR/VR manner (S16).

Thereafter, the user may perform a predetermined activity corresponding to the output AR/VR content information, and the customer device 100 may transmit information about the activity (activity information) to the server 500 (S17). .

The server 500 transmits information corresponding to the activity information (eg, advertisement, product, benefit information, etc.) to the customer device 100 (S18). Then, the customer device 100 outputs the received product/advertisement/benefit information (S19).

When the embodiment of FIG. 7 is applied, based on the activating information provided by the customer device 100, the server 500 provides an extended service provided by an adjacent store or a store registered as a store of interest by the user in the customer device 100. It can be output as AR/VR content information.

In one embodiment, an interactive game combining context information such as a specific time, season, weather, etc., surrounding objects, digital code photos, and the like, and extended services such as product purchase inducement may be provided to the customer device 100 .

In particular, when the customer device 100 photographs a store after executing an application/software on an app or web to perform a predetermined promotion, a user may be notified that an ESG activity is possible with the current extended service. For example, when the message “Please take a picture of the heat shelter in front of the store~” is output to the customer device 100, the user can take a picture of the heat shelter space in front of the store and transmit the captured image to the server 500 . In addition, the server 500 may provide rewards for predetermined activities of the user as in S18 to S19.

Of course, in this process, the server 500 may provide location information about the heat shelter to the smartphones of people who seek the heat shelter by sharing the uploaded captured image.

The extended service may also be provided in the form of a game. In one embodiment, AR/VR content information in which a predetermined product moves inside the store is output on the screen of the customer device 100 in the store. When the user touches a product in the corresponding content or moves along the corresponding product, the user can move to the shelf of the corresponding product, and when purchasing the corresponding product, the server 500 provides benefits such as a discount or “1 + 1” to the customer device (100) can be provided.

8 is a diagram illustrating a process in which a customer device outputs user benefit information as AR/VR content information when a user visits a store according to an embodiment of the present invention. The customer devices 100a and 100b are devices possessed by two users visiting the same store. Reference numeral 100a is a customer device logged in with information of a first user, and reference numeral 100b is a customer device logged in with information of a second user.

Both users may visit the store and photograph a specific identifier using the customer devices 100a and 100b to recognize them. The identifier may be a store logo, CI, BI, or the like, and may be identification information of a specific shelf. Alternatively, when the user takes out the customer device 100a or 100b, the customer device 100a or 100b may transmit location information to the server 500 and receive AR/VR content information related to the store from the server 500.

In FIG. 8 , parts except for 20b and 21b are images taken by the customer devices 100a and 100b of actual photos of the store as an embodiment. In this case, in the embodiment of FIG. 8, only 20b and 21b are AR/VR contents. That is, the customer devices 100a and 100b may receive AR/VR content information for displaying 20b/21b in augmented reality from the server 500 .

Unlike this, in another embodiment, all parts including 20b and 21b in FIG. 8 are AR/VR content information generated by the server 500 based on actual photos of the store. In this case, in the embodiment of FIG. 8 , all information displayed on the display units 100a and 100b is AR/VR content. That is, the customer devices 100a and 100b may receive, from the server 500, AR/VR content information including an image or video including 20b/21b on a picture of a store.

Even if the two users are in the same store, images output from the customer devices 100a and 100b may be different. For example, when the first user registers a piece of cake as a product of interest, the customer device 100a displays a piece of cake sold in a store as shown in 20b and displays the location of the corresponding product as shown in 20a and an arrow. Further, at the bottom of 20b, promotion information "1 + 1 gift" is output.

When the second user registers coffee as a product of interest, the customer device 100b displays caffe latte sold in the store as 21b and displays the location of the corresponding product as 21a and an arrow. Further, promotion information of “10% discount” is displayed at the bottom of 21b.

As shown in FIG. 8 , the server 500 may provide customized product promotion information to a corresponding customer according to various customer needs.

Also, the customer's needs may be generated based on health care information considering the customer's health. When a customer checks his or her blood sugar level as a matter of concern for his or her health, promotional information on low-sugar products suitable for the customer's needs can be provided to the customer in a customized manner.

8 reduces the process of finding a product by a customer directly finding a product in the store or making an inquiry to a worker, and allows the customer to easily find the product. Previously, new product information or recommended products were exposed to customers through price cards (paper, display, etc.) to inform the location of products.

However, this method has limitations in the information it can provide. For example, even if a customer searches for the location of a product category, it is not easy to find a related event product such as beer or ice cream, so it may frequently occur that the product is not the target product when checking out at the POS.

8 describes an embodiment in which a customer visiting a store can easily find desired products (products of interest, trend products), etc. using AR/VR contents, and easily recognize event products and new products.

Customers can recognize the store logo, CI/BI/shelf design, code, etc.) through customer devices (AR platform) such as mobile devices, and the number of customers in the store, current time, location, and special conditions of the customer When conditions based on context information such as the like are satisfied, the server 500 transmits AR augmented content (product/quantity, marketing, image information, etc.) to the customer device so that the customer can check it.

The user may purchase a product displayed on the customer device 100a or 100b or may proceed with a procedure such as reservation/pickup/gift.

The server 500 may provide customized AR/VR content information to the user's customer devices 100a and 100b based on information on products purchased by the user in the past or products of interest.

In one embodiment, the server 500 may transmit AR/VR content information so that the AR environment is configured differently according to the information of the customer possessing the customer device 100a or 100b. At this time, the server 500 may generate AR/VR content information by collecting the customer's past purchase history, special event products of the held card, interest products of the customer, membership level, and the like as customer information. In addition, the server 500 may generate AR/VR content information for recommending products customized to the user by reflecting the group to which the customer belongs (according to age group/product taste/region, etc.).

In addition, depending on the user's access, products can be promoted through a speaker installed on the shelf or an app push method. For example, the customer device 100 may transmit user identification information of the customer device 100 to the guide device 300 disposed around the shelf.

When the guide device 300 notifies a discount event through a speaker or sends a message to the display unit 140 of the customer device 100 in a push method, the display unit 140 of the customer device 100 displays a discount product, Information on recommended products and the like may be output.

This can provide customized and differentiated services for each customer within the store, which can induce customers to visit the store. In addition, the customer's data accumulated according to the customer's purchase becomes a standard for continuously recommending products or benefits to customers. Through this, stores can also improve their profits, and they can also prepare advertisements or promotional products tailored to the store or to customers visiting the store (targeted for 30s, students in their teens, etc.).

When the embodiment of FIG. 8 is applied, when a customer enters a store, information on 1+1 and 2+1 events and events can be provided to a customer device owned by the customer based on the customer's existing purchase/experience data. there is. In one embodiment, the server 500 may transmit AR/VR content information for guiding a location of a corresponding product and a service zone in a customized manner to a customer device.

Customers who own a customer device, an AR platform displaying AR/VR content information on products subject to events or discounts, can easily and quickly recognize target products, range of benefits, and location of products. To this end, as shown in 100a/100b of FIG. 8, information on products and benefits is intuitively displayed using augmented arrows, augmented product design contents, messages, and the like.

As shown in FIG. 8, when a user who owns a customer device 100a or 100b displaying AR/VR content information purchases a product at a store, a result corresponding to the purchase action is displayed in the virtual space (Metaverse, etc.) in response to this. can This allows customers' purchasing behavior in the virtual space to be converted into new content, allowing a number of customers to perform new activities in the virtual space based on the corresponding content.

For example, content such as a tree growing in a virtual space (metaverse) may be displayed in response to the user's purchase. When content is displayed in the form of a growing tree managed by a specific individual, organization, or community, ESG activities such as special benefits for the community or planting trees in proportion to the content can be carried out.

FIG. 9 is a diagram illustrating a process in which content related to a product specialized for a corresponding space is displayed on the customer device when the customer device moves to a space where a product is displayed according to an embodiment of the present invention.

As an example, product information included in AR/VR content information may be exposed on the customer device 100 possessed by the customer so that the customer can quickly recognize a specific product when the customer moves near the shelf where the corresponding product is placed. . In addition, the AR/VR content information may include content that recommends products included in various promotions, for example, 1+2 products, mix-and-match products, etc. to be purchased simultaneously.

Through AR/VR content information, it is possible to guide the location of event products that require advertising. In addition, according to the request of the advertiser/partner, the server 500 includes the event/service zone information in the AR/VR content information and transmits the AR/VR content only to the customer device 100 of the corresponding store only for stores that meet specific conditions. Allow advertisements or event information to be exposed.

9 is a diagram illustrating a process of performing augmented reality based on a location of a customer device in a store according to an embodiment of the present invention.

A customer (user) who is a visitor to the store may use the customer device 100 to photograph a shelf in the store or obtain an image (video) or identification information of the shelf in the store (S31). A customer may directly photograph the shelf using the camera 130 of the customer device 100 .

Alternatively, when the customer device is an HMD or glasses (glasses), the customer moves toward the store or looks at the store while wearing the customer device 100, and a photograph is taken. In addition, in order to obtain the identification information of the shelf, the customer device 100 may use the information related to the shelf transmitted by the guide device 300 (S32). The guide device 300 may provide identification information of the shelf to the customer device 100 .

In addition, the customer device 100 transmits the customer-side generated information including the captured image or video or shelf identification information and meta information thereto to the server 500 (S33). The meta-information includes various information such as information on whether the corresponding counter is a ramen stand, a stationery stand, or a fresh food stand, or a current status in which products are placed in the corresponding stand unlike before.

The server 500 transmits AR/VR content information to the customer device 100 using the captured image (or video) or shelf identification information, meta information therefor, and identification information of the customer device 100 (S34). . At this time, the server 500 uses the identification information of the customer device 100 to search the database 520 for information such as products frequently purchased by the customer or products registered as interested products by the customer, and customized to the customer. AR/VR content information can be created.

As a result, the display unit 140 of the customer device 100 arranges and outputs the received AR/VR content information on the currently photographed image or video of the shelf (S35). Alternatively, the control unit 150 of the customer device 100 may generate a new image/video by combining AR/VR content information with an existing image/video and output it to the display unit 140. Alternatively, when the customer device 100 transmits only the identification information of the shelf, the customer device 100 may output AR/VR content information to the display unit 140 .

If only the identification information of the shelf is transmitted, the server 500 may generate AR/VR content information including previously stored images or videos of the shelf indicated by the corresponding identification information.

Here, when the user selects AR/VR content information displayed on the customer device 100, detailed information related to the product or promotion is output (S36).

The AR/VR content information may be output as an augmented reality image, including an image of a shelf placed on or adjacent to a currently photographed image of a shelf. It may vary according to user identification information set in the customer device 100 of the AR/VR content information.

As an example, the AR/VR content information may include images of products that the user is interested in or frequently purchased at the currently photographed stand, or products that can be recommended to the user. This is referred to as user specific goods information.

Also, the AR/VR content information may include images of products currently being photographed and promoted (promoted) on the shelf. This is referred to as Promotion Goods Information. This may be applied when a store intends to promote a specific product or when a server conducts a promotion to a specific group of stores.

In addition, promotional product information or user-specified product information is not limited to one product. A plurality of products may be displayed, and may include a product category of a higher concept in which a plurality of products are grouped.

That is, specific ramen may be included in the AR/VR content information, discount information on all types of ramen, or discount information on ramen types of a specific company may be included in the AR/VR content information.

For the AR/VR content information, as shown in FIG. 8 , user-specific product information or promotional product information is displayed in a large size among products displayed on a corresponding shelf.

The user can check user-specific product information or promotional product information expanded to the AR/VR content information, and select detailed view to check detailed information about the product.

The server 500 may generate AR/VR content information according to users or stores. Alternatively, the server 500 may generate AR/VR content information separately from the user/store. In this case, AR/VR content information may be equally provided according to all stores set by the server 500, stores in a certain area, or stores belonging to a certain group.

That is, AR/VR content information displayed on the user's customer device 100 can be equally applied to all stores. Alternatively, AR/VR content information may be equally applied to all stores in the region to which the store belongs. Alternatively, AR/VR content information may be equally applied to stores grouped according to sales or customers of stores.

In addition, the server 500 may generate AR/VR content information customized according to the store's sales or customers.

That is, the server 500 may generate AR contents including customized fact-based data considering the type of store, commercial area, recent sales/customer tendency, and recent trends as AR/VR content information. Also, the AR/VR content information may be exposed on the display unit 140 of the customer device 100 to assist the user in selecting a product.

The AR/VR content information includes promotional product information optimized for stores of the same type or similar layout as the corresponding store, or stores in a commercial area similar to the corresponding store. Accordingly, the server may generate the same AR/VR content information for all stores, or, on the other hand, may generate customized information according to the equilibrium/business district/region/main customer class.

AR/VR content information displayed through AR augmentation may include the latest trend data on related products to the user. As an example, when a user executes an AR/VR application on the customer device 100 and recognizes a shelf or a specific product or selects a product popped up from AR/VR content information, trend data related to the product (SNS, You can check reviews on the blog, purchasing trends compared to the same product group, nutrition information, product use/usage guide, etc.).

Also, the AR/VR application may display products that are not currently placed in the store. In this case, when the customer requests the arrangement of the corresponding product through the customer device 100, the server 500 may notify the manager of the store of the fact and instruct the arrangement of the product.

When the embodiment of FIG. 9 is applied, when a customer visits a store and runs an app through a customer device such as a mobile device to recognize a shelf, recommended products for each store, event products, or customized products are exposed and the customer is interested. You can easily purchase products, new products and trending products.

In addition, AR/VR content information may be generated in advance by a store device of a store and transmitted to the server 500 . For example, the AR/VR content information may include recommendations for event products of the store and AR advertisements of the store, and based on this, the store may independently implement a new business model. That is, the store operator provides AR/VR content information including AR advertisements to the server 500, and through this, it is possible to present a new revenue generation model.

In addition, since customized products are recommended to customers, the customer's convenience can be increased, the process of searching for products can be shortened, and customer-centered recommendation services can be provided throughout the purchase process, thereby optimizing the customer's experience.

When the number of product purchases by customers using the AR/VR content information increases, the server 500 may provide recommendation information to a store device of the store to adjust the arrangement of the corresponding product in the store or to place an additional order.

That is, when the number of purchases of a recommended product increases according to a customer's past purchase history in the corresponding store, it is possible to adjust the arrangement of the corresponding product in the corresponding store or place an additional order.

Meanwhile, when detailed information is output in S36 and the customer inputs reaction (like, comment) information on the output information using the customer device 100, the reaction information is transmitted to the server 500 (S37). The server 500 may collect the received response information and provide product recommendation or guide to the store (S38).

Alternatively, the server 500 may generate report information on a promotion result of a corresponding product to the product manufacturer using the response information.

The embodiment of FIG. 9 is summarized as follows.

The communication unit 110 of the customer device 100 transmits to the server 500 customer-side generated information including any one or more of shelf identification information in the store, location information of the customer device in the store, condition information, or image information ( S33).

Then, the communication unit 110 receives from the server 500 AR/VR content information corresponding to identification information of the shelf, corresponding to location information, satisfying condition information, or combinable to image information (S34). Then, the control unit 150 of the customer device 100 outputs AR/VR content information to the display unit 140 of the customer device 100 (S35).

Here, as an example, the image information is an image captured by the camera 130 of the customer device 100 on a shelf in a store or one or more products displayed on the shelf.

And, in this case, the AR/VR content information includes any one or more of information on a shelf or product, promotion information of the store, or interactive game content provided with a reward or metaverse content provided by the server 500.

In addition, as shown in S32 , the communication unit 110 of the customer device 100 may receive shelf-related information or store location information from the guide device 300 installed in the store or store.

The identification information of the shelf includes barcodes, QR codes, etc. disposed on the shelf.

The condition information includes any one or more of a product, a discount condition of the product, a location of a store that sells the product, or identification information of a user set in a customer device. For example, the user may input "Ramen". Alternatively, the user may input "1 + 1 gift".

10 is a diagram illustrating a process in which a store or product manufacturer generates information about a product or discount and provides the information to a server according to an embodiment of the present invention.

10 shows that a store manager can create an AR/VR product information and advertisement event exposure zone for his or her store using a store device (which can provide AR platform functions), and can set regular/irregular, time, target customers, etc. Including specified examples.

In addition, advertisers can expose AR/VR product information and advertising events in areas approved by the store manager through devices (PR devices, promotional devices) for their promotions (which can provide AR platform functions). let it be

In order for the server 500 to generate AR/VR content information or to provide AR/VR content information to a specific store/specific area within the store/specific customer, information on the store, product, shelf location (area), and customer is required. .

Store devices owned by store operators or PR devices 700 installed at product manufacturers provide product information, store information, information on a specific area within the store, product information, or customer information necessary to provide AR/VR content information. AR/VR basic information, which is information necessary for generating AR/VR contents such as information, is generated (S41).

This information is transmitted to the server 500 (S42). The server 500 stores AR/VR basic information. Thereafter, when the customer device 100 transmits customer-side generated information to the server 500 as shown in FIG. 9 (S33), AR/VR content information may be provided using the stored AR/VR basic information.

For example, it is assumed that the AR/VR basic information is a 1+1 discount on "choco milk" for stores in Seoul. When a user with a customer device 100 visits a store in Seoul or moves near a milk-fresh food stand in a store in Seoul, the server 500 provides a 1+1 discount for "choco milk". AR/VR content information is transmitted to the customer device 100.

For example, it is assumed that the AR/VR basic information is a discount of "Youth chocolate milk 1+1" for "teenage students" at stores across the country. When a user carrying the customer device 100 visits a store in Seoul, the server 500 receiving the user's age information transmitted by the customer device 100 provides the customer device 100 with youth chocolate milk 1+ AR/VR content information for a 1" discount is transmitted to the customer device 100.

Meanwhile, the server 500 transmits AR/VR content information to a plurality of customer devices 100, collects customer reaction information, and transmits it to the store device/PR device 700.

For example, the reaction information may include data on customer reactions (likes, comments, etc.)/taste for a new product, and a store manager or product manufacturer may use the customer reaction information to establish a new promotion strategy.

Therefore, the AR/VR platform can be used as a product promotion platform and communication platform.

Methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the medium may be those specially designed and configured for the present invention or those known and usable to those skilled in computer software.

As described above, although the present invention has been described by the limited embodiments and drawings, the present invention is not limited to the above embodiments, and those skilled in the art in the field to which the present invention belongs can make various modifications and variations from these descriptions. this is possible It should be understood that the foregoing embodiments are illustrative in all respects and not restrictive, and the scope of the present invention will be indicated by the following claims rather than the foregoing detailed description. And it should be construed that all changes and transformable forms derived from the meaning and scope of the claims, as well as their equivalent concepts, are included in the scope of the present invention.

Therefore, the scope of the present invention should not be limited to the described embodiments and should not be defined, and should be defined by the following claims and equivalents to these claims.

100: customer device 300: guide device
500: server 700: store device/PR device

Claims (21)

Transmitting any one or more of store identification information or location information or condition information or image information of the customer device to a server by a communication unit of the customer device;
receiving, by the communication unit, AR/VR content information corresponding to the identification information of the store or corresponding to the location information or the identification information of the store corresponding to the condition information or the image information from the server; and
A method for providing product or store information based on augmented reality, comprising outputting, by the controller of the customer device, the AR/VR content information to a display unit of the customer device.
According to claim 1,
The image information
The camera of the customer device is a signboard of the store, a visual medium advertising the store, or an image taken around the customer device,
The AR/VR content information is based on augmented reality, including any one or more of product information sold in the store, promotional information of the store, interactive game content provided with a reward, or metaverse content provided by the server. to provide product or store information.
According to claim 1,
After the output step
displaying, by the control unit, an input interface for requesting a reservation purchase of the product or inputting a receiving address or a delivery address of the product on the display unit; and
The method of providing information of a product or store based on augmented reality, comprising the step of transmitting the information input from the input interface to the server by the communication unit.
Transmitting, by a communication unit of a customer device, any one or more of store identification information in a store or in-store location information or condition information or image information of the customer device to a server;
receiving, by the communication unit, AR/VR content information corresponding to the identification information of the shelf, corresponding to the location information, satisfying the condition information, or combinable to the image information from the server; and
A method for providing product or store information based on augmented reality, comprising outputting, by the controller of the customer device, the AR/VR content information to a display unit of the customer device.
According to claim 4,
The image information
An image captured by a camera of the customer device of a shelf of the store or one or more products displayed on the shelf,
The AR / VR content information is based on augmented reality, including any one or more of information on the shelf or the product, promotion information of the store, or interactive game content provided with a reward or metaverse content provided by the server. How to provide product or store information.
According to claim 4,
Prior to the sending step,
Further comprising receiving, by the communication unit, information related to the counter or location information in the store from a guide device installed in the store or the counter, providing product or store information based on augmented reality.
According to any one of claims 1 or 4,
The condition information provides product or store information based on augmented reality, including any one or more of a product or a discount condition of the product, a location of a store that sells the product, or identification information of a user set in the customer device. How to.
receiving customer-side generated information from a customer device by a communication unit of the server;
generating AR/VR content information by the controller of the server using the customer-side generated information; and
A method for providing product or store information based on augmented reality, comprising transmitting the AR/VR content information to a customer device by the communication unit.
According to claim 8,
The customer-side generated information includes at least one of store identification information or location information or condition information or image information of the customer device,
The AR/VR content information is information corresponding to the identification information of the store or information corresponding to the identification information of the store corresponding to the location information or the condition information or the image information, and product or store information based on augmented reality. How to provide.
According to claim 9,
The image information
The customer device is a signboard of the store or a visual medium advertising the store or an image taken around the customer device,
The AR/VR content information is based on augmented reality, including any one or more of product information sold in the store, promotional information of the store, interactive game content provided with a reward, or metaverse content provided by the server. to provide product or store information.
According to claim 9,
The method of providing product or store information based on augmented reality, further comprising receiving, by the communication unit, a reservation purchase request for a product or information on a destination or delivery location of a product from the customer device.
According to claim 8,
The customer-side generated information includes any one or more of shelf identification information in a store, in-store location information of the customer device, condition information, or image information,
The AR/VR content information corresponds to the identification information of the shelf, corresponds to the location information, satisfies the condition information, or information of a product or store based on augmented reality, which is information that can be combined with the image information. How to provide.
According to claim 8,
The image information
An image captured by a camera of the customer device of a shelf of the store or one or more products displayed on the shelf,
The AR / VR content information includes any one or more of the interactive game content provided with the shelf or the product information or the promotional information of the store or the reward provided or the metaverse content provided by the server,
According to any one of claims 9 or 12,
The condition information provides product or store information based on augmented reality, including any one or more of a product or a discount condition of the product, a location of a store that sells the product, or identification information of a user set in the customer device. How to.
According to claim 8,
Prior to the receiving step
Further comprising the step of receiving AR / VR basic information from a store device or a promotion device by the communication unit of the server,
The generating step is a step in which the controller generates AR/VR content information using the AR/VR basic information.
a display unit outputting information and receiving an input corresponding to selection of the information;
a communication unit that transmits customer-side generated information to a server and receives AR/VR content information from the server; and
A customer device including a controller for controlling the display unit to output the AR/VR content information.
According to claim 16,
The customer-side generated information includes any one or more of store identification information or location information of the customer device or identification information of a shelf in the store or in-store location information of the customer device or condition information or image information, the customer device .
According to claim 16,
The image information
The camera of the customer device is a signboard of the store, a visual medium advertising the store, or an image taken around the customer device, or
An image captured by a camera of the customer device of a shelf of the store or one or more products displayed on the shelf,
The AR / VR content information includes any one or more of information on products sold in the store or displayed on the shelf, promotional information of the store, interactive game contents for which rewards are provided, or metaverse contents provided by the server , customer device.
a communication unit that receives customer-side generated information from a customer device and transmits AR/VR content information to the customer device; and
A server comprising a control unit generating the AR/VR content information using the customer-side generated information.
According to claim 19,
If the customer-side generated information is any one or more of store identification information or location information or condition information or image information of the customer device, the control unit corresponds to the store identification information or the location information or the condition information or the image information. Generating the AR / VR content information corresponding to the identification information of the store corresponding to the information;
If the customer-side generated information is any one or more of identification information of the counter in the store, location information or condition information, or image information in the store of the customer device, the control unit may correspond to the identification information of the counter, correspond to the location information, or A server that satisfies the condition information or generates the AR/VR content information that is information that can be combined with the image information.
According to claim 19,
The communication unit of the server receives AR / VR basic information from a store device or a promotion device,
The control unit generates AR / VR content information using the AR / VR basic information.

Images