KR20210014280A - Method for shopping virtual product in offline virtual product display store and system therefore - Google Patents

Abstract

Disclosed are a method for shopping a virtual product in an offline virtual product display store and a system thereof. A method for shopping a virtual product according to an embodiment of the present invention comprises the steps of: recognizing, in real time, each of customers moving in an offline virtual product display store; matching each of the customers with a virtual shopping cart of each of the customers; and providing a first virtual product to a virtual shopping cart of a first customer, when the first virtual product is selected by the first customer from among the virtual products displayed on at least one display means of the offline virtual product display store.

Description

Virtual product shopping method and system at offline virtual product display store {METHOD FOR SHOPPING VIRTUAL PRODUCT IN OFFLINE VIRTUAL PRODUCT DISPLAY STORE AND SYSTEM THEREFORE}

The present invention relates to a technology for shopping virtual products in an offline virtual product display store, and more specifically, recognizes a customer shopping for a virtual product displayed in an offline virtual product display store, and corresponds to a virtual product selected by the customer. It relates to a virtual product shopping method and system that can be provided to a customer's virtual shopping cart.

With the development of the Internet, as an Internet purchasing system for purchasing goods online has been developed and commercialized, it is possible to purchase goods without having to go to an offline shopping space. However, in Internet shopping malls, not only goods delivery accidents frequently occur, but the reliability of the goods purchased is often a problem due to the nature of online. Accordingly, many customers feel the need to use offline shopping, and offline shopping spaces such as large marts are expanding to secure such customers.

In general, offline shopping uses a cart prepared in the store to put the product to be purchased by a customer while browsing the store. At this time, the corresponding large mart installs terminals in various places within the store, advertisements for goods located around the terminal, discount information for each payment card company for each product, discounts implemented by each product manufacturer or large mart. By displaying product information such as event information, customers are encouraged to purchase.

Embodiments of the present invention recognize a customer shopping for a virtual product displayed in an offline virtual product display store, and provide a virtual product selected by the customer to the virtual shopping cart of the customer, and the same System.

A virtual product shopping method according to an embodiment of the present invention includes the steps of real-time recognizing each customer moving in an offline virtual product display store; Matching each of the customers with a virtual shopping cart of each of the customers; And providing the first virtual product to the virtual shopping cart of the first customer when a first virtual product is selected by a first customer among virtual products displayed on at least one display means of the offline virtual product display store. Includes steps.

The matching of the virtual shopping cart includes real-time recognition of the gaze direction of each of the customers through an image photographed by at least one photographing means installed in the offline virtual product display store, and the recognition of the at least one display means. The virtual shopping carts of each of the customers may be displayed on a display means positioned in the gaze direction of each of the customers, corresponding to the positions of the customers.

The matching of the virtual shopping cart may include overlaying at least one virtual shopping cart of each of the customers and at least one of the most products displayed on the corresponding display means on a display means positioned in the gaze direction of each of the customers to display. have.

The matching of the virtual shopping carts includes displaying the virtual shopping carts of each of the customers with a preset first transparency on a display means positioned in the gaze direction of each of the customers, and a call command for the virtual shopping cart of the second customer When inputted, the virtual shopping cart of the second customer may be displayed with a preset second transparency lower than the first transparency.

The matching of the virtual shopping carts may include overlaying and displaying identification information for identifying each of the customers on the virtual shopping carts of each of the customers.

In the matching of the virtual shopping carts, the virtual shopping carts of each of the customers may be displayed on a separate display means for displaying the virtual shopping carts corresponding to the locations of the customers.

In the providing of the virtual shopping cart, when the first virtual product is selected by the first customer and a preset gesture is recognized through an image captured by at least one photographing means installed in the offline virtual product display store The first virtual product may be provided as a virtual shopping cart of the first customer.

Recognizing each of the customers in real time may include: receiving customer information including personal bio information of each of the customers; Allocating customer identification information of each of the customers based on the received customer information; And recognizing each of the customers moving in the offline virtual display store in real time based on the assigned customer identification information and an image captured by at least one photographing means installed in the offline virtual product display store. I can.

In the providing of the virtual shopping cart, when the actual sample product displayed in the offline virtual product display store is selected by the first customer, a virtual product set in advance corresponding to the actual sample product is provided to the first customer. Can be provided by cart.

A virtual product shopping system according to an embodiment of the present invention includes: a recognition unit for real-time recognition of each customer moving in an offline virtual product display store; A matching unit for matching each of the customers and each of the virtual shopping carts; And providing the first virtual product to the virtual shopping cart of the first customer when a first virtual product is selected by a first customer among virtual products displayed on at least one display means of the offline virtual product display store. It includes a control unit.

The recognition unit recognizes the gaze direction of each of the customers in real time through an image captured by at least one or more photographing means installed in the offline virtual product display store, and the matching unit recognizes each of the recognized customers among the at least one display means. The virtual shopping carts of each of the customers may be displayed on a display means positioned in the gaze direction of, corresponding to the respective locations of the customers.

The matching unit may display by overlaying at least one virtual shopping cart of each of the customers and at least one most products displayed on the corresponding display means on a display means positioned in the gaze direction of each of the customers.

The matching unit displays the virtual shopping carts of each of the customers with a preset first transparency on display means positioned in the gaze direction of each of the customers, and when a call command for the virtual shopping cart of a second customer is input, the second The customer's virtual shopping cart may be displayed with a second preset transparency lower than the first transparency.

The matching unit may overlay and display identification information for identifying each of the customers on the virtual shopping cart of each of the customers.

The matching unit may display the virtual shopping carts of each of the customers on a separate display means for displaying the virtual shopping carts corresponding to the locations of the customers.

When the first virtual product is selected by the first customer and a preset gesture is recognized through an image captured by at least one or more photographing means installed in the offline virtual product display store, the control unit selects the first virtual product. It may be provided as a virtual shopping cart of the first customer.

The recognition unit receives customer information including personal bio information of each of the customers, allocates customer identification information of each of the customers based on the received customer information, and displays the assigned customer identification information and the offline virtual product Each of the customers moving in the offline virtual display store may be recognized in real time based on images captured by at least one photographing means installed in the store.

According to embodiments of the present invention, a customer shopping for a virtual product displayed in an offline virtual product display store may be recognized, and a virtual product selected by the customer may be provided to the virtual shopping cart of the customer.

According to embodiments of the present invention, when virtual products entered into a customer's virtual shopping cart are paid by a corresponding customer, a real product corresponding to each of the corresponding virtual products may be automatically delivered to a delivery destination set by the customer.

According to embodiments of the present invention, since virtual products are put in the virtual shopping cart, there is no need to drag the actual shopping cart, and thus, the complexity in the store can be reduced.

1 shows an exemplary diagram for explaining the present invention.
2 is a flowchart illustrating an operation of a virtual product shopping method according to an embodiment of the present invention.
3 is a flowchart illustrating an operation according to an embodiment of step S210 shown in FIG. 2.
4 shows the configuration of a virtual product shopping system according to an embodiment of the present invention.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in a variety of different forms, only these embodiments are intended to complete the disclosure of the present invention, and common knowledge in the technical field to which the present invention pertains. It is provided to completely inform the scope of the invention to those who have, and the invention is only defined by the scope of the claims.

The terms used in this specification are for describing exemplary embodiments, and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used herein, "comprises" and/or "comprising" refers to the recited component, step, operation, and/or element being one or more of the other elements, steps, operations and/or elements. It does not exclude presence or addition.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in advance that are generally used are not interpreted ideally or excessively unless explicitly defined specifically.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. The same reference numerals are used for the same elements in the drawings, and duplicate descriptions for the same elements are omitted.

Embodiments of the present invention recognize a customer shopping for a virtual product displayed in an offline virtual product display store in which virtual products are displayed on a display means, and provide a virtual product selected by the customer to the virtual shopping cart of the customer. The point is to do it.

In this case, according to the present invention, when virtual products entered into the customer's virtual shopping cart are paid by the customer, the actual products corresponding to each of the virtual products may be automatically delivered to a delivery destination set by the customer.

Further, according to the present invention, a virtual shopping cart of a corresponding customer may be displayed on a display means positioned in the gaze direction by recognizing the gaze direction of the customer, and may be displayed by overlaying virtual products and a virtual shopping cart.

In this case, the present invention prevents the virtual products from being covered by the virtual shopping cart by increasing the transparency of the virtual shopping cart and overlaying the virtual shopping cart, and when attempting to put the virtual product in the virtual shopping cart, the transparency of the virtual shopping cart It can also be controlled to lower. For example, the present invention controls a virtual shopping cart with a preset first transparency having high transparency, and when a virtual product is selected by a first customer, the virtual shopping cart of the first customer is set with a preset second transparency having low transparency. By controlling to, it is possible to help the first customer to put the virtual product in his or her virtual shopping cart. Of course, such transparency control may be set by a business operator or an individual who provides the present invention, and is not limited to displaying a virtual shopping cart by overlaying it with virtual products and adjusting the transparency, and basically does not display a virtual shopping cart. When a command for displaying a virtual shopping cart is input by a customer in a state in which the virtual shopping cart is not displayed, the user may control to display the virtual shopping cart of the customer.

Furthermore, the present invention may display a virtual shopping cart of each customer on a separately provided display means for displaying a virtual shopping cart.

Here, the present invention provides identification information for identifying each customer so that each customer can check their virtual shopping cart when displaying a virtual shopping cart, for example, customer name, customer nickname, customer face information, and set by the customer. An avatar or the like may be overlaid on a virtual shopping cart and displayed.

Furthermore, in the present invention, after the first virtual product is selected by the first customer, when a preset gesture is recognized through an image captured by a photographing means installed in the offline virtual product display store, the first virtual product is transferred to the first customer. It can be offered or packed in a virtual shopping cart.

Further, the present invention allocates customer identification information to each of the customers entering the offline virtual product display store, and photographs the assigned customer identification information and shooting means installed in the offline virtual product display store, for example, by a camera or CCTV. Based on the image, it is possible to recognize each of the customers moving in the offline virtual product display store.

At this time, in the present invention, in recognizing each of customers moving in an offline virtual product display store, bio information of each customer, for example, fingerprint information, iris information, body information, face information, as well as clothes color information, etc. It may be used, and may reflect device identification information transmitted from a wireless identification device, for example, RFID, which is mapped to customer identification information paid at an offline virtual product display store when a customer enters.

In this case, the present invention may recognize in real time a customer moving in an offline virtual display store based on image information of a customer captured by a photographing means mapped with customer identification information and a deep learning-based learning model. Here, the method of mapping the customer identification information and the customer's image information is by taking an image of the customer when entering the offline virtual product display store and assigning the customer identification information, and the customer identification information and the customer's face information or body information, or You can map clothes color information, etc. Of course, the method of mapping the customer identification information and the customer's image information is not limited thereto, and any possible method may be applied.

The present invention will be described with reference to FIGS. 1 to 4 as follows.

1 shows an exemplary diagram for explaining the present invention.

As shown in FIG. 1, the system includes at least one photographing means 100 installed in an offline virtual product display store and a server 200 for recognizing each of customers moving in the offline virtual product display store.

Here, the offline virtual product display store is a store that displays virtual products corresponding to real products on the screen of the display means by using a display means instead of a display case that displays products.If necessary, a real product, for example, a real sample product is displayed. You can also display it.

Although not shown in FIG. 1, the offline virtual product display store may further include an input means, for example, a store terminal, for receiving information of a customer entering the corresponding store, and customer information, for example, through the store terminal , By receiving the customer's phone number, fingerprint information, iris information, and face information, it is possible to allocate customer identification information from the customer based on the customer information through communication with the server.

At least one or more photographing means 100 may capture an image in an offline virtual product display store in real time and provide it to the server 200, and the provided image is processed by the server 200 in an offline virtual product display store. You can recognize each of the customers moving in.

That is, the server 200 includes customer information entering the offline virtual product display store, for example, the customer's phone number and/or fingerprint information, iris information, face information, body information (eg, palm information, key information, etc.). Including) and the like, and assigns customer identification information to the customer based on the received customer information.

Here, the server 200 may map and store the customer identification information and the device identification information of the wireless identification device when a wireless identification device corresponding to the customer identification information, for example, RFID is paid to the customer, and the customer When the information includes the customer's face information and clothes color information, the customer identification information and the customer's face information and/or clothes color information may be mapped and stored. In this case, the clothes color information may be divided into information on the color of the clothes on the top and information on the color of the clothes on the bottom.

Further, when entering the offline virtual product display store, the server 200 may allocate temporary customer identification information that can be allocated and used in the corresponding offline virtual product display store, as described above, and each of the customers can determine the method in advance. By accessing the provided service server and registering customer information, fixed customer identification information for the customer information may be allocated. In this case, the customer may pre-register payment information for performing payment for the purchased virtual products. Of course, the fixed customer identification information may be used equally in all offline virtual product display stores, that is, each customer uses an application that provides the method of the present invention, for example, a smart phone app. , By registering personal information such as customer name, customer phone number, fingerprint information, body information, iris information, and face information in the server 200, it responds to the customer's personal information based on the personal information registered in the server 200 Fixed customer identification information is allocated, and the allocated customer identification information can be mapped to the customer's personal information and stored in a database.

That is, when the server 200 receives customer information entering the corresponding store from the offline virtual product display store, it searches for the customer information from the database, and if matching customer information exists, the customer identification information previously allocated for the customer information is retrieved. It can be assigned to customers entering the store.

Furthermore, the server 200 includes customer identification information and an image captured by at least one photographing means 100 installed in an offline virtual product display store, for example, among customer's face information, clothes color information, and body information in the image. Each of the customers moving in the offline virtual product display store may be recognized on the basis of at least one.

At this time, the server 200 maps the customer identification information in the offline virtual product display store to the corresponding customer identification information and reflects the device identification information received from the wireless identification device when the wireless identification device is paid to the customer moving in the offline virtual display store. It can also be recognized in real time.

Furthermore, the server 200 may recognize in real time customers moving in an offline virtual display store based on a deep learning-based learning model. For example, the server 200 may recognize in real time a customer moving in an offline virtual display store based on image information of a customer captured by at least one or more photographing means mapped with customer identification information and a deep learning-based learning model. I can. That is, the server 200 accurately recognizes the image information of the customer, for example, face information, clothes color information, etc., mapped with the customer identification information using a deep learning-based learning model, thereby moving in the offline virtual display store. Each customer can be clearly identified and recognized.

Furthermore, the server 200 may display virtual products on display means provided in an offline virtual product display store, and display virtual shopping carts of each customer on the display means. In this case, the server 200 may display the customer's virtual shopping cart on the display unit in response to the customer's current location based on the customer recognition result. In one embodiment, the server 200 may display the virtual shopping cart at a location corresponding to each location of customers on a display means separately provided to display the virtual shopping cart. In another embodiment, the server 200 recognizes the customer's gaze direction through an image photographed by the photographing means, and places the virtual products and the customer's virtual shopping cart on the display means located in the customer's gaze direction. It can be displayed by overlaying it on a location corresponding to the location of. In this case, the server 200 may adjust the transparency of the virtual shopping cart displayed by overlaying the virtual products. For example, the server 200 controls the virtual shopping cart with a preset first transparency having high transparency, and when a virtual product is selected by the first customer, the virtual shopping cart of the first customer has a second preset transparency with low transparency. Can be controlled by Of course, the method of displaying a virtual shopping cart by overlaying with virtual products may be displayed only in a specific area, or may be displayed in a specific arrangement method. This arrangement method may be performed by a business operator or an individual providing the technology of the present invention. Can be determined.

Here, when the server 200 displays the virtual shopping cart, identification information for identifying each customer, for example, customer name, customer nickname, customer face information, and customer information so that each customer can check their virtual shopping cart. The avatar set by this may be displayed by overlaying it on the virtual shopping cart. Depending on the situation, a list of virtual products contained in the virtual shopping cart or corresponding images of each of the virtual products may be displayed together.

Furthermore, the server 200 may provide different colors, shapes, and the like for each customer when displaying the virtual shopping cart so that it is easy to check the user's virtual shopping cart.

The server 200 processes an image captured by the photographing means 100 to clearly recognize each of the customers included in the image, or by clearly recognizing a gesture for selecting a virtual product and putting it in a virtual shopping cart, Virtual products can be easily shopped in offline virtual product display stores.

Depending on the situation, the present invention may display their own virtual shopping cart on their own smartphone through augmented reality (AR) using an application pre-installed on each terminal of the customer, for example, a smartphone. Through augmented reality using a smart phone, virtual products that the customer wants to purchase can be easily checked using their smart phone, and purchases and payments can also be made quickly.

Of course, the server 200 removes information on the virtual product from the virtual shopping cart based on the information on the canceled virtual product if it is canceled at the request of the customer among the virtual products in the virtual shopping cart by each customer. May be. Furthermore, returns may also be made through an application for the service, and various existing methods related to returns may be applied.

If necessary, the present invention displays real products for not only virtual products but also some products, and when a real product is selected through image processing, information or images on the real product may be put in a virtual shopping cart. For example, the present invention displays an actual sample product in an offline virtual product display store, and when a predetermined gesture or motion of putting the actual sample product in a virtual shopping cart by a customer is recognized or detected by the photographing means, the actual sample A virtual product set in advance corresponding to a product or an actual sample product may be put in the customer's virtual shopping cart. Here, the pre-promised gesture may include a gesture in which the customer picks up and releases the actual sample product, or a gesture to throw, and the like, and the promised gesture may be determined by a business operator or an individual providing the technology of the present invention.

2 is a flowchart illustrating an operation of a virtual product shopping method according to an embodiment of the present invention.

Referring to FIG. 2, in the virtual product shopping method according to an embodiment of the present invention, each customer moving in an offline virtual product display store is recognized in real time (S210).

Here, step S210, as shown in FIG. 3, customer information including personal bio information of a customer entering an offline virtual product display store, for example, customer name, customer phone number, customer face information, fingerprint information, Customer information including at least one of the iris information is received (S310).

Here, step S310 may be received through a store terminal provided in an offline virtual product display store, and the received customer information may be transmitted to the server.

When customer information is received in step S310, customer identification information is assigned to the customer based on the received customer information (S320).

Here, step S320 may allocate temporary customer identification information that can be allocated and used in the offline virtual product display store based on the customer information received when entering the offline virtual product display store, and each of the customers selects the corresponding method in advance. By accessing the provided service server and registering customer information, fixed customer identification information for the customer information may be allocated. Of course, the fixed customer identification information may be used equally in all offline virtual product display stores, that is, each customer uses an application that provides the method of the present invention, for example, a smart phone app. Fixed customer identification information corresponding to the customer's personal information based on the personal information registered in the server by registering personal information such as customer name, customer phone number, fingerprint information, body information, iris information, and face information on the server Is assigned, and the assigned customer identification information can be mapped to the customer's personal information and stored in the database.

That is, in step S320, when customer information entering the corresponding store is received from the offline virtual product display store, the corresponding customer information is searched from the database, and if matching customer information exists, the customer identification information previously allocated for the corresponding customer information is stored in the corresponding store Can be assigned to customers who enter

If the customer identification information is assigned to the customer entering the offline virtual product display store in step S320, the assigned customer identification information and an image taken by at least one or more photographing means installed in the offline virtual product display store, for example, in an image Each of the customers moving in the offline virtual product display store is recognized based on at least one of the customer's face information, clothes color information, and body information (S330).

Here, in step S330, when a wireless identification device is provided by mapping to the customer identification information in the offline virtual product display store, the customer moving in the offline virtual display shop is recognized in real time by reflecting the device identification information received from the wireless identification device. You may.

Furthermore, in step S330, a customer moving in an offline virtual display store may be recognized in real time based on image information of the customer captured by at least one or more photographing means mapped with the customer identification information and a deep learning-based learning model. . That is, step S230 accurately recognizes customer image information, such as face information and clothes color information, mapped with customer identification information using a deep learning-based learning model, so that each customer moving in an offline virtual display store Can be recognized clearly.

Referring to FIG. 2 again, if each of the customers moving in the offline virtual product display store is recognized in real time in step S210, each of the customers and each of the virtual shopping carts are matched (S220).

Here, in step S220, virtual products may be displayed on display means provided in the offline virtual product display store, and virtual shopping carts of each customer may also be displayed on the display means.

Further, in step S220, the virtual shopping cart of the customer may be displayed on the display unit in response to the current location of the customer based on the customer recognition result. In this case, step S220 may display the virtual shopping cart at a location corresponding to each of the customers on a display means separately provided to display the virtual shopping cart, and the customer's gaze direction through an image captured by the photographing means. Recognizing and displaying the virtual products and the virtual shopping cart of the corresponding customer on the display means located in the direction of the customer's gaze by overlaying them on a location corresponding to the location of the customer. Depending on the situation, step S220 may adjust the transparency of the virtual shopping cart that is displayed overlaid with the virtual products. For example, in step S220, the virtual shopping cart is controlled with a preset first transparency having high transparency, and when a virtual product is selected by the first customer, the virtual shopping cart of the first customer is changed to a preset second transparency having low transparency. Can be controlled by

Further, in step S220, when displaying the virtual shopping cart, identification information for identifying each of the customers, for example, customer name, customer nickname, customer face information, by the customer, so that each customer can check their virtual shopping cart. A set avatar or the like may be displayed by overlaying the virtual shopping cart, or different colors and shapes may be provided for each customer when the virtual shopping cart is displayed.

In the virtual shopping cart matched with each of the customers in step S220, the movement of each of the customers is recognized in real time, and the virtual shopping cart of each of the customers may also be moved and displayed according to the positions of the customers.

In step S220, a virtual shopping cart matched with each of the customers is displayed on the display means, and any of the virtual products displayed on at least one or more display means of the offline virtual product display store, for example, the first virtual product When the customer, for example, is selected by the first customer, the first virtual product is provided or stored in the virtual shopping cart of the first customer (S230).

At this time, step S230 recognizes that the first customer has selected the first virtual product, and after the first virtual product is selected by the first customer, a gesture preset by the first customer, for example, a throwing gesture, is applied to the photographing means. When recognized through an image captured by the user, the first virtual product may be provided or stored in a virtual shopping cart of the first customer. Of course, step S230 is not limited to the above description, and by using a selection button for selecting a virtual product, the first virtual product selected by the first customer may be put in the virtual shopping cart of the first customer.

Through this process, after each customer's virtual shopping cart contains virtual products, when the customer performs payment for the virtual products in the virtual shopping cart, automatic payment is performed by the customer's payment card registered in advance on the server. Alternatively, the customer may directly perform payment through a payment terminal installed in an offline virtual product display store. When virtual products contained in a virtual shopping cart are paid by a customer, a process of delivering real products corresponding to the virtual products to a product destination set by the customer may be automatically performed.

As described above, the virtual product shopping method according to an embodiment of the present invention recognizes a customer shopping for a virtual product displayed in an offline virtual product display store, and provides the virtual product selected by the customer to the virtual shopping cart of the customer. I can.

In addition, in the virtual product shopping method according to an embodiment of the present invention, when the virtual products in the customer's virtual shopping cart are paid by the customer, the actual products corresponding to each of the virtual products are automatically delivered to the delivery destination set by the customer. Can be.

In addition, in the virtual product shopping method according to an embodiment of the present invention, since virtual products are placed in the virtual shopping cart, there is no need to drag the actual shopping cart, and thus, complexity in the store can be reduced.

4 illustrates the configuration of a virtual product shopping system according to an embodiment of the present invention, and illustrates the configuration of the server shown in FIG. 1. Of course, the system of the present invention is not limited to a server, and may be composed of at least one or more devices or systems capable of performing the present invention.

Referring to FIG. 4, a system 400 according to an embodiment of the present invention includes a recognition unit 410, a matching unit 420, a control unit 430, and a database (DB) 440.

The DB 440 stores all kinds of data related to the present invention. For example, the DB 440 includes customer information of each customer, customer identification information mapped with customer information, device identification information mapped with customer identification information, information on offline virtual product display stores, customer payment cart information, and purchase. All kinds of information for servicing the present invention, such as history information, delivery address information, and the like may be stored.

The recognition unit 410 recognizes in real time each of the customers moving in the offline virtual product display store.

At this time, the recognition unit 410 includes at least one of customer information including personal bio information of a customer entering an offline virtual product display store, for example, customer name, customer phone number, customer face information, fingerprint information, and iris information. Receives customer information including one, assigns customer identification information to the customer based on the received customer information, and the assigned customer identification information and an image taken by at least one shooting means installed in the offline virtual product display store For example, based on at least one of the customer's face information, clothes color information, and body information in the image, each of the customers moving in the offline virtual product display store may be recognized.

Further, the recognition unit 410 maps to the corresponding customer identification information in the offline virtual product display store, and reflects the device identification information received from the wireless identification device when a wireless identification device is provided to detect customers moving in the offline virtual display shop. It can be recognized in real time, and based on the customer's image information captured by at least one or more shooting means mapped with customer identification information and a deep learning-based learning model, customers moving in the offline virtual display store can be recognized in real time. I can.

Furthermore, the recognition unit 410 may recognize the direction of each customer's gaze in real time through an image photographed by a photographing means installed in an offline virtual product display store, and recognize a gesture of a customer who selects a virtual product in real time. You may.

When each of the customers moving in the offline virtual product display store is recognized by the recognition unit 410 in real time, the matching unit 420 matches each of the customers with a virtual shopping cart of each of the customers.

In this case, the matching unit 420 may display virtual products on display means provided in the offline virtual product display store, and display virtual shopping carts of each customer on the display means.

At this time, the matching unit 420 may display the customer's virtual shopping cart on the display means in response to the customer's current location based on the customer recognition result, and the customer can display the virtual shopping cart on the display means separately provided to display the virtual shopping cart. A virtual shopping cart may be displayed at a location corresponding to each location, and the customer's gaze direction is recognized through the image captured by the photographing means, and the virtual products and the corresponding customer are displayed on the display means located in the customer's gaze direction. The virtual shopping cart can be displayed by overlaying it on a location corresponding to the location of the customer.

Furthermore, the matching unit 420 may adjust the transparency of the virtual shopping cart displayed by being overlaid with the virtual products, and identify each customer so that each customer can check their virtual shopping cart when displaying the virtual shopping cart. For example, the customer's name, customer nickname, customer face information, and avatar set by the customer may be displayed overlaid on the virtual shopping cart. When displaying the virtual shopping cart, the color and shape are different for each customer. It can also be provided.

The controller 430 displays a virtual shopping cart matched with each of the customers on the display means, and any virtual product, for example, a first virtual product, among virtual products displayed on at least one or more display means of the offline virtual product display store When selected by a certain customer, for example, a first customer, a first virtual product is provided or contained in a virtual shopping cart of the first customer.

At this time, the controller 430 recognizes that the first customer has selected the first virtual product, and after the first virtual product is selected by the first customer, a gesture preset by the first customer, for example, a throwing gesture, is photographed. When recognized through an image captured by means, the first virtual product may be provided or put in a virtual shopping cart of the first customer.

Although the description thereof is omitted in the system of FIG. 4, each component constituting FIG. 4 may include all the contents described in FIGS. 1 to 3, which is obvious to those skilled in the art.

The apparatus described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It can be implemented using one or more general purpose computers or special purpose computers, such as a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to behave as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. Can be embodyed. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described by the limited embodiments and drawings, various modifications and variations are possible from the above description by those of ordinary skill in the art. For example, the described techniques are performed in a different order from the described method, and/or components such as a system, structure, device, circuit, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and claims and equivalents fall within the scope of the claims to be described later.

Claims (17)

Recognizing in real time each of the customers moving in the offline virtual product display store;
Matching each of the customers with a virtual shopping cart of each of the customers; And
Providing the first virtual product to the virtual shopping cart of the first customer when a first virtual product is selected by a first customer among virtual products displayed on at least one display means of the offline virtual product display store
Virtual product shopping method comprising a.
The method of claim 1,
The step of matching the virtual shopping cart
Recognizing the gaze direction of each of the customers in real time through an image photographed by at least one photographing means installed in the offline virtual product display store, and located in the gaze direction of each of the recognized customers among the at least one display means And displaying a virtual shopping cart of each of the customers on a display means corresponding to the location of each of the customers.
The method of claim 2,
The step of matching the virtual shopping cart
The virtual product shopping method, characterized in that for displaying by overlaying at least one virtual shopping cart of each of the customers and at least one most products displayed on the corresponding display means on a display means positioned in the gaze direction of each of the customers.
The method of claim 2,
The step of matching the virtual shopping cart
When a virtual shopping cart of each of the customers is displayed in a predetermined first transparency on a display means positioned in the gaze direction of each of the customers, and a call command for a virtual shopping cart of a second customer is input, the virtual shopping cart of the second customer Displaying a shopping cart with a second preset transparency lower than the first transparency.
The method of claim 2,
The step of matching the virtual shopping cart
And displaying identification information for identifying each of the customers overlaid on the virtual shopping cart of each of the customers.
The method of claim 1,
The step of matching the virtual shopping cart
And displaying the virtual shopping carts of each of the customers on a separate display means for displaying the virtual shopping carts corresponding to the locations of the customers.
The method of claim 1,
The step of providing the virtual shopping cart
When the first virtual product is selected by the first customer and a preset gesture is recognized through an image captured by at least one photographing means installed in the offline virtual product display store, the first virtual product is displayed as the first Virtual product shopping method, characterized in that providing a customer's virtual shopping cart.
The method of claim 1,
Recognizing each of the customers in real time
Receiving customer information including personal bio information of each of the customers;
Allocating customer identification information of each of the customers based on the received customer information; And
Recognizing each of the customers moving in the offline virtual display store in real time based on the assigned customer identification information and an image captured by at least one photographing means installed in the offline virtual product display store
Virtual product shopping method comprising a.
The method of claim 1,
The step of providing the virtual shopping cart
When the actual sample product displayed in the offline virtual product display store is selected by the first customer, a virtual product set in advance corresponding to the actual sample product is provided to the virtual shopping cart of the first customer. How to shop for products.
A recognition unit for real-time recognition of each customer moving in the offline virtual product display store;
A matching unit for matching each of the customers and each of the virtual shopping carts; And
When a first virtual product is selected by a first customer among virtual products displayed on at least one display means of the offline virtual product display store, a control unit that provides the first virtual product to the virtual shopping cart of the first customer
Virtual product shopping system comprising a.
The method of claim 10,
The recognition unit
Recognizing the gaze direction of each of the customers in real time through an image captured by at least one or more photographing means installed in the offline virtual product display store,
The matching unit
And displaying a virtual shopping cart of each of the customers on a display means positioned in the gaze direction of each of the recognized customers among the at least one display means, corresponding to the locations of the customers.
The method of claim 11,
The matching unit
A virtual product shopping system, characterized in that for displaying by overlaying at least one virtual shopping cart of each of the customers and at least one most products displayed on the corresponding display means on a display means positioned in the gaze direction of each of the customers.
The method of claim 11,
The matching unit
When a virtual shopping cart of each of the customers is displayed in a predetermined first transparency on a display means positioned in the gaze direction of each of the customers, and a call command for a virtual shopping cart of a second customer is input, the virtual shopping cart of the second customer A virtual product shopping system, characterized in that displaying a shopping cart with a second preset transparency lower than the first transparency.
The method of claim 11,
The matching unit
And displaying identification information for identifying each of the customers overlaid on the virtual shopping cart of each of the customers.
The method of claim 10,
The matching unit
And displaying the virtual shopping carts of each of the customers in a separate display means for displaying the virtual shopping carts corresponding to the locations of the customers.
The method of claim 10,
The control unit
When the first virtual product is selected by the first customer and a preset gesture is recognized through an image captured by at least one photographing means installed in the offline virtual product display store, the first virtual product is displayed as the first Virtual product shopping system, characterized in that provided to the customer's virtual shopping cart.
The method of claim 10,
The recognition unit
Receive customer information including personal bio information of each of the customers, allocate customer identification information of each of the customers based on the received customer information, and install the assigned customer identification information and the offline virtual product display store A virtual product shopping system, characterized in that it recognizes in real time each of the customers moving within the offline virtual display store based on images captured by at least one photographing means.

Images