KR102261518B1 - Custumer recognition method for tracking the custumer in offline virtual product display store and system therefore - Google Patents

Abstract

Disclosed are a customer recognition method and system for customer tracking in an offline virtual product display store. A customer recognition method according to an embodiment of the present invention includes: receiving customer information including personal bio information of a customer entering an offline virtual product display store; allocating customer identification information to the customer based on the received customer information; and recognizing the customer moving in the offline virtual display store in real time based on the assigned customer identification information and an image captured by at least one photographing means installed in the offline virtual product display store.

Description

CUSTUMER RECOGNITION METHOD FOR TRACKING THE CUSTUMER IN OFFLINE VIRTUAL PRODUCT DISPLAY STORE AND SYSTEM THEREFORE

The present invention relates to a technology for recognizing a customer in an offline virtual product display store, and more particularly, to accurately recognize each customer in order to track each customer shopping for a virtual product displayed in an offline virtual product display store. It relates to a customer recognition method and a system for the same.

With the development of the Internet, as the Internet purchasing system for purchasing items online has been developed and commercialized, it has become possible to purchase items without having to go to an offline shopping space. However, in Internet shopping malls, not only do shipping accidents occur frequently, but also, the reliability of purchased products is often a problem due to the nature of the online shopping mall. Accordingly, the number of customers who feel the need to use offline shopping has increased, and offline shopping spaces such as large marts are being expanded to secure such customers.

In general, offline shopping uses a cart prepared in a store to put items that a customer wants to purchase while looking around the store. At this time, the large mart installs the terminal in various places within the store, and advertises the products located in the vicinity of the terminal, discount information by payment card company for each product, and discounts implemented by each product manufacturer or large mart By displaying product information such as event information, it also inspires customers to purchase.

Embodiments of the present invention provide a customer recognition method and system capable of accurately recognizing each customer in order to track each customer shopping for a virtual product displayed in an offline virtual product display store.

A customer recognition method according to an embodiment of the present invention includes: receiving customer information including personal bio information of a customer entering an offline virtual product display store; allocating customer identification information to the customer based on the received customer information; and recognizing the customer moving in the offline virtual display store in real time based on the assigned customer identification information and an image captured by at least one photographing means installed in the offline virtual product display store.

In the recognizing step, the customer moving in the offline virtual display store may be recognized in real time by reflecting device identification information that is mapped to the customer identification information and transmitted from the wireless identification device provided to the customer.

The step of recognizing the customer moving in the offline virtual display store based on at least one of the customer identification information and the customer's face information, body information, and clothes color information photographed by the at least one or more photographing means. can be recognized in real time.

In the recognizing step, the customer moving in the offline virtual display store can be recognized in real time based on the customer's image information captured by the photographing means mapped with the customer identification information and a deep learning-based learning model. can

Furthermore, a customer recognition method according to an embodiment of the present invention includes: registering customer information including personal bio-information of each customer in a database; generating customer identification information of each of the customers based on the registered customer information; and mapping the registered customer information and the generated customer identification information and storing the mapping in the database, wherein the allocating includes comparing the received customer information with the registered customer information and matching customer information In the case in which is present, customer identification information mapped to the received customer information and stored may be allocated to the customer.

The bio-information may include at least one of iris information, fingerprint information, and face information of the customer.

A customer recognition system according to an embodiment of the present invention includes: a receiving unit for receiving customer information including personal bio information of a customer entering an offline virtual product display store; an allocator for allocating customer identification information to the customer based on the received customer information; and a recognition unit configured to recognize the customer moving in the offline virtual display store in real time based on the assigned customer identification information and an image captured by at least one photographing means installed in the offline virtual product display store.

The recognition unit may recognize the customer moving in the offline virtual display store in real time by reflecting device identification information that is mapped to the customer identification information and transmitted from the wireless identification device provided to the customer.

The recognition unit may recognize the customer moving in the offline virtual display store in real time based on at least one of the customer identification information and the customer's face information, body information, and clothes color information photographed by the photographing means have.

The recognition unit may recognize the customer moving in the offline virtual display store in real time based on the customer's image information captured by the photographing means mapped with the customer identification information and a deep learning-based learning model .

Furthermore, the customer recognition system according to an embodiment of the present invention registers customer information including personal bio information of each customer in a database, and generates customer identification information of each customer based on the registered customer information, Further comprising a control unit for mapping the registered customer information and the generated customer identification information to store in the database, wherein the allocating unit compares the received customer information with the registered customer information and there is matching customer information In this case, the customer identification information mapped to the received customer information and stored may be allocated to the customer.

The bio-information may include at least one of iris information, fingerprint information, and face information of the customer.

According to embodiments of the present invention, each of the customers can be accurately recognized in real time in order to track each of the customers who shop for a virtual product displayed in an offline virtual product display store.

According to embodiments of the present invention, by accurately recognizing each of the customers entering the offline virtual product display store, when each customer selects products displayed in the offline virtual product display store and puts them in the virtual cart, the corresponding customer's virtual card is stored. You can put exactly the virtual product that the customer has chosen.

1 shows an exemplary view for explaining the present invention.
2 is a flowchart illustrating a customer recognition method according to an embodiment of the present invention.
3 shows the configuration of a customer recognition system according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in a variety of different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments, and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” refers to a referenced component, step, operation and/or element of one or more other components, steps, operations and/or elements. The presence or addition is not excluded.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in the meaning commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. The same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

Embodiments of the present invention make it a gist of accurately recognizing each of the customers in real time in order to track each of the customers shopping for the displayed virtual product in an offline virtual product display store in which the virtual products are displayed on the display.

Here, the present invention allocates customer identification information to each customer entering the offline virtual product display store, and records the assigned customer identification information and a photographing means installed in the offline virtual product display store, for example, a camera or CCTV. Based on the image, each of the customers moving in the offline virtual product display store may be recognized.

Further, in the present invention, in recognizing each customer moving in the offline virtual product display store, each customer's bio information, for example, fingerprint information, iris information, body information, face information, as well as clothing color information, etc. may be additionally used. Alternatively, the wireless identification device mapped to customer identification information paid in the offline virtual product display store when the customer enters, for example, device identification information transmitted from RFID may be reflected.

Furthermore, the present invention may recognize a customer moving in an offline virtual display store in real time based on a deep learning-based learning model and image information of the customer captured by the photographing means mapped with the customer identification information. Here, as a method of mapping customer identification information and customer image information, when entering an offline virtual product display store and assigning customer identification information, the customer identification information and the customer's face information or body information or Clothing color information and the like can be mapped. Of course, a method of mapping customer identification information and customer image information is not limited thereto, and all possible methods may be applied.

The present invention will be described with reference to FIGS. 1 to 3 as follows.

1 shows an exemplary view for explaining the present invention.

As shown in FIG. 1 , the system includes at least one photographing means 100 installed in the offline virtual product display store and a server 200 for recognizing each of customers moving in the offline virtual product display store.

Here, the offline virtual product display store is a store that displays a virtual product corresponding to a real product on the screen of the display unit using a display unit instead of a display case where the product is displayed, and the actual product may be displayed if necessary.

Although not shown in FIG. 1 , the offline virtual product display store may further include an input means for receiving customer information entering the store, for example, a store terminal, and may include customer information through the store terminal, for example. , by receiving the customer's phone number, fingerprint information, iris information, face information, and the like, it is possible to allocate customer identification information to the customer based on the customer information through communication with the server.

At least one photographing means 100 may capture an image in the offline virtual product display store in real time and provide it to the server 200, and the provided image is processed in the server 200 through image processing in the offline virtual product display store. can recognize each of the customers moving from

That is, the server 200 provides customer information entering the offline virtual product display store, for example, the customer's phone number and/or fingerprint information, iris information, face information, and body information (eg, palm information, key information, etc.). ) and the like, and allocates customer identification information to the customer based on the received customer information.

Here, the server 200 may map and store the corresponding customer identification information and the device identification information of the wireless identification device when the wireless identification device corresponding to the corresponding customer identification information, for example, RFID is paid to the customer, and stores the customer identification information. When the information includes the customer's face information and clothes color information, the customer identification information may be mapped and stored with the customer's face information and/or clothes color information. In this case, the clothes color information may be divided into upper clothes color information and lower clothes color information.

Furthermore, when entering the offline virtual product display store, the server 200 may allocate temporary customer identification information that can be allocated and used in the corresponding offline virtual product display store as described above, and each customer can use the corresponding method in advance. By accessing the provided service server and registering customer information, fixed customer identification information for the corresponding customer information may be assigned. Of course, the static customer identification information may be equally used in all offline virtual product display stores, that is, each customer can use an application eg a smart phone app to provide the method of the present invention, eg customer information eg , by registering personal information, such as customer name, customer phone number, fingerprint information, body information, iris information, and face information, in the server 200, responding to the customer's personal information based on the personal information registered in the server 200 fixed customer identification information may be allocated, and the allocated customer identification information may be mapped to the customer's personal information and stored in a database.

That is, when the server 200 receives customer information entering the store from the offline virtual product display store, it searches for the customer information from the database and, if matching customer information exists, the customer identification information allocated in advance for the customer information It can be assigned to customers entering the store.

Furthermore, the server 200 may include customer identification information and an image captured by at least one or more photographing means 100 installed in the offline virtual product display store, for example, among the customer's face information, clothes color information, and body information in the image. Each of the customers moving within the offline virtual product display store may be recognized based on at least one.

At this time, the server 200 maps the customer identification information to the corresponding customer identification information in the offline virtual product display store and reflects the device identification information received from the wireless identification device when the wireless identification device is provided to provide a customer moving in the offline virtual display store. It can also be recognized in real time.

Furthermore, the server 200 may recognize a customer moving in the offline virtual display store in real time based on a deep learning-based learning model. For example, the server 200 may recognize a customer moving in an offline virtual display store in real time based on a deep learning-based learning model and image information of the customer captured by at least one photographing means that is mapped with the customer identification information. can That is, the server 200 uses a deep learning-based learning model to accurately recognize the image information of the customer mapped with the customer identification information, for example, face information, clothes color information, etc. Each customer can be clearly identified and recognized.

2 is a flowchart illustrating a customer recognition method according to an embodiment of the present invention.

Referring to FIG. 2 , a customer recognition method for customer tracking in an offline virtual product display store according to an embodiment of the present invention includes customer information including personal bio information of a customer entering the offline virtual product display store. , receives customer information including at least one of customer name, customer phone number, customer's face information, fingerprint information, and iris information (S210).

Here, step S210 may be received through a store terminal provided in the offline virtual product display store, and the received customer information may be transmitted to the server.

When customer information is received by step S210, customer identification information is allocated to the corresponding customer based on the received customer information (S220).

Here, in step S220, based on the customer information received when entering the offline virtual product display store, temporary customer identification information that can be allocated and used in the offline virtual product display store may be allocated, and each customer can use the corresponding method in advance. By accessing the provided service server and registering customer information, fixed customer identification information for the corresponding customer information may be assigned. Of course, the static customer identification information may be equally used in all offline virtual product display stores, that is, each customer can use an application eg a smart phone app to provide the method of the present invention, eg customer information eg , by registering personal information such as customer name, customer phone number, fingerprint information, body information, iris information, and face information in the server, fixed customer identification information corresponding to the customer's personal information based on the personal information registered in the server may be allocated, and the allocated customer identification information may be mapped to the customer's personal information and stored in the database.

That is, in step S220, when customer information entering the store is received from the offline virtual product display store, the customer information is retrieved from the database and, if matching customer information exists, the customer identification information allocated in advance for the corresponding customer information is applied to the corresponding store. can be assigned to customers entering through

When customer identification information is allocated to a customer entering the offline virtual product display store in step S220, the allocated customer identification information and an image captured by at least one or more photographing means installed in the offline virtual product display store, for example, within the image Each of the customers moving in the offline virtual product display store is recognized based on at least one of the customer's face information, clothes color information, and body information ( S230 ).

Here, in step S230, when the wireless identification device is provided by mapping to the corresponding customer identification information in the offline virtual product display store, the device identification information received from the wireless identification device is reflected to recognize the customer moving in the offline virtual display store in real time You may.

Further, in step S230, the customer moving in the offline virtual display store can be recognized in real time based on the customer's image information captured by at least one or more photographing means mapped with the customer identification information and a deep learning-based learning model. . That is, step S230 uses a deep learning-based learning model to accurately recognize customer image information that is mapped to customer identification information, for example, face information, clothing color information, etc., so that each customer moving in the offline virtual display store can be clearly distinguished and recognized.

As described above, the customer recognition method according to an embodiment of the present invention may accurately recognize each customer in real time in order to track each customer shopping for a virtual product displayed in an offline virtual product display store.

In addition, the customer recognition method according to an embodiment of the present invention accurately recognizes each customer entering the offline virtual product display store, so that when each customer selects products displayed in the offline virtual product display store and puts them in a virtual cart, the corresponding customer You can put exactly the virtual product of your choice on the virtual card of the customer.

3 shows the configuration of a customer recognition system according to an embodiment of the present invention, and shows the configuration of the server shown in FIG. Of course, the system of the present invention is not limited to a server, and may consist of at least one device or system capable of performing the present invention.

Referring to FIG. 3 , the system 300 according to an embodiment of the present invention includes a receiver 310 , an allocator 320 , a recognizer 330 , a controller 340 , and a database (DB) 350 . .

The DB 350 stores all kinds of data related to the present invention. For example, the DB includes all kinds of information for servicing the present invention, such as customer information of each customer, customer identification information mapped with customer information, device identification information mapped with customer identification information, and information on offline virtual product display stores. Information may be stored.

The receiving unit 310 includes customer information including personal bio information of a customer entering the offline virtual product display store, for example, customer name, customer phone number, customer's face information, fingerprint information, and iris information. Receive customer information.

The allocator 320 allocates customer identification information to the corresponding customer based on the customer information received by the receiver 310 .

Here, the allocator 320 may allocate temporary customer identification information that can be allocated and used in the offline virtual product display store based on the customer information received when entering the offline virtual product display store, and each of the customers in advance By accessing a service server providing the method and registering customer information, fixed customer identification information for the corresponding customer information may be assigned. For example, each customer sends personal information such as customer name, customer phone number, fingerprint information, body information, iris information, and face information to the server using the application of the present invention, for example, a smart phone app. By registering, fixed customer identification information corresponding to the customer's personal information is allocated based on the personal information registered in the server, and the allocated customer identification information is mapped to the customer's personal information and stored in the DB 350 .

That is, when the customer information entering the store is received from the offline virtual product display store, the allocator 320 searches for the customer information from the DB 350, and if there is matching customer information, it is pre-allocated for the customer information. Customer identification information may be assigned to a customer entering the store.

The recognition unit 330 may be configured to detect the assigned customer identification information and an image captured by at least one or more photographing means installed in the offline virtual product display store, for example, at least one of the customer's face information, clothes color information, and body information in the image. based on the recognition of each customer moving within the offline virtual product display store.

Here, the recognition unit 330 maps the corresponding customer identification information to the corresponding customer identification information in the offline virtual product display store and reflects the device identification information received from the wireless identification device when the wireless identification device is provided to identify the customer moving in the offline virtual display store. It can also be recognized in real time.

Furthermore, the recognition unit 330 recognizes the customer moving in the offline virtual display store in real time based on the deep learning-based learning model and image information of the customer captured by at least one photographing means mapped with the customer identification information in real time. can do.

The control unit 340 registers customer information including personal bio information of each customer in the DB 350, generates customer identification information of each customer based on the registered customer information, and generates registered customer information and the generated customer information. The customer identification information is mapped and stored in the DB 350 . Of course, the control unit 340 may perform various types of control performed in the system of the present invention.

Although the description of the system of FIG. 3 is omitted, each component constituting FIG. 3 may include all the contents described in FIGS. 1 to 2 , which is apparent to those skilled in the art.

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. may be embodied in The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (12)

Customer information including personal bio information of a customer entering an offline virtual product display store in which a virtual product corresponding to a real product is displayed on the screen of the display unit using a display unit instead of a display case in which the receiving unit displays the product - the customer receiving information including at least one of face information, clothes color information, body information, phone number, fingerprint information, and iris information of the corresponding customer through an input means provided in the offline virtual product display store;
allocating customer identification information to the customer based on the received customer information in an allocator; and
Recognizing the customer moving in the offline virtual display store in real time based on the assigned customer identification information and an image captured by at least one photographing means installed in the offline virtual product display store by a recognition unit
including,
registering customer information including personal bio-information of each customer in a database in the control unit;
generating, in the control unit, customer identification information of each of the customers based on the registered customer information; and
Map the registered customer information and the generated customer identification information in the control unit and store in the database
further comprising,
The saving step
When the customer information includes the customer's face information and the clothes color information, the generated customer identification information is mapped to the customer's face information and clothes color information and stored, and the clothes color information is combined with the upper clothes color information and the lower clothes information Stored separately by color information,
The allocating step
By comparing the received customer information with the registered customer information, if there is matching customer information, the customer identification information mapped to the received customer information and stored is allocated to the customer,
The recognizing step
Using a deep learning-based learning model, the customer's face information and clothing color information are recognized from the customer's image information captured by the photographing means mapped with the customer identification information, and the recognized face information and clothes color A customer recognition method for recognizing the customer moving in the offline virtual display store in real time using information and face information and clothes color information received through the input means.
According to claim 1,
The recognizing step
The customer recognition method according to claim 1, wherein the customer moving in the offline virtual display store is recognized in real time by reflecting device identification information mapped to the customer identification information and transmitted from the wireless identification device provided to the customer.
delete delete delete delete Customer information including personal bio information of a customer who enters an offline virtual product display store that displays a virtual product corresponding to a real product on the screen of the display means using a display means instead of a showcase for displaying products - the customer information is a receiving unit for receiving the customer's face information, clothes color information, body information, phone number, fingerprint information, and iris information through an input means provided in the offline virtual product display store;
an allocator for allocating customer identification information to the customer based on the received customer information; and
A recognition unit that recognizes the customer moving in the offline virtual display store in real time based on the assigned customer identification information and an image captured by at least one photographing means installed in the offline virtual product display store
including,
Register customer information including personal bio information of each customer in a database, generate customer identification information of each customer based on the registered customer information, and map the registered customer information and the generated customer identification information control unit to store in the database
further comprising,
the control unit
When the customer information includes the customer's face information and the clothes color information, the generated customer identification information is mapped to the customer's face information and clothes color information and stored, and the clothes color information is combined with the upper clothes color information and the lower clothes information Stored separately by color information,
the allocator
By comparing the received customer information with the registered customer information, if there is matching customer information, the customer identification information mapped to the received customer information and stored is allocated to the customer,
the recognition unit
Using a deep learning-based learning model, the customer's face information and clothing color information are recognized from the customer's image information captured by the photographing means mapped with the customer identification information, and the recognized face information and clothes color A customer recognition system for recognizing the customer moving in the offline virtual display store in real time using information and face information and clothes color information received through the input means.
8. The method of claim 7,
the recognition unit
The customer recognition system, characterized in that by reflecting the device identification information that is mapped to the customer identification information and transmitted from the wireless identification device provided to the customer, the customer moving in the offline virtual display store is recognized in real time.
delete delete delete delete

Images