KR20200103257A - blockchain-based system for processing marketing information interlocked with biometric authentication by smart mirrors - Google Patents

Abstract

The present invention generally relates to a technology for processing marketing information by performing a customer′s biometric authentication (biometric authentication according to the FIDO international standard) with an IoT device installed in a smart mirror installed in each store of a shopping facility and interlocking with it. According to the present invention, there is an advantage in that the smart mirror can provide personalized services for each shopping facility customer by performing and utilizing the customer′s biometric authentication according to the FIDO international standard. In addition, according to the present invention, in a large-scale shopping facility equipped with a number of stores, customers′ shopping facilities are technically supported so that customers can view products as they wish, compare them, and finally purchase the products they like most effectively. There is an advantage that can increase user satisfaction. In addition, according to the present invention, since it is possible to collect customer preference information for various products in an uncontaminated state, there is an advantage in that it is possible to accurately obtain customer taste trends. In particular, it is possible to increase the reliability of information and increase the effect of information sharing by managing the information with a block chain.

Description

Blockchain-based system for processing marketing information interlocked with biometric authentication by smart mirrors}

The present invention generally relates to a technology for processing marketing information by performing biometric authentication of a customer with an IoT device installed in a smart mirror installed in each store of a shopping facility and interlocking with it.

In particular, in the present invention, a smart mirror installed in each store of a shopping facility identifies a customer through biometric authentication, and when customer preference is recognized, the product information and customer fitting video are stored and a corresponding QR code is provided to the customer. Through the code, customers can collect and search their preferred product information, store and accumulate marketing information generated in multiple smart mirrors in a blockchain, and a marketing information processing server receives marketing information from the blockchain. It relates to marketing information processing technology that detects customer taste trends by analyzing big data and recommends products to customers through smart mirrors or smartphones based on this.

Currently, there is fierce competition between online shopping (including mobile shopping) and offline shopping. Each has advantages and disadvantages, but online shopping, which is more advantageous in terms of user convenience and price competitiveness, is dominating the market, and offline shopping is in a very difficult position. People do not completely give up offline shopping in that they can touch and wear products with their own hands, and have the pleasure of meeting and interacting with people, but they complain of inconvenience in terms of shopping convenience.

Accordingly, in order to increase the convenience of offline shopping, large-scale shopping facilities in which a large number of stores are arranged in a fairly large area are increasing in recent years. Stores are placed in large areas such as premium outlets and Starfield, and they are substantially enlarged by linking department stores and shopping centers. As the shopping facilities became larger, the convenience of shopping was certainly increased from the customer's point of view. This is because while maintaining the advantage of offline, that you can experience the product directly, the disadvantage of having to walk a lot to browse various stores has been resolved.

However, as a result, a new problem has emerged. As the convenience of shopping has increased, customers who visit shopping facilities tend to shop while browsing more stores, and it has become difficult to manage the large amount of information obtained in the process. For example, a customer who wants to buy a piece of clothing will try on multiple products while browsing multiple stores. The intention is to remember the good things among these products, and then choose the one that I like the most and has the right price. However, in reality, the first thing I tried on is memorable and it is difficult to go back to the store, so I choose appropriately from the products I wear later.

This problem does not appear in online shopping, but occurs only in offline shopping, and as a result, consumers have accumulated unsatisfactory experiences with offline shopping. However, since this is inherent in the behavioral pattern of offline shopping, there has been widespread recognition that it cannot be solved.

In addition, in offline shopping, it has been understood that trends are created by experts and spread to customers through various media. On the other hand, efforts to detect which products or characteristics customers have preferences were relatively inadequate. This is due to the limitation of the offline shopping environment that interactions (interactions) with customers are inconvenient. Therefore, conventionally, a method of collecting information from customers in the form of a survey or collecting sales performance information has been adopted.

However, there is a problem in that it is difficult to believe the information extracted from the survey results because people do not respond properly.

In addition, sales performance is different from the customer's preference trend. In general, because the factor of price plays a large role in purchasing behavior, it is not possible to extract a preference trend from sales performance. Even though customers actually liked it, the price was a little more expensive than competing products, so information about the product that did not reach the final purchase stage is not reflected. In this case, in terms of the customer's preference trend, it is reasonable that the product that was not purchased due to price is treated more importantly than the product actually purchased.

However, all of these problems have been accepted as essential limitations of offline shopping. Accordingly, the trend created by experts was unilaterally delivered to customers through the media, and the trend information obtained from online shopping was used as an auxiliary. However, the trend created by experts is different from the actual trend of customers, and the main users of online shopping and offline shopping are different, so it is not very helpful to use one of the trend information to the other.

An object of the present invention is to provide a technology for processing marketing information by performing biometric authentication of a customer with an IoT device installed in a smart mirror generally installed in each store of a shopping facility and interlocking with it.

In particular, it is an object of the present invention to provide a QR code corresponding to the smart mirror installed in each store of a shopping facility to the customer through biometric authentication and to store the product information and customer fitting video when the customer preference is recognized and to store the corresponding product information and , Through QR code, customers can collect and view their preferred product information, store and accumulate marketing information generated in a number of smart mirrors in a blockchain, and a marketing information processing server retrieves marketing information from the blockchain. It provides marketing information processing technology that detects customer taste trends by analyzing big data and recommends products to customers through smart mirrors or smartphones based on this.

The present invention for achieving the above object is a block chain-based marketing information processing system that is linked to the biometric authentication of the customer accompanying the identification of the customer's product wear and preference made by a plurality of smart mirrors installed in each store of a shopping facility. It is installed in each store for customer fitting, acquires customer identification information through biometric authentication for customers in front of the mirror, and creates a fitting image and provides it to an external image storage server when customer satisfaction is recognized. And it receives the fitting video URL, generates a QR code by combining its own store information and the fitting video URL, and displays it on the display screen, and generates block data by combining customer identification information, store information, product information, and fitting video URL. It performs blockchain update according to a preset schedule, and when a QR code is obtained from the customer's mobile terminal, the fitting image corresponding to the fitting image URL of the QR code is inquired from an external image storage server and displayed on the display screen. A smart mirror of 100; When a fitting video is received from outside through the network, it is stored in a database and a response is provided with the corresponding fitting video URL, and when a fitting video URL is received from the outside through the network, the corresponding fitting video is retrieved from the database and a response is provided. Storage server 300; Installed on the customer's mobile terminal 401, acquires the QR code displayed on the display screen of the smart mirror using the camera of the mobile terminal, manages the list of one or more obtained QR codes, and allows the user to The fitting image corresponding to the fitting image URL of the selected QR code is acquired from the video storage server and displayed on the terminal screen, and the store information screen corresponding to the store information of the QR code selected by the user from the list of QR codes is displayed on the screen of the mobile terminal. Displaying the customer interworking agent unit 400; Through the blockchain, a number of combinations of customer identification information, store information, product information, and fitting video URL (hereinafter referred to as'customer preference information package') generated in response to customer satisfaction from multiple smart mirrors are acquired, and a number of A number of fitting images corresponding to the fitting image URL of the customer preference information package are acquired from the video storage server, and customer trend information and store preference information are analyzed by analyzing customer identification information, product information, and multiple fitting images of a number of customer preference information packages. It is configured to include; marketing information processing server 500 for extracting.

In this case, the smart mirror 100 includes: a store information storage unit 110 for storing store information on stores in which the corresponding smart mirror is installed; A display unit 120 for configuring a reflective screen and an electronic display screen for the corresponding smart mirror; A customer biometric authentication processing unit 130 for acquiring customer identification information through biometric authentication for a customer existing in front of the smart mirror using at least one biometric sensor provided in the corresponding smart mirror; A fitting image generator 140 that photographs the front of the smart mirror using a camera provided in the corresponding smart mirror and generates a fitting image including a state in which a customer is fitting a product at a corresponding time point; When the customer satisfaction is recognized, the fitting image generated by the fitting image generator 140 is provided to an external image storage server, and a fitting image URL is received in response, and generated in response to the product that the customer expressed satisfaction. A customer preference processing unit 150 for displaying the generated QR code on the display unit 120; A QR code generation unit 160 for generating a QR code by combining store information and a fitting image URL under the control of the customer preference processing unit 150; A customer QR inquiry unit 170 that provides a fitting image URL of the QR code to an external image storage server when a QR code is obtained from the customer's mobile terminal, receives the fitting image in response to the QR code, and displays the fitting image on the display screen; Blockchain mirror processing unit that creates block data by combining customer identification information, store information, product information, and fitting video URL, and performs block chain update according to a preset schedule to participate in information sharing on the customer preferred distributed ledger 200 (180); may be configured to include.

In this case, the customer biometric authentication processing unit 130 includes: a fingerprint sensing unit 131 for performing authentication processing by recognizing a customer's fingerprint with a fingerprint sensor in order to perform biometric authentication for the customer; An iris sensing unit 132 for performing authentication processing by recognizing a customer's iris with a camera in order to perform biometric authentication for the customer; A face recognition unit 133 for performing authentication processing by recognizing a customer's face with a camera in order to perform biometric authentication for the customer; A voice recognition unit 134 for performing authentication processing by recognizing a voice of a micro customer in order to perform biometric authentication for the customer; By collecting and combining the authentication processing results of the fingerprint sensing unit, iris sensing unit, face recognition unit, and voice recognition unit, biometric authentication is performed for the customer in the front of the smart mirror and the customer database when the biometric authentication is successful It may be configured to include; multiple authentication unit 135 for obtaining customer identification information from.

In addition, the customer interlocking agent unit 400 may include a customer information storage unit 410 for storing personal information of a user of a mobile terminal in which the customer interlocking agent unit 400 is installed; Access to the personal information stored in the customer information storage unit 410 is controlled, and if the user biometric authentication of the mobile terminal on which the customer interlocking agent unit 400 is installed is successful using the biometric sensor of the mobile terminal, the customer information storage unit ( A customer information management unit 420 that provides personal information stored in 410 to the outside; QR code recognition unit 430 for recognizing the QR code displayed on the display unit of the smart mirror using the camera of the mobile terminal; A QR list management unit 440 for list management of one or more QR codes obtained from a plurality of stores through the QR code recognition unit 430; The QR list management unit 440 provides the fitting image URL of the specific QR code provided with the user selection input from the list of QR codes managed by the image storage server, and receives the fitting image from the image storage server in response to display on the mobile terminal screen A preference inquiry providing unit 450; Including; a store information guide unit 460 for displaying a store information screen corresponding to store information of a specific QR code provided with a user selection input from a list of QR codes managed by the QR list management unit 440 on the mobile terminal screen; Can be.

In addition, the marketing information processing server 500 may include a block chain server processing unit 510 participating in information sharing on the customer preferred distributed ledger 200 through a block chain; A customer preference information collection unit 520 for acquiring a plurality of customer preference information packages that are a combination of customer identification information, store information, product information, and fitting video URL from the customer preference distributed ledger 200 shared through the blockchain; A customer preference image collection unit 530 for providing each fitting image URL included in a plurality of customer preference information packages to an image storage server and receiving a fitting image from the image storage server in response; Customer visiting a shopping facility by combining and analyzing customer identification information and product information of a plurality of customer preference information packages obtained by the customer preference information collection unit 520 and a plurality of fitting images received by the customer preference image collection unit 530 A customer trend information extracting unit 540 for extracting customer trend information that they prefer; A store preference information extracting unit that extracts store preference information of customers visiting the shopping facility by analyzing a combination of customer identification information, store information, and product information of a plurality of customer preference information packages obtained by the customer preference information collection unit 520 ( 550); may be configured including.

According to the present invention, there is an advantage that a smart mirror can provide a personalized service for each customer of a shopping facility by performing and utilizing a customer's biometric authentication according to the FIDO (Fast IDentity Online) international standard.

In addition, according to the present invention, in a large-scale shopping facility equipped with a number of stores, customers' shopping facilities are technically supported so that customers can view products as they wish, compare them, and finally purchase the products they like most effectively. There is an advantage that can increase user satisfaction.

In addition, according to the present invention, since it is possible to collect customer preference information for various products in an uncontaminated state, there is an advantage of accurately obtaining customer taste trends. In particular, it is possible to increase the reliability of information and increase the effect of information sharing by managing it with a block chain.

[Fig. 1] is a diagram conceptually showing a blockchain-based marketing information processing system linked to biometric authentication of a smart mirror according to the present invention.
[Fig. 2] is a flow chart showing the overall operation process of the blockchain-based marketing information processing system according to the present invention.
[Figure 3] is a block diagram showing the internal functional configuration of a smart mirror in the present invention.
[Fig. 4] is a flow chart showing a process related to a smart mirror in the present invention.
[Figure 5] is a block diagram showing the internal functional configuration of the customer interworking agent unit in the present invention.
6 is a block diagram showing the internal functional configuration of a marketing information processing server in the present invention.
[Fig. 7] is a flow chart showing the operation process of the marketing information processing server in the present invention.

Hereinafter, the present invention will be described in detail with reference to the drawings.

[Fig. 1] is a diagram conceptually showing a block chain-based marketing information processing system linked to biometric authentication of a smart mirror according to the present invention, and [Fig. 2] is a block linked to biometric authentication of a smart mirror according to the present invention It is a flow chart showing the overall operation process of the chain-based marketing information processing system.

Referring to Fig. 1, the blockchain-based marketing information processing system according to the present invention includes a plurality of smart mirrors 100, a blockchain subsystem, an image storage server 300, a customer interlocking agent unit 400, and marketing. It is configured to include an information processing server (500). In this case, the image storage server 300 and the marketing information processing server 500 may be the same device or separate devices according to implementation examples.

First, the smart mirror 100 is installed in each store of an offline shopping facility and is very similar in appearance to a general mirror. When a customer browses the store and finds a product they like (eg, clothes, bags, shoes, accessories, etc.), they try to fit in front of the smart mirror 100 to see if the product suits them well. The smart mirror 100 may be installed for each store, or may be used in common by several stores, and may be implemented in the form of a kiosk equipped with functions such as store information.

In the present invention, the smart mirror 100 obtains customer identification information by performing biometric authentication on the customer who is fitting in front of itself, through which the reaction that the customer represents for a specific product in a specific store is classified as a customer (eg: Age, gender) can be classified and analyzed.

When a customer shows satisfaction after fitting a product, the smart mirror 100 recognizes it. Although the camera can read customer's facial expressions or gestures to see if they indicate satisfaction, it is possible to configure a specific button installed on the smart mirror 100 to be operated by the customer, and the store staff can check the customer's reaction and check the smart mirror ( 100) can also be configured.

In the case of recognizing customer satisfaction, the smart mirror 100 generates a fitting image by photographing the customer wearing the product with a camera, and provides the fitting image to the image storage server 300 through the network. do. The video storage server 300 responds and transmits a fitting video URL to this. The smart mirror 100 creates a QR code (digital tag) 161 by combining its own store information and the fitting image URL and displays it on its own display screen, which is displayed by the customer on a mobile terminal (eg, a smartphone) 401 ) To put it in. Conceptually, this corresponds to storing fitting results that the customer liked in the store.

In addition, the smart mirror 100 generates block data by combining customer identification information, store information, product information, and fitting video URL, and performs block chain update according to a preset schedule, which is the marketing information processing server 500 It is intended to be able to perform marketing information analysis by collecting this information. Customer identification information, store information, product information, and fitting video URL included in the block data can be important marketing data because the customer has shown satisfaction after performing a fitting. Since the sales performance information used in the past is contaminated by the factor of price, the purity is inferior to measure customer preference. On the other hand, the satisfaction of the customer with the fitting result is directly related to the customer's preference. Technical configuration that can increase the accuracy of the information analysis result by reflecting such data in the present invention, although such data were not reflected in the prior art when the customer liked it but did not make an actual purchase based on the absolute price level or relative price comparison Present.

In addition, a description is given of a situation in which the customer visits the store again later, presents the mobile terminal 401, and presents the QR code 161 that he had previously received at the store. At this time, the smart mirror 100 obtains a QR code 161 by photographing the screen of the customer's mobile terminal 401 using, for example, a camera, and analyzes the QR code 161 to obtain a fitting image URL. The fitting image corresponding to the fitting image URL is inquired from the image storage server 300 and displayed on the display screen. A desirable result desired in the present invention is that a customer purchases a corresponding product by viewing an old fitting image displayed on the screen of the smart mirror 100.

In the present invention, a plurality of smart mirrors 100 installed in a shopping facility collectively collect a data combination of customer identification information, store information, product information, and fitting image URL corresponding to the fitting when the customer displays satisfaction with the fitting result. As a result, a blockchain is used to share with the marketing information processing server 500.

Blockchain generally has the concept of a public digital transaction ledger in which transaction information generated on a network is duplicated and stored distributedly as many as the number of nodes shared among network participants. Blockchain technology is currently being actively applied around cryptocurrency.In the present invention, blockchain technology is used to share fitting-related information generated in the smart mirror 100, and for this purpose, a number of smart mirrors ( 100) and the marketing information processing server 500 participate in the digital ledger management of the blockchain as a kind of peer. In this specification, the digital ledger of the blockchain is referred to as the customer preferred distributed ledger 200 to indicate that it is a distributed ledger that manages information related to fittings that indicate satisfaction by customers.

The customer-preferred distributed ledger 200 is a form in which a number of transaction data is composed of each block and connected to each other, and one block is peers that connect to the blockchain in the corresponding blockchain update cycle (eg, 30 seconds). That is, a plurality of smart mirrors 100 correspond to newly generated block data in response to a fitting event indicating satisfaction of a customer in each store. Referring to FIG. 1, such a block may include user identification information, store information, product information, and fitting image URL. Due to the nature of the blockchain, which is based on a distributed network, data once entered in the customer preferred distributed ledger 200 cannot be modified without permission, so it is reliable.

The image storage server 300 is a device that receives and manages fitting images from a plurality of smart mirrors 100. To this end, when receiving a fitting image from the outside through a network, the image storage server 300 stores the fitting image in the fitting image database 310 and provides a response with a fitting image URL corresponding thereto. The fitting image URL corresponds to information for accessing the fitting image. In addition, when the video storage server 300 receives a fitting video URL from the outside through a network, the video storage server 300 finds a fitting video corresponding thereto from the fitting video database 310 and provides a response.

The customer connection agent unit 400 is installed in the form of an app, for example, on a mobile terminal 401 (eg, a smartphone) carried by the customer, and the customer reports products at various stores while walking around the shopping facility. The fitting information is stored and provided for information inquiry for later use by customers as needed.

The customer interlocking agent unit 400 uses the camera of the mobile terminal 401 to display the QR code 161 displayed on the display screen by the smart mirror 100 in response to the customer displaying satisfaction with the fitting result at a specific store. To obtain it. When a customer walks around several stores and looks at products, the customer interlocking agent unit 400 obtains one or more QR codes 161, and manages them as a list of QR codes.

When a customer launches the app at an appropriate time and looks at the QR code list and selects a specific QR code item from among them, the customer linkage agent unit 400 analyzes the QR code 161 selected by the user to obtain a fitting image URL. The fitting image corresponding thereto is obtained from the image storage server 300 and displayed on the terminal screen. Through this, customers will be able to revive their memories of the products they liked. In addition, the customer interworking agent unit 400 analyzes the QR code 161 selected by the user to obtain store information and displays a store information screen corresponding thereto on the terminal screen. Customers can recall the store's memory and identify the store's location (e.g., Area A-324) so it can be easily reached.

The marketing information processing server 500 acquires a number of combinations of data, preferably customer identification information, store information, product information, and fitting video URL, generated in response to customer satisfaction from a plurality of smart mirrors 100 through a block chain. do. In the present specification, a combination of data generated in response to a single customer satisfaction is referred to as a'customer preference information package'. When a plurality of smart mirrors 100 installed in shopping facilities show satisfaction after fitting a product at each corresponding store, the customer preference information package corresponding thereto is made into block data and reflected in the block chain. Accordingly, in the customer preference distributed ledger 200 of the blockchain, a plurality of customer preference information packages that have been individually generated throughout the shopping facility are accumulated and collected.

In addition, the marketing information processing server 500 obtains a plurality of fitting image URLs by analyzing a plurality of customer preference information packages, and acquires a plurality of fitting images corresponding to them from the image storage server 300. It is secured because it is possible to grasp which product attributes satisfy customers by analyzing the fitting image.

Then, the marketing information processing server 500 analyzes information obtained from a plurality of customer preference information packages, that is, customer identification information, store information, product information, and fitting images to extract customer trend information and store preference information. Customer trend information is the result of collectively analyzing which product attributes the customers of the shopping facility like, and store preference information is a collective analysis of which stores among the many stores belonging to the shopping facility show favorability of customers. One result.

Next, the overall operation process of the blockchain-based marketing information processing system according to the present invention will be described with reference to [Fig. 2].

Step (S100): First, when a customer visits a store and fits a specific product in front of the smart mirror 100, the smart mirror 100 of the store performs biometric authentication for the customer who is fitting in front of the customer. To identify. When the customer shows satisfaction with the current fitting result, the smart mirror 100 generates a fitting image and provides it to the image storage server 300. The image storage device 300 provides a fitting image URL, which is information for accessing the fitting image corresponding thereto.

Step (S110): The customer interaction agent unit 400 installed in the customer's mobile terminal 401 is a customer preference event, that is, a QR code corresponding to the customer expressing satisfaction with a specific fitting result in the corresponding store (161) ). Preferably, the smart mirror 100 combines information related to the customer preference event and displays the QR code 161 on the mirror screen, and the customer interlocking agent unit 400 uses the camera of the mobile terminal 401 to display the QR code. (161) is photographed and identified. As the customer walks around the various stores of the shopping facility and browses the products, the identification of the QR code 161 is repeated, and the customer interworking agent unit 400 manages the list of QR codes.

Step (S120): The smart mirror 100 generates block data from a combination of data related to a customer preference event, preferably customer identification information, store information, product information, and fitting video URL, and this is a marketing information processing server 500 ) To be shared with the blockchain. At this time, text data such as customer identification information, store information, product information, and fitting video URL were selected so that the size of the block data can be reduced to increase the operation efficiency of the block chain. For example, if a fitting video or image was put into block data, the operation efficiency of the block chain would have decreased considerably.

Step (S130): The customer interworking agent unit 400 installed in the customer's mobile terminal 401 checks a list of QR codes 161 acquired at various stores, and a specific QR code item for the QR code list Choose In response to such customer manipulation, the customer interface agent unit 400 analyzes the QR code 161 selected by the user to obtain a fitting image URL, obtains a fitting image corresponding thereto from the image storage server 300 and displays it on the terminal screen. . Through this, the customer connection agent unit 400 provides a fitting image inquiry for each QR code item in the QR code list. In addition, the customer interworking agent unit 400 analyzes the QR code 161 selected by the user to obtain store information and displays a store information screen corresponding thereto on the terminal screen.

Step (S140): The marketing information processing server 500 collects information related to an event (ie, a customer preference event) in which a customer has indicated satisfaction in a plurality of smart mirrors 100 through a block chain. In this specification, as an example of such information, a data combination of customer identification information, store information, product information, and fitting video URL is presented, and one of these combinations is referred to as a'customer preference information package' more than once.

Through this, in a number of stores scattered in shopping facilities, customers try on various products and consider some of them as purchase candidates, and related information is sent to the marketing information processing server 500. Is collected. The marketing information processing server 500 extracts a number of customer preference information packages from the blockchain, and analyzes and processes them to extract information related to customer preference trends for product attributes (eg, color, atmosphere, style, etc.). Alternatively, it is possible to extract information for distinguishing between stores that customers are interested in and stores that are turned away.

Step (S150): In addition, the marketing information processing server 500 obtains a plurality of fitting image URLs from the customer preference information package related to the customer preference event, and acquires a plurality of fitting images corresponding to them from the image storage server 300 do. By analyzing the fitting image through the video analysis server, the fitting image is secured because it is possible to grasp which product attributes are satisfying customers.

Step (S160): The marketing information processing server 500 extracts customer trend information based on the collected information of the customer preference event. Based on a number of customer preference information packages (i.e., customer identification information, store information, product information, fitting video URL) and a number of fitting images collected through the blockchain, preferably based on customer identification information, product information, and fitting images. It statistically analyzes what kind of product or product attribute people prefer and extracts customer trend information through it.

Step (S170): Further, the marketing information processing server 500 extracts store preference information based on the collected information of the customer preference event. Among a number of customer preference information packages collected through the blockchain (i.e., customer identification information, store information, product information, fitting video URL), preferably based on customer identification information, store information, and product information, It statistically analyzes which product represents the preference and extracts store preference information through it.

Step (S180): When the customer later visits the store again and presents the mobile terminal 401 and presents the QR code 161 that he had previously received at the store, the smart mirror 100 of the store The QR code 161 is provided from the returning customer and a fitting image corresponding thereto is displayed on the screen.

When a returning customer finds a QR code item that they previously received after fitting a corresponding store from the QR code list through the menu operation of the customer linkage agent unit 400 and displays it on the screen of the mobile terminal 401, the smart mirror (100) acquires a QR code 161 by photographing the screen of the mobile terminal 401 using a camera, analyzes the QR code 161 to obtain a fitting image URL, and a fitting image corresponding to the fitting image URL Is inquired from the image storage server 300 and displayed on the display screen.

[Fig. 3] is a block diagram showing the internal functional configuration of the smart mirror 100 installed in each store in the present invention, and [Fig. 4] is a flow chart showing a process related to the smart mirror 100 in the present invention.

3, the smart mirror 100 includes a store information storage unit 110, a display unit 120, a customer biometric authentication processing unit 130, a fitting image generation unit 140, and a customer preference processing unit 150 , A QR code generation unit 160, a customer QR inquiry unit 170, and a block chain mirror processing unit 180.

First, the store information storage unit 110 stores store information about the store in which the corresponding smart mirror 100 is installed, such as store name, store location, and handling products. When installing the smart mirror 100, these information may be stored, or an external store management server (not shown) may download it. In addition, store information may be directly stored, or a token (eg, a mirror identifier) that can obtain store information by accessing an external server may be stored.

The display unit 120 is a configuration that allows the corresponding smart mirror 100 to be used as a mirror, and has a reflective screen to illuminate the person in front of it, and additionally, a guide image (e.g., QR code) or a video playback screen (e.g. : A fitting image) is provided with an electronic display screen for displaying.

The customer biometric authentication processing unit 130 uses one or more biometric sensors 131 to 134 provided in the corresponding smart mirror 100 to receive customer identification information through biometric authentication for customers in front of the smart mirror 100. Acquire.

In general, it is preferable that the smart mirror 100 has multiple biometric authentication in consideration of the fact that a shopping facility has colorful lighting, a difference between a dark place and a bright place, and a lot of ambient noise. This is because when only one biometric authentication method is supported, the smart mirror 100 will frequently fail to identify customers if the surrounding environment of the store is an unfavorable condition for the biometric authentication method. For example, if the smart mirror 100 supporting only the iris recognition method is installed in a store or dark store where lighting is colorful and rapidly changing, the customer identification failure rate will be high. Likewise, if the smart mirror 100 supporting only the voice recognition method is installed in a store where music is noisy, the customer identification failure rate will be high. Since stores in shopping facilities have different environments, if only one biometric authentication method is supported, a problem will inevitably occur in many stores.

Accordingly, the present invention proposes that the customer biometric authentication processing unit 130 supports multiple biometric authentication. To this end, the customer biometric authentication processing unit 130 performs authentication processing by recognizing the customer's iris with a plurality of biometric authentication units, such as a fingerprint sensing unit 131 that recognizes a customer's fingerprint with a fingerprint sensor and performs authentication processing. It includes an iris sensing unit 132 to perform, a face recognition unit 133 to perform authentication processing by recognizing a customer's face with a camera, and a voice recognition unit 134 to perform authentication processing by recognizing a customer's voice with a microcomputer. Done.

In addition, the customer biometric authentication processing unit 130 also includes a multi-authentication unit 135 that synthesizes the plurality of biometric authentication results and derives a biometric authentication result for the customer. The multiple authentication unit 135 collects individual authentication processing results of the fingerprint sensing unit 131, the iris sensing unit 132, the face recognition unit 133, and the voice recognition unit 134, and combines these results with each other. That is, among them, the authentication processing result that failed at all in biometric authentication is removed, the remaining authentication processing results are compared with each other to check whether there is a conflict, and if there is no problem, the biometric authentication result for the customer is output. At this time, when the biometric authentication is successful, customer identification information is obtained from the customer database.

The fitting image generating unit 140 photographs the front of the smart mirror 100 using a camera provided in the corresponding smart mirror 100 and generates a fitting image containing a customer fitting a product at a corresponding time point.

The customer preference processing unit 150 provides the image storage server 300 through the fitting image generated by the fitting image generation unit 140 when it is recognized that the customer has expressed satisfaction after fitting a product, and In response, a fitting image URL provided by the image storage server 300 is received. Then, the customer preference processing unit 150 displays the QR code 161 generated by the following QR code generation unit 160 on the display unit 120 in response to the product for which the customer has displayed satisfaction.

The QR code generation unit 160 generates a QR code by combining store information and a fitting image URL under the control of the customer preference processing unit 150. In this specification, the QR code is used as a term representing a digital tag, and may be implemented as an NFC tag, an RFID tag, a QR code, a two-dimensional barcode, or the like.

The customer QR inquiry unit 170 presents the mobile terminal 401 by revisiting the corresponding visit, and when the customer presents the QR code 161 that was previously received at the store, The QR code 161 is obtained from the mobile terminal 401, the URL of the fitting image of the QR code 161 is provided to the image storage server 300, and the fitting image is received in response to the received image and displayed on the display screen. It is a function that easily redisplays the fitting image of the product that the customer once expressed satisfaction with through the QR code 161 after a certain period of time. It is highly likely to be connected.

The blockchain mirror processing unit 180 generates customer identification information, store information, product information, and information generated as a customer displays satisfaction in the fitting result in front of the smart mirror 100 during a preset block chain update period (eg, 1 hour). Block data (transaction data) is generated by collecting data combinations of fitting video URLs, and blockchain updates are performed according to a preset schedule (e.g., once every hour) to share information on the customer preferred distributed ledger 200. Participate.

Next, a process related to the smart mirror 100 in the present invention will be described with reference to FIG. 4.

Steps (S200 to S220): A customer who visits the store tries to wear it in front of the smart mirror 100 to check whether the products sold in the store (eg, clothes, bags, shoes, accessories, etc.) are suitable for them. When the customer expresses satisfaction with the fitting result, the smart mirror 100 recognizes that fact. At this time, the smart mirror 100 uses a biometric sensor (eg, a fingerprint sensor, an iris sensor, a face recognition sensor, a voice recognition sensor) to obtain the biometric information of the customer standing in front of the user and performing the fitting, and obtains the biometric information of the customer. Obtain customer identification information through biometric authentication used.

At this time, the biometric information for each customer is acquired by the customer information management unit 420 of the customer interlocking agent unit 400 using the hardware (eg, fingerprint sensor, camera) of the mobile terminal 401 and is converted to a customer database through the network. It may be configured to transmit or provide a smart mirror 100 in a short distance.

Step (S230): The smart mirror 100 generates a fitting image by photographing the customer wearing the product with a camera, and provides the fitting image to the image storage server 300 through a network. The image storage server 300 stores and manages the fitting image in the fitting image database 310, and responds and transmits a fitting image URL, which is information for accessing the fitting image.

Steps (S240, S250): Next, the smart mirror 100 generates a QR code 161 by combining its store information and the fitting image URL, and displays it on its own display screen, which is displayed by the customer on a mobile terminal (eg: It is intended to be carried using a smartphone) (401). Conceptually, this corresponds to storing fitting results that the customer liked in the store.

Accordingly, the customer takes a picture of the QR code displayed on the display unit 120 of the smart mirror 100 using his or her mobile terminal 401. The QR code recognition unit 430 of the customer interlocking agent unit 400 installed in the mobile terminal 401 identifies the QR code 161 from the camera photographed image, and the QR list management unit 440 is thus obtained from various stores. Save and manage them as a list.

Step (S260): In addition, the smart mirror 100 generates block data (transaction data) by combining customer identification information, store information, product information, and fitting video URL, and on a preset schedule (eg, once an hour). Blockchain update is performed accordingly. This is to enable the analysis of marketing information by sharing information on the fitting result indicating satisfaction of the customer to the marketing information processing server 500. The customer identification information, store information, product information, and fitting video URL included in the block data are important marketing data because they are the items that show satisfaction after the customer has done the fitting.

Steps (S270 to S290): When the customer revisits the visit and presents the mobile terminal 401 and presents the QR code 161 that he had previously received at the store, the customer of the smart mirror 100 The QR inquiry unit 170 obtains the QR code 161 from the mobile terminal screen of the customer using a camera, provides the fitting image URL included in the QR code 161 to the image storage server 300 and responds The fitting image is received and displayed on the display screen of the smart mirror 100.

[Fig. 5] is a block diagram showing the internal functional configuration of the customer interworking agent unit 400 in the present invention.

In the present invention, the customer interworking agent unit 400 is installed in the form of, for example, an app on a mobile terminal 401 (eg, a smart phone, a wearable computer) carried by the customer, and the customer moves around the shopping facility. It is a configuration that stores information that has been viewed and fitted products in several stores, and provides it to information inquiry so that customers can use it later if necessary. Referring to [Fig. 5], the customer connection agent unit 400 includes a customer information storage unit 410, a customer information management unit 420, a QR code recognition unit 430, a QR list management unit 440, a preference inquiry providing unit (450), it is configured to include a store information guide 460.

First, the customer information storage unit 410 stores personal information on the user of the mobile terminal 401 in which the customer interlocking agent unit 400 is installed.

The customer information management unit 420 controls access to personal information stored in the customer information storage unit 410. When the user biometric authentication of the mobile terminal 401 in which the customer interlocking agent unit 400 is installed by using the biometric sensor (not shown) of the mobile terminal 401 is successful, the customer information management unit 420 is Personal information stored in 410 is provided to the outside so that the smart mirror 100 or the marketing information processing server 500 can utilize it. At this time, it may be configured to transmit to a customer database (not shown) through a network (eg, the Internet), or may be configured to provide the smart mirror 100 through a local area network (eg, Bluetooth, NFC).

The QR code recognition unit 430 photographs the QR code 161 displayed on the display unit 120 by the customer preference processing unit 150 of the smart mirror 100 with the camera of the mobile terminal 401 and recognizes it through image analysis. do. Conceptually, the customer stores the fitting result that he liked in the store in his mobile terminal 401.

The QR list management unit 440 manages a list of one or more QR codes obtained from a plurality of stores through the QR code recognition unit 430. When a customer goes around several stores and looks at the products, the QR code recognition unit 430 acquires one or more QR codes 161, and the QR list management unit 440 manages them as a QR code list.

The preference inquiry providing unit 450 provides the fitting image URL included in the QR code item to the image storage server 300 when the user selects a specific QR code item from the QR code list managed by the QR list management unit 440, and In response, a fitting image is received from the image storage server 300 and displayed on the mobile terminal screen. It provides an opportunity for customers to explore the shopping facilities, for example, while relaxing at a coffee shop, to look at fitting videos that they have stored and to think about what to buy. This is also a technical configuration for promoting sales in stores.

The store information information unit 460 displays a store information screen corresponding to store information included in a specific QR code item selected by the user from the QR code list managed by the QR list management unit 440 on the mobile terminal screen.

[Fig. 6] is a block diagram showing the internal functional configuration of the marketing information processing server 500 in the present invention, and [Fig. 7] is a flow chart showing the operation process of the marketing information processing server 500 in the present invention.

In the present invention, the marketing information processing server 500 collects a plurality of customer preference information packages generated by a plurality of smart mirrors 100 through a block chain and statistically analyzes them to extract customer trend information and store preference information. 6, the marketing information processing server 500 includes a block chain server processing unit 510, a customer preference information collection unit 520, a customer preference image collection unit 530, and a customer trend information extraction unit 540. , It is configured to include a store preference information extraction unit (550).

First, the blockchain server processing unit 510 participates in sharing information on the customer preferred distributed ledger 200 of the blockchain by performing a blockchain update according to a preset schedule (eg, once every hour).

The customer preference information collection unit 520 acquires a plurality of customer preference information packages, which are a combination of customer identification information, store information, product information, and fitting video URL from the customer preference distributed ledger 200 shared through a block chain. These customer preference information packages are information generated by a plurality of smart mirrors 100 installed in shopping facilities in response to a fitting event indicating satisfaction of customers in each store.

The customer preference image collection unit 530 analyzes a plurality of customer preference information packages obtained by the customer preference information collection unit 520 to obtain a plurality of fitting image URLs, and provides these fitting image URLs to the image storage server 300 In response, a plurality of fitting images are received from the image storage server 300. It is secured because it is possible to grasp which product attributes satisfy customers by analyzing the fitting image.

The customer trend information extraction unit 540 stores customer identification information and product information of a plurality of customer preference information packages obtained by the customer preference information collection unit 520, and a plurality of fitting images received by the customer preference image collection unit 530. Combination analysis is performed to extract customer trend information preferred by customers visiting shopping facilities. Customer trend information is the result of collectively analyzing which product attributes the customers of shopping facilities like.

The store preference information extraction unit 550 combines and analyzes customer identification information, store information, and product information of a plurality of customer preference information packages obtained by the customer preference information collection unit 520 to provide store preference information of customers visiting the shopping facility. Extract. The store preference information is a result of collectively analyzing which stores among a number of stores belonging to a shopping facility to which store customers favor.

Next, an operation process of the marketing information processing server 500 in the present invention will be described with reference to FIG. 7.

Step (S300): First, the marketing information processing server 500 performs blockchain update according to a preset schedule to participate in sharing information on the customer preferred distributed ledger 200 of the blockchain.

Steps (S310, S320): The marketing information processing server 500 acquires a plurality of customer preference information packages generated by a plurality of smart mirrors 100 from the customer preference distributed ledger 200 of the blockchain. At this time, the customer preference information package includes customer identification information, store information, product information, and fitting video URL. In addition, the marketing information processing server 500 analyzes a plurality of customer preference information packages to obtain a plurality of fitting image URLs, provides these fitting image URLs to the image storage server 300, and responds to the image storage server 300. A plurality of fitting images are received from.

Step (S330): The marketing information processing server 500 includes customer identification information and product information of a plurality of customer preference information packages obtained by the customer preference information collection unit 520, and a plurality of the customer preference image collection unit 530 received. By combining and analyzing the fitting images of, we extract customer trend information preferred by customers visiting shopping facilities. Customer trend information is a result obtained by collectively analyzing the decisions made by customers of a shopping facility after actually having their own thoughts, indicating what product attributes people like.

Step (S340): In addition, the marketing information processing server 500 visits the shopping facility by analyzing a combination of customer identification information, store information, and product information of a plurality of customer preference information packages obtained by the customer preference information collection unit 520 It extracts information about the store preferences of customers who do. The store preference information is a result of collectively analyzing which stores among a number of stores belonging to a shopping facility to which store customers favor.

On the other hand, the present invention can be implemented in the form of a computer-readable code on a nonvolatile computer-readable recording medium. Various types of storage devices exist as such nonvolatile recording media, such as hard disk, SSD, CD-ROM, NAS, magnetic tape, web disk, cloud disk, etc. It can be implemented and executed. In addition, the present invention may be implemented in the form of a computer program stored in a medium in order to execute a specific procedure in combination with hardware.

100: smart mirror
110: store information storage unit
120: display unit
130: customer biometric authentication processing unit
131: fingerprint sensing unit
132: iris sensing unit
133: face recognition unit
134: voice recognition unit
135: multiple authentication unit
140: fitting image generator
150: Customer preference processing unit
160: QR code generation unit
161: QR code
170: customer QR inquiry unit
180: Blockchain mirror processing unit
200: Customer preferred distributed ledger
300: video storage server
310: Video DB
400: Customer connection agent department
401: mobile terminal
410: customer information storage unit
420: Customer Information Management Department
430: QR code recognition unit
440: QR list management unit
450: Preference inquiry provider
460: store information information
500: Marketing information processing server
510: Blockchain server processing unit
520: Customer preference information collection department
530: Customer preference video collection unit
540: Customer trend information extraction unit
550: Store preference information extraction unit

Claims (7)

As a blockchain-based marketing information processing system linked to the customer's biometric authentication associated with the customer's product wearing and preference identification, which is performed in a number of smart mirrors installed in each store of a shopping facility,
It is installed in each store for customer fitting, and it acquires customer identification information through biometric authentication for customers in front of the mirror, and when customer satisfaction is recognized, a fitting image is created and provided to an external image storage server, and fitting It receives the video URL, generates a QR code by combining its own store information and the fitting video URL, and displays it on the display screen, and generates block data by combining customer identification information, store information, product information, and fitting video URL. Blockchain updates are performed according to a preset schedule, and when a QR code is obtained from a customer's mobile terminal, a fitting image corresponding to the fitting image URL of the QR code is inquired from the external image storage server and displayed on the display screen. A smart mirror of 100;
When a fitting video is received from outside through the network, it is stored in a database and a response is provided with the corresponding fitting video URL, and when a fitting video URL is received from the outside through the network, the corresponding fitting video is retrieved from the database and a response is provided. Storage server 300;
Installed on the customer's mobile terminal 401, acquires the QR code displayed on the display screen of the smart mirror using the camera of the mobile terminal, manages the list of one or more obtained QR codes, and manages the list of QR codes. The fitting image corresponding to the fitting image URL of the QR code selected by the user is acquired from the image storage server and displayed on the terminal screen, and a store information screen corresponding to the store information of the QR code selected by the user from the list of the QR code is displayed on the mobile device. A customer interworking agent unit 400 displayed on the screen of the terminal;
Through the blockchain, a number of combinations of customer identification information, store information, product information, and fitting video URL (hereinafter referred to as'customer preference information package') generated in response to customer satisfaction from multiple smart mirrors are acquired, and the plurality of A number of fitting images corresponding to the fitting image URL of the customer preference information package of are obtained from the image storage server, and customer trend information is analyzed by analyzing customer identification information, product information, and multiple fitting images of the plurality of customer preference information packages. Marketing information processing server 500 to extract;
Blockchain-based marketing information processing system linked to the biometric authentication of the smart mirror, including.
The method according to claim 1,
The smart mirror 100,
Store information storage unit 110 for storing store information on stores in which the corresponding smart mirror is installed;
A display unit 120 for configuring a reflective screen and an electronic display screen for the corresponding smart mirror;
A customer biometric authentication processing unit 130 for acquiring customer identification information through biometric authentication for a customer existing in front of the smart mirror using at least one biometric sensor provided in the corresponding smart mirror;
A fitting image generator 140 that photographs the front of the smart mirror using a camera provided in the corresponding smart mirror and generates a fitting image including a state in which a customer is fitting a product at a corresponding time point;
When the customer satisfaction is recognized, the fitting image generated by the fitting image generating unit 140 is provided to an external image storage server, and a fitting image URL is received in response, in response to the product for which the customer expressed satisfaction. A customer preference processing unit 150 for displaying the generated QR code on the display unit 120;
A QR code generation unit 160 for generating a QR code by combining the store information and the fitting image URL under the control of the customer preference processing unit 150;
A customer QR inquiry unit 170 that provides a fitting image URL of the QR code to an external image storage server when a QR code is obtained from the customer's mobile terminal, receives the fitting image in response to the QR code, and displays the fitting image on the display screen;
Blockchain mirror processing unit that creates block data by combining customer identification information, store information, product information, and fitting video URL, and performs block chain update according to a preset schedule to participate in information sharing on the customer preferred distributed ledger 200 (180);
Blockchain-based marketing information processing system linked to biometric authentication of a smart mirror, comprising: a.
The method according to claim 2,
The customer interlocking agent unit 400,
A QR code recognition unit 430 for recognizing the QR code displayed on the display unit of the smart mirror using a camera of the mobile terminal;
A QR list management unit 440 for listing one or more QR codes obtained from a plurality of stores through the QR code recognition unit 430;
In the QR code list managed by the QR list management unit 440, a fitting image URL of a specific QR code provided with a user selection input is provided to the image storage server, and a fitting image is received from the image storage server in response to the mobile A preference inquiry providing unit 450 displayed on the terminal screen;
A store information guide unit 460 for displaying a store information screen corresponding to store information of a specific QR code provided with a user selection input from the list of QR codes managed by the QR list management unit 440 on the mobile terminal screen;
Blockchain-based marketing information processing system linked to biometric authentication of a smart mirror, comprising: a.
The method of claim 3,
The customer interlocking agent unit 400,
A customer information storage unit 410 for storing personal information on a user of a mobile terminal in which the customer interlocking agent unit 400 is installed;
Access to the personal information stored in the customer information storage unit 410 is controlled, and when the user biometric authentication of the mobile terminal on which the customer interlocking agent unit 400 is installed is successful using a biometric sensor of the mobile terminal, the customer information A customer information management unit 420 that provides personal information stored in the storage unit 410 to the outside;
Blockchain-based marketing information processing system linked to the biometric authentication of the smart mirror, characterized in that it is configured to further include.
The method of claim 3,
The marketing information processing server 500,
Blockchain server processing unit 510 participating in information sharing on the customer preferred distributed ledger 200 through a block chain;
A customer preference information collection unit 520 for acquiring a plurality of customer preference information packages that are a combination of customer identification information, store information, product information, and fitting video URL from the customer preference distributed ledger 200 shared through a block chain;
A customer preference image collection unit 530 for providing each fitting image URL included in the plurality of customer preference information packages to the image storage server and receiving a fitting image from the image storage server in response;
The shopping facility is analyzed by combining customer identification information and product information of a plurality of customer preference information packages obtained by the customer preference information collection unit 520 and a plurality of fitting images received by the customer preference image collection unit 530. A customer trend information extracting unit 540 for extracting customer trend information preferred by visiting customers;
Blockchain-based marketing information processing system linked to biometric authentication of a smart mirror, comprising: a.
The method of claim 5,
The customer biometric authentication processing unit 130,
A fingerprint sensing unit 131 for performing authentication processing by recognizing a customer's fingerprint with a fingerprint sensor in order to perform biometric authentication for the customer;
An iris sensing unit 132 for performing authentication processing by recognizing a customer's iris with a camera in order to perform biometric authentication for the customer;
A face recognition unit 133 for performing authentication processing by recognizing a customer's face with a camera in order to perform biometric authentication for the customer;
A voice recognition unit 134 for performing authentication processing by recognizing a voice of a micro customer in order to perform biometric authentication for the customer;
When the authentication process results of the fingerprint sensing unit, iris sensing unit, face recognition unit, and voice recognition unit are collected and combined with each other, biometric authentication is performed for the customer in front of the smart mirror and the biometric authentication is successful, the customer A multiple authentication unit 135 for obtaining customer identification information from a database;
Blockchain-based marketing information processing system linked to the biometric authentication of the smart mirror, characterized in that configured to include.
The method of claim 5,
The marketing information processing server 500,
Store preference information extraction for extracting store preference information of customers visiting the shopping facility by analyzing a combination of customer identification information, store information, and product information of a plurality of customer preference information packages obtained by the customer preference information collection unit 520 Part 550;
Blockchain-based marketing information processing system linked to the biometric authentication of the smart mirror, characterized in that it is configured to further include.

Images