TWI760521B - Smart store shopping system and purchasing method using thereof - Google Patents

Abstract

A smart store shopping system and purchasing method using thereof are disclosed. The smart store shopping system includes an identification access device, a virtual shopping cart module, a tracking device, and a check-out device. When a consumer enters the smart store, a shopping code is created for the consumer to start the virtual shopping cart module. And, using a body image taken during entering the smart store, it is possible to track the movement and moving path of the consumer in the smart store. All products taken by the consumer in the smart store will be automatically identified and added to the virtual shopping cart. After checkout, the check-out device would check whether the physical products at the time of checkout are as the same as the purchased product information recorded in the virtual shopping cart, and the check-out device would mark separately for the products paid and unpaid.

Description

Smart store shopping system and shopping method using the same

The present invention relates to a smart store shopping system and a shopping method using the same, in particular to a smart store shopping system that automatically detects the commodities purchased by consumers, and a shopping method using the smart store shopping system.

When consumers go to the store to buy products, after they choose the products they want to buy, they mostly use the clerk to operate the point of sale terminal (POS, Point of Sales) to check out the product and collect fees, or wait for the clerk to make coffee or Other freshly brewed drinks. However, with the advancement of technology, there have been various forms of smart stores that do not require additional staff to perform checkout.

For example, there are fingerprint or palm print identification, or mobile phone membership account as the identification mechanism for entering and exiting the store, so as to understand the information of consumers entering the store. After consumers have purchased the goods, they will check out through the self-checkout counter by scanning the RFID tags attached to the goods.

In an embodiment of the present invention, a shopping system for a smart store is provided, which includes an in-out identification device, a virtual shopping cart module, a dynamic tracking device, and a checkout device. The entry-exit recognition device will capture and photograph the facial images of consumers entering the smart store, and analyze the facial images to create and transmit shopping codes. The communication connection of the virtual shopping cart module identifies the incoming and outgoing devices, and the virtual shopping cart module receives and transmits the shopping code. The dynamic tracking device communicates with the virtual shopping cart module, and after receiving the shopping code, captures and tracks a body image of the consumer corresponding to the shopping code, and detects at least one product purchased by the consumer corresponding to the body image data, and at least one product data corresponding to the shopping code is sent to the virtual shopping cart module.

The checkout device is communicatively connected to the virtual shopping cart module, the checkout device captures a checkout face image and a product image, and transmits the checkout face image to the virtual shopping cart module. Next, receive from the virtual shopping cart module at least one commodity data that is identical to the shopping code corresponding to the checkout facial image and the admission facial image. The checkout device will analyze the product image to obtain and display the product information, and after receiving the checkout signal, calculate the checkout amount of the product information. After completing the payment request, compare the product information that has been requested and at least one product information. In addition, create a payment note for at least one product data corresponding to the product information that has been requested for payment, create an unpaid note for at least one product data that does not correspond to the product information for which payment has been completed, and then create a payment note and a non-payment note. Transfer to virtual shopping cart module.

In addition, in one embodiment of the present invention, a shopping method is provided, which includes: an entry-exit device captures an entry facial image of a consumer entering a smart store; the entry-exit device analyzes the entry facial image to create and transmit a shopping code to a virtual shopping cart module; the virtual shopping cart module stores the facial image and shopping code after receiving the entry facial image and the shopping code, and transmits the shopping code to the dynamic tracking device; the dynamic tracking device captures and tracks the corresponding consumption according to the shopping code One of the body images is detected, and at least one product data selected by the consumer corresponding to the body image is detected, and the at least one product data corresponding shopping code is sent to the virtual shopping cart module; the checkout device captures and sends the checkout The facial image is sent to the virtual shopping cart module, and at least one commodity data corresponding to the shopping code corresponding to the admission facial image of the checkout facial image is received from the virtual shopping cart module; the checkout device captures the commodity image and analyzes it product image to obtain product information; after receiving the checkout signal, the checkout device calculates the checkout amount of the product information and, after the payment is completed, compares the product information and at least one product information after the payment is completed; and the checkout device will Create a payment note corresponding to at least one item of product information that has been requested for payment, create a payment note for at least one item of product information that is not corresponding to the item information that has been requested for payment, and send the payment note and the non-payment note to the virtual machine Shopping cart mod.

Through the above-mentioned smart store shopping system and shopping method, when a consumer enters the smart store, a shopping code will be created for the consumer to activate the virtual shopping cart module, and the shopping code will be used as the consumer code of the virtual shopping cart . In addition, the body image taken after entering the store corresponds to the shopping code, so as to track the movement and dynamics of consumers in the smart store. After consumers purchase products in the smart store, the dynamic tracking device will identify the purchased product information, and transmit the purchased product information corresponding to the consumer's shopping code to the virtual shopping cart module for storage. That is to say, all shopping behaviors of consumers in the smart store will be automatically identified and added to the virtual shopping cart. Finally, after checkout, it will compare whether the physical product at checkout is the same as the purchased product information originally recorded in the virtual shopping cart, and note the product information that has been checked out and that have not been checked out.

In this way, consumers can enter the store for consumption by taking a facial image, and use the tracking body image to record the items purchased by the consumer. After the purchase is completed, you can check out through the checkout device at the self-checkout counter. Through the photographed facial images and body images, the information of the products purchased by consumers in the store can be recorded, and the purchased products can be stored. Further, the captured facial images and body images can also be used to track and analyze the shopping behavior from the time consumers enter the smart store to the time they leave the smart store, so as to identify whether all the products that consumers have purchased have been purchased. Payment, to achieve the function of double comparison.

The detailed features and advantages of the present invention are described in detail in the following embodiments, and the content is sufficient to enable any person familiar with the relevant art to understand the technical content of the present invention and implement it accordingly, and according to the content disclosed in this specification, the scope of the patent application and the drawings , any person skilled in the related art can easily understand the related objects and advantages of the present invention.

Please refer to FIGS. 1 to 2 . FIG. 1 is a schematic diagram of a shopping system in a smart store according to an embodiment of the present invention, and FIG. 2 is a schematic block diagram of a shopping system in a smart store according to an embodiment of the present invention. The smart store system of this embodiment includes an in-out identification device 10 , a virtual shopping cart module 20 , a dynamic tracking device 30 and a checkout device 40 . All modules or devices in the smart store shopping system can process or control data through one host, or each module and device can be executed by multiple hosts. The virtual shopping cart module 20 is communicatively connected to the identification access device 10 , the dynamic tracking device 30 and the checkout device 40 , respectively.

When a consumer wants to enter the smart store 1 for consumption, the identification entry and exit device 10 captures an entry face image of the consumer entering the smart store 1 . After analyzing the entrance face image, a shopping code is created according to the entrance face image, and the entrance face image and the shopping code are sent to the virtual shopping cart module 20 . In this embodiment, after the entrance facial image is acquired, the image analysis technology can be used to distinguish different facial images by analyzing and acquiring facial feature data. The used image analysis and recognition technology may be known, and will not be repeated here.

In this embodiment, the virtual shopping cart module 20 can be implemented in conjunction with the host set in the smart store 1, or can also be implemented by a host set independently. The virtual shopping cart module 20 receives and stores the facial image and the shopping code from the entry and exit identification device 10 . When the virtual shopping cart module 20 receives the entry face image and the shopping code, it will create a personal virtual shopping cart for the consumer according to the shopping code in the virtual shopping cart module 20, so as to carry out subsequent product purchase records. Then, the shopping code will be transmitted to the dynamic tracking device 30 to activate the dynamic tracking device 30 to start the movement and dynamic tracking of the consumer.

After receiving the shopping code, the dynamic tracking device 30 will shoot and start tracking the body image of the consumer corresponding to the shopping code, and detect at least one item of product information that the consumer corresponding to the body image has purchased. And at least one commodity data corresponding shopping code is sent to the virtual shopping cart module 20 .

For example, when a consumer takes a commodity from the shelf in the smart store 1, the dynamic tracking device 30 will identify the information of the commodity taken, and learn the information of the consumer who takes the commodity according to the body image. The shopping code is then associated with the product information and the shopping code and sent to the virtual shopping cart module 20 . At this time, the consumer's virtual shopping cart in the virtual shopping cart module 20 will store a piece of commodity data that has been purchased. If the consumer puts it back on the shelf after taking it, the dynamic tracking device 30 will also identify it accordingly, and notify the virtual shopping cart module 20 of the removal information, so as to delete the purchased product data originally recorded in the virtual shopping cart. The dynamic tracking device 30 will continue to track all the movements and activities of the consumer in the smart store 1 until the consumer leaves the smart store 1 .

After the consumer completes the purchase, the checkout device 40 can be used for checkout. The consumer uses the checkout device 40 to take a checkout face image and a product image. At this time, the checkout device 40 will transmit the checkout facial image to the virtual shopping cart module 20, and receive from the virtual shopping cart module 20 the same shopping code corresponding to the admission facial image of the checkout facial image. At least one product information. Here, the method of comparing whether the facial images are the same may be analysis and recognition by using facial feature data through image analysis, or various conventional recognition methods, which will not be repeated here.

The checkout device 40 also analyzes the product image to obtain and display product information, and after receiving the checkout signal, calculates the checkout amount of the product information. And after the payment is completed, compare the product information for which the payment is completed and at least one product information. Create a payment note for at least one product information corresponding to the product information that has been requested for payment, create an unpaid note for at least one item of product information that is not corresponding to the product information for which payment has been completed, and transmit the payment note and the non-payment note to the virtual shopping cart module 20.

After the consumer uses the checkout device 40 to photograph the product image of the purchased product and completes the checkout, the checkout device 40 will compare the checkout product with the purchased product in the original virtual shopping cart to confirm Which items have completed checkout and which items have not yet completed checkout.

In this way, consumers can make purchases in the store by taking facial images and body images, and after the purchase is completed, they can check out through the checkout device 40 at the self-checkout counter. Through the photographed facial images and body images, the information of the products purchased by consumers in the store can be recorded, and the purchased products can be stored. Further, the captured facial image and body image can also be used to track and analyze the shopping behavior of the consumer from entering the smart store 1 to leaving the smart store 1 . In addition, it is also possible to identify whether all the goods purchased by consumers have been paid for when purchasing, so as to achieve the function of double comparison.

In further application, the in-out recognition device 10 can capture an outgoing facial image when the consumer is about to leave the smart store 1 , and transmit the outgoing facial image to the virtual shopping cart module 20 . The virtual shopping cart module 20 will confirm whether the at least one item of product information corresponding to the shopping code corresponding to the entrance facial image that is the same as the exit facial image has an unpaid note according to the exit facial image, and if so, transmit the unpaid checkout Notify to identify incoming and outgoing devices. That is to say, for the consumer who wants to leave the venue, the facial image is photographed and identified, and it is confirmed whether there is any item marked as unchecked in the virtual shopping cart corresponding to the consumer, and if so, the unsettled item is sent. The reminder of the bill notification is sent to the identification entry and exit device 10 .

After receiving the notification, the identification access device 10 can use the screen display, light warning, sound warning or other various prompting means to remind the consumer that there is a product that has not yet been checked out. Please complete the checkout or return the product to the Leave in place and then leave.

Next, the identification of the in-out device 10 will be further described. Please refer to FIG. 1 to FIG. 3 at the same time. FIG. 3 is a schematic diagram of an ingress and egress identification device according to an embodiment of the present invention. The identification access device 10 includes an identification camera 11 , a box door 12 , a display device 13 and a controller 14 . 1 and FIG. 3 of the present embodiment are illustrated with two sets of identification entry and exit devices 10 as an example, but in other embodiments, only one set of identification entry and exit devices 10 may be installed depending on the requirements of space and passenger flow, or Install more identification entry and exit devices 10 . Each set of identification entry and exit devices 10 includes two identification cameras 11 for capturing facial images when entering and leaving the venue. However, in other implementations, only one identification camera 11 may be provided, and a rotating or moving structure may be used, so that the identification camera 11 can capture facial images in different directions.

The controller 14 is connected to the identification camera 11 , the gate 12 and the display device 13 . The recognition camera 11 captures and obtains an image of the entrance face, and sends the image to the controller 14 for image analysis. Then, the controller 14 will create a shopping code according to the facial image of the entrance, and transmit the shopping code to the virtual shopping cart module 20 . At the same time, the control gate 12 is opened to allow consumers to enter the smart store for consumption. In this embodiment, identification can also be performed in combination with member data, so as to record the consumption data of each member or perform deduction and checkout according to the member's account, which will be described in detail later. However, no matter whether you are a member or not, you can take a facial image, give a shopping code and create a virtual shopping cart for shopping in the smart store 1.

When the consumer completes the checkout and wants to leave the venue, he also needs to identify the entry and exit device 10 . The recognition camera 11 captures and obtains an outgoing face image, and sends the image to the controller 14 for image analysis. When the controller 14 receives the outgoing facial image, it will transmit the outgoing facial image to the virtual shopping cart module 20 for the aforementioned comparison and confirmation. If an unchecked notification is received, the controller 14 will control the display device 13 to display a warning message and control the gate 12 to close. The warning message can be text, patterns, numbers, sound effects, etc., and the present invention is not limited thereto. For example, the display device 13 may display "You still have xx products that have not been checked out, please return the products to their original positions, or proceed to checkout".

Next, it will be explained how consumers who have registered as members come to shop for consumption. The smart store shopping system of this embodiment may include the member server 50 to communicate with the device 10 for identifying the entry and exit. The member server 50 includes a database 51 and a verification module 52 . The database 51 stores plural member data, and each member data may include member number, name, gender, date of birth, card data, facial image and payment account data. The verification module 52 is connected to the database 51 .

When the consumer wants to enter the smart store 1 and takes a face shot at the entry-exit device 10 , the entry-exit device 10 can transmit the entry face image to the member server 50 . The member server 50 will compare the face images of each member profile in the database 51 when receiving the face images for admission. When the facial images that are the same as the admission facial images are compared among the facial images of all members, the corresponding member information is found. Then, the member server 50 transmits the member number corresponding to the face image to the identification access device 10 . The identification entry and exit device 10 will be able to use the membership number as the consumer's shopping code.

In this way, consumers who are already members can use the membership number to associate consumption behaviors and records, and even store these records in the database 51, so as to analyze the consumption habits of the members. Even, members can review their past consumption content. For example, the member server 50 may store the member data corresponding to the time when the consumer enters the smart store 1 as the entry time, so as to record the date and time when each member enters the smart store 1 . Member data corresponding to the purchasing process in the smart store 1 (such as purchasing flow, purchasing items, products originally intended to be purchased but not purchased in the end, etc.) may also be stored in the database 51 . Thereby, it can be used as the consumption time or habit analysis of members, and further application can be carried out. For example, if almost all consumers go to the smart store 1 to buy coffee at a specific time, they can prompt coffee-related promotions or messages before the specific time. Or for products that consumers have bought or want to buy, consumers can also be reminded of relevant preferential activity information when there is a promotion, so as to increase consumers' willingness to come to the store for consumption.

Next, please continue to refer to FIG. 1 and FIG. 2 , the dynamic tracking device 30 of this embodiment includes a plurality of cameras 31 and a processor 32 . The camera 31 is disposed in the smart store 1 and is used for photographing and obtaining the image of the consumer's body and the image of at least one product purchased. The plurality of cameras 31 can be installed anywhere in the smart store 1, and the cameras 31 can be installed even on the shelves. In this way, no matter where consumers go to the smart store 1 , they can be photographed by the camera 31 .

The processor 32 is connected to the camera 31 . The camera 31 will continue to capture images in the smart store 1 and transmit all the images to the processor 32 for analysis and processing. After receiving the image of the camera 31, the processor 32 will analyze the image of the product taken by the consumer in the image to obtain the corresponding product information, and according to the body image, transmit the product data corresponding to the consumer's shopping code to the virtual shopping Car mod 20. At this time, the product information will be added to the consumer's virtual shopping cart as the purchased product.

In addition, if the consumer puts the product back on the shelf and does not want to buy it, the processor 32 analyzes the image captured by the camera 31 to indicate that when the product is put back on the shelf, it will also analyze the image of the product put back by the consumer in the image to obtain Corresponding product data, and according to the body image, a message for deleting the product data and the shopping code corresponding to the consumer are sent to the virtual shopping cart module 20 . In this way, the product information in the consumer's virtual shopping cart can be updated in real time.

In other embodiments, in addition to the above-mentioned method of using the camera 31 to shoot and perform image analysis, a weight sensor can also be used on the shelf. The product at the corresponding location is purchased, and the product purchased by the consumer is then determined. In combination with the image of the consumer's body captured by the camera 31, the identity of the consumer who purchased the commodity is determined, so as to increase or decrease commodity data in the consumer's virtual shopping cart.

In addition, in this embodiment, the multimedia device 60 may be further set to communicate with the dynamic tracking device 30 . Consumers can use the multimedia device 60 to operate various billing information to be paid, or purchase various tickets and the like. After the multimedia device 60 receives the payment information (eg, the bill fee to be paid, the to-be-paid fee of the purchased ticket), the payment information is sent to the dynamic tracking device 30 . The dynamic tracking device 30 transmits the payment information corresponding to the shopping code corresponding to the body image to the virtual shopping cart module 20 according to the body image captured when the multimedia device 60 is operated. In this way, this payment information can be added to the consumer's virtual shopping cart. If the consumer forgets to input the payment information to pay the bill during the checkout process, the billing device 40 can also be used to remind the consumer about the information to be paid and whether to pay the bill.

Next, please refer to FIG. 1 , FIG. 2 and FIG. 4 . FIG. 4 is a schematic diagram of a checkout device according to an embodiment of the present invention. The checkout device 40 of this embodiment includes a first camera 41 , a second camera 42 , a touch screen 43 and a control module 44 . The control module 44 is connected to the first camera 41 , the second camera 42 and the touch screen 43 . The first camera 41 is used to capture and obtain the checkout face image, and the control module 44 analyzes the checkout face image. Then, the control module 44 will transmit the checkout face image to the virtual shopping cart module 20 to perform the aforementioned checkout and compare whether there is any unchecked product data in the virtual shopping cart. If consumers are not registered members, they can only use stored-value cards, credit cards, etc. for checkout and debit. But if you are a member, you can have more payment methods, which will be explained in detail next.

In the case where the consumer is a member, the control module 44 can transmit the image of the checkout face to the member server 50 to confirm whether the consumer who wants to checkout is a member, and if so, receive the corresponding checkout face The payment account information of the members of the video. Here, the checkout face image is sent to the member server 50 to compare and confirm whether it is a member. In addition, the checkout face image can also be sent to the virtual shopping cart module 20 by sending the checkout face image to the virtual shopping cart module. The obtained shopping code is used to analyze whether it is a member, and if so, the payment account information of the member corresponding to the shopping code (ie, the member code) is obtained from the member server 50 .

Then, the commodity to be purchased is photographed through the second camera 42 to obtain an image of the commodity. The control module 44 receives and analyzes the product image obtained from the second camera 42 to obtain product information, and transmits the product information to the touch screen 43 for display. When the touch screen 43 receives the checkout signal, the control module 44 will calculate the checkout amount according to the commodity information, and request the payment according to the payment account information. When the control module 44 completes the payment request, it can control the touch screen 43 to display the checkout completion information.

For example, after entering the store, the consumer can purchase the desired product, and after the purchase is completed, the customer can proceed to the checkout device 40 to check out. When it is time to check out, the consumer can first touch the touch screen 43 to start the checkout, or the first camera 41 can automatically start the checkout after sensing that there is a consumer standing at a specific position for a specific time. . When the checkout procedure is started, the first camera 41 will capture the face of the customer to obtain a face image of the checkout. The control module 44 analyzes the checkout face image, and transmits the checkout face image to the member server 50 to receive payment account data corresponding to the checkout face image. After receiving the payment account data, the control module 44 will temporarily store the data for subsequent checkout and payment request.

Next, after completing the facial recognition process, the consumer can place the product to be purchased in the checkout scanning area 45, and the second camera 42 will photograph the checkout scanning area 45 to obtain an image of the product. The control module 44 receives and analyzes the product image obtained from the second camera 42 to obtain product information, and transmits the product information to the touch screen 43 for display. The consumer can confirm whether the identified product information is correct through the touch screen 43 . And if there are commodities that you don't want to buy, you can delete the commodities you don't want to buy through the touch screen 43, or increase the number of commodities you want to buy.

When all the commodities to be purchased are input, the consumer can input the checkout signal through the touch screen 43 . The checkout signal can be an option button displayed on the touch screen 43 for the consumer to click. After the consumer inputs the checkout signal, the control module 44 will calculate the checkout amount according to the commodity information, and perform payment request according to the payment account information received previously. After the control module 44 completes the payment request, it will control the touch screen 43 to display the checkout completion information.

In some embodiments, when the consumer inputs the checkout signal, the commodity information to be checked out and the commodity information received by the virtual shopping cart module 20 can be compared at the same time. If there are products in the product data, but also in the product information to be checked out, a prompt can be popped up at this time to remind the consumer that there are still some products to be checked out.

In this embodiment, the payment account data in the member data may be a cloud account data, and the cloud account data may correspond to a financial account, a stored value card account or a debit account. For example, if the cloud account information is set to correspond to the financial account, when the payment is requested at checkout, a request for payment will be sent to the bank server of the financial account, and it will be completed after receiving the return signal of successful payment. Please pay. If the cloud account information is set to correspond to the stored-value card account, when making a checkout request, the card server of the stored-value card account will confirm whether there is enough stored-value amount for debiting, and if so, it will be sent to the card server of the stored-value card account. The card server sends a request for payment, and when it receives a successful payment request signal back, it completes the payment request.

If the cloud account information is set to correspond to the billing account, the billing amount will be sent to the billing account's server when the billing account is requested. If the billing amount meets the billing conditions of the server, the server will store the billing amount in the billing account, and send a payment request success signal to the billing device 40 . When the checkout device 40 receives the returned payment request success signal, the payment request is completed. Here, the billing account can be set to be linked to the consumer's salary account, and the server can be set to periodically (eg, on the 5th of each month) to obtain the payment corresponding to the billing amount currently stored in the billing account from the salary account. In addition, the server can reset the billing amount in the billing account to zero after confirming that the billing amount has been obtained.

In other embodiments, the payment account information can also be a mobile payment information or a credit card information. When the checkout device 40 wants to request money, it requests money from the corresponding institution or bank according to the mobile payment information or credit card information.

Next, please refer to FIG. 1 to FIG. 5 . FIG. 5 is a flowchart of a shopping method executed by the smart store shopping system according to an embodiment of the present invention, respectively. The shopping method of this embodiment is applied to the aforementioned smart store shopping system, so the components, structures and connection relationships of the smart store shopping system will not be repeated here.

When consumers want to enter the smart store 1 for consumption, they will first go to the identification entry and exit device 10 to perform store entry identification. The consumer will first stand in front of the entry-exit identification device 10, so that the entry-exit identification device 10 captures an entry face image of the consumer who wants to enter the smart store (step S10). Then, the in-out recognition device 10 analyzes the facial image of the entrance to create and transmit the shopping code to the virtual shopping cart module 20 (step S11 ). At the same time, the identification access device 10 can control the gate 12 to open, so that consumers can enter the smart store for consumption.

Next, after receiving the face image and the shopping code, the virtual shopping cart module 20 stores the face image and the shopping code, and transmits the shopping code to the dynamic tracking device 30 (step S12 ). When a consumer purchases a product in the store, the dynamic tracking device 30 captures and tracks the corresponding consumer's body image according to the shopping code, and detects at least one item of information about the product purchased by the consumer corresponding to the body image. , and transmit at least one product data corresponding shopping code to the virtual shopping cart module 20 (step S13 ). After the consumer completes the purchase of the commodity, the consumer can proceed to the checkout device 40 to perform the checkout.

The checkout device captures and transmits the checkout facial image to the virtual shopping cart module, and receives from the virtual shopping cart module at least one item of product data that is identical to the shopping code corresponding to the admission facial image of the checkout facial image (step S14). At the same time, the consumer places the purchased product in the checkout scanning area 45, and the checkout device 40 captures an image of the product, and analyzes the image of the product to obtain product information (step S15). After the input of all commodities to be checked out is completed, the consumer can input the checkout signal through the touch screen 43 . After receiving the checkout signal, the checkout device 40 calculates the checkout amount of the commodity information and completes the payment request (step S16 ). After the payment is completed, the commodity information for which the payment is completed is compared with at least one commodity information (step S17 ). The checkout device 40 creates a payment note for at least one product data corresponding to the product information for which payment has been completed, creates an unpaid note for at least one product data that does not correspond to the product information for which payment has been completed, and adds the payment note and the pending payment. The payment note is sent to the virtual shopping cart module 20 (step S18). In this way, the purchase and checkout of the product purchased by the consumer are completed.

In this way, consumers can make purchases in the store by taking a facial image, and after the purchase is completed, they can check out through the checkout device 40 at the self-checkout counter. Through the photographed facial images and body images, the information of the products purchased by consumers in the store can be recorded, and the purchased products can be stored. Further, the captured facial image and body image can also be used to track and analyze the shopping behavior of the consumer from entering the smart store 1 to leaving the smart store 1 . In addition, it is also possible to identify whether all the goods purchased by consumers have been paid for when purchasing, so as to achieve the function of double comparison.

Further, when the consumer has purchased the commodities to be purchased and completed the checkout and wants to leave the smart store 1 , in this embodiment, it is set to still need to go through the identification entry and exit device 10 to leave. When the consumer is about to leave the smart store 1 , the in-out recognition device 10 captures an outgoing facial image, and transmits the outgoing facial image to the virtual shopping cart module 20 (step S19 ). The virtual shopping cart module 20 confirms whether at least one item of product data of the shopping code corresponding to the entry facial image that is the same as the exit facial image has an unpaid note according to the exit facial image (step S20 ). If there is, send a notification of non-payment to the identification entry and exit device 10 (step S21 ). And after receiving the non-checkout notification, the identification entry and exit device 10 controls the display device 13 to display a warning message and controls the gate 12 to close (step S21a). If not, the gate 12 is opened to let the consumer leave the smart store 1 (step S22 ).

Next, please refer to FIG. 1 to FIG. 6 at the same time. FIG. 6 is a flowchart of identifying members in a shopping method according to an embodiment of the present invention. In step S11, the identification and confirmation of whether the member is a member can be performed again. After obtaining the face image of the customer's entrance in step S10 , the identification entry and exit device 10 transmits the face image of the entrance to the member server 50 (step S111 ). The member server 50 compares the facial images of the plurality of member data stored in the database 51 (step S112 ). When the face image is compared to the entrance face image, the membership number corresponding to the face image is sent to the identification entry and exit device 10 (step S113 ). The identification entry and exit device 10 uses the membership number as the shopping code, and transmits the shopping code to the virtual shopping cart module 20 (step S114 ). Then, step S12 is executed continuously.

In this way, consumers who are already members can use the membership number to associate consumption behaviors and records, and even store these records in the database 51, so as to analyze the consumption habits of the members. Even, members can review their past consumption content. For example, the member server 50 may store the member data corresponding to the time when the consumer enters the smart store 1 as the entry time, so as to record the date and time when each member enters the smart store 1 . Member data corresponding to the purchasing process in the smart store 1 (such as purchasing flow, purchasing items, products originally intended to be purchased but not purchased in the end, etc.) may also be stored in the database 51 . Thereby, it can be used as the consumption time or habit analysis of members, and further application can be carried out. For example, if almost all consumers go to the smart store 1 to buy coffee at a specific time, they can prompt coffee-related promotions or messages before the specific time. Or for products that consumers have bought or want to buy, consumers can also be reminded of relevant preferential activity information when there is a promotion, so as to increase consumers' willingness to come to the store for consumption.

During the shopping process of the consumer in the smart store 1, before executing the checkout procedure in step S15, the multimedia device 60 may also be used to input payment information such as bills or tickets. FIG. 7 is a flow chart of inputting payment information in a multimedia device in a shopping method according to an embodiment of the present invention. As shown in FIG. 7 , between step S13 and step S14 , the consumer can use the multimedia device to input payment information such as payment or ticket purchase. The multimedia device 60 receives the payment information, and transmits the payment information to the dynamic tracking device 30 (step S30 ). The dynamic tracking device 30 transmits the shopping code corresponding to the payment information to the virtual shopping cart module 20 according to the body image when the multimedia device 60 is operated (step S31 ).

Please refer to FIG. 8 again. FIG. 8 is a flow chart of member checkout and payment in a shopping method according to an embodiment of the present invention. When the consumer is a member and takes the goods to be purchased to checkout at the checkout device 40, when the consumer touches the touch option (eg checkout) on the touch screen 43, the customer enters the checkout Account collection procedure. In the aforementioned step S16, the checkout device 40 may transmit the checkout face image to the member server 50 (step S16a). The member server 50 compares the facial image stored in the database 51, and when the facial image is compared to the checkout facial image, transmits the payment account data corresponding to the facial image to the checkout device 40 (step S16b). After receiving the checkout signal, the checkout device 40 calculates the checkout amount and requests the checkout amount according to the payment account data (step S16c). Then, after the payment is completed, the checkout device 40 displays payment completion information (step S16d). Then continue to execute step S17.

In this way, for registered members, after the purchase is completed, they can check out through the checkout device 40 at the self-checkout counter, and complete the payment from the set payment account by taking a facial image. In this way, consumers can purchase and pay for commodities in the smart store 1 even if they do not carry any mobile devices, cash, credit cards or stored-value cards.

Although the present invention is disclosed above by the above-mentioned embodiments, it is not intended to limit the present invention. Anyone skilled in the art can make some changes and modifications without departing from the spirit and scope of the present invention. The scope of patent protection shall be determined by the scope of the patent application attached to this specification.

1‧‧‧Smart Store10‧‧‧Identifying Access Device 11‧‧‧Recognition Camera12‧‧‧Cassette Door13‧‧‧Display Device 14‧‧‧Controller20‧‧‧Virtual Shopping Cart Module30‧‧‧ Motion Tracking Device 31‧‧‧Camera 32‧‧‧Processor 40‧‧‧Checkout Device 41‧‧‧First Camera 42‧‧‧Second Camera 43‧‧‧Touch Screen 44‧‧‧Control Module 45 ‧‧‧Checkout Scanning Area 50‧‧‧Membership Server 51‧‧‧Database 52‧‧‧Authentication Module 60‧‧‧Multimedia Device S10‧‧‧Recognition of Entry and Exit Device to photograph the entrance of consumers who want to enter the smart store Facial image S11‧‧‧Recognizing the entry and exit device analyzes the entry facial image to create and transmit the shopping code to the virtual shopping cart module S12‧‧‧The virtual shopping cart module stores the entry face after receiving the entry facial image and the shopping code Part of the image and shopping code, and send the shopping code to the dynamic tracking device S13‧‧‧The dynamic tracking device captures and tracks the corresponding consumer's body image according to the shopping code, and detects the at least a product data, and send at least one product data corresponding shopping code to the virtual shopping cart module S14. The checkout device captures and transmits the checkout face image to the virtual shopping cart module, and receives from the virtual shopping cart module At least one product data corresponding to the shopping code corresponding to the face image of the checkout face image S15‧‧‧The checkout device captures the image of the product, and analyzes the image of the product to obtain the product information S16‧‧‧The checkout device receives the product After the checkout signal, calculate the checkout amount of the product information and complete the payment request When the payment account information corresponding to the face image is sent to the checkout device S16c‧‧‧After receiving the checkout signal, the checkout device calculates the payment amount and performs the payment of the payment amount according to the payment account information S16d‧‧‧Complete After the payment is requested, the checkout device will display the payment completion information S17‧‧‧Compare the product information that has been requested and at least one product information Create a payment note, create a non-payment note for at least one product information that does not correspond to the product information that has been requisitioned, and send the payment note and the non-payment note to the virtual shopping cart module S19‧‧‧Identifying the in-out device for consumers When you want to leave the smart store, take an image of the exiting face, and send the image of the exiting face to the virtual shopping cart module S20‧‧‧The virtual shopping cart module confirms the entry that is the same as the exiting facial image according to the image of the exiting face Whether there is an unpaid remark in at least one item of the shopping code corresponding to the facial image S21‧‧‧Send unpaid notification to the identification in-out device S21a‧‧‧The identification in-out device controls the display device to display after receiving the unpaid notification Warning message and control the gate to close S22‧‧‧Open the gate to let consumers leave the smart store S111‧‧‧Identify the entry and exit devices Send the admission face image to the member server S112‧‧‧The member server compares the face images of the plural member data stored in the database S113‧‧‧When the face image is compared to the admission face image, send the corresponding The membership number of the facial image is sent to the identification entry and exit device S114‧‧‧The identification entry and exit device uses the membership number as the shopping code, and sends the shopping code to the virtual shopping cart module S30‧‧‧Multimedia device receives the payment information, and sends the payment information to The dynamic tracking device S31‧‧‧The dynamic tracking device transmits the shopping code corresponding to the payment information to the virtual shopping cart module according to the body image when operating the multimedia device

[Fig. 1] is a schematic diagram of a shopping system in a smart store according to an embodiment of the present invention; [Fig. 2] is a block diagram of a shopping system in a smart shop according to an embodiment of the present invention; [Fig. 3] is a schematic diagram of an in-out device identification according to an embodiment of the present invention ; [FIG. 4] is a schematic diagram of a checkout device according to an embodiment of the present invention; [FIG. 5] is a flowchart of a shopping method performed by a smart store shopping system using an embodiment of the present invention; [FIG. 6] is a schematic diagram of an embodiment of the present invention A flow chart of identifying members in a shopping method according to an embodiment of the present invention; [Fig. 7] is a flow chart of inputting payment information in a multimedia device in a shopping method according to an embodiment of the present invention; and [Fig. 8] is a flow chart of members in a shopping method according to an embodiment of the present invention. Checkout payment flow chart.

1‧‧‧Smart Store

10‧‧‧Identification of entry and exit devices

11‧‧‧Identifying the camera

12‧‧‧Cassette Door

13‧‧‧Display device

31‧‧‧Camera

40‧‧‧Checkout device

60‧‧‧Multimedia Installation

Claims (18)

A shopping system for a smart store, comprising: an entry and exit device for photographing an entry face image of a consumer entering a smart store, and analyzing the entry face image to create and transmit a shopping code; a virtual shopping cart module, The virtual shopping cart module is communicatively connected to the identification access device, and the virtual shopping cart module receives and transmits the shopping code; a dynamic tracking device is communicatively connected to the virtual shopping cart module, and after receiving the shopping code, shoots and tracks the shopping code corresponding to the shopping code. A body image of a consumer, and detecting at least one commodity data corresponding to the body image selected by the consumer, and sending the at least one commodity data corresponding to the shopping code to the virtual shopping cart module; a multimedia The device is communicatively connected to the dynamic tracking device. After the multimedia device receives a payment information, it transmits the payment information to the dynamic tracking device, and the dynamic tracking device corresponds to the payment information according to the body image when the multimedia device is operated. The shopping code is sent to the virtual shopping cart module; and a checkout device is communicatively connected to the virtual shopping cart module, the checkout device shoots a checkout face image and a product image, and transmits the checkout face image to the virtual shopping cart module, and receive from the virtual shopping cart module the at least one commodity data of the shopping code corresponding to the entry facial image that is the same as the checkout facial image, and analyze the commodity image to obtain And display a product information, after receiving a checkout signal, calculate a checkout amount of the product information and after completing the payment, compare the product information and the at least one product information after the payment is completed, and will correspond to the Create a payment note for the at least one product information of the product information in the payment request, which will not correspond to completed An unpaid note is created for the at least one product information of the product information requested for payment, and the payment note and the unpaid note are sent to the virtual shopping cart module. The shopping system for a smart store as described in item 1 of the patent application scope, wherein the identification entry and exit device captures an outgoing facial image when the consumer is about to leave the smart store, and transmits the outgoing facial image to the virtual shopping a cart module, the virtual shopping cart module confirms whether the at least one commodity data of the shopping code corresponding to the entrance facial image which is the same as the exit facial image has the unpaid remark according to the exit facial image , if any, send a non-checkout notification to the ID device. The shopping system for a smart store as described in item 2 of the scope of the application, wherein the identification entry and exit device comprises: at least one identification camera for capturing and obtaining the entrance facial image and the exit facial image; a gate; and a display device ; and a controller connected to the identification camera, the gate and the display device, when the controller receives the face image for admission, analyzes the face image for admission to create and transmit the shopping code, and controls the gate to open, When the controller receives the departure facial image, it transmits the departure facial image to the virtual shopping cart module, and after receiving the non-checkout notification, controls the display device to display a warning message and controls the gate closure. The shopping system for a smart store as described in item 1 of the patent application scope, wherein the dynamic tracking device includes: a plurality of cameras arranged in the smart store for capturing and obtaining the image of the consumer's body and the image of the at least one product purchased; and a processor connected to the cameras, the processor receives and analyzes the at least one product A product image is obtained to obtain the corresponding at least one product data, and according to the body image, the at least one product data corresponding to the shopping code is sent to the virtual shopping cart module. The smart store shopping system described in item 1 of the scope of the application further includes a member server that communicates with the identification access device, and the member server includes: a database storing a plurality of member data, each member data Including a facial image and a membership number; and a verification module, connected to the database, and comparing the facial images in the database when receiving the admission facial image from the identification entry and exit device, when the When the face image is compared to the entrance face image, the membership number corresponding to the face image is sent to the identification access device, so that the identification access device can use the membership number as the shopping code. The shopping system for a smart store as described in item 5 of the patent application scope, wherein each of the member information further includes a payment account information, and the checkout device includes: a first camera for capturing and obtaining the checkout face image; a second camera for capturing and obtaining the image of the product; a touch screen for displaying the product information and receiving the checkout signal; and a control module for connecting the first camera, the second camera and the touch screen, the control module analyzes the checkout face image, and transmits the checkout face image to the member server to receive the payment account information corresponding to the checkout face image, the control module from the first The two cameras receive and analyze the product image to obtain the product information, and transmit the product information to the touch screen screen, the control module receives the checkout signal from the touch screen, calculates the checkout amount according to the commodity information, and collects money according to the payment account information. After the checkout is completed, the control module controls the checkout amount The touch screen displays a checkout completion message. The smart store shopping system as described in claim 6, wherein the payment account data is a cloud account data, and the cloud account data corresponds to a financial account, a stored value card account or a debit account. The smart store shopping system described in claim 7, wherein the billing account is linked to a salary account, and a server of the billing account periodically obtains a billing amount stored in the billing account from the salary account. The smart store shopping system as described in item 6 of the scope of application, wherein the payment account information is a mobile payment information or a credit card information. A shopping method includes: an entry-exit device photographing a face image of a consumer entering a smart store; the entry-exit device analyzes the entry face image to create and transmit a shopping code to a virtual shopping cart module ; After receiving the entry facial image and the shopping code, the virtual shopping cart module stores the entry facial image and the shopping code, and transmits the shopping code to a dynamic tracking device; the dynamic tracking device is based on the shopping code Shooting and tracking a corresponding body image of the consumer, and detecting at least one commodity data corresponding to the consumer purchased by the body image, and sending the at least one commodity data corresponding to the shopping code to the virtual shopping car module; a multimedia device receives a payment information, and transmits the payment information to the dynamic tracking device; The dynamic tracking device transmits the payment information corresponding to the shopping code to the virtual shopping cart module according to the body image when operating the multimedia device; a checkout device captures and transmits a checkout face image to the virtual shopping cart module, and receives from the virtual shopping cart module the at least one commodity data of the shopping code corresponding to the entry facial image that is the same as the checkout facial image; the checkout device captures a commodity image and analyzes it The product image is used to obtain a product information; after receiving a checkout signal, the checkout device calculates a checkout amount of the product information and completes the payment; ; and the checkout device creates a payment note for the at least one product information corresponding to the product information for which the payment has been completed, and creates an unpaid payment for the at least one product information that does not correspond to the product information for which the payment has been completed. note, and transmit the payment note and the unpaid note to the virtual shopping cart module. The shopping method as described in item 10 of the scope of application, further comprising: when the consumer is about to leave the smart store, the in- and out-recognition device captures an outgoing facial image, and transmits the outgoing facial image to the virtual shopping cart module; the virtual shopping cart module confirms whether the at least one commodity data of the shopping code corresponding to the entrance facial image which is the same as the exit facial image has the unpaid remark according to the exit facial image ; and, if any, sending a non-checkout notification to the ID device. The shopping method as described in item 11 of the scope of the application further comprises: after receiving the non-checkout notice, the identification access device controls a display device to display a warning message and controls a gate to close. According to the shopping method described in item 11 of the scope of the application, the device for identifying entry and exit includes: at least one identification camera for capturing and obtaining the entry facial image and the exit facial image; a gate; a display device; and a The controller is connected to the identification camera, the gate and the display device, when the controller receives the face image for admission, analyzes the face image for admission to create and transmit the shopping code, and controls the gate to open, when the control When receiving the outgoing facial image, the device transmits the outgoing facial image to the virtual shopping cart module, and controls the display device to display a warning message and controls the gate to close after receiving the non-checkout notification. The shopping method as described in claim 10, wherein the dynamic tracking device comprises: a plurality of cameras disposed in the smart store for photographing and acquiring the body image of the consumer and the at least one product purchased an image; and a processor connected to the cameras, the processor receives and analyzes the at least one product image to obtain the corresponding at least one product data, and transmits the at least one product data corresponding to the shopping code according to the body image to the virtual shopping cart module. The shopping method as described in claim 10, wherein the step of analyzing the entrance face image by the identification entry and exit device to create and transmit the shopping code to the virtual shopping cart module comprises: the identification entry and exit device transmits the entrance face Send an image to a member server; the member server compares a face image of a plurality of member data stored in a database; when the face image is compared to the admission face image, the corresponding face image is sent The membership number of the external image is sent to the identification in-out device; and the identification in-out device takes the membership number as the shopping code, and transmits the shopping code to the virtual shopping cart module. The shopping method as described in claim 15, wherein each member information further includes a payment account information, and the checkout device includes: a first camera for photographing and obtaining the checkout face image; a first Two cameras are used to shoot and obtain the image of the product; a touch screen is used to display the product information and receive the checkout signal; and a control module is connected to the first camera, the second camera and the touch screen, The receiving and transmitting the checkout face image to the member server to receive the payment account data corresponding to the checkout face image, the control module receiving and analyzing the product image from the second camera to obtain the product information, and send the product information to the touch screen, the control module receives the checkout signal from the touch screen, calculates the checkout amount according to the product information, and requests money according to the payment account information, the After completing the payment request, the control module controls the touch screen to display a checkout completion message. The shopping method according to claim 16, wherein the payment account data is a cloud account data, and the cloud account data corresponds to a financial account, a stored value card account or a debit account. The shopping method described in claim 17, wherein the billing account is linked to a salary account, and a server of the billing account periodically obtains a billing amount stored in the billing account from the salary account.

Images