KR20230038948A - Manless showcase and method for controlling the same - Google Patents

Abstract

The present invention relates to an unmanned stand. The unmanned stand includes: an authentication unit for performing user authentication for a user who wishes to use an unmanned stand; a product management unit which determines whether there is a withdrawal product selected by a customer and withdrawn from the unmanned stand among products displayed on the unmanned stand, after user authentication is performed; and a control unit which displays payment-related information on the withdrawal product on a screen, if it is determined that the withdrawal product exists, wherein the payment-related information may include detail information of the withdrawal product and payment request information for requesting payment of the amount corresponding to the detail information.

Description

Unmanned shelf and its driving method {MANLESS SHOWCASE AND METHOD FOR CONTROLLING THE SAME}

The present application relates to an unmanned shelf and a driving method thereof, and particularly to an artificial intelligence (AI)-based unmanned shelf and a driving method thereof. In addition, the present disclosure relates to an unmanned sales system including an unmanned shelf.

Frequently, convenience stores, marts, etc. are equipped with shelves (for example, refrigerated shelves) on which products (products) such as beverages and refrigerated food are displayed. A shelf refers to a place where products/objects are displayed and sold in a store, etc., and is otherwise called a display case, a sales stand, a showcase, and the like.

In general, when a customer wants to purchase a specific product in a counter, the customer takes out the product he wishes to purchase from among the products displayed in the counter and brings it to the cashier. Then, the cashier (checkout attendant) located at the checkout counter measures the price of the product by scanning the barcode of the product brought by the customer. Afterwards, the customer purchases the product by performing payment such as credit card payment or cash payment based on the measured price.

By the way, such a conventional shelf is simply used for displaying products, and when checking stock, it is generally done by a shelf manager (for example, an employee) manually checking stock, which takes a long time There is a problem in that accuracy is low and it is difficult to grasp the inventory of products in the shelf in real time.

In addition, since counters are conventionally used only for product display and payment for products in the counters is made by a cashier, a cashier must be kept 24 hours a day to sell products 24 hours a day, resulting in high labor costs.

The background technology of the present application is disclosed in Korean Patent Publication No. 1985-0005258.

The present application is intended to solve the problems of the prior art described above, and the problems caused by the inventory identification using a conventional shelf are manually performed by a shelf manager (eg, an employee) (ie, it takes a long time, the accuracy is low, , It is an object of the present invention to provide an unmanned shelf, a method of driving the same, and an unmanned sales system that can solve the problem of not knowing in real time the inventory of products in the shelf.

The present application is to solve the above-mentioned problems of the prior art, and provides an unmanned shelf, a driving method thereof, and an unmanned sales system that can solve the problem of high labor costs when selling products displayed on a conventional shelf 24 hours a day. intended to do

However, the technical problem to be achieved by the embodiments of the present application is not limited to the technical problems described above, and other technical problems may exist.

As a technical means for achieving the above technical problem, the unmanned stand according to an embodiment of the present application includes an authentication unit performing user authentication for a user who wishes to use the unmanned stand; a product management unit that determines whether there is a withdrawal product selected by the customer and withdrawn from the unmanned shelf among products displayed at the unmanned shelf after the user authentication is performed; and a control unit displaying payment-related information for the withdrawal product on a screen when it is determined that the withdrawal product exists, wherein the payment-related information includes details of the withdrawal product and an amount corresponding to the details information. Payment request information for requesting payment may be included.

In addition, the product management unit may control the door of the unmanned store to be openable so that products in the unmanned store can be withdrawn only when the user authentication is confirmed as normal.

In addition, when the product management unit determines that the door is closed after detecting the door open in a state in which the door is controlled to be openable, the product management unit takes pictures of the products displayed on the unmanned shelf. It is possible to determine whether the product to be withdrawn exists by acquiring and analyzing a product photographic image from an internal camera installed in the area.

In addition, the product management unit compares a first product shooting image obtained from the internal camera before the door opening is detected and a second product shooting image obtained from the internal camera after the door closing is detected, It may be determined whether the withdrawal product exists.

In addition, the product management unit takes a learning image in a plurality of product types as an input value and takes product information matched with each of the product types in the plurality of product types as an output value based on a deep learning-based pre-learning item, It is possible to generate detail information including product information of the withdrawal product determined using the image.

In addition, when it is determined that the payment for the payment-related information has been completed, the unmanned counter according to an embodiment of the present application recognizes the product for which the payment has been completed as a sold product, and displays sales information related to the product for sale. A provision unit provided to the management server may be further included. Here, the sales information may include sales product-related data, sales product-related customer data for customer-customized analysis, and inventory data about remaining products remaining on the unmanned shelf due to the sale of the sales product.

In addition, when it is determined that the payment has been completed, the providing unit performs stock checking on the remaining products in the unmanned shelf, and as a result of performing the stock checking, the stock quantity of any one product type among the product types of the remaining products is determined. If the quantity is less than the set quantity, an order request message for inventory management for replenishment of the product of any one product type for the unmanned shelf may be provided to the management server.

In addition, the management server may be capable of performing at least one of identification of preferred products by customer, identification of customer consumption patterns, identification of sales status, identification of inventory, and inventory management through analysis of the sales information provided from the providing unit.

On the other hand, the unmanned sales system according to an embodiment of the present application, the unmanned shelf according to an embodiment of the present application described above; And it may include a management server that transmits and receives data through the network with the unmanned counter, analyzes information provided from the unmanned counter, and performs management of the unmanned counter.

In addition, the driving method of the unmanned store according to an embodiment of the present application may include, in an authentication unit, performing user authentication for a customer who wishes to use the unmanned store; Determining, in a product management unit, whether there is a withdrawal product selected by the customer and withdrawn from the unmanned shelf among products displayed at the unmanned shelf after the user authentication is performed; and displaying, by the control unit, payment-related information for the withdrawal product on a screen, if it is determined that the withdrawal product exists, wherein the payment-related information includes detailed information of the withdrawal product and the detailed information. Payment request information for requesting payment of a corresponding amount may be included.

The above-described problem solving means are merely exemplary and should not be construed as intended to limit the present disclosure. In addition to the exemplary embodiments described above, additional embodiments may exist in the drawings and detailed description of the invention.

According to the above-described problem solving means of the present application, by providing an artificial intelligence / deep learning-based unmanned shelf, it is possible to enable a manager to immediately grasp the stock of products in an unmanned shelf in real time, and to more accurately and quickly identify inventory. There is an effect.

According to the above-described problem solving means of the present application, based on user authentication, door lock ON / OFF control of unmanned shelf, determination of the existence of withdrawal products based on artificial intelligence / deep learning, identification of detailed information including product information of withdrawal products, By providing an unmanned shelf that enables automatic payment processing for withdrawal products, TOTAL CASHIER-FREE is possible at a relatively low cost. This present application can solve the problem that labor costs are high as a cashier must be placed 24 hours a day for 24-hour product sales in the case of a conventional shelf.

However, the effects obtainable herein are not limited to the effects described above, and other effects may exist.

1 is a diagram showing a schematic configuration of an unmanned sales system according to an embodiment of the present invention.
2 is a view for explaining a process of determining whether a product management unit withdrawn exists in an unmanned shelf according to an embodiment of the present invention.
3 is a view showing an example of products displayed on an unmanned shelf according to an embodiment of the present disclosure.
4 is an operational flowchart of a method for driving an unmanned shelf according to an embodiment of the present disclosure.

Hereinafter, embodiments of the present application will be described in detail so that those skilled in the art can easily practice with reference to the accompanying drawings. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. And in order to clearly describe the present application in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the present specification, when a part is said to be “connected” to another part, it is not only “directly connected”, but also “electrically connected” or “indirectly connected” with another element in between. "Including cases where

Throughout the present specification, when a member is referred to as being “on,” “above,” “on top of,” “below,” “below,” or “below” another member, this means that a member is located in relation to another member. This includes not only the case of contact but also the case of another member between the two members.

Throughout the present specification, when a certain component is said to "include", it means that it may further include other components without excluding other components unless otherwise stated.

1 is a diagram showing a schematic configuration of an unmanned sales system 100 according to an embodiment of the present invention. Hereinafter, the unmanned sales system 100 according to an embodiment of the present application will be referred to as the present system 100 for convenience of description.

Referring to FIG. 1, the present system 100 is a system capable of selling products unmanned using an unmanned shelf 10, which can be referred to as a computer vision-based unmanned sales system capable of 24-hour business support. . In this application, the term product may be expressed interchangeably with terms such as product.

The unmanned shelf 10 is an unmanned shelf 10 according to an embodiment of the present application, and hereinafter may be referred to as the present unmanned shelf 10 for convenience of description. In addition, the unmanned shelf 10 may be differently expressed as an artificial intelligence (AI)-based unmanned shelf, an unmanned sales stand, an unmanned vending machine, an unmanned showcase, an unmanned sales showcase, and the like.

The system 100 may include an unmanned store 10 (this unmanned store) and a management server 20 .

The unmanned shelf 10 may be installed in a place where it is desired to sell and manage products unattended, a place requiring 24-hour sales support related to product sales, and the like.

Illustratively, the unattended shelf 10 may be a shelf installed in a fitness exercise facility. Fitness facilities may refer to exercise-related facilities such as golf courses, gyms, gyms, and tennis courts. However, this is only one example to help the understanding of the present application, and is not limited thereto. The unmanned shelf 10 may be installed not only in fitness facilities, but also in major tourist attractions, subway stations, public toilets, convenience stores, marts, stores, and the like. That is, the unmanned shelf 10 may be installed in any place where various products are displayed and sold unattended.

Various types of products (ie, products having a plurality of product types, products classified into a plurality of product types) may be displayed on the unmanned shelf 10 . Here, the product type may include at least one of product type, size (capacity), and brand name. For example, 10 products are displayed in the unmanned shelf 10, and the type of the first product and the second product among the 10 displayed products are different from each other, among the type, size (capacity) and brand name of the two products. At least one may mean different.

For example, when the unmanned shelf 10 is installed in a fitness facility, the unmanned shelf 10 includes products having a plurality of product types, 300ml bottled water of brand a, 500ml bottled water of brand a, and 500ml bottled water of brand a. Beverages, b-brand beverages, c-brand exercise creams, and c-brand exercise taping products may be displayed.

As another example, when the unmanned shelf 10 is installed in a public restroom, products having a plurality of product types, such as tissue paper, wet wipes, soap, toothpaste, and the like, may be displayed on the unmanned shelf 10. As another example, when the unmanned shelf 10 is installed in a convenience store, the unmanned shelf 10 includes products having a plurality of product types, 500ml bottled water of brand a, 500ml bottled water of brand b, beverage of brand c, c Brand alcohol, etc. may be displayed.

The unmanned stand 10 performs user authentication on the customer 1 who wishes to use the unmanned stand 10, and after the user authentication is performed, among the products displayed on the unmanned stand 10, the using customer ( It is determined whether there is a withdrawal product selected by 1) and withdrawn from the unmanned counter 10, and when it is determined that there is a withdrawal product, information related to payment for the withdrawal product may be displayed on the screen. A more detailed description of the unmanned shelf 10 will be described later.

The unmanned counter 10 may display information related to payment for withdrawal products on the screen of the kiosk 40, for example. The unmanned counter 10 may be interlocked with the kiosk 40 through the network 30 . Through this, the unmanned counter 10 displays payment-related information on the screen of the kiosk 40, so that the customer 1 withdraws from the unmanned counter 10 through the kiosk 40 and pays for the withdrawal product. you can make it

The kiosk 40 refers to a touch screen type information delivery system installed in a public place, and as the kiosk 40 applied to the system 100, a pre-installed kiosk or a kiosk provided by the present application may be applied. . In addition, as the kiosk 40 applied to the present system 100, a kiosk having a payment service providing function such as a simple payment function, a card payment function, and a cash payment function may be applied.

In the above-described example of the present application, when payment-related information is displayed on the screen of the kiosk 40, it has been exemplified that payment for withdrawal products is made through the kiosk 40 based on this, but is not limited thereto. As another example, the unmanned counter 10 may display payment-related information on the screen of a screen display device (not shown) connected to the unmanned counter 10 through the network 30, rather than the screen of the kiosk 40. .

Here, the screen display device (not shown) means a terminal capable of displaying payment-related information, and is exemplarily PCS (Personal Communication System), GSM (Global System for Mobile communication), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (WCode Division Multiple Access), Wibro (Wireless Broadband Internet) terminal, It may include all types of wired and wireless communication devices such as a smart phone, a smart pad, a tablet PC, a laptop computer, a wearable device, and a desktop PC, but is not limited thereto.

In addition, the screen display device (not shown) may be, for example, a device having a payment service providing function such as a simple payment function, a card payment function, and a cash payment function. According to this, the kiosk 40 may be understood as a concept included in, for example, a screen display device (not shown).

For example, the unmanned shelf 10 may display payment-related information on the screen of a screen display device (not shown), and based on this, the customer 1 using the screen display device (not shown) can display the unmanned shelf 10 Payment can be made for withdrawal products withdrawn from A more detailed description of the unmanned shelf 10 will be described later.

The customer 1 refers to a user who wants to purchase a product displayed on the unmanned shelf 10, and may be referred to as a customer, a product buyer, or a consumer.

The management server 20 may refer to a server operated/managed by the manager 2 . The manager 2 is a user who installs, operates, manages inventory, and analyzes information related to products for sale (for example, sales information and sales data to be described later) of the unmanned shelf 10, and may be otherwise referred to as a seller. can

The management server 20 may transmit and receive data through the unmanned shelf 10 and the network 30 . The management server 20 may analyze information (for example, sales information) provided from the unmanned stand 10 and perform management of the unmanned stand 10 .

The management server 20, for example, when receiving sales information from the unmanned shelf 10, analyzes the provided sales information to identify preferred products for each customer, identify customer consumption patterns, and sell current status of products in the unmanned shelf 10 in real time. It is possible to perform real-time identification, identification of product inventory in the unmanned shelf 10, inventory management, and the like.

In this system 100, the network 30 includes, for example, a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) network, a World Interoperability for Microwave Access (WIMAX) network, the Internet, and a local area LAN (LAN). Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth network, NFC (Near Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB ( Digital Multimedia Broadcasting) networks, wireless communication networks, etc. may be included, but are not limited thereto. In addition, in the present system 100, the network 30 may be a cloud.

In the present system 100, the unmanned counter 10, the management server 20, and the kiosk 40 (or screen display device) can interlock with each other through the network 30 to transmit and receive data.

In one example with reference to FIG. 1, it is illustrated as a case in which one unmanned shelf 10 is included in the system 100, but this is only one example to help the understanding of the present application, but is not limited thereto, and the present system ( 100) may include a plurality of unmanned shelves. In this case, the management server 20 may manage each of the plurality of unmanned counters by interworking with the plurality of unmanned counters through the network 30 . Hereinafter, the unmanned shelf 10 will be described in more detail.

The unattended shelf 10 may include an authentication unit 11 , a product management unit 12 , a control unit 13 and a provision unit 14 . At this time, the components of the authentication unit 11, product management unit 12, control unit 13, and provision unit 14 may be collectively expressed as a gateway unit of an unmanned sales system.

The authentication unit 11 may perform user authentication for a user 1 who wishes to use the unmanned counter 10 . User authentication may be performed by, for example, at least one of image analysis-based authentication and QR code recognition-based authentication.

Regarding image analysis-based authentication, the authentication unit 11 may obtain a user-captured image of the user 1 for user authentication, for example. At this time, the authentication unit 11 includes a camera provided on one area of the outer surface (eg, front) of the unmanned shelf 10, a camera provided on one area of the outer surface (eg, front) of the kiosk 40, and a tablet PC. A user photographed image of the customer 1 may be obtained through at least one of cameras provided in the same screen display device (not shown).

At this time, it is exemplarily assumed that the unmanned shelf 10 is installed in a fitness facility, and that only members registered as members of the fitness facility can use the unmanned shelf 10. In this case, the unmanned counter 10 may include a member management database (not shown), although not shown in the drawing, for user authentication. The member management database (not shown) may have pre-registered and stored member information of users (that is, members) who can use the fitness gym or the unmanned counter 10 . Here, member information may include, for example, images, names, ages, etc. of users, but is not limited thereto and may be set in various ways by a setting manager.

The authenticating unit 11 is configured to use customer 1 corresponding to a previously obtained user-photographed image based on member information (particularly, member image) of users (members) pre-registered in a member management database (not shown). User authentication (in other words, member authentication) can be performed for the user.

When the user-captured image is acquired, the authentication unit 11 compares the user-captured image with images of a plurality of members pre-stored in a member management database (not shown), so that the user 1 corresponding to the user-captured image manages the member. It may be checked whether the user is a member already registered in a database (not shown). At this time, the authentication unit 11 may use feature point matching technology as an example when comparing images of a plurality of members and images taken by a user, but is not limited thereto, and recognizes various images to determine whether objects in the two images match. technology, face recognition technology, etc. may be applied. At this time, as the image recognition technology (or face recognition technology), various technologies previously known or developed in the future may be applied.

As a result of comparison between the two images, the authentication unit 11 confirms that the user 1 corresponding to the user-captured image is a member previously registered in the member management database (not shown) (ie, an image matching the user-captured image). If exists in the member management database), it may be determined that user authentication (for example, member authentication) for the using customer 1 corresponding to the user-captured image is normal authentication.

In relation to QR code recognition-based authentication, the authentication unit 11 is, for example, a screen display provided on one area of the outer surface (eg, the front) of the unmanned shelf 10, the outer surface (eg, the front) of the kiosk 40 A QR code for user authentication may be displayed on the screen display unit on at least one of a screen display unit provided in one area of the screen display unit and a screen display unit provided in a screen display device (not shown) such as a tablet PC. In this case, the screen display unit is a screen that can be displayed and may mean a screen such as a touch panel or a monitor.

When the using customer 1 intends to perform QR code recognition-based authentication, the user 1 takes a picture (photographs) of the QR code displayed on the screen display unit using a portable terminal possessed by the customer 1 to obtain information about the QR code. recognition can be performed. Here, the portable terminal is a terminal having portability/mobility and may mean all types of wired and wireless communication devices such as smart phones, tablet PCs, and wearable devices.

When the using customer 1 recognizes the QR code with his or her mobile terminal, user authentication (identity authentication) for the using customer 1 can be performed through the mobile terminal. For example, the customer using 1 recognizes the QR code and then inputs the user's personal information (eg, name, phone number, or simple password set in conjunction with financial payment approval) through a mobile terminal. Authentication (identity authentication) can be made. When the contents of the personal information entered by the user and the actual customer information on the portable terminal possessed by the customer 1 match, the authentication unit 11 performs user authentication (for example, identity authentication) for the customer 1. ) can be determined to be normal authentication.

Through the authentication method based on QR code recognition, it is possible to authenticate users (members) already registered in the member management database (not shown), as well as users who are not registered in the member management database (not shown). Authentication for (that is, authentication for customers using the unmanned counter 10 for the first time) may also be performed.

After user authentication (member authentication or identity authentication) is performed by the authentication unit 12 in the product management unit 12, the user 1 selects the products displayed on the unmanned shelf 10 and displays the unmanned shelf 10 ) It is possible to determine whether there is a product withdrawn from (ie, a product withdrawn).

The product manager 12 may control the open/closed state of the door of the unmanned shelf 10 . In other words, the product management unit 12 may control ON/OFF of the door lock of the unmanned store 10 . When the door lock is ON, the door of the unmanned stand 10 is controlled to be openable, and when the door lock is OFF, the door of the unmanned stand 10 may be controlled to be closed (ie, the door cannot be opened).

The product management unit 12 allows withdrawal of products (displayed products, displayed products) in the unmanned shelf 10 only when user authentication (member authentication or identity authentication) is confirmed as normal authentication in the authentication unit 11. The door of the unmanned shelf 10 may be controlled to be openable. That is, the product management unit 12 controls the door lock of the unmanned display stand 10 to be ON only when user authentication is confirmed as normal, so that the door of the unmanned display stand 10 can be opened. can

When the door of the unmanned shelf 10 is controlled to be openable, the customer 1 opens the door of the unmanned shelf 10 and selects a product that he or she wishes to purchase from among the products displayed in the unmanned shelf 10. After taking it out, the door of the unmanned shelf 10 can be closed. Here, a product taken out from the unmanned counter 10 by the customer 1 as the customer 1 desires to purchase the product may be referred to as a withdrawal product. Exemplarily, when 10 beverages are displayed as products displayed in the unmanned shelf 10, and the customer 1 takes out to purchase two beverages, the withdrawal product is unattended by the customer 1. It may refer to the corresponding two beverages taken out from the shelf 10.

When the product management unit 12 determines that the door is closed after detecting the door open in a state where the door of the unmanned shelf 10 is controlled to be openable, the product manager 12 photographs the products displayed on the unmanned shelf 10. It is possible to determine whether there is a product to be withdrawn by acquiring and analyzing a product photographing image from an internal camera installed in one area inside the unmanned shelf 10 . At this time, the fact that the door is closed after the door is sensed while the door is controlled to be openable means that the customer 1 opens the door of the unmanned shelf 10 to purchase a product and takes out the desired product. It can be predicted that he would have taken the act of closing the door afterwards. Accordingly, in this case, the product management unit 12 may obtain a product photographed image from an internal camera to determine whether there is a product to be withdrawn.

The product management unit 12 performs an analysis of comparing a first product photographed image obtained from the internal camera before the door opening is detected and a second product photographed image obtained from the internal camera after the door is closed. It is possible to determine whether a withdrawal product exists. Here, before the door opening is sensed (or when the first product photographed image is acquired) may exemplarily mean a time when the door of the unmanned shelf 10 is controlled to be in an openable state. In addition, after the door closing is detected (or when the second product shooting image is acquired) may exemplarily mean a time after a preset time (eg, 2 seconds) from the time when the closing of the door is detected. there is.

As described above, the product management unit 12 may determine whether there is a product to be withdrawn using a product photographing image obtained from an internal camera installed in one area inside the unmanned shelf 10 .

Inside the unmanned shelf 10, an internal camera may be installed in one area to take pictures of products displayed on the unmanned shelf 10. At this time, the number of internal cameras corresponding to the number of shelves in the unmanned shelf 10 may be installed. For example, when there are 5 shelves in the unmanned display stand 10, one internal camera may be installed per shelf in the unmanned display stand 10, so that a total of 5 internal cameras may be installed. That is, one camera may be installed at the top of the unmanned shelf 10 for each shelf.

For example, each internal camera may be installed on top of products displayed on each shelf so that the displayed products may be photographed from above. However, it is not limited thereto, and each internal camera may be installed anywhere in the internal area of the unmanned shelf 10 as long as it can recognize all products displayed on each shelf.

FIG. 2 is a diagram for explaining a process in which the product management unit 12 determines whether a withdrawal product exists in an unmanned shelf 10 according to an embodiment of the present disclosure.

Referring to FIG. 2 , the product management unit 12 obtains a first product shooting image 21 and a second product shooting image (from an internal camera installed in one area inside the unmanned shelf 10) in order to determine whether there is a product to be withdrawn. 22) can be obtained. Here, the first product shooting image 21 may be an image obtained before the door opening is sensed, and the second product shooting image 22 may be an image obtained after the door closing is sensed.

The product management unit 12 may compare the obtained first product captured image 21 and the second product captured image 22 to determine whether a product to be withdrawn exists.

As a specific example, the product manager 12 may confirm that a total of 10 beverages (including water) are displayed as products displayed in the unmanned shelf 10 through the first product photographed image 21 . In other words, the product management unit 12 may identify products corresponding to 10 beverages in the corresponding image through image analysis of the first product photographed image 21 . Meanwhile, according to the second product photographed image 22 , it can be confirmed that a total of 8 beverages are displayed as displayed products in the unmanned shelf 10 . That is, the product management unit 12 may identify products corresponding to 8 beverages in the corresponding image through image analysis of the second product photographed image 22 .

As a result of the product management unit 12 comparing the first product shooting image 21 and the second product shooting image 22 with each other, two drinks 31 and 32 among 10 drinks in the first product shooting image 21 It can be confirmed that is not found in the second product photographed image 22 . In this case, with respect to the two beverages 31 and 32 products found in the first product photographed image 21 but not found in the second product photographed image 22, the product management unit 12 determines the corresponding two beverages. (31, 32) It is possible to recognize the product as a withdrawal product selected by the customer 1 and withdrawn from the unmanned counter 10. That is, the product manager 12 takes out the corresponding two beverage products 31 and 32 from the unmanned shelf 10 in hopes of purchasing them among the products displayed on the unmanned shelf 10 by the customer 1 using the product. (withdrawal product). In this case, the product management unit 12 may determine that a product to be withdrawn exists (in particular, two products to be withdrawn) exist as a result of comparative analysis of the two product photographed images 21 and 22 .

When the product management unit 12 performs image analysis on each of the two product captured images 21 and 22 or analysis on comparison between the two product captured images 21 and 22, various image analysis algorithms known or developed in the future are used. A comparative analysis based on this can be performed.

If it is determined that there is a product to be withdrawn, then the product management unit 12 takes the learning image in a plurality of product types as an input value and the product information matched with each learning image in the plurality of product types as an output value based on deep learning. Based on the details (that is, the pre-learned deep learning model), it is possible to generate detailed information including product information of the product to be withdrawn using the product photographed image.

In particular, the product management unit 12 may generate detail information including product information of the product to be withdrawn through the pre-learned deep learning model by providing an image of the product to be withdrawn as an input value of the pre-learned deep learning model. . At this time, referring to FIG. 2 , for example, an image of a product to be retrieved provided as an input value of a pre-learned deep learning model was found in the first product shooting image 21 but not in the second product shooting image 22. It may refer to images of two beverages 31 and 32 that have not been found. That is, the image of the withdrawal product provided as an input value may refer to a partial image obtained by extracting only portions corresponding to each of the two beverages 31 and 32 from the entire image of the first product photographing image 21 .

The product information of the product to be withdrawn in the detail information generated by the product management unit 12 may include information about the product name and price of the product to be withdrawn. In addition, the details information may include product information (product name, price), quantity (number), and information on the total payment amount of the product withdrawn. In addition, as an example, the detailed information includes user ID (ID of the user), device name (ID of the unmanned device), information on the branch where the unmanned stand is installed (store information), time when the door of the unmanned stand was opened, and the door of the unmanned stand Information about the closing time and the time when the history information was created may be included.

The deep learning model used by the product management unit 12 inputs a plurality of learning images for each of a plurality of product types (ie, different products having at least one different product type, size (capacity), and brand name) as input values. , and may be a pre-learned model that has been trained to have product information (product name, price) matched with each of a plurality of learning images as an output value. According to this pre-learned deep learning model, when an image of a new product is given as an input value, product information (product name, price) of the product in the input image can be derived.

As an example, the deep learning model may mean an artificial intelligence (AI) algorithm model, a machine learning (machine learning) model, a neural network model (artificial neural network model), a neuro fuzzy model, and the like. However, it is not limited thereto, and deep learning models applicable to this system 100 are illustratively convolutional neural networks (Convolution Neural Network, CNN), Recurrent Neural Network (RNN), deep learning Various neural network models previously known or developed in the future, such as a deep neural network, may be applied.

The product management unit 12 may use a deep learning model (ie, a pre-learned deep learning model) learned through the collected data (learning data) to detect a product to be withdrawn (a product withdrawn). If it is determined that there is a product to be withdrawn, the product management unit 12 may generate detailed information including product information of the product to be withdrawn using a pre-learned deep learning model.

According to the foregoing, for example, the product management unit 12 can determine whether or not there is a withdrawal product through comparative analysis between the two product photographed images 21 and 22, and deep learning-based pre-learning items (ie, pre-learning deep learning model), it is possible to recognize information about the type of product to be withdrawn (ie, product information of the product to be withdrawn or detail information including product information). This application provides for the unmanned shelf 10 using deep learning (deep learning model) based vision recognition technology.

When the product management unit 12 determines that a product to be withdrawn exists, the control unit 13 may display information related to payment for the product to be withdrawn on the screen.

At this time, as described above, for example, if the unmanned counter 10 and the kiosk 40 are interlocked with the network 30, the controller 13 displays payment-related information on the screen of the kiosk 40, and the unmanned counter 10 ) and the screen display device (not shown) are interlocked with the network 30, payment related information can be displayed on the screen of the screen display device (not shown). As another example, when the unmanned shelf 10 is a shelf having a function of providing payment services such as a simple payment function, a card payment function, and a cash payment function on its own, the control unit 13 may display a screen provided in the unmanned shelf 10 itself. Payment-related information may be displayed on the display unit.

The payment-related information may include detail information of the withdrawal product and payment request information for requesting payment of an amount corresponding to the detail information. As described above, the detail information of the withdrawal product is information generated by the product management unit 12, and may include product information (product name, price), quantity (number), and information on the total payment amount of the withdrawal product. The payment request information may include confirmation information for confirming the total payment amount and payment method selection information for selecting a calculation method for the total payment amount. Here, the payment method selection information may include simple payment, card payment, cash payment, etc., but is not limited thereto, and various payment methods may be applied.

For example, when the authentication unit 11 performs user authentication, if user authentication (eg, member authentication) based on a user-captured image is normally performed, this means that only members are authenticated, so the control unit 13 operates an unmanned display stand. When the closure of (10) is detected, the details information of the withdrawal product is displayed on the screen, and when the user confirms the displayed detail information (ie, when the user inputs the confirmation information confirming the total payment amount) ) By additionally providing payment request information on the screen, it is possible to induce payment for withdrawal products (that is, induce users to pay for withdrawal products).

As another example, when user authentication is performed by the authentication unit 11, user authentication (eg, member authentication) based on a user photographed image is normally performed, but at this time, user authentication is performed in a member management database (not shown). Assume that financial payment approval information (that is, payment information set so that payment is automatically made when a user is authenticated with a user-captured image) is stored in advance in relation to a user 1 . In this case, the control unit 13, when detecting the closing of the door of the unmanned shelf 10, displays detail information of the withdrawal product on the screen, and when the user confirms the displayed detail information (i.e., confirms the total payment amount) When a user inputs confirmation information to be requested), payment for a withdrawal product may be automatically completed based on financial payment approval information pre-stored in a member management database (not shown).

As another example, when user authentication is performed by the authentication unit 11, user authentication (eg, identity authentication) based on QR code recognition is normally performed, but at this time, after recognizing the QR code, for example, the user Assume that user authentication is performed using a simple password set in association with financial payment approval as personal information. In this case, the control unit 13, when detecting the closing of the door of the unmanned shelf 10, displays detail information of the withdrawal product on the screen, and when the user confirms the displayed detail information (i.e., confirms the total payment amount) When the user inputs the confirmation information to be confirmed), the payment for the withdrawal product can be automatically completed based on the information (simple password) previously authenticated through QR code recognition.

When it is determined that the payment for the payment-related information displayed on the screen has been completed, the provision unit 14 recognizes the withdrawal product for which the payment has been completed as a sold product, and transfers the sales information related to the product to the management server 20. can provide In addition, if it is determined that the payment is completed, for example, the providing unit 14 may provide payment information to the portable terminal possessed by the customer 1 .

Here, the sales information includes data related to sales products, customer data related to sales products for customer-tailored analysis (or provision of customized information, service provision), and residual products remaining on the unmanned shelf 10 due to the sale of sales products. Inventory data may be included.

Sales product-related data is information on the details of products sold (that is, information about withdrawal products, which means product information (product name, price), quantity (number), and total payment amount) of products sold It may include sales time information (ie, payment completion time information), and installation information of the unmanned stand 10 where the sale product was displayed. Installation information of the unmanned shelf 10 may include installation location (address) information, store (branch, store) information, and the like. The management server 20 analyzes/predicts product sales demand for each store through analysis using data related to sales products, aggregates sales information for each store, information about time zones in which products sell well, and which products are sold well in a geographical area. It is possible to grasp information such as sales status and trends in customer visits.

Customer data related to a product for sale may refer to information about a customer who purchases a product for sale. Customer data may include, for example, information such as age and gender of the customer, excluding information such as name or mobile phone number to ensure anonymity. The management server 20 can identify information such as products preferred by a specific age group, preferred products according to gender, customer preferences, analysis of customer consumption patterns, etc. through analysis using customer data, and customized service can be provided.

Inventory data is information about the remaining products displayed in the unmanned shelf 10 after the sale of the sale product, and may include information such as the type and number of the remaining products. The management server 20 can grasp information such as products with high preference and products with low preference through analysis using inventory data, and also enables efficient operation of the unmanned shelf 10 by identifying inventory in real time. can do.

In addition, the sales information may include weather information when the sale product is sold (when payment is completed). To this end, the management server 20 or the unmanned counter 10 may be linked with a weather server (not shown) through the network 30 .

For example, when the unmanned sales stand 10 is interlocked with a weather server (not shown), the unmanned sales stand 10 presents weather information (weather information at the time payment is completed) from the weather server when it is determined that payment is completed. may be received and provided to the management server 20 together with sales information. As another example, when the management server 20 is interlocked with the weather server, the management server 20 receives sales information from the unmanned shelf 10 (at this time, the sales information includes only sales product-related data, customer data, and inventory data). may be available), the weather information may be obtained by receiving the weather information at the time of receiving the sales information (ie, the time when the payment is completed) from the weather server through the network 30 . The management server 20 may grasp information such as preferred products by temperature/season/humidity through analysis using weather information.

As described above, the management server 20 analyzes the sales information provided from the providing unit 14 to identify preferred products for each customer, identify customer consumption patterns, identify product sales status in the real-time unmanned shelf 10, and real-time unmanned It may be possible to perform at least one of product stock checking and stock management in the shelf 10 .

In addition, when it is determined that the payment is completed, the provider 14 checks the inventory of the remaining products in the unmanned shelf 10, and as a result of checking the inventory, the inventory quantity of any one product type among the product types of the remaining products. If the quantity is less than or equal to the preset quantity, an order request message for inventory management for replenishing a product of any one product type for the unmanned shelf 10 may be provided to the management server 20 . The manager 2 can efficiently manage inventory through the order request message for inventory management.

At this time, the provision unit 14 performs stock identification using the previously-trained deep learning model when performing inventory identification, or a product photographed image obtained from an internal camera installed inside the unmanned shelf 10 (ie, the first Inventory may be identified through image analysis of the first/second product image).

For example, the providing unit 14 may perform stock checking whenever door closing is sensed. However, it is not limited thereto, and the providing unit 14 may perform stock checking in response to, for example, when a stock checking request signal is received from the manager 2 .

Illustratively, it is assumed that four product types (first to fourth product types) are displayed as a plurality of product types in the unmanned shelf 10 . At this time, in the unmanned shelf 10, as the first product type, 10 500ml bottled water of brand a, as the second product type, 10 exercise creams of brand c, and as the third product type, brand c for exercise Assume that there are 10 taping products (large size) and 10 sports taping products (small size) of c brand as the fourth product type. Also, assume that the preset quantity is set to three for each of the four product types.

Here, among a total of 40 products displayed on the unmanned shelf 10, as the product corresponding to the first product type (ie, 500ml bottled water of brand a) is sold to a large number of customers, 3 items remain in the unmanned shelf 10 Let's say there is. In this case, the provision unit 14 may provide an order request message for inventory management to the management server 20, since it is determined that the inventory quantity of the first product type is 3 or less, which is a preset quantity, as a result of performing the inventory check. there is. At this time, the order request message is illustratively, 'There are only 3 products of the first product type left on the unmanned shelf (10) from the existing 10. Please supply more'.

According to the present application, as the unmanned shelf 10 can provide an order request message for inventory management to the management server 20 according to the stock level after performing inventory checking on its own, when identifying stock conventionally displayed on the shelf It is possible to solve problems (that is, it takes a long time, the accuracy is low, and it is difficult to grasp the inventory of products in the rack in real time) of the method of manually checking inventory by a rack manager (for example, an employee). That is, according to the present application, the unmanned shelf 10 can provide an order request message for inventory management to the management server 20, so that the manager 2 can immediately grasp the product inventory in the unmanned shelf 10 in real time. can do. In addition, according to the present application, as the unmanned shelf 10 performs inventory identification through image analysis of a pre-learned deep learning model or an acquired product photographic image, more accurate inventory identification is possible.

In addition, the present application automatically controls the door lock ON / OFF of the unmanned shelf 10 based on user authentication, determines whether there is a product withdrawn, identifies detailed information including product information of the product withdrawn, and processes payment for the product withdrawn By providing an unmanned shelf 10 that allows to store, it is possible to completely unmanned (TOTAL CASHIER-FREE) at a relatively low cost, and accordingly, in the case of a conventional shelf, a cashier must be placed 24 hours a day to sell products 24 hours a day, resulting in high labor costs. You can solve the problem you are having.

This unmanned shelf 10 may be, for example, a shelf (vending machine) installed in a fitness facility. A plurality of various product types (ie, a plurality of product types) such as beverages, exercise cream, and exercise taping products may be displayed in the unmanned shelf 10 .

The present system 100 can provide various products (products of a plurality of product types) through one platform (ie, this unmanned shelf). That is, the present system 100 has the advantage of being able to sell products of various packages on one unmanned shelf 10 .

3 is a view showing an example of products displayed on an unmanned shelf 10 according to an embodiment of the present application.

Referring to FIG. 3 , the unmanned shelf 10 may be, for example, a shelf capable of displaying on three floors because it has three shelves. If the unmanned shelf 10 is, for example, a shelf installed in a gym, etc., beverages, protein supplements of different capacities, exercise tapings of different sizes, and the like may be displayed on each of the shelves as products having a plurality of product types. Placement positions of products in the unmanned display stand 10 may be variously set/changed by a user (eg, a manager) who manages the unmanned display stand 10 .

Meanwhile, facial images of members using fitness facilities may be pre-registered in a member management database (not shown) in the unmanned shelf 10 . At this time, when the user-captured image is acquired, the authentication unit 11 performs a matching analysis with images registered in a member management database (not shown), so that the user 1 corresponding to the user-captured image is registered in the member management database ( (not shown) may perform user authentication as to whether or not a member already registered with the user is correct. When it is determined that the member is correct as a result of performing user authentication, the product management unit 12 may control the door lock of the unmanned display stand 10 to be ON to control the door of the unmanned display stand 10 to be openable. Thereafter, the customer 1 can open the door of the unmanned counter 10, select and take out a product to purchase, and close the door of the unmanned counter 10. Thereafter, when the unmanned shelf 10 detects the closing of the door of the unmanned shelf 10, image analysis or pre-learned deep learning model is used to determine whether or not there is a product to be withdrawn, or to provide detailed information including product information of the product to be withdrawn. Derivation can be performed, and based on the derived detail information, it can be provided so that payment for withdrawal products can be made. For example, the unmanned counter 10 may provide automatic payment for the withdrawal product taken out by the customer 1 when closing of the door is sensed.

This unmanned shelf 10 combines artificial intelligence (AI) technology to perform recognition of displayed products through an internal camera, and determines that the product has been sold as payment for the product withdrawn among the displayed products is completed. If so, sales information related to sold products may be provided to the management server 20 . Through this, the unmanned shelf 10 enables the manager 2 of the management server 20 to analyze the received sales information (including sales product related data, customer data, inventory data, etc.) It can help to perform various analysis, inventory management, etc. efficiently. In addition, the unmanned shelf 10 performs inventory check and when it is determined that the quantity of a specific product in the unmanned shelf 10 is insufficient, the management server 20 provides an order request message for inventory management to the manager 2 to inventory the inventory. You can place an order request.

The present application proposes the present system 100, which is an unmanned sales system based on deep learning (deep learning model) or artificial intelligence (AI). In addition, the present system 100 may be a system for managing a cloud (or network) based AI sales stand (ie, an unmanned shelf). The system 100 and the management server 20 are based on information (sales information) obtained from the unmanned shelf 10, such as customer analysis, inventory management, order management, payment management, digital signage management, S/W platform management, etc. can be performed.

The system 100 may support simple payment commonly referred to as QR code scanning. The system 100 may provide a simple product sales service and enable easier product operation/management. In addition, the present system 100 may provide a web-based management system capable of real-time sales and stock checking through the unmanned shelf 10 . In addition, the system 100 enables creation of customer group/personally customized advertisements through customer (user) identification based on analysis of sales information, and enables customer attraction through coupons and vouchers.

This system 100 analyzes sales information so that the manager 2 can determine sales volume/quantity by product, product sales ranking, device (unmanned shelf) sales status, total sales volume/amount, total sales volume/number, number of orders, etc. information can be grasped.

In the present system 100, the unmanned shelf 10 may perform product recognition by processing and labeling a product photographed image when it is acquired, and using a pre-learned deep learning model. The manager 2 may be otherwise referred to as a seller, and may perform installation, operation, inventory management, and analysis of sales information (sale data) of the unmanned shelf 10 through the management server 20 . The using customer 1 is a customer who uses the unmanned shelf 10 and may be referred to as a consumer, and for example, purchases products displayed on the unmanned shelf 10 through simple mobile payment (simple payment through QR code recognition). can

This unmanned shelf 10 can provide a high product (product) recognition rate with an accuracy of 99% or more through deep learning-based computer vision recognition technology. In addition, the present unmanned shelf 10 has a simple device configuration, so the shelf price is cheaper than conventional vending machines (eg, mechanical and RFID methods). In addition, the present unmanned shelf 10 has an advantage in that additional variable costs do not occur because inventory can be identified based on product (product) image learning. That is, as the present unmanned shelf 10 does not require human intervention in inventory management, it has the advantage of being able to operate the unmanned shelf 10 at a low cost.

In addition, the present system 100 can analyze the current status of sales associated with customer data by considering customer data as sales information. That is, the system 100 can analyze sales information including customer data, analyze preferences and consumption patterns for each customer, and achieve operational efficiency through real-time inventory identification.

Conventional counters (sales counters, vending machines, etc.) have difficulties in being completely unmanned (TOTAL CASHIER-FREE), and there are limitations in spreading and distributing counters due to high-cost installation problems. There is a problem in that the cost required for installation/maintenance of conventional racks requires a high budget compared to the cost required for installation/maintenance of the deep learning-based unmanned rack 10 provided by the present application. Since the present application can manage and maintain the unmanned shelf 10 based on deep learning, it is possible to spread at a lower cost.

In addition, this unmanned shelf 10 can be interlocked with simple payment and unmanned kiosks. As the unmanned counter 10 is interlocked with simple payment services such as Kakao Pay and Payco, for example, it can be provided so that the customer 1 can make a payment more easily and conveniently when paying for a product. This unmanned shelf 10 can be linked with major simple payment services that have recently been expanding their use offline. Through this, the unmanned counter 10 can make payment automatically at the same time as the door of the unmanned counter 10 is closed, so that faster and more convenient payment can be supported.

In addition, the present unmanned counter 10 may provide payment to be made through linkage with a pre-installed unmanned kiosk or dedicated kiosk. As this unmanned shelf 10 can be easily linked with existing unmanned kiosks, the service of customer stores can be made more efficient. Accordingly, customers who are already accustomed to unmanned kiosks can easily use this unmanned kiosk without any inconvenience. can make it possible In addition, the unmanned counter 10 may support payment services such as credit card payment through the kiosk 40 .

The unmanned shelf 10 may control the door of the unmanned shelf 10 to be opened, for example, when the customer 1 performs user authentication through QR code recognition. Thereafter, the customer 1 can open the door of the unmanned counter 10 to take out a desired product and close the door. When the unmanned counter 10 detects that the door of the unmanned counter 10 is closed, automatic payment for the product taken out by the customer 1 can be made through simple mobile payment.

This system 100 can be affiliated with simple payment providers. This system 100 can further increase customer accessibility through cooperation with more simple payment providers. Simple payment operators may exemplarily include Kakao Pay, Payco, Naver Pay, etc., but are not limited thereto, and the system 100 provides various simple payment service functions that currently exist or are developed in the future. can be affiliated with

This system 100, for example, when the unmanned counter 10 is equipped with a payment service providing function by itself, the payment can be made on its own without the need for a kiosk 40 or a screen display device (not shown), allowing customers to use Payment can be made more conveniently. In addition, this system 100 can expand partnerships with various simple payment providers, and can be easily used by overseas users (overseas customers) through partnerships with global simple payment providers.

The present application can maximize space utilization by installing the unmanned shelf 10 in an existing store. This unmanned shelf 10 may be installed in a relatively small area. Therefore, since the electrical/wiring work has already been done for the installation of the existing system (conventional shelf installation) in the existing store, the present application does not need additional budget to install the unmanned shelf 10, maximizing space utilization. can do.

In addition, the present application can provide a publicity effect for customers as a new product sales channel. Through the provision of this system 100, the present application can provide new products tailored to customer composition by season and store to be sold through deep learning learning (deep learning product learning) without restrictions. In addition, the present application may provide data for analyzing the sales effect of a new product through the provision of sales information.

In addition, the present application can enhance manager and customer convenience with a mobile-based service. The present application can increase customer convenience with existing service apps or mobile simple payments. In addition, the present application may increase the efficiency of management tasks, and illustratively, real-time inventory check and price promotion for same-day inventory exhaustion may be performed immediately.

In addition, the present application can contribute to increasing sales through linkage with existing systems and big data. The present application may interwork with, for example, a conventional remote store management system related to product sales. We will perform analysis based on integrated information (ie, sales information) by integrating payment data (i.e., sales product-related data) with market information data, weather, events, anonymized customer data based on customer characteristics, etc. Based on this, it is possible to acquire (create) basic data for increasing sales based on customer analysis.

In addition, the present application can secure / increase customer accessibility through linkages with major tourist attractions, golf courses, gyms, gyms, etc. In addition, the present application can be made possible to sell products outside of business hours through the provision of the unmanned shelf 10, and thus an effective business extension effect can be expected.

The authentication unit 11 of the unmanned shelf 10 may perform identification/identification of a user (customer) through integration with a face recognition function. By identifying the customer through user authentication, the unmanned shelf 10 determines (approves) whether to open the door (door) of the unmanned shelf 10 to the customer 1 (ie, whether to allow the door to be opened). )can do. When user authentication is confirmed as normal authentication, the unmanned shelf 10 may control the door of the unmanned shelf 10 to open the door so that the customer can take out products from the unmanned shelf 10 .

The management server 20 may perform business using customer information, for example, by analyzing sales information provided from the unmanned counter 10 . For example, the management server 20 may receive sales information including a user-captured image from the unmanned shelf 10, and in this case, image analysis for analyzing customer face characteristics based on the user-captured image (Face features) For example, feature point analysis) may be performed to identify characteristics such as age and gender of the corresponding customer, and analysis information may be anonymized as an example. In addition, the management server 20 integrates and links the identified customer information (for example, age, gender, etc.) with sales information (sales product related data, inventory data, etc.) to secure more useful customer data. can

The unmanned counter 10 and the management server 20 may grasp (recognize) information of the user 1 corresponding to the user-captured image by, for example, performing image analysis on the obtained user-captured image. At this time, as the information of the customer 1 that can be identified (recognized), whether or not he is a regular customer (or whether he is a new customer), gender, age, current emotion, whether he is wearing glasses, whether he is wearing a mask, etc. can be identified. . The information of the customer 1 may be more easily utilized when the management server 20 performs customer-tailored analysis using the sales information.

The system 100 and the management server 20 can perform more diverse analyzes by combining existing data of the customer company with sales information provided by the unmanned counter 10 . For example, the management server 20 includes information (eg, age, age, etc.) of the user 1 identified through image analysis of a user-captured image, various market information data, weather/event information, sales information, It is possible to comprehensively acquire electronic payment data and conduct multiple analyzes on the acquired data to predict trends and identify sales trends. Based on this, the management server 20 may derive and provide information such as aggregating sales by store, identifying trends in customer visits, benchmarking with other sales counters (other unmanned counters), and recommending methods for increasing sales.

In the unmanned shelf 10, for example, a deep learning model for object recognition as a deep learning model, for example, a learning model based on detectron 2, etc. may be applied. In addition, in performing product recognition (object recognition) using a deep learning model, the unmanned shelf 10 can process acquired data (ie, product image) for a more accurate recognition rate and can automate this processing. there is.

Hereinafter, based on the details described above, the operation flow of the present application will be briefly reviewed.

4 is an operational flowchart of a method for driving an unmanned shelf according to an embodiment of the present disclosure.

The driving method of the unmanned shelf illustrated in FIG. 4 may be performed by the previously described unmanned shelf 10 . Therefore, even if the contents are omitted below, the description of the unmanned counter 10 can be equally applied to the description of the driving method of the unmanned counter.

Referring to FIG. 4 , in step S11, the authentication unit may perform user authentication for a user who wishes to use the unmanned counter.

Next, in step S12, the product management unit may determine whether there is a withdrawal product selected by the customer and withdrawn from the unmanned counter among the products displayed at the unmanned counter after the user authentication is performed in step S11.

At this time, in step S12, the product management unit may control the door of the unmanned store to be openable so that products in the unmanned store can be taken out only when the user authentication is confirmed as normal.

In addition, in step S12, if the product management unit determines that the door is closed after the door is detected while the door is controlled to be openable, products displayed on the unmanned shelf are photographed in one area inside the unmanned shelf. It is possible to determine whether there is a product to be withdrawn by acquiring and analyzing a product photographed image from an internal camera installed in .

In addition, in step S12, the product management unit compares the first product photographed image obtained from the internal camera before the door opening is detected with the second product photographed image obtained from the internal camera after the door is closed. can determine if it exists.

In addition, in step S12, the product management unit takes the learning image in a plurality of product types as an input value and takes product information based on deep learning based on the product information matched with each of the product types as an output value. It is possible to create detail information including product information of the determined withdrawal product using the image.

Next, in step S13, if it is determined that the withdrawal product exists in step S12, the controller may display payment related information for the withdrawal product on the screen. Here, the payment-related information may include detail information of the withdrawal product and payment request information for requesting payment of an amount corresponding to the detail information.

In addition, although not shown in the drawing, in the driving method of the unmanned counter according to an embodiment of the present application, after step S13, when the provider determines that the payment for the payment-related information has been completed, the product for which the payment has been completed is sold. Recognizing the product as a product for sale and providing sales information related to the product for sale to a management server.

In this case, the sales information may include data related to sales products, customer data related to sales products for customer-customized analysis, and inventory data regarding remaining products remaining on the unmanned shelf due to sales of the sales products.

In addition, in the providing step, if it is determined that the payment has been completed, the providing unit performs stock checking on the remaining products in the unmanned shelf, and as a result of performing the stock checking, the stock quantity of any one product type among the product types of the remaining products is determined. If it is less than the set quantity, an order request message for inventory management for replenishment of the product of any one product type for the unmanned shelf may be provided to the management server.

At this time, the management server may be able to perform at least one of identification of preferred products by customer, identification of customer consumption patterns, identification of sales status, identification of inventory, and inventory management through analysis of the sales information provided from the provider.

In the foregoing description, steps S11 to S13 may be further divided into additional steps or combined into fewer steps, depending on an embodiment of the present invention. Also, some steps may be omitted if necessary, and the order of steps may be changed.

The driving method of the unmanned rack according to an embodiment of the present application may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to act as one or more software modules to perform the operations of the present invention, and vice versa.

In addition, the above-described driving method of the unmanned shelf may be implemented in the form of a computer program or application stored in a recording medium and executed by a computer.

The above description of the present application is for illustrative purposes, and those skilled in the art will understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present application. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present application is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts thereof should be construed as being included in the scope of the present application.

100: unmanned sales system
1: Customer 2: Manager
10: unmanned shelf
11: authentication unit 12: product management unit
13: control unit 14: provision unit
20: management server
30: Network
40: kiosk

Claims (5)

As an unmanned stall,
an authentication unit performing user authentication for a customer who wishes to use the unmanned counter;
a product management unit that determines whether there is a withdrawal product selected by the customer and withdrawn from the unmanned shelf among products displayed at the unmanned shelf after the user authentication is performed; and
When it is determined that the withdrawal product exists, a control unit displaying information related to payment for the withdrawal product on a screen,
The payment related information includes detail information of the withdrawal product and payment request information requesting payment of an amount corresponding to the detail information. According to claim 1,
The product management department,
The unmanned shelf controlling the door of the unmanned shelf to be openable so that products in the unmanned shelf can be withdrawn only when the user authentication is confirmed as normal authentication. According to claim 2,
The product management department,
When it is determined that the door is closed after the door is opened while the door is controlled to be openable, the product from an internal camera installed in one area inside the unmanned shelf is photographed. An unmanned shelf that determines whether the withdrawal product exists by acquiring and analyzing a photographed image. According to claim 3,
The product management department,
It is determined whether the product to be withdrawn exists by comparing a first product photographed image obtained from the internal camera before the door opening is detected and a second product photographed image obtained from the internal camera after the door closing is detected. That's what you do, unmanned maedae. According to claim 3,
The product management department,
The retrieval determined using the product photographed image based on a deep learning-based pre-learning item with a learning image in a plurality of product types as an input value and product information matching each of the learning images in the plurality of product types as an output value. Create detailed information including product information of the product;
When it is determined that the payment for the payment-related information has been completed, a provision unit recognizing the withdrawal product for which payment has been completed as a sold product and providing sales information related to the sale product to a management server;
The sales information includes data related to sales products, customer data related to sales products for customer-customized analysis, and inventory data about remaining products remaining on the unmanned shelf due to the sale of the sales products,
The provisioning department,
If it is determined that the payment has been completed, inventory of the remaining products in the unmanned shelf is checked, and as a result of inventory checking, if the stock quantity of any one product type among the product types of the remaining products is less than a preset quantity, the An order request message for inventory management for product replenishment of the one product type for the unmanned shelf is provided to the management server.

Images