WO2020010821A1

WO2020010821A1 - Method and device for restocking management for automatic vending machine, and user terminal

Info

Publication number: WO2020010821A1
Application number: PCT/CN2018/125570
Authority: WO
Inventors: 黄鼎隆; 斯科特•马修•罗伯特; 马咪娜
Original assignee: 深圳码隆科技有限公司
Priority date: 2018-07-09
Filing date: 2018-12-29
Publication date: 2020-01-16
Also published as: CN108922026B; CN108922026A

Abstract

A method and device for restocking management for an automatic vending machine, and a user terminal. The method comprises: receiving payment completed information generated when a user makes a payment on the basis of payment information, updating an inventory record on the basis of a merchandise shopping list (S30); comparing the merchandise quantities corresponding to merchandise types in the updated inventory record with preset inventory thresholds (S40); and if the merchandise quantities corresponding to the merchandise types are less than the preset inventory thresholds corresponding to the merchandise types, then generating a restocking list comprising the merchandise types and the merchandise quantities corresponding to the merchandise types, thus facilitating restocking according to the restocking list (S50). When a user finishes shopping, by means of an image recognition technique, with respect to the result of comparing a recognized inventory record with preset thresholds, merchandises that need to be restocked are determined intelligently, thus implementing by means of image recognition the intelligent determination of the merchandises that need to be restocked in a restocking process; the method is simple, the efficiency of restocking is high, and user experience is great.

Description

Replenishment management method, device and user terminal of vending machine

This application is based on a Chinese patent application with an application number of 201810745691.6 and an application date of July 9, 2018, and claims its priority. The entire content of this application is hereby incorporated into this application as a whole.

Technical field

The present application relates to the technical field of computer vision product identification, and more particularly, to a replenishment management method, device, and user terminal for a vending machine.

Background technique

Material circulation is the most basic element of human society. The retail industry or related industries that are directly sold to consumers (hereinafter referred to as the retail industry or retail enterprises), as an important means of current material circulation, need to employ a large number of cashiers to achieve shopping settlement. Make the entry. After the entry is completed, complete the payment with the consumer.

The automatic vending container solves the shortcomings of manual entry and manual settlement in the retail industry. It is a consumer-oriented unmanned sales and automatic settlement sales method.

At present, the existing replenishment management methods of vending machines in vending containers are all managers who perform statistics on all the remaining different types of goods in the vending machine, and finally obtain the total quantity, thereby manually Statistics learned that which products need to be replenished and which products do not need to be replenished. During the replenishment process, it is easy to make mistakes due to manual statistics by the staff. The method is cumbersome, occupies a lot of labor, the replenishment efficiency is low, and the user experience not good.

Application content

In view of this, the present application provides a replenishment management method, device, and user terminal of a vending machine to solve the shortcomings of the prior art.

To solve the above problems, this application provides a replenishment management method for a vending machine, including:

Collecting a comparison image of the goods in the container of the vending machine, and identifying the comparison image of the goods, to determine the inventory record corresponding to the comparison image of the goods; the inventory record includes the product category and the number of products corresponding to the product category ;

Receiving an end-of-shopping instruction, collecting an update image of the goods in the container according to the end-of-shopping instruction, identifying the update image of the goods, generating a product purchase list based on the recognition result, and generating a payment corresponding to the product purchase list Information so that users can make payments based on the payment information;

Receiving payment completion information generated after a user pays according to the payment information, and updating the inventory record according to the product shopping list;

Comparing the number of products corresponding to each product category in the updated inventory record with a preset inventory threshold corresponding to the product category;

If the number of products corresponding to the product category is less than the preset inventory threshold corresponding to the product category, a replenishment list including the product category and the number of products corresponding to the product category is generated, so that Replenishment based on the replenishment list.

Preferably, the "collecting a control image of the goods in the container of the vending machine, and identifying the control image of the goods, and determining an inventory record corresponding to the control image of the goods; said inventory record includes the product category and the same as the said The number of products corresponding to the product category "includes:

Receiving a shopping start instruction, and starting a shopping process according to the shopping start instruction, and receiving a gravity sensing signal returned by a gravity sensor in each cell in the shopping warehouse;

Collecting a product image in the cell according to the gravity sensing signal through a camera corresponding to the cell;

Identify the product image based on the product comparison image, determine the product category of the target product captured by the user and the number of products corresponding to the product category, and according to the product category and destination of the target product The said product quantity is played in real time through a voice module to prompt the user.

Preferably, the “identifying the product image based on the product comparison image, determining the product category of the target product captured by the user and the number of products corresponding to the product category, and according to the target product ’s After the product category and the quantity of the product are played in real time through a voice module to prompt the user, the method further includes:

Collect in real time a cell area change image in each of the cells;

Extracting a key frame containing the shopping gesture of the user and a time stamp corresponding to the key frame in the cell area change image;

Identifying the key frame, if the key frame is a return key frame corresponding to a return action;

Determining a gesture outline of a return gesture in each of the return key frames;

Taking a screenshot of a gestured product in the return key frame according to the outline;

Constructing a return movement trajectory based on the gesture product screenshot based on the timestamp corresponding to the key frame;

Determining a cell area where the target product is placed according to the return movement trajectory, and determining whether the returned product is placed in a cell area corresponding to the product category based on a preset product cell placement area;

If not, it is played in real time through the voice module to prompt the user to place an error in the placement area.

Preferably, after “if not, playing through the voice module in real time to prompt the user to place an error in the area”, the method further includes:

Controlling the LED indicator of the cell area corresponding to the product category to light up to prompt the user to place the target product in the cell area corresponding to the LED indicator.

Preferably, the "receiving an end-of-shopping instruction, collecting an update image of the goods in the container according to the end-of-shopping instruction, identifying the update image of the goods, generating a product purchase list based on the recognition result, and generating a The payment information corresponding to the product purchase list so that the user can pay according to the payment information "includes:

Receiving an end-of-shopping instruction, and acquiring an updated image of the goods in the container according to the end-of-shopping instruction;

Performing grayscale processing on the updated product image to obtain a grayscale image;

Binarizing the grayscale image to obtain a binary grayscale image;

Obtaining a product outline of the product in the binary grayscale image;

Extracting and including screenshots of the goods according to the outline of the goods;

Determining an update record in the screenshot of the product through a pre-trained image recognition model; the update record includes a product category and an updated number of products corresponding to the product category;

Compare the update record with the inventory record, and generate a product purchase list according to the comparison result; and generate payment information corresponding to the product purchase list so that the user can make payment according to the payment information.

In addition, in order to solve the above problems, the present application provides a replenishment management device for a vending machine, including: a collection module, a receiving module, a comparison module, and a replenishment module;

The acquisition module is configured to collect a comparison image of the goods in the container of the vending machine, identify the comparison image of the goods, and determine an inventory record corresponding to the comparison image of the goods. The number of products corresponding to the product category;

The receiving module is configured to receive an end-of-shopping instruction, collect an updated image of the goods in the container according to the end-of-shopping instruction, identify the updated image of the goods, generate a product purchase list based on the recognition result, and generate a product purchase list. The payment information corresponding to the product purchase list, so that the user can pay according to the payment information;

The receiving module is further configured to receive payment completion information generated after a user pays according to the payment information, and update the inventory record according to the shopping list of the product;

The comparison module is configured to compare the number of products corresponding to each product category in the updated inventory record with a preset inventory threshold corresponding to the product category;

The replenishment module is configured to generate, if the quantity of the product corresponding to the product category is less than the preset inventory threshold corresponding to the product category, including the product category and the product category corresponding to the product category. A replenishment list of the quantity of the goods to facilitate replenishment based on the replenishment list.

Preferably, it further comprises a receiving module, a gravity sensing module and a recognition module;

The receiving block is further configured to receive a shopping start instruction, and start a shopping process according to the shopping start instruction, and receive a gravity sensing signal returned by a gravity sensor in each cell in the shopping warehouse;

The gravity sensing module is configured to collect a product image in the cell through a camera corresponding to the cell according to the gravity sensing signal;

The identification module is configured to identify the product image based on the product comparison image, determine the product category of the target product captured by the user and the number of products corresponding to the product category, and according to the target product The product category and the number of products are played in real time through a voice module to prompt the user.

In addition, in order to solve the above problem, the present application also provides a user terminal including a memory and a processor, where the memory is used to store a replenishment management program of the vending machine, and the processor runs the replenishment of the vending machine. Managing a program to cause the user terminal to execute the replenishment management method of the vending machine as described above.

In addition, in order to solve the above-mentioned problem, the present application also provides a computer-readable storage medium, where the computer-readable storage medium stores a replenishment management program of the vending machine, and the replenishment management program of the vending machine is processed The implementation of the dispenser implements the replenishment management method of the vending machine as described above.

The present invention provides a replenishment management method, device and user terminal for a vending machine. Among them, the method provided in the present application obtains the inventory record by identifying the comparison image of the goods in the container, and then at the end of shopping, collects the updated image of the goods in the container again, and determines the product in combination with the comparison image of the goods after identification The purchase list is convenient for payment; after the payment is completed, the updated inventory records are compared with preset inventory thresholds, so that the products that should be replenished are determined and a replenishment list is obtained, so as to prompt the management to perform the corresponding Product replenishment. After the user finishes shopping, the image recognition technology is used to intelligently determine the products that need to be replenished by comparing the identified inventory record with a preset threshold value. This realizes the use of image recognition during the replenishment process. Intelligently determine the products that need to be replenished. The method is simple, the replenishment efficiency is high, and the user experience is good.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic structural diagram of a hardware operating environment involved in a solution of an embodiment of a replenishment management method of an automatic vending machine;

2 is a schematic flowchart of a first embodiment of a replenishment management method of a vending machine of this application;

3 is a schematic flowchart of a second embodiment of a replenishment management method for a vending machine of this application;

4 is a schematic flowchart of steps after step S80 to step S20 in the third embodiment of the replenishment management method of the vending machine of this application;

5 is a detailed flowchart of step S20 of the fourth embodiment of the replenishment management method of the vending machine of the present application;

6 is a functional module schematic diagram of a replenishment management device of a vending machine provided by the present application;

FIG. 7 is a functional module schematic diagram of a replenishment management device of another vending machine provided by the present application.

The implementation, functional characteristics and advantages of the purpose of this application will be further described with reference to the embodiments and the drawings.

detailed description

The embodiments of the present application are described in detail below, in which the same or similar reference numerals indicate the same or similar elements or elements having the same or similar functions.

In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality" is two or more, unless it is specifically and specifically defined otherwise.

In this application, the terms "installation," "connected," "connected," and "fixed" should be understood broadly unless otherwise specified and limited, for example, they may be fixed connections or removable connections Or integrated; it can be mechanical or electrical; it can be directly connected or indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific situations.

It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

As shown in FIG. 1, FIG. 1 is a schematic structural diagram of a hardware operating environment of a terminal involved in a solution according to an embodiment of the present application.

In the embodiment of the present application, the terminal may be a PC provided in an automatic container machine, or may be a mobile terminal device such as a smart phone, a tablet computer, an e-book reader, an MP3 player, an MP4 player, and a portable computer. In addition, it may be a computer hardware device provided in the automatic container machine itself.

As shown in FIG. 1, the terminal may include a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to implement connection and communication between these components. The user interface 1003 may include a display screen, an input unit such as a keyboard, a remote controller, and the optional user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory, such as a magnetic disk memory. The memory 1005 may optionally be a storage device independent of the foregoing processor 1001.

In addition, the terminal includes an image acquisition device, which may specifically be a camera, a camera, and the like.

Optionally, the terminal may further include an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. In addition, the mobile terminal may be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, which will not be repeated here.

Those skilled in the art can understand that the terminal shown in FIG. 1 does not constitute a limitation on the terminal, and may include more or fewer components than those shown in the figure, or combine some components, or arrange different components.

As shown in FIG. 1, the memory 1005 as a computer-readable storage medium may include an operating system, a data interface control program, a network connection program, and a replenishment management program of a vending machine.

The present invention provides a replenishment management method, device and user terminal for a vending machine. Wherein, the method realizes that in the process of replenishment, intelligently determines the products that need to be replenished through image recognition, the method is simple, the replenishment efficiency is high, and the user experience is good.

Example 1:

Referring to FIG. 2, a first embodiment of the present application provides a replenishment management method for a vending machine, including:

Step S10: Collect a comparison image of the goods in the container of the vending machine, identify the comparison image of the goods, and determine an inventory record corresponding to the comparison image of the goods; the inventory record includes a commodity category and a correspondence to the commodity category The number of items

As mentioned above, the method provided in this embodiment is applied to a vending machine or an vending scene with an open shopping environment. For example, in an open vending machine, after opening the cabinet door, the user can enter it for selection. Commodity, settlement after closing the cabinet door.

As mentioned above, the container is an area where the merchandise is displayed inside the vending machine, and the interior can be divided into different display areas, or a display layer or a display grid.

As mentioned above, the goods comparison image is an image collected by a camera inside the vending machine for the goods displayed in the container before the shopping process is performed, for further comparison. The internally collected image can be an image collection of the goods in the entire container through a camera, or a camera can be set for each area to perform image collection of the goods in the container.

As mentioned above, through image recognition, the quantity of goods corresponding to each variety in the goods comparison image is identified, and an inventory record is obtained.

As mentioned above, image recognition can be neural network recognition. By training an artificial neural network model in advance, a corresponding variety, that is, a corresponding quantity is identified.

Step S20: Receive an end-of-shopping instruction, collect an update image of the goods in the container according to the end-of-shopping instruction, identify the update image of the goods, generate a product purchase list based on the recognition result, and generate a purchase list with the product Corresponding payment information, so that the user can make payment according to the payment information;

As described above, the shopping end instruction may be an end instruction generated by a user closing a warehouse door of a shopping warehouse, or may be a shopping end instruction issued by a user using a smart terminal connected to a vending machine. In short, this instruction means that the user ends the shopping and performs the settlement.

As described above, the product update image is an image of all the products displayed in the shopping warehouse collected by the user after the shopping is completed, and the image is an image of the internal product situation of the user after the shopping is completed. For example, when a user takes out a bottle of carbonated beverage, one bottle of carbonated beverage will be missing from the product update image.

Step S30: Receive payment completion information generated by the user after paying according to the payment information, and update the inventory record according to the product shopping list;

As mentioned above, the payment completion information is information generated after the user makes a payment based on the payment information.

As mentioned above, the inventory record is changed according to the product shopping list, and the item removed from the user is the updated inventory record.

Step S40: Compare the number of products corresponding to each product category in the updated inventory record with a preset inventory threshold corresponding to the product category;

Step S50, if the quantity of the product corresponding to the product category is less than the preset inventory threshold corresponding to the product category, a replenishment including the product category and the quantity of the product corresponding to the product category is generated Checklist to facilitate replenishment according to the replenishment list.

As mentioned above, the preset inventory threshold is the inventory threshold of the product preset by the manager. Each product corresponds to one or more preset inventory thresholds. For example, the preset inventory threshold of carbonated beverages is 5 bottles. When the number of products is 4 bottles, it is less than the preset inventory threshold, and then a replenishment list corresponding to the carbonated beverage is generated, and the management is notified to replenish according to the replenishment list.

The method provided in this embodiment obtains the inventory record by identifying the comparison image of the goods in the container, and then collects the updated image of the goods in the container again at the end of shopping. After the identification, the purchase of the goods is determined by combining the comparison image of the goods List to facilitate payment; after the payment is completed, the updated inventory records are compared with preset inventory thresholds, so as to identify the products that should be replenished and get a replenishment list, in order to prompt the management to carry out the corresponding product Replenishment. In this embodiment, after the user finishes shopping, the image recognition technology is used to intelligently determine the product that needs to be replenished by comparing the identified inventory record with a preset threshold value. Identifying and intelligently judging the products that need to be replenished, the method is simple, the replenishment efficiency is high, and the user experience is good.

Example 2:

Referring to FIG. 3, a second embodiment of the present application provides a replenishment management method for a vending machine. Based on the first embodiment shown in FIG. 2 described above, the step S10 is "collecting a comparison of goods in the container of the vending machine." Image, and identify the product comparison image to determine the inventory record corresponding to the product comparison image; the inventory record includes the product category and the number of products corresponding to the product category, and includes:

Step S60: Receive a shopping start instruction, and start a shopping process according to the shopping start instruction, and receive a gravity sensing signal returned by a gravity sensor in each cell in the shopping warehouse;

As mentioned above, the shopping start instruction may be that the user's shopping movement is sensed by an infrared sensor provided in the shopping warehouse, and when the user extends his arm into the shopping warehouse, the corresponding infrared sensor is triggered to generate a shopping start instruction.

As mentioned above, the shopping process is a process in which a user selects a product and takes out a corresponding product.

As mentioned above, the shopping warehouse of the vending machine contains a plurality of individual cells, and each cell is a storage area, and each storage area can place similar products. The cells in multiple shopping bins make up a complete shopping bin.

As mentioned above, each cell is correspondingly provided with a gravity sensor. When the user grabs the product, the data of the gravity sensor placed at the bottom or side of the product is triggered to generate a gravity sensor signal to determine that the product is Removed by the user.

Step S70: Collect a product image in the cell through a camera corresponding to the cell according to the gravity sensing signal;

Step S80: identify the product image based on the product comparison image, determine the product category of the target product captured by the user and the number of products corresponding to the product category, and according to the product of the target product The category and the number of goods are played in real time through a voice module to prompt the user.

As described above, according to the gravity sensing signal sent by the gravity sensor, the camera is controlled to collect the product images in the cells, and each cell may correspond to one or more cameras individually, or multiple cells may correspond to one or more cameras. camera.

In the foregoing, image acquisition is performed by a camera to obtain a product image, and by identifying the product image, based on the comparison of the product image and the control image of the product, the relevant information of the target product captured by the user, such as the quantity, the name of the product, etc., is determined. When the user does not capture one product in one variety, or multiple products in one variety, or a different number of different varieties, the name of the product captured by the user is broadcasted in real time through the voice module, such as " You got a cola "or" You got a cola ". In this embodiment, by performing image-based recognition through the system when the user is shopping, and capturing different products through the system, real-time broadcast of the product captured by the user is prompted to inform the user of the variety and quantity of the product captured, Provides instructions for users 'shopping, improves user experience and shopping efficiency, and places users' mistakes and mistakes.

Example 3:

Referring to FIG. 4, a third embodiment of the present application provides a replenishment management method for a vending machine. Based on the second embodiment shown in FIG. 3 described above, the step S80 is based on the comparison image of the goods, The product image is identified, the product category of the target product captured by the user and the number of products corresponding to the product category are determined, and the real-time playback is performed by a voice module according to the product category and the number of products of the target product to After prompting the user, it also includes:

Step S90: Collect a cell area change image in each of the cells in real time;

As described above, the shopping warehouse includes a plurality of independent cells, and a camera is used to collect a change image of the internal cell area.

Step S100, extracting a key frame containing a shopping gesture of the user and a time stamp corresponding to the key frame in the cell area change image;

As mentioned above, a timestamp is a complete, verifiable piece of data that can exist before a certain time, usually a sequence of characters that uniquely identifies the time at a certain moment. The data generated by digital signature technology includes the original file information, signature parameters, signature time and other information. It is widely used in intellectual property protection, contract signing, financial accounting, electronic quotation and bidding, and stock trading.

As mentioned above, the shopping gesture refers to the corresponding movement of the hand when the user makes a purchase, such as the shape of the hand when grasping.

Step S110: identify the key frames, and if the key frames are return key frames corresponding to a return action, determine a gesture outline of a return gesture in each of the return key frames;

As described above, the key frames are identified, and it is determined whether the key frames are corresponding to the return action. The specific judgment method can be identified through a pre-trained neural network in image recognition, thereby determining that the user is a return action.

As mentioned above, the gesture contour is based on each pixel point in each key frame. Specifically, contours of high-difference edges corresponding to the pixel points can be obtained.

Step S120: Take a screenshot of the gestured product in the return key frame according to the contour;

Step S130: Construct a return movement trajectory based on the gesture product screenshot based on the timestamp corresponding to the key frame;

Step S140: Determine a cell area where the target product is placed according to the return movement trajectory, and determine whether the returned product is placed in a cell area corresponding to the product category based on a preset product cell placement area;

As described above, based on the contour, a corresponding gesture product screenshot is obtained.

As described above, after taking a screenshot of a gestured product, a movement trajectory can be established based on the time stamp, and the movement trajectory of the product during the return process, that is, the area to be placed can be obtained based on the movement trajectory.

Step S150: if not, play through the voice module in real time to prompt the user that there is an error in the placement area;

In step S160, the LED indicator of the cell area corresponding to the product category is controlled to light up, so as to prompt the user to place the target product in the cell area corresponding to the LED indicator.

As mentioned above, the voice module may be a speaker in a vending machine. Play the prompt information according to the system feedback information. If the returned product is not placed in the area corresponding to the product category, the voice module will prompt "This product is placed incorrectly, please place it in the correct position, that is, some area".

In addition, the LED indicator is used to prompt the user, and the light is displayed at the correct position, so that the user can place the product in the correct position according to the LED indicator.

In this embodiment, the image recognition technology is used to determine the location of the returned goods when the user returns the goods. In the actual shopping process, if the returned goods are placed in the wrong area, the goods in the shopping warehouse will be placed. It is confusing and inconvenient for other shopping consumers. Therefore, this embodiment uses image recognition technology to determine the placement of returned goods. If an error occurs, it prompts through multiple prompting methods to improve the user experience. The user's shopping process provides convenience, improves the user's shopping efficiency, and makes the shopping process more intelligent.

Example 4:

Referring to FIG. 5, a fourth embodiment of the present application provides a replenishment management method for a vending machine. Based on the first embodiment shown in FIG. 2 described above, the step S20, "Receive a shopping end instruction, Instructs to collect the updated image of the goods in the container, identify the updated image of the goods, generate a product purchase list according to the recognition result, and generate payment information corresponding to the product purchase list, so as to facilitate the user based on the payment information Make a payment "includes:

Step S21: Receive a shopping end instruction, and collect an updated image of the goods in the container according to the shopping end instruction;

Step S22: performing grayscale processing on the updated product image to obtain a grayscale image;

As mentioned above, a grayscale digital image is an image with only one sampled color per pixel. This type of image is usually displayed as a grayscale from the darkest black to the brightest white, although in theory this sample can be of different shades of any color, or even different colors at different brightnesses. Gray images are different from black and white images. In the field of computer graphics, black and white images have only two colors: black and white. Gray images have many levels of color depth between black and white.

Step S23: Binarize the grayscale image to obtain a binary grayscale image;

A binarized image is obtained by selecting a grayscale image of 256 brightness levels through an appropriate threshold value and still reflecting the overall and local characteristics of the image. As mentioned above, converting the original image into a binary image can further improve the image processing speed, improve the system's ability to recognize contours, and improve efficiency.

Step S24, obtaining a product outline of the product in the binary grayscale image;

Step S20, extracting and including screenshots of the goods according to the outline of the goods;

As described above, obtaining a screenshot of the product and further identifying the screenshot improves the image recognition efficiency, avoids the slow speed, low efficiency, and inability to save the high-resolution pictures in the process of data interaction and image recognition. This improves efficiency and reduces recognition time.

Step S26: Determine an update record in the screenshot of the product through a pre-trained image recognition model; the update record includes a product category and an updated quantity of the product corresponding to the product category;

As mentioned above, Artificial Neural Networks (ANNs) are also referred to as Neural Networks (NNs) or Connection Models for short. It is a kind of distributed and parallel information processing that imitates the behavioral characteristics of animal neural networks. Algorithmic mathematical model. This kind of network depends on the complexity of the system and adjusts the interconnection relationship between a large number of internal nodes to achieve the purpose of processing information.

As mentioned above, in this embodiment, the artificial neural network is pre-trained to realize the recognition of the screenshot of the goods.

As mentioned above, the updated record in the product screenshot is the product record of which products are missing from the original control image in the product screenshot.

In step S27, the update record is compared with the inventory record, and a product purchase list is generated based on the comparison result; and payment information corresponding to the product purchase list is generated, so that the user can perform the payment according to the payment information. Pay.

As mentioned above, the generated product purchase list is based on the comparison between the update record and the inventory record. What is missing is the variety and quantity of the product captured by the user. The user then makes payment according to the payment information corresponding to the list.

In addition, the present application provides a replenishment management device for a vending machine, including: a collection module 10, a receiving module 20, a comparison module 30, and a replenishment module 40;

The acquisition module 10 is configured to collect a comparison image of the goods in the container of the vending machine, identify the comparison image of the goods, and determine an inventory record corresponding to the comparison image of the goods; the inventory record includes the product category and the The number of commodities corresponding to the commodity category;

The receiving module 20 is configured to receive an end-of-shopping instruction, collect an updated image of the goods in the container according to the end-of-shopping instruction, identify the updated image of the goods, generate a purchase list of the goods according to the recognition result, and generate Payment information corresponding to the product purchase list, so that a user can pay according to the payment information;

The receiving module 20 is further configured to receive payment completion information generated after a user pays according to the payment information, and update the inventory record according to the shopping list of the product;

The comparison module 30 is configured to compare the number of products corresponding to each product category in the updated inventory record with a preset inventory threshold corresponding to the product category;

The replenishment module 40 is configured to generate, if the number of products corresponding to the product category is less than the preset inventory threshold corresponding to the product category, including the product category and the corresponding product category corresponding to the product category. A replenishment list describing the quantity of goods, so as to facilitate replenishment according to the replenishment list.

Preferably, it further includes a gravity sensing module 60 and a recognition module 70;

The receiving block 20 is further configured to receive a shopping start instruction and start a shopping process according to the shopping start instruction, and receive a gravity sensing signal returned by a gravity sensor in each cell in the shopping warehouse;

The gravity sensing module 50 is configured to collect a product image in the cell through a camera corresponding to the cell according to the gravity sensing signal;

The identification module 60 is configured to identify the product image based on the product comparison image, determine the product category of the target product captured by the user and the number of products corresponding to the product category, and according to the target The commodity category and the quantity of the commodity are played in real time through a voice module to prompt the user.

In addition, the present application also provides a user terminal including a memory and a processor, where the memory is used to store a replenishment management program of the vending machine, and the processor runs the replenishment management program of the vending machine to enable all The user terminal executes the replenishment management method of the vending machine as described above.

In addition, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a replenishment management program for a vending machine, and the replenishment management program for the vending machine is implemented by a processor such that The replenishment management method of the vending machine described above.

The above-mentioned serial numbers of the embodiments of the present application are merely for description, and do not represent the superiority or inferiority of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods in the above embodiments can be implemented by means of software plus a necessary universal hardware platform, and of course, also by hardware, but in many cases the former is better. Implementation. Based on such an understanding, the technical solution of the present application, in essence, or a part that contributes to the existing technology, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM) as described above. , Magnetic disk, optical disc), including a number of instructions to enable a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the embodiments of the present application.

The above are only preferred embodiments of the present application, and thus do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of the application, or directly or indirectly used in other related technical fields Are included in the scope of patent protection of this application.

Claims

A replenishment management method for a vending machine, which includes:

Collecting a comparison image of the goods in the container of the vending machine, and identifying the comparison image of the goods, to determine the inventory record corresponding to the comparison image of the goods; the inventory record includes the product category and the number of products corresponding to the product category ;

Receiving an end-of-shopping instruction, collecting an update image of the goods in the container according to the end-of-shopping instruction, identifying the update image of the goods, generating a product purchase list based on the recognition result, and generating a payment corresponding to the product purchase list Information so that users can make payments based on the payment information;

Receiving payment completion information generated after a user pays according to the payment information, and updating the inventory record according to the product shopping list;

Comparing the number of products corresponding to each product category in the updated inventory record with a preset inventory threshold corresponding to the product category;

If the number of products corresponding to the product category is less than the preset inventory threshold corresponding to the product category, a replenishment list including the product category and the number of products corresponding to the product category is generated, so that Replenishment based on the replenishment list.
The replenishment management method for a vending machine according to claim 1, wherein the "collecting a comparison image of the goods in the container of the vending machine, and identifying the comparison picture of the goods, and determining the comparison picture of the goods Corresponding inventory records; said inventory records include the product category and the number of products corresponding to the product category ", including:

Receiving a shopping start instruction, and starting a shopping process according to the shopping start instruction, and receiving a gravity sensing signal returned by a gravity sensor in each cell in the shopping warehouse;

Collecting a product image in the cell according to the gravity sensing signal through a camera corresponding to the cell;

Identify the product image based on the product comparison image, determine the product category of the target product captured by the user and the number of products corresponding to the product category, and according to the product category and destination of the target product The said product quantity is played in real time through a voice module to prompt the user.
The replenishment management method for a vending machine according to claim 2, wherein the "identification of the product image based on the product comparison image, and determining the product category and the target product of the target product captured by the user After the number of products corresponding to the product category is played in real time through a voice module according to the product category and the number of products of the target product to prompt the user, the method further includes:

Collect in real time a cell area change image in each of the cells;

Extracting a key frame containing the shopping gesture of the user and a time stamp corresponding to the key frame in the cell area change image;

Identifying the key frame, if the key frame is a return key frame corresponding to a return action;

Determining a gesture outline of a return gesture in each of the return key frames;

Taking a screenshot of a gestured product in the return key frame according to the outline;

Constructing a return movement trajectory based on the gesture product screenshot based on the timestamp corresponding to the key frame;

Determining a cell area where the target product is placed according to the return movement trajectory, and determining whether the returned product is placed in a cell area corresponding to the product category based on a preset product cell placement area;

If not, it is played in real time through the voice module to prompt the user to place an error in the placement area.
The replenishment management method for a vending machine according to claim 3, wherein after "if not, then playing through the voice module in real time to prompt the user to place an error in the placement area", further comprising:

Controlling the LED indicator of the cell area corresponding to the product category to light up to prompt the user to place the target product in the cell area corresponding to the LED indicator.
The replenishment management method for a vending machine according to claim 1, wherein the "receiving a shopping end instruction, collecting an update image of the goods in the container according to the shopping end instruction, and updating the image of the goods Identifying, generating a product purchase list according to the recognition result, and generating payment information corresponding to the product purchase list, so that the user can pay according to the payment information "includes:

Receiving an end-of-shopping instruction, and acquiring an updated image of the goods in the container according to the end-of-shopping instruction;

Performing grayscale processing on the updated product image to obtain a grayscale image;

Binarizing the grayscale image to obtain a binary grayscale image;

Obtaining a product outline of the product in the binary grayscale image;

Extracting and including screenshots of the goods according to the outline of the goods;

Determining an update record in the screenshot of the product through a pre-trained image recognition model; the update record includes a product category and an updated number of products corresponding to the product category;

Compare the update record with the inventory record, and generate a product purchase list according to the comparison result; and generate payment information corresponding to the product purchase list so that the user can make payment according to the payment information.
A replenishment management device for a vending machine, comprising: a collection module, a receiving module, a comparison module, and a replenishment module;

The acquisition module is configured to collect a comparison image of the goods in the container of the vending machine, identify the comparison image of the goods, and determine an inventory record corresponding to the comparison image of the goods. The number of products corresponding to the product category;

The receiving module is configured to receive an end-of-shopping instruction, collect an updated image of the goods in the container according to the end-of-shopping instruction, identify the updated image of the goods, generate a product purchase list based on the recognition result, and generate a product purchase list. The payment information corresponding to the product purchase list, so that the user can pay according to the payment information;

The receiving module is further configured to receive payment completion information generated after a user pays according to the payment information, and update the inventory record according to the shopping list of the product;

The comparison module is configured to compare the number of products corresponding to each product category in the updated inventory record with a preset inventory threshold corresponding to the product category;

The replenishment module is configured to generate, if the quantity of the product corresponding to the product category is less than the preset inventory threshold corresponding to the product category, including the product category and the product category corresponding to the product category. A replenishment list of the quantity of the goods to facilitate replenishment based on the replenishment list.
The replenishment management device for a vending machine according to claim 6, further comprising: a receiving module, a gravity sensing module, and an identification module;

The receiving block is further configured to receive a shopping start instruction, and start a shopping process according to the shopping start instruction, and receive a gravity sensing signal returned by a gravity sensor in each cell in the shopping warehouse;

The gravity sensing module is configured to collect a product image in the cell through a camera corresponding to the cell according to the gravity sensing signal;

The identification module is configured to identify the product image based on the product comparison image, determine the product category of the target product captured by the user and the number of products corresponding to the product category, and according to the target product The product category and the number of products are played in real time through a voice module to prompt the user.
A user terminal, comprising a memory and a processor, wherein the memory is used to store a replenishment management program of a vending machine, and the processor runs the replenishment management program of the vending machine to enable the user The terminal executes the replenishment management method of the vending machine according to any one of claims 1-5.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores a replenishment management program of a vending machine, and the replenishment management program of the vending machine is implemented by a processor when executed as claimed in the claims The replenishment management method for a vending machine according to any one of 1-5.