WO2020047918A1

WO2020047918A1 - Self-service vending method, apparatus, and system, electronic device, and computer storage medium

Info

Publication number: WO2020047918A1
Application number: PCT/CN2018/108000
Authority: WO
Inventors: 黄鼎隆; 斯科特·马修·罗伯特; 傅恺; 李天宇
Original assignee: 深圳码隆科技有限公司
Priority date: 2018-09-06
Filing date: 2018-09-27
Publication date: 2020-03-12
Also published as: CN109190706A

Abstract

The present application provides a self-service vending method, apparatus, and system, an electronic device, and a computer storage medium. The self-service vending method is applied to a product identification server, and comprises: receiving a first product image and a second product image transmitted by a controller in a self-service vending machine, wherein the first product image and the second product image are respectively images of products in a cabinet acquired via a camera apparatus when the self-service vending machine senses, via an infrared sensor, that a cabinet door has been opened and closed; respectively identifying, according to the first product image, the second product image, and a product identification model, first product information and second product information of products in the cabinet when the cabinet door is open and when closed; and generating a pre-purchase list on the basis of the first product information and the second product information, wherein the pre-purchase list comprises a product name, a product quantity, a product unit price, and a product total price. In the present application, a product identification server cooperates with a self-service vending machine, thereby perfoming vending quickly and efficiently, and greatly improving user shopping experience.

Description

Unmanned sales method, device, system, electronic equipment and computer storage medium

Cross-reference to related applications

This application claims the priority of a Chinese patent application filed on September 6, 2018 with the priority number of 2018110406471 and the name "unmanned sales method, device and system", the entire content of which is incorporated herein by reference. Applying.

Technical field

The present application relates to the technical field of self-service sales, and in particular, to an unmanned sales method, device, system, electronic device, and computer storage medium.

Background technique

With the enhancement of national strength and the rapid development of the national economy, people's shopping methods have become more and more diverse. Among them, self-service shopping has become an increasingly popular shopping method, which has greatly facilitated people's lives.

The existing self-service shopping method is usually implemented by a self-service vending machine. As long as the user selects his favorite item and inputs corresponding coins from the coin port, he can buy the selected item. However, the items provided by this self-service vending machine are limited, usually only beverages such as Coke and Sprite, and require relatively complicated operations by the user, making the user's shopping inefficient.

Summary of the Invention

In view of this, the purpose of this application includes providing an unmanned vending method, device, system electronic equipment and computer storage medium, which can realize a fast and efficient vending service through the cooperation of a commodity recognition server and an unmanned vending machine. , Greatly improving the user's shopping experience.

In a first aspect, an embodiment of the present application provides an unmanned sales method, which is applied to a commodity identification server. The method includes:

Receive the first product image and the second product image sent by the controller in the unmanned vending machine; the first product image and the second product image are respectively when the unmanned vending machine senses that the door of the cabinet is opened and closed through the infrared sensor. Commodity images in the box collected by the camera device;

Identifying the first product information and the second product information in the cabinet when the cabinet door is opened and closed according to the first product image, the second product image, and the product recognition model;

Based on the first product information and the second product information, a pre-shopping list is generated; the pre-shopping list includes: product name, product quantity, product unit price, and total price.

With reference to the first aspect, the embodiment of the present application provides a first possible implementation manner of the first aspect, further including:

Obtaining product data samples;

Dividing product data samples into training data samples and verification data samples;

The neural network model is trained by training data samples to generate a product recognition model.

With reference to the first aspect, the embodiment of the present application provides a second possible implementation manner of the first aspect, further including:

The verification data samples are input into the product recognition model to verify the accuracy of the product recognition model.

With reference to the first aspect, the embodiment of the present application provides a third possible implementation manner of the first aspect, in which, according to the first product image, the second product image, and the product recognition model, the box when the cabinet door is opened and closed is identified The first and second product information in the body specifically include:

Input the first product image into the product recognition model, and use the first output result of the product recognition model as the first product information;

The second product image is input into the product recognition model, and the second output result of the product recognition model is used as the second product information.

With reference to the first aspect, the embodiment of the present application provides a fourth possible implementation manner of the first aspect, wherein generating a pre-shopping list based on the first product information and the second product information specifically includes:

Comparing the first product information with the second product information;

Determine whether the first product information is the same as the second product information;

If yes, then it is determined that the user has not made a purchase;

If not, a pre-shopping list is generated based on different product information in the first product information and the second product information.

With reference to the first aspect, the embodiment of the present application provides a fifth possible implementation manner of the first aspect. After generating the pre-shopping list based on the first product information and the second product information, the method further includes:

Sending the pre-shopping list to the controller, so that the controller generates payment information according to the pre-shopping list;

Receive the payment success information generated by the user after the user completes the payment operation according to the payment information.

With reference to the first aspect, the embodiment of the present application provides a sixth possible implementation manner of the first aspect. After generating the pre-shopping list based on the first product information and the second product information, the method further includes:

Receiving user information sent by the controller;

Send the pre-shopping list to the user's mobile terminal according to the user information, so that the user makes payment according to the pre-shopping list;

Receive the payment success information sent by the mobile terminal after the user completes the payment.

With reference to the first aspect, the embodiment of the present application provides a seventh possible implementation manner of the first aspect, further including:

Record the sales information of the unmanned vending machine based on the payment success information;

Determine current inventory status and replenishment information based on sales information;

Send the replenishment information to the mobile terminal of the manager.

With reference to the first aspect, the embodiment of the present application provides an eighth possible implementation manner of the first aspect, further including:

If the payment fails, save the pre-shopping list;

When the user purchases again, the pre-shopping list is sent to the user's mobile terminal, so that the user can perform the purchase operation only after paying according to the pre-shopping list.

With reference to the first aspect, the embodiment of the present application provides a ninth possible implementation manner of the first aspect, where the number of camera devices is multiple, and the multiple camera devices are disposed at different positions of the cabinet.

With reference to the first aspect, the embodiment of the present application provides a tenth possible implementation manner of the first aspect, further including:

Obtain a sales inquiry request of the unmanned vending machine within the target time period, and the sales inquiry request is a request sent by the manager's mobile terminal;

Based on the sales situation query request to determine the sales situation of the unmanned vending machine within the target time period;

The sales situation is sent to the manager's mobile terminal for the manager to view.

In a second aspect, an embodiment of the present application further provides an unmanned sales device, including:

An image receiving module configured to receive a first product image and a second product image sent by a controller in the unmanned vending machine; the first product image and the second product image are the unmanned vending machine sensing the cabinet door through an infrared sensor, respectively When opening and closing, the product images in the box collected by the camera device;

An image recognition module configured to identify the first product information and the second product information in the cabinet when the cabinet door is opened and closed according to the first product image, the second product image, and the product recognition model;

The list generation module is configured to generate a pre-shopping list based on the first product information and the second product information; the pre-shopping list includes: product name, product quantity, product unit price and total price.

In a third aspect, an embodiment of the present application further provides an unmanned vending system, including: an unmanned vending machine and a commodity identification server;

The unmanned vending device of the second aspect is installed on the commodity identification server;

The product identification server communicates with the unmanned vending machine;

The unmanned vending machine includes: a vending machine body and a controller, a camera device and an infrared sensor installed in the vending machine body; the vending machine body includes a cabinet door and a box body.

According to a fourth aspect, an embodiment of the present application further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes The computer program implements the method according to any one of the first aspects.

In a fifth aspect, an embodiment of the present application further provides a computer storage medium on which a computer program is stored, and when the computer runs the computer program, the steps of the method according to any one of the first aspects are performed.

The embodiments of the present application bring the following beneficial effects:

The unmanned vending method provided in the embodiment of the present application is applied to a commodity recognition server, and the method includes: receiving a first commodity image and a second commodity image sent by a controller in the unmanned vending machine; a first commodity image and a second commodity image; The product images are the product images in the box collected by the camera when the unmanned vending machine senses the cabinet door is opened and closed through the infrared sensor; the cabinet is identified based on the first product image, the second product image, and the product recognition model. The first product information and the second product information in the box when the door is opened and closed; based on the first product information and the second product information, a pre-shopping list is generated; the pre-shopping list includes: product name, product quantity, product unit price and total price. Through the cooperation of the product identification server and the unmanned vending machine, this application can realize a fast and efficient sales service and greatly improve the shopping experience of users.

Other features and advantages of the present application will be explained in the following description, and partly become apparent from the description, or be understood by implementing the present application. The objectives and other advantages of the present application are achieved and obtained in the structures particularly pointed out in the description, the claims, and the drawings.

In order to make the above-mentioned objects, features, and advantages of this application more comprehensible, preferred embodiments are described below in conjunction with the accompanying drawings and described in detail below.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the specific implementations of the present application or the technical solutions in the prior art, the drawings used in the specific implementations or prior art descriptions will be briefly introduced below. Obviously, the appended The drawings are some implementations of the present application. For those of ordinary skill in the art, other drawings can be obtained according to these drawings without paying creative labor.

FIG. 1 is a flowchart of an unmanned sales method provided by Embodiment 1 of this application; FIG.

FIG. 2 is a flowchart of another unmanned sales method provided by Embodiment 1 of the present application; FIG.

FIG. 3 is a flowchart of another unmanned sales method provided by Embodiment 1 of the present application; FIG.

4 is a flowchart of another unmanned sales method provided by Embodiment 1 of the present application;

5 is a flowchart of another unmanned sales method provided by Embodiment 1 of the present application;

FIG. 6 is a flowchart of another unmanned sales method provided by Embodiment 1 of the present application; FIG.

7 is a flowchart of another unmanned sales method provided by Embodiment 1 of the present application;

8 is a schematic diagram of an unmanned sales device provided in Embodiment 2 of the present application;

FIG. 9 is a schematic diagram of an unmanned sales system provided in Embodiment 3 of the present application; FIG.

FIG. 10 is a schematic diagram of an electronic device according to a fourth embodiment of the present application.

detailed description

In order to make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the application, but not all The examples. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

The existing self-service vending machines provide limited items, usually only beverages such as Coke and Sprite, and require relatively complicated operations by the user, making the user's shopping inefficient. Based on this, the embodiments of the present application provide an unmanned vending method, device and system, which can realize fast and efficient vending services through the cooperation of a product identification server and an unmanned vending machine, and greatly improve the physical shopping experience of users.

In order to facilitate the understanding of this embodiment, a method for selling unmanned goods disclosed in the embodiment of the present application is first introduced in detail.

Embodiment one:

An embodiment of the present application provides an unmanned sales method, which is applied to a commodity identification server. As shown in FIG. 1, the method includes the following steps:

S11: Receive the first product image and the second product image sent by the controller in the unmanned vending machine.

The first commodity image and the second commodity image are respectively the commodity images in the box collected by the camera when the unmanned vending machine senses that the cabinet door is opened and closed through the infrared sensor.

In specific implementation, when the user opens the cabinet door of the unmanned vending machine, the infrared sensor in the unmanned machine can sense that the cabinet door is opened. At this time, the controller in the unmanned machine sends a control instruction to the camera device. The camera device is controlled to collect the commodity image inside the cabinet when the cabinet door is opened, that is, the first commodity image. In the embodiment of the present application, a plurality of camera devices may be provided, and the camera devices may be respectively installed in a plurality of positions in the box, so as to improve the accuracy of subsequent image recognition. Similarly, when a user removes an item from the cabinet and closes the cabinet door, the infrared sensor receives a sensing signal and sends a signal to close the cabinet door to the controller. The controller sends a control instruction to the camera device again to control the camera device. Collect the commodity image inside the cabinet when the cabinet door is closed, that is, the second commodity image.

S12: Identify the first product information and the second product information in the cabinet when the cabinet door is opened and closed according to the first product image, the second product image, and the product recognition model.

A commodity recognition server stores a commodity recognition model in advance, and the commodity recognition model is obtained by training a neural network model by using commodity data samples. The specific training process is as follows, as shown in FIG. 3, and will not be repeated here.

After the commodity recognition server obtains the first commodity image and the second commodity image sent by the unmanned vending machine, the first commodity image and the second commodity image are respectively input into the commodity recognition model, thereby obtaining the first commodity image corresponding to the first commodity image. A piece of product information, and a piece of second product information corresponding to the second product image.

In a preferred way of practical application, the first product image is input into the product recognition model, and the first output result of the product recognition model is used as the first product information;

S13: Generate a pre-shopping list based on the first product information and the second product information.

The pre-shopping list includes: product name, product quantity, product unit price and total price. The specific process of generating a pre-shopping list includes the following steps, as shown in Figure 2:

S21: Compare the first product information with the second product information.

S22: Determine whether the first product information is the same as the second product information.

If yes, step S23 is performed: it is determined that the user has not made a purchase.

If not, step S24 is executed: a pre-shopping list is generated according to different product information in the first product information and the second product information.

In the embodiment of the present application, the first product information and the second product information corresponding to when the cabinet door is opened and when the cabinet door is closed are compared to determine whether they are the same. If they are the same, it means that the user has not taken the product, that is, No shopping, if different, it means that the user has removed the product, and the second product information corresponding to the second product image must be less than the first product information corresponding to the first product image. Pre-shopping list is generated from different product information in the two product information.

The training process of the product recognition model involved in this method is described in detail below. The trained product recognition model can quickly identify the product image to determine the above-mentioned product information. The specific model training process includes the following steps, see figure 3 shows:

S31: Acquire a product data sample.

First, a large number of product images and product information are obtained as product data samples. The product images include: images of various products from various angles, and the product information includes: a variety of product information such as the product name and product weight corresponding to the image.

S32: Divide the product data samples into training data samples and verification data samples.

Further, the above commodity data samples are divided into training data samples and verification data samples. Preferably, 70% of the data is used to train the model, and 30% of the data is used to verify the accuracy of the model. This application does not specifically limit the above values, and other values may also be specifically used.

S33: Train the neural network model through training data samples to generate a product recognition model.

After the product data samples are divided into training data samples and verification data samples, the training data samples are input into a neural network model for training, thereby obtaining a product recognition model.

After training the product recognition model, the following steps are also included, as shown in Figure 4:

S41: Input the verification data sample into the product recognition model to verify the accuracy of the product recognition model.

When the verification result is not ideal, the parameters of the product recognition model are adjusted, and the product recognition model is continuously optimized to improve the accuracy of the product recognition model.

After generating the pre-shopping list based on the first product information and the second product information, the following steps are further included, as shown in FIG. 5:

S51: Send the pre-shopping list to the controller, so that the controller generates payment information according to the pre-shopping list.

After generating the pre-shopping list, the product identification server sends the pre-shopping list to the controller in the unmanned vending machine, and further causes the controller to generate payment information according to the pre-shopping list, so that the user makes payment according to the payment information.

S52: Receive payment success information generated by the user after the user completes the payment operation according to the payment information.

After the user completes the payment according to the payment information, the controller sends the user payment success information to the product identification server, so that the product identification server performs subsequent recording and analysis.

Or, in another implementation manner, after generating the pre-shopping list based on the first product information and the second product information, the method further includes the following steps, as shown in FIG. 6:

S61: Receive user information sent by the controller.

Before the user opens the cabinet door, first perform the code scanning operation through the mobile terminal to open the cabinet door of the unmanned vending machine through this operation. During this process, the controller will automatically receive the user's user information, such as the user's mobile phone. Number, which is the above user information. After the user completes the retrieval operation, the user information of the user is sent to the product identification server.

S62: Send the pre-shopping list to the user's mobile terminal according to the user information, so that the user makes payment according to the pre-shopping list.

Further, the product identification server sends the pre-shopping list to the user's mobile terminal according to the mobile phone number, so that the user completes the payment operation.

S63: Receive payment success information sent by the mobile terminal after the user completes the payment.

After the user completes the payment according to the above pre-shopping list, the user receives the payment success information for subsequent recording and analysis.

In addition, if the payment fails, the pre-shopping list is saved; when the user purchases again, the pre-shopping list is sent to the user's mobile terminal, so that the user can perform the purchase operation only after paying according to the pre-shopping list.

After the product identification server receives the user's payment success information, it further includes the following steps, as shown in FIG. 7:

S71: Record the sales situation information of the unmanned vending machine according to the payment success information.

After receiving the user's payment success information, the commodity identification server records the commodity information in the contract to collect statistics on the sales situation of the unmanned vending machine.

S72: Determine the current inventory situation and replenishment information based on the sales situation information.

Further, through the above-mentioned sales situation information and historical inventory information, it is possible to know the current amount of goods remaining, and determine the name and quantity of goods that the unmanned vending machine needs to replenish, that is, the above-mentioned stock situation and replenishment information.

S73: Send the replenishment information to the mobile terminal of the manager.

After determining the replenishment information of the unmanned vending machine, it is sent to the mobile terminal of the manager, so that the manager can perform the replenishment operation in a timely manner, thereby satisfying the user's shopping needs.

The unmanned sales method of this application further includes the following steps:

(1) Obtaining the sales query request of the unmanned vending machine within the target time period, and the sales query request is a request sent by the manager's mobile terminal;

(2) Determine the sales situation of the unmanned vending machine within the target time period based on the sales situation inquiry request;

(3) The sales situation is sent to the mobile terminal of the manager so that the manager can view it.

The unmanned vending method provided in the embodiment of the present application is applied to a commodity recognition server, and the method includes: receiving a first commodity image and a second commodity image sent by a controller in the unmanned vending machine; a first commodity image and a second commodity image; The product images are the product images in the box collected by the camera when the unmanned vending machine senses the cabinet door is opened and closed through the infrared sensor; the cabinet is identified based on the first product image, the second product image, and the product recognition model. The first product information and the second product information in the box when the door is opened and closed; based on the first product information and the second product information, a pre-shopping list is generated; the pre-shopping list includes: product name, product quantity, product unit price and total price. The embodiment of the present application can realize a fast and efficient sales service through the cooperation of a product identification server and an unmanned vending machine, which greatly improves the shopping experience of users.

Embodiment two:

An embodiment of the present application provides an unmanned sales device. As shown in FIG. 8, the device includes an image receiving module 81, an image recognition module 82, and a list generating module 83.

The image receiving module 81 is configured to receive a first commodity image and a second commodity image sent by a controller in the unmanned vending machine; the first commodity image and the second commodity image are respectively sensed by the unmanned vending machine through an infrared sensor. When the cabinet door is opened and closed, the product image in the box is collected by the camera device; the image recognition module 82 is configured to recognize when the cabinet door is opened and closed according to the first product image, the second product image, and the product recognition model. The first product information and the second product information in the box; the list generating module 83 is configured to generate a pre-shopping list based on the first product information and the second product information; the pre-shopping list includes: product name, product quantity, product unit price And the total price.

Optionally, the device is further configured to: obtain commodity data samples; divide the commodity data samples into training data samples and verification data samples; train the neural network model through the training data samples to generate a commodity recognition model.

Optionally, the device is further configured to: input a verification data sample into a commodity recognition model to verify the accuracy of the commodity recognition model.

Optionally, the image recognition module is further configured to: input the first product image into the product recognition model, and use the first output result of the product recognition model as the first product information; input the second product image into the product recognition model, and input the product The second output of the recognition model is used as the second product information.

Optionally, the list generation module is further configured to: compare the first product information with the second product information; determine whether the first product information is the same as the second product information; if so, determine that the user has not made a purchase; if not , Based on different product information in the first product information and the second product information, a pre-shopping list is generated.

Optionally, the device is further configured to: send the pre-shopping list to the controller, so that the controller generates payment information according to the pre-shopping list; receiving the payment sent by the controller after the user completes the payment operation according to the payment information is successful information.

Optionally, the device is further configured to: receive user information sent by the controller; send a pre-shopping list to the user's mobile terminal according to the user information, so that the user makes payment according to the pre-shopping list; and move after receiving the user's payment Payment success information sent by the terminal.

Optionally, the device is further configured to: record the sales situation information of the unmanned vending machine according to the payment success information; determine the current inventory situation and the replenishment information according to the sales situation information; and move the replenishment information to the manager Send by terminal.

Optionally, the device is further configured to: if the payment fails, save the pre-shopping list; when the user purchases again, send the pre-shopping list to the user's mobile terminal so that after the user makes payment according to the pre-shopping list, To make this purchase.

Optionally, the number of the imaging devices is multiple, and the multiple imaging devices are disposed at different positions of the cabinet.

Optionally, the device is further configured to: obtain a sales situation query request of the unmanned vending machine within a target time period, the sales situation query request is a request sent by a manager's mobile terminal; and determine the unmanned vending machine based on the sales situation query request Sales in the target time period; send sales to the mobile terminal of the manager for the manager to view.

In the unmanned vending device provided by the embodiment of the present application, each module has the same technical characteristics as the aforementioned unmanned vending method, and therefore, the above functions can also be implemented. For the specific working process of each module in the device, refer to the foregoing method embodiments, and details are not described herein again.

Embodiment three:

An embodiment of the present application further provides an unmanned vending system. As shown in FIG. 9, the system includes an unmanned vending machine 91 and a commodity identification server 92. An unmanned vending device 921 as described in the second embodiment is installed on the commodity recognition server 92; the commodity recognition server 92 is in communication with the unmanned vending machine 91; the unmanned vending machine 91 includes a vending machine body 911 and a vending machine body The controller 912, the camera 913, and the infrared sensor 914 in the 911; the vending machine body 911 includes a cabinet door and a cabinet.

In addition, the above system also includes mobile terminals, such as: smart phones, tablet computers, and so on.

In the unmanned vending system provided by the embodiment of the present application, each module has the same technical characteristics as the aforementioned unmanned vending method, and therefore, the above functions can also be implemented. For specific working processes of each module in this system, refer to the foregoing method embodiments, and details are not described herein again.

Embodiment 4:

An embodiment of the present application provides an electronic device. Referring to FIG. 10, the electronic device includes: a processor 100, a memory 101, a bus 102, and a communication interface 103. The processor 100, the communication interface 103, and the memory 101 are connected through the bus 102; The processor 100 is configured to execute an executable module, such as a computer program, stored in the memory 101. When the processor executes the program, the steps of the method as described in the method embodiment are implemented.

The memory 101 may include high-speed random access memory (RAM, Random Access Memory), and may also include non-volatile memory (non-volatile memory), such as at least one disk memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 103 (which can be wired or wireless), and the Internet, a wide area network, a local network, a metropolitan area network, and the like can be used.

The bus 102 may be an ISA bus, a PCI bus, an EISA bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only a two-way arrow is used in FIG. 10, but it does not mean that there is only one bus or one type of bus.

The memory 101 is configured to store a program, and the processor 100 executes the program after receiving the execution instruction. The method executed by the apparatus defined by the flow process disclosed in any one of the foregoing embodiments of the present application may be configured in the processor 100 Or implemented by the processor 100.

The processor 100 may be an integrated circuit chip and has a signal processing capability. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 100 or an instruction in the form of software. The above-mentioned processor 100 may be a general-purpose processor, including a central processing unit (CPU), a network processor (Network Processor) (NP), etc .; it may also be a digital signal processor (Digital Signal Processing, DSP) ), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Various methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed. A general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in combination with the embodiments of the present application may be directly implemented by a hardware decoding processor, or may be performed by using a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, and the like. The storage medium is located in the memory 101, and the processor 100 reads information in the memory 101 and completes the steps of the foregoing method in combination with its hardware.

In another embodiment of the present application, a computer storage medium is further provided, and a computer program is stored thereon, and when the computer runs the computer program, the method of any one of the foregoing method embodiments is executed. step.

The computer program product of the unmanned sales method provided in the embodiment of the present application includes a computer-readable storage medium storing a non-volatile program code executable by the processor, and the program code includes instructions configured to execute the foregoing For specific implementation of the method described in the method embodiment, refer to the method embodiment, and details are not described herein again.

Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the devices and electronic devices described above can refer to the corresponding processes in the foregoing method embodiments, and are not repeated here.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more components configured to implement a specified logical function. Executable instructions. It should also be noted that in some alternative implementations, the functions noted in the blocks may also occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented in a dedicated hardware-based system that performs the specified function or action. , Or it can be implemented with a combination of dedicated hardware and computer instructions.

In the description of this application, it should be noted that the terms "center", "upper", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. The indicated orientation or positional relationship is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing this application and simplified description, and does not indicate or imply that the device or element referred to must have a specific orientation, a specific orientation Construction and operation are therefore not to be construed as limitations on the present application. In addition, the terms "first", "second", and "third" are only configured for descriptive purposes, and should not be construed to indicate or imply relative importance.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some communication interfaces, devices or units, which may be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each of the units may exist separately physically, or two or more units may be integrated into one unit.

If the functions are implemented in the form of software functional units and sold or used as independent products, they may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of this application is essentially a part that contributes to the existing technology or a part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present application. The foregoing storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes .

Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present application, and are used to describe the technical solutions of the present application, but not to limit them. The scope of protection of the present application is not limited to this. The embodiments describe this application in detail. Those of ordinary skill in the art should understand that anyone skilled in the art can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in this application. Or you can easily think of changes, or equivalent replacements of some of the technical features; and these modifications, changes, or replacements do not deviate the essence of the corresponding technical solution from the spirit and scope of the technical solutions of the embodiments of the present application, which should be covered in this application Within the scope of protection. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Industrial applicability

The unmanned sales method, device, system, electronic equipment and computer storage medium provided in the embodiments of the present application. By applying the technical solution of the present application, the fast and efficient sales service can be realized through the cooperation of the product identification server and the unmanned vending machine, which greatly improves the shopping experience of the user.

Claims

An unmanned sales method is characterized in that it is applied to a commodity identification server, and the method includes:

Receive the first product image and the second product image sent by the controller in the unmanned vending machine; the first product image and the second product image are the unmanned vending machine sensing the door opening and When closed, the image of the goods in the box collected by the camera;

Identifying the first product information and the second product information in the cabinet when the cabinet door is opened and closed according to the first product image, the second product image, and the product recognition model;

Based on the first product information and the second product information, a pre-shopping list is generated; the pre-shopping list includes: a product name, a quantity of the product, a unit price of the product, and a total price.
The method according to claim 1, further comprising:

Obtaining product data samples;

Dividing the product data samples into training data samples and verification data samples;

A neural network model is trained by using the training data samples to generate the product recognition model.
The method according to claim 2, further comprising:

The verification data sample is input into the commodity recognition model to verify the accuracy of the commodity recognition model.
The method according to any one of claims 1 to 3, wherein, according to the first product image, the second product image, and a product recognition model, all the doors are identified when the cabinet door is opened and closed. The first product information and the second product information in the box specifically include:

Inputting the first product image into the product recognition model, and using the first output result of the product recognition model as the first product information;

Inputting the second product image into the product recognition model, and using the second output result of the product recognition model as the second product information.
The method according to any one of claims 1 to 4, wherein the generating a pre-shopping list based on the first product information and the second product information specifically includes:

Comparing the first commodity information with the second commodity information;

Determining whether the first product information is the same as the second product information;

If yes, then it is determined that the user has not made a purchase;

If not, the pre-shopping list is generated based on different product information in the first product information and the second product information.
The method according to any one of claims 1 to 5, after the generating a pre-shopping list based on the first product information and the second product information, further comprising:

Sending the pre-shopping list to the controller, so that the controller generates payment information according to the pre-shopping list;

Receiving payment success information generated by the controller after the user completes a payment operation according to the payment information.
The method according to any one of claims 1 to 5, after the generating a pre-shopping list based on the first product information and the second product information, further comprising:

Receiving user information sent by the controller;

Sending the pre-shopping list to the mobile terminal of the user according to the user information, so that the user makes payment according to the pre-shopping list;

Receive payment success information sent by the mobile terminal after the user completes the payment.
The method according to claim 6 or 7, further comprising:

Record the sales situation information of the unmanned vending machine according to the payment success information;

Determine the current inventory situation and replenishment information according to the sales situation information;

Sending the replenishment information to the mobile terminal of the manager.
The method according to claim 6 or 7, further comprising:

If the payment fails, saving the pre-shopping list;

When the user purchases again, the pre-shopping list is sent to the user's mobile terminal, so that the user can perform the purchase operation only after making payment according to the pre-shopping list.
The method according to claim 1, wherein the number of the imaging devices is plural, and the plural imaging devices are disposed at different positions of the box.
The method according to any one of claims 1 to 9, further comprising:

Obtaining a sales status query request of the unmanned vending machine within a target time period, where the sales status query request is a request sent by a manager's mobile terminal;

Determining a sales situation of the unmanned vending machine within a target time period based on the sales situation query request;

And sending the sales situation to the mobile terminal of the manager, so that the manager can check.
An unmanned vending device, comprising:

An image receiving module configured to receive a first commodity image and a second commodity image sent by a controller in the unmanned vending machine; the first commodity image and the second commodity image are respectively an unmanned vending machine and an infrared sensor; When the cabinet door is sensed to open and close, the image of the product in the box collected by the camera device;

An image recognition module configured to identify first product information and second product information in the cabinet when a cabinet door is opened and closed according to the first product image, the second product image, and a product recognition model;

The list generation module is configured to generate a pre-shopping list based on the first product information and the second product information; the pre-shopping list includes: a product name, a quantity of the product, a unit price of the product, and a total price.
An unmanned vending system, comprising: an unmanned vending machine and a commodity identification server;

The unmanned sales device according to claim 9 is installed on the commodity identification server;

The commodity identification server is communicatively connected with the unmanned vending machine;

The unmanned vending machine includes a vending machine body and a controller, a camera device, and an infrared sensor installed in the vending machine body; the vending machine body includes a cabinet door and a box body.
An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and is characterized in that the processor implements the foregoing when the computer program is executed. The method according to any one of claims 1 to 11.
A computer storage medium, characterized in that a computer program is stored thereon, and the computer executes the steps of the method according to any one of claims 1 to 11 when the computer runs the computer program.