WO2020010827A1

WO2020010827A1 - Open type vending machine-based shopping settlement method and device

Info

Publication number: WO2020010827A1
Application number: PCT/CN2019/070038
Authority: WO
Inventors: 黄鼎隆; 斯科特马修•罗伯特; 马咪娜; 王海涵
Original assignee: 深圳码隆科技有限公司
Priority date: 2018-07-09
Filing date: 2019-01-02
Publication date: 2020-01-16
Also published as: CN109035629A

Abstract

Provided are an open type vending machine-based shopping settlement method and device. The method comprises: receiving a door opening sensing signal and acquiring a first commodity display image in a shopping cabin (S10); identifying the first commodity display image to determine first commodity information (S20); acquiring commodity shopping images during the shopping of a user (S30); receiving a door closing sensing signal (S40); identifying the commodity shopping images to determine second commodity information (S50); and determining a shopping list according to the comparison result of the second commodity information and the first commodity information so as to carry out settlement according to the shopping list (S60). By means of the method and device, shopping lists are determined and generated by utilizing the image identifying technology in an open type shopping environment, guarantee is provided for final generation of more accurate shopping lists; the settlement precision is improved; the labor cost is lowered; the settlement duration is shortened; the settlement efficiency is increased; the shopping process is simple; and user experience is improved.

Description

Shopping settlement method and device based on open vending machine

This application is based on a Chinese patent application with an application number of 201810744665.1 and an application date of July 9, 2018, and claims its priority. The entire content of this application is hereby incorporated into this application as a whole.

Technical field

The present application relates to the field of image recognition technology, and more particularly, to a shopping settlement method and device based on an open vending machine.

Background technique

Material circulation is the most basic element of human society. The retail industry or an open related industry that sells directly to consumers (hereinafter referred to as the retail industry or retail enterprises) as an important means of current material circulation needs to employ a large number of cashiers to realize the purchase and sale of goods. The shopping items are counted piece by piece. After the statistics are completed, the payment is completed with the consumer.

The automatic vending container solves the shortcomings of manual statistics and manual settlement in the retail industry. It is a consumer-oriented unmanned sales and automatic settlement sales method. Through the selection of the user, the types and total value of the products selected by the user can be counted for the user to settle and complete the payment process. However, the current vending container can only perform automatic settlement based on the user's selection, and cannot accurately identify the user's shopping behavior and obtained in an open retail shopping environment, thereby achieving settlement.

In short, in the existing open shopping environment, only one-by-one inspection, statistics, and total price calculation of the goods purchased by consumers can be performed manually, which consumes a lot of labor costs, has long settlement time, low efficiency, and accuracy. Poor, and the user experience is poor.

Application content

In view of this, the present application provides a shopping settlement method and device based on an open vending machine to solve the shortcomings of the prior art.

In order to solve the above problems, this application provides a shopping settlement method based on an open vending machine, including:

Receiving a door opening sensor signal generated based on the opening of a cabinet door of a shopping warehouse based on a vending machine, and collecting a first product display image in the shopping warehouse according to the door opening sensor signal;

Identifying the first product display image, and determining first product information corresponding to the first product display image;

During a user shopping process, collecting a product shopping image in the shopping warehouse;

Receiving a door closing sensor signal generated based on the cabinet door closing;

Identifying the product shopping image, and determining second product information in the product shopping image;

Determine a shopping list according to a comparison result between the second product information and the first product information, so as to facilitate settlement according to the shopping list.

Preferably, before the “receiving the door closing sensor signal generated based on the closing of the cabinet door”, the method further includes:

Receiving an unmanned sensor signal returned by an infrared sensor in a shopping area in the vending machine, and establishing a time stamp corresponding to the unmanned sensor signal;

If the time elapsed from the time point when the time stamp of the unmanned sensor signal was received reaches a preset time threshold, it is determined that the user has left the shopping area, and the shopping cart of the vending machine is closed. Cabinet door.

Preferably, the "collecting a shopping image of goods in the shopping warehouse during a user's shopping process" includes:

During the user's shopping process, receiving the gravity sensing information returned by the gravity sensors corresponding to different product display areas in the shopping warehouse;

According to the gravity sensing information, image collection is performed on the commodity display area corresponding to the gravity sensor to obtain the commodity shopping image.

Collecting a continuous shooting shopping image of a shopping action of the user;

Identifying the continuous shooting shopping image, and determining whether the user performs a return operation on the returned merchandise;

If yes, determining the placement position of the user for the returned product according to the continuous shooting shopping image;

If the placement position is inconsistent with the preset product placement area corresponding to the returned product, a position error prompt is played through a voice module, so as to prompt the user that the returned product is placed in the wrong position.

Preferably, after the "if the placement position is inconsistent with the preset product placement area corresponding to the returned merchandise, playing a position error prompt through a voice module" further includes:

Controlling the LED indicator light corresponding to the preset product display area to light up to prompt the user to place the returned product in the preset product display area corresponding to the LED indicator.

Preferably, after “determining a shopping list according to a comparison result between the second product information and the first product information, so as to facilitate settlement according to the shopping list”, the method further includes:

After the user performs settlement according to the shopping list, collecting a second product display image in a shopping warehouse of the vending machine;

Identifying the second product display image, and determining second product information corresponding to the second product display image;

If the number of products corresponding to the product variety in the second product information is less than a preset product replenishment threshold corresponding to the product variety, determining that the product corresponding to the product variety needs to be restocked;

Generate a replenishment list based on the product corresponding to the product variety, so as to facilitate replenishment according to the replenishment list.

In addition, in order to solve the above problems, the present application also provides a shopping settlement device based on an open vending machine, including: a receiving module, an identification module, a collection module, and a settlement module;

The receiving module is configured to receive a door opening sensor signal generated by opening a cabinet door of a shopping warehouse based on a vending machine, and collect a first product display image in the shopping warehouse according to the door opening sensor signal;

The identification module is configured to identify the first product display image and determine first product information corresponding to the first product display image;

The collection module is configured to collect a shopping image of goods in the shopping warehouse during a user shopping process;

The receiving module is further configured to receive a door closing sensor signal generated based on the cabinet door closing;

The identification module is further configured to identify the product shopping image and determine second product information in the product shopping image;

The settlement module is configured to determine a shopping list according to a comparison result between the second product information and the first product information, so as to facilitate settlement according to the shopping list.

In addition, in order to solve the above problem, the present application also provides a user terminal including a memory and a processor, where the memory is used to store a shopping settlement program based on an open vending machine, and the processor runs the open automatic The shopping settlement program of the vending machine causes the user terminal to execute the shopping settlement method based on the open vending machine as described above.

In addition, in order to solve the above-mentioned problems, the present application also provides a computer-readable storage medium, where the computer-readable storage medium stores a shopping settlement program based on an open vending machine, and the shopping based on the open vending machine When the settlement program is executed by the processor, the shopping settlement method based on the open vending machine is realized as described above.

The present application provides a shopping settlement method and device based on an open vending machine. Wherein, the method provided in the present application collects display images in the shopping warehouse through the door opening sensor signal to obtain the first product information, and when the user finishes shopping and closes the cabinet door, the second product information is obtained through image acquisition and identification, The first product information and the second product information are compared, and a shopping list is determined according to the comparison result, so as to facilitate settlement based on the list. Through the shopping settlement method based on the open vending machine provided in this application, it is realized that in an open shopping environment, by using image recognition technology to confirm the products the user has taken from the shopping warehouse, and generate a shopping list, image recognition Technology provides accurate guarantee for ultimately generating a more accurate settlement list, greatly improving the accuracy of settlement, reducing labor costs, shortening settlement time, high settlement efficiency, simple shopping process, and improving user experience.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic structural diagram of a hardware operating environment involved in a solution of an embodiment of a shopping settlement method based on an open vending machine;

2 is a schematic flowchart of a first embodiment of a shopping settlement method based on an open vending machine in this application;

3 is a schematic flowchart of a second embodiment of a shopping settlement method based on an open vending machine in this application;

4 is a schematic flowchart of a third embodiment of a shopping settlement method based on an open vending machine in this application;

5 is a schematic flowchart of a fourth embodiment of a shopping settlement method based on an open vending machine in this application;

6 is a schematic flowchart of a fifth embodiment of a shopping settlement method based on an open vending machine in this application;

FIG. 7 is a functional block diagram of a shopping settlement device based on an open vending machine of the present application.

The implementation, functional characteristics and advantages of the purpose of this application will be further described with reference to the embodiments and the drawings.

detailed description

The embodiments of the present application are described in detail below, in which the same or similar reference numerals indicate the same or similar elements or elements having the same or similar functions.

In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality" is two or more, unless it is specifically and specifically defined otherwise.

In this application, the terms "installation," "connected," "connected," and "fixed" should be understood broadly unless otherwise specified and limited, for example, they may be fixed connections or removable connections Or integrated; it can be mechanical or electrical; it can be directly connected or indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific situations.

It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

As shown in FIG. 1, FIG. 1 is a schematic structural diagram of a hardware operating environment of a terminal involved in a solution according to an embodiment of the present application.

In the embodiment of the present application, the terminal may be a PC provided in an automatic container machine, or may be a mobile terminal device such as a smartphone, a tablet computer, an e-book reader, an MP3 player, an MP4 player, and a portable computer. In addition, it may be a computer hardware device provided in the automatic container machine itself.

As shown in FIG. 1, the terminal may include a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to implement connection and communication between these components. The user interface 1003 may include a display screen, an input unit such as a keyboard, a remote controller, and the optional user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory, such as a magnetic disk memory. The memory 1005 may optionally be a storage device independent of the foregoing processor 1001. In addition, the terminal also includes an image acquisition device 1006, which may specifically be a camera, a camera, and the like. The image acquisition device 1006 is connected to the processor 1001 through a communication bus 1002. In addition, the terminal also includes a gravity sensor 1007. The gravity sensor is connected to the processor 1001 through a communication bus 1002. Optionally, the terminal may further include an RF (Radio Frequency) circuit, an audio circuit, a WiFi module, and the like. In addition, the mobile terminal may be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, which will not be repeated here.

Those skilled in the art can understand that the terminal shown in FIG. 1 does not constitute a limitation on the terminal, and may include more or fewer components than those shown in the figure, or combine some components, or arrange different components.

As shown in FIG. 1, the memory 1005 as a computer-readable storage medium may include an operating system, a data interface control program, a network connection program, and a shopping settlement program based on an open vending machine.

The present application provides a shopping settlement method and device based on an open vending machine. Wherein, the method realizes that in an open shopping environment, by using image recognition technology to confirm the products the user has taken from the shopping warehouse and generate a shopping list, the image recognition technology provides accuracy for eventually generating a more accurate settlement list Guarantee, greatly improving the accuracy of settlement, reducing labor costs, shortening settlement time, high settlement efficiency, simple shopping process, and improving user experience.

Example 1:

Referring to FIG. 2, a first embodiment of the present application provides a shopping settlement method based on an open vending machine, including:

Step S10: Receive a door opening sensor signal generated based on the opening of a cabinet door of the shopping warehouse of the vending machine, and collect a first product display image in the shopping warehouse according to the door opening sensor signal;

As mentioned above, the method provided in this embodiment is applied to a vending machine or an vending scene with an open shopping environment. For example, in an open vending machine, after opening the cabinet door, the user can enter it for selection. Commodity, settlement after closing the cabinet door.

As mentioned above, the shopping warehouse is an area where the merchandise is displayed inside the vending machine, and the interior can be divided into different display areas, or a display layer or a display grid.

As described above, the first product display image is an image collected by a camera inside the vending machine for the products displayed in the shopping warehouse before the shopping process is performed, for further comparison. The internally collected images can be image collection of the goods in the entire container through a camera, or a camera can be set for each area to perform image collection of the goods in the shopping warehouse.

Step S20: identify the first product display image, and determine first product information corresponding to the first product display image;

As mentioned above, through image recognition, the quantity of goods corresponding to each variety in the goods comparison image is identified, and an inventory record is obtained.

As mentioned above, image recognition can be neural network recognition. By training an artificial neural network model in advance, a corresponding variety, that is, a corresponding quantity is identified.

As mentioned above, the first product information may include, but is not limited to, information such as product variety, product specifications, and product quantity.

As described above, in this embodiment, before a user purchases a product displayed in a container, image collection is performed, and the collected image is identified to determine the product information contained in the image, and the product variety and corresponding Quantity, control data for further comparison.

Step S30: during the shopping process of the user, collect a shopping image of the goods in the shopping warehouse;

As mentioned above, the user shopping process is a shopping process initiated for the user to start, for example, starting with an infrared sensor, or connecting a smart mobile terminal with a vending machine through a network by the user to implement the starting shopping process. During the shopping process, the image in the shopping warehouse is collected as the product shopping image, and the real-time image recognition can be used to obtain the product variety and quantity captured by the user.

Step S40: Receive a door closing sensor signal generated based on the cabinet door closing;

As mentioned above, in this embodiment, the cabinet door of the vending machine is provided with corresponding sensors, such as an infrared sensor, a gravity sensor or an acceleration sensor, etc., and the sensor is used to obtain a door closing sensor signal generated when the user closes the cabinet door after shopping. , This signal is an instruction determination for the user to end taking the product.

In addition, it is also possible for the user to end the shopping process and close the cabinet door through the mobile terminal or operate the vending machine, or the user can send a signal to end the shopping process through a voice or image recognition action instruction, thereby closing the cabinet door.

Step S50: identifying the product shopping image, and determining second product information in the product shopping image;

As mentioned above, the second product information is data information obtained by identifying a product shopping image, where the product shopping image can be acquired once or multiple times during the user's shopping process, and different regions can be based on different regions. Placed goods are imaged. For the acquisition of product shopping images in the shopping warehouse, image recognition can be used to acquire an image after each change in shopping information through image recognition to obtain a product shopping image. The product shopping image for image recognition in S50 can be the latest or The most recently acquired image.

As mentioned above, the second product information may include, but is not limited to, data information such as the current product type, the number of products, and the product specifications of the user in the shopping warehouse after shopping.

Step S60: Determine a shopping list according to a comparison result between the second product information and the first product information, so as to facilitate settlement according to the shopping list.

As described above, the second product information is compared with the first product information to generate a comparison result, and a shopping list may be further determined based on the result. When comparing the second product information with the first product information, the second product information may be compared with the corresponding product variety and quantity in the first product information to determine the reduced product variety and the corresponding quantity. In this way, the variety and quantity of the goods taken by the user are determined, and a shopping list corresponding to the variety of goods is generated, so that the user can make settlement according to the list.

The method provided in this embodiment collects a display image in a shopping warehouse through a door opening sensor signal to obtain first product information, and when the user finishes shopping and closes the cabinet door, acquires second product information through image collection and identification, and The first product information and the second product information are compared, and a shopping list is determined according to the comparison result, so as to facilitate settlement based on the list. With the shopping settlement method based on the open vending machine provided in this embodiment, it is realized that in an open shopping environment, by using image recognition technology to confirm the products that the user has taken from the shopping warehouse, and generate a shopping list, image Identification technology provides accurate guarantee for the final generation of a more accurate settlement list, which greatly improves the accuracy of settlement, reduces labor costs, shortens settlement time, high settlement efficiency, simple shopping process, and improved user experience.

Example 2:

Referring to FIG. 3, a second embodiment of the present application provides a shopping settlement method based on an open vending machine. Based on the first embodiment shown in FIG. 2 above, the step S40, Before the "door closing sensor signal", it also includes:

Step S70: Receive an unmanned sensing signal returned by an infrared sensor in a shopping area in the vending machine, and establish a time stamp corresponding to the unmanned sensing signal;

As mentioned above, the shopping area is an area in terms of the open shopping environment in the vending machine. In the open vending machine, after opening the cabinet door, there will be a shopping area for the user to carry out shopping of the selected products. In the shopping area, it is determined that the user starts shopping, and when leaving the shopping area, it is determined that the user ends shopping.

As described above, the infrared sensor is disposed in a position in the vending machine that can sense whether the user enters the shopping area, and when the user leaves the shopping area, a door closing sensor signal is generated.

Timestamp (timestamp), a complete, verifiable piece of data that can represent a piece of data that existed before a certain time, usually a sequence of characters that uniquely identifies the time of a moment. The data generated by digital signature technology includes the original file information, signature parameters, signature time and other information. It is widely used in intellectual property protection, contract signing, financial accounting, electronic quotation and bidding, and stock trading.

As mentioned above, the time stamp is a time point that characterizes the unmanned sensor signal.

In step S80, if the time elapsed from the time point when the time stamp of the unmanned sensor signal is received reaches a preset time threshold, it is determined that the user has left the shopping area and shut down the vending machine. Cabinet door of shopping warehouse.

As described above, the system automatically records the time after the unmanned sensor signal is generated, that is, the duration reached from the time point. If the time reaches the preset time threshold, it means that the time when the user leaves the shopping area reaches the preset time threshold. If the time is reached, it is determined that the user has left the shopping area and the shopping process is ended, the door of the shopping warehouse is automatically closed.

In addition, the cabinet door can also be closed manually by the user.

In this embodiment, in an open shopping environment, an unmanned sensor signal used to characterize the user leaving the shopping area is obtained through an infrared sensor, and the duration of the received signal is recorded, if the duration reaches a preset time Threshold, it is determined that the user leaves the shopping area and ends the shopping process, then the cabinet door is automatically controlled to close, which improves the convenience of the vending machine, and provides the automatic door closing function for the open vending machine, thereby enabling automatic The vending machine is more intelligent, preventing the problem of being unable to make further settlement due to the user leaving the cabinet door and forgetting to close it.

Example 3:

Referring to FIG. 4, a third embodiment of the present application provides a shopping settlement method based on an open vending machine. Based on the first embodiment shown in FIG. 2 described above, the step S30 is, The "shopping image of goods in the shopping warehouse" includes:

Step S31: During the user's shopping process, receive the gravity sensing information returned by the gravity sensors corresponding to different product display areas in the shopping warehouse;

As mentioned above, the product display area is an area where products are classified according to specifications and varieties. For example, there are containers in the shopping warehouse. Each container is provided with a plurality of separated cells, and each cell is placed in each cell. A product that is different from other products, each cell is a product display area.

In the above, a corresponding gravity sensor can be set in each product display area to sense the change of gravity induction in the product display area. If the user grabs the product in the area, the change in quality reaches a threshold, and the gravity is triggered. The sensor generates a gravity sensing message.

In step S32, according to the gravity sensing information, image collection is performed on the product display area corresponding to the gravity sensor to obtain the product shopping image.

As mentioned above, in this embodiment, a corresponding image acquisition device, such as a camera, is provided in each product display area. Each camera captures an image of a product display area correspondingly. After receiving the gravity sensing information, according to the gravity sensing information, an image acquisition device corresponding to a corresponding product display area is turned on, and images of the product display area are collected to obtain a product shopping image. For specific product display areas, targeted image acquisition is performed, and image acquisition is triggered based on data changes from the gravity sensor, which makes the settlement of goods more intelligent, improves the accuracy and precision of commodity settlement, and provides users with The self-service shopping provides convenience.

Example 4:

Referring to FIG. 5, a fourth embodiment of the present application provides a shopping settlement method based on an open vending machine. Based on the first embodiment shown in FIG. 2 above, the step S40, Before the "door closing sensor signal", it also includes:

Step S90: Collect continuous shooting shopping images of the shopping actions of the user;

As mentioned above, the continuous shooting shopping image may be a continuous shooting still image, or a dynamic GIF animation or video file.

Step S100: identify the continuous shooting shopping image, and determine whether the user performs a return operation on the returned merchandise;

As described above, the continuous shooting image is identified, so as to determine whether the user performs a return operation of a returned product based on the recognition result.

The identification process can include:

S1. Extract key frames of the continuous shooting shopping image, and construct a timestamp corresponding to each of the key frames;

S2. Perform threshold segmentation on the temporally continuous key frames to obtain a binary image, including:

S2-1, converting the temporally continuous key frames of each frame into an HSV color space to obtain an HSV image;

S2-2. Perform a histogram analysis on the HSV image to obtain a threshold range corresponding to the HSV image;

S2-3. Set the pixel values of the pixel values in the three dimensions of H, S, and V within the threshold range to 1, and set the pixel values of the three dimensions of H, S, and V not in the threshold. Pixels in the range are set to 0 to obtain the binary image after the segmentation.

S3. Using the hand image of the user's uncaptured product as a comparative seed image, comparing the segmented binary image to frame all the key frames, which is different from the comparative seed image. Image of the target product to obtain a target keyframe with the product.

S4. Recognize the target key frame through a pre-trained artificial neural network, and obtain the variety and quantity of the commodities included in the target key frame.

S5. Construct a movement trajectory of the commodity according to the timestamp; and obtain a movement vector of the commodity movement according to the movement trajectory;

S6. According to the movement vector, determine that the product is a returned product and move to a vending machine;

S7. If the product is a returned product and moved to a vending machine, it is determined that the user performs a return operation on the returned product;

S8. If the product is not a returned product and has not been moved to the vending machine, it is determined that the user has not performed a return operation on the returned product;

Step S110: if yes, determine the placement position of the user for the returned product according to the continuous shooting shopping image;

As described above, the placement position is determined based on the continuous shooting shopping image. The relative coordinates of the image corresponding to each product display area can be defined based on the overall image collected before the shopping process begins. Position, based on the relative coordinates of the image, to obtain the relative coordinates of the placement position.

In step S120, if the placement position is not consistent with the preset product placement area corresponding to the returned product, a position error prompt is played through a voice module, so as to prompt the user that the returned product is placed in the wrong position.

As mentioned above, the preset product placement area is the correct classified placement position corresponding to each product.

As mentioned above, the display position can be further compared with the preset product display area, and the coordinate ranges of the two can be compared. If the coordinates match, or the coordinates of the display position belong to the coordinate range of the preset product display area . If they are inconsistent, or not within their range, an error prompt is played through the voice module to remind the user that the location is the wrong placement, for example: "You placed the product in the wrong location, please place it in the correct XX location, Thank you!"

In step S130, the LED indicator corresponding to the preset commodity display area is controlled to light up to prompt the user to place the returned product in the preset commodity display area corresponding to the LED indicator.

As described above, each preset commodity display area is provided with an LED indicator, and when the user places the returned product, the LED indicator of the correct preset commodity display area may be lighted up. In this embodiment, after the error prompt is played through the voice module, the LED indicator corresponding to the correct product display area can be turned on, or the color of the indicator can be changed, or a strobe prompt can be performed to prompt the user to return the returned product. The product is placed in the preset product placement area corresponding to the LED indicator. In this embodiment, image recognition technology is used to identify the continuous shooting shopping image of the user's shopping action, so as to determine whether the user performs a return operation, and through image recognition, determine whether the location of the user's returned merchandise is placed correctly. The original preset product placement area, if not, will be prompted through the voice module, and the LED indicator corresponding to the preset product placement area will be used to remind the user to place the returned product in the correct position. , To realize the intelligent monitoring of the placement of goods, to provide convenience for users shopping, to prevent the situation where users randomly place products to affect the shopping process of other users.

Example 5:

Referring to FIG. 6, a fifth embodiment of the present application provides a shopping settlement method based on an open vending machine. Based on the first embodiment shown in FIG. 2 described above, the step S60 is “based on the second product information and After the comparison result of the first product information determines a shopping list so as to facilitate settlement according to the shopping list, the method further includes:

Step S140: After the user performs settlement according to the shopping list, collect a second product display image in a shopping warehouse of the vending machine;

Step S150: identify the second product display image, and determine second product information corresponding to the second product display image;

As described above, after the settlement is completed, a second product display image is collected, and the image may be an entire image in the shopping warehouse or an image for each shopping cell. By identifying the second product display image, second product information of the product included in the image can be obtained. The second commodity information includes information such as a variety of the commodity and a quantity corresponding to the variety.

Step S160: if the number of products corresponding to the product variety in the second product information is less than a preset product replenishment threshold corresponding to the product variety, determine that the product corresponding to the product variety needs to be restocked;

As mentioned above, the preset product replenishment threshold is the variety of each product, and there is a preset product replenishment threshold. The preset product replenishment threshold of each product can be different or the same. When the number of products of a product is less than this, When the preset product replenishment threshold corresponding to the product is determined, the product needs to be replenished. For example, Cola's preset replenishment threshold for products is 5 bottles and 4 bottles in the vending machine. At this time, it is determined that Cola needs to be replenished.

Step S170: Generate a replenishment list based on the product corresponding to the product variety, so as to facilitate replenishment according to the replenishment list.

As mentioned above, the replenishment list is a list of replenishment for operation and maintenance personnel. When a replenishment is required, a replenishment list is generated, the contact information of the operation and maintenance personnel is obtained, and a replenishment list is generated for the administrator. In this embodiment, the image recognition technology is used to determine whether the product reaches the preset product replenishment threshold, and if there is a product that needs to be replenished, a replenishment list is generated to facilitate replenishment according to the replenishment list, so that Maintenance personnel's replenishment work provides convenience and can be targeted for replenishment according to the replenishment list.

In addition, referring to FIG. 7, the present application also provides a shopping settlement device based on an open vending machine, including: a receiving module 10, an identification module 20, a collection module 30, and a settlement module 40;

The receiving module 10 is configured to receive a door opening sensor signal generated by opening a cabinet door of a shopping warehouse based on a vending machine, and collect a first product display image in the shopping warehouse according to the door opening sensor signal;

The identification module 20 is configured to identify the first product display image and determine first product information corresponding to the first product display image;

The collection module 30 is configured to collect a product shopping image in the shopping warehouse during a user shopping process;

The receiving module 10 is further configured to receive a door closing sensor signal generated based on the cabinet door closing;

The identification module 20 is further configured to identify the product shopping image and determine second product information in the product shopping image;

The settlement module 40 is configured to determine a shopping list according to a comparison result between the second product information and the first product information, so as to facilitate settlement according to the shopping list.

In addition, the present application also provides a user terminal including a memory and a processor, where the memory is configured to store a shopping settlement program based on the open vending machine, and the processor runs the shopping settlement based on the open vending machine. A program to cause the user terminal to execute the shopping settlement method based on the open vending machine as described above.

In addition, the present application also provides a computer-readable storage medium, where the computer-readable storage medium stores a shopping settlement program based on an open vending machine, and the shopping settlement program based on the open vending machine is processed by a processor When executed, the shopping settlement method based on the open vending machine is realized as described above.

The above-mentioned serial numbers of the embodiments of the present application are merely for description, and do not represent the superiority or inferiority of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods in the above embodiments can be implemented by means of software plus a necessary universal hardware platform, and of course, also by hardware, but in many cases the former is better. Implementation. Based on such an understanding, the technical solution of the present application, in essence, or a part that contributes to the existing technology, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM) as described above. , Magnetic disk, optical disc), including a number of instructions to enable a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the embodiments of the present application.

The above are only preferred embodiments of the present application, and thus do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of the application, or directly or indirectly used in other related technical fields Are included in the scope of patent protection of this application.

Claims

A shopping settlement method based on an open vending machine is characterized in that it includes:

Receiving a door opening sensor signal generated based on the opening of a cabinet door of a shopping warehouse based on a vending machine, and collecting a first product display image in the shopping warehouse according to the door opening sensor signal;

Identifying the first product display image, and determining first product information corresponding to the first product display image;

During a user shopping process, collecting a product shopping image in the shopping warehouse;

Receiving a door closing sensor signal generated based on the cabinet door closing;

Identifying the product shopping image, and determining second product information in the product shopping image;

Determine a shopping list according to a comparison result between the second product information and the first product information, so as to facilitate settlement according to the shopping list.
The shopping settlement method based on the open vending machine according to claim 1, wherein before "receiving a door closing sensor signal generated based on closing of said cabinet door", further comprising:

Receiving an unmanned sensor signal returned by an infrared sensor in a shopping area in the vending machine, and establishing a time stamp corresponding to the unmanned sensor signal;

If the time elapsed from the time point when the time stamp of the unmanned sensor signal was received reaches a preset time threshold, it is determined that the user has left the shopping area, and the shopping cart of the vending machine is closed. Cabinet door.
The shopping settlement method based on the open vending machine according to claim 1, wherein the "collecting a shopping image of goods in the shopping warehouse during a user's shopping process" comprises:

During the user's shopping process, receiving the gravity sensing information returned by the gravity sensors corresponding to different product display areas in the shopping warehouse;

According to the gravity sensing information, image collection is performed on the commodity display area corresponding to the gravity sensor to obtain the commodity shopping image.
The shopping settlement method based on the open vending machine according to claim 1, wherein before "receiving a door closing sensor signal generated based on closing of said cabinet door", further comprising:

Collecting a continuous shooting shopping image of a shopping action of the user;

Identifying the continuous shooting shopping image, and determining whether the user performs a return operation on the returned merchandise;

If yes, determining the placement position of the user for the returned product according to the continuous shooting shopping image;

If the placement position is inconsistent with the preset product placement area corresponding to the returned product, a position error prompt is played through a voice module, so as to prompt the user that the returned product is placed in the wrong position.
The shopping settlement method based on the open vending machine according to claim 4, characterized in that, "if the placement position is not consistent with the preset product placement area corresponding to the returned product, play it through a voice module. "Position error prompt" also includes:

Controlling the LED indicator light corresponding to the preset product display area to light up to prompt the user to place the returned product in the preset product display area corresponding to the LED indicator.
The method according to claim 1, wherein the "determining a shopping list based on a comparison result between said second product information and said first product information, so as to facilitate After the shopping list is settled ", it also includes:

After the user performs settlement according to the shopping list, collecting a second product display image in a shopping warehouse of the vending machine;

Identifying the second product display image, and determining second product information corresponding to the second product display image;

If the number of products corresponding to the product variety in the second product information is less than a preset product replenishment threshold corresponding to the product variety, determining that the product corresponding to the product variety needs to be restocked;

Generate a replenishment list based on the product corresponding to the product variety, so as to facilitate replenishment according to the replenishment list.
A shopping settlement device based on an open vending machine is characterized in that it includes: a receiving module, an identification module, a collection module, and a settlement module;

The receiving module is configured to receive a door opening sensor signal generated by opening a cabinet door of a shopping warehouse based on a vending machine, and collect a first product display image in the shopping warehouse according to the door opening sensor signal;

The identification module is configured to identify the first product display image and determine first product information corresponding to the first product display image;

The collection module is configured to collect a shopping image of goods in the shopping warehouse during a user shopping process;

The receiving module is further configured to receive a door closing sensor signal generated based on the cabinet door closing;

The identification module is further configured to identify the product shopping image and determine second product information in the product shopping image;

The settlement module is configured to determine a shopping list according to a comparison result between the second product information and the first product information, so as to facilitate settlement according to the shopping list.
A user terminal, comprising a memory and a processor, the memory is configured to store a shopping settlement program based on an open vending machine, and the processor runs the shopping settlement program based on the open vending machine to The user terminal is caused to execute the shopping settlement method based on the open vending machine according to any one of claims 1-6.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores a shopping settlement program based on an open vending machine, and when the shopping settlement program based on an open vending machine is executed by a processor A shopping settlement method based on any one of claims 1 to 6 is realized.