CN112507762A - Intelligent vending machine control method and system based on commodity image recognition - Google Patents
Intelligent vending machine control method and system based on commodity image recognition Download PDFInfo
- Publication number
- CN112507762A CN112507762A CN201910872072.8A CN201910872072A CN112507762A CN 112507762 A CN112507762 A CN 112507762A CN 201910872072 A CN201910872072 A CN 201910872072A CN 112507762 A CN112507762 A CN 112507762A
- Authority
- CN
- China
- Prior art keywords
- commodity
- image
- vending machine
- displaying
- shelf
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 230000003993 interaction Effects 0.000 claims description 3
- 230000001419 dependent effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Control Of Vending Devices And Auxiliary Devices For Vending Devices (AREA)
Abstract
The application provides an intelligent vending machine control method based on commodity image recognition, which comprises the following steps: collecting a commodity shelf image and displaying the collected commodity shelf image, wherein the commodity shelf image can be dragged, rotated and zoomed so as to be convenient for viewing commodities in the commodity shelf; when detecting that the commodity in the commodity frame image is selected, acquiring the image of the selected commodity, and identifying the acquired commodity image by using a deep convolution network training method; judging whether the identified commodity is a sold commodity; if yes, displaying the commodity label of the selected commodity; if not, displaying prompt information. The identification mode of the application is more intelligent and automatic, the identification range is wider, and the transaction is more rapid and convenient.
Description
Technical Field
The invention relates to the technical field of automatic vending equipment, in particular to an intelligent vending machine control method and system based on commodity image recognition.
Background
Unmanned vending machines are increasingly widely applied to the market, and most of the unmanned vending machines in the market identify commodities based on electronic tags or manual input of customers. When the electronic tag is used, the electronic tag is added to each commodity, and then the target object is automatically identified through the radio frequency signal and relevant data is obtained. This method is labor intensive and prone to error. Based on manual input of customers, the user is required to input the goods channel number of the goods to be purchased, and the method requires the user to input continuous numbers, so that errors are easy to occur.
Disclosure of Invention
Aiming at the problems in the prior art, the application provides an intelligent vending machine control method and system based on commodity image recognition, and the method and system are used for solving the technical problems.
In a first aspect, the application provides an intelligent vending machine control method based on commodity image recognition, which includes the following steps:
s10: acquiring a commodity shelf image and displaying the acquired commodity shelf image, wherein the commodity shelf image can be dragged and/or rotated and/or zoomed so as to be convenient for viewing commodities in the commodity shelf;
s20: when detecting that the commodity in the commodity shelf image is selected, acquiring the image of the selected commodity and identifying the acquired commodity image;
s30: judging whether the identified commodity is a sold commodity;
if yes, displaying the commodity label of the selected commodity; if not, displaying prompt information;
in step S20, the recognizing the captured commodity image includes: and generating an image of the sold commodity, training the image of the sold commodity by using a deep convolutional network training method, acquiring a trained model parameter, and identifying the acquired commodity image by using the trained model parameter.
In an embodiment according to the first aspect, in step S10, the commodity shelf image is a commodity shelf perspective view, a commodity shelf front view and/or a commodity shelf video.
In one embodiment according to the first aspect, the item label comprises an item name and an item price.
In one embodiment according to the first aspect, in step S20, the image of the item sold is generated by:
and generating an image of the sold commodity in a man-machine interaction mode, or generating the image of the sold commodity in a scanning mode.
In one embodiment according to the first aspect, when the identified article is a vended article, further comprising the steps of:
s40: displaying selectable payment modes;
s50: executing a payment process when any payment mode is detected to be selected;
s60: and when the payment is judged to be successful, controlling the selected commodity to be delivered.
In one embodiment according to the first aspect, after the identified product is not a vended product and the indication information is displayed, the product shelf image acquired in step S10 is displayed.
In a second aspect, the present application provides a smart vending machine control system based on commodity image recognition, including:
the display module is used for displaying the commodity shelf image, and displaying the commodity label of the commodity and prompt information;
the image acquisition device is used for acquiring the commodity shelf image and acquiring the image of the selected commodity;
the recognition module is used for training the images of the sold commodities by utilizing a deep convolutional network training method to obtain trained model parameters and recognizing the selected commodities according to the trained model parameters;
the judging module is used for judging whether the identified commodity is a selling commodity or not;
the control module is used for controlling according to the judgment result of the judgment module;
and if the identified commodity is judged to be a sold commodity, displaying a commodity label of the commodity, and if not, displaying prompt information.
In one embodiment according to the second aspect, the identification module further comprises:
and the storage submodule is used for storing the trained model parameters.
In one embodiment according to the second aspect, the image capture module is further configured to capture an image of the item being sold.
In a third aspect, the present application provides a smart vending machine that employs the smart vending machine control method according to the first aspect.
Compared with the prior art, the method has the following advantages:
the intelligent vending machine control method is based on commodity image recognition, and abandons the mode of electronic tag recognition and input of a channel number of a commodity to be purchased in the prior art, and firstly obtains model parameters of the sold commodity by using a deep convolution network training method, stores the model parameters in the intelligent vending machine in advance, and recognizes the image of the selected commodity by using the model parameters. The identification mode is more intelligent and automatic, the identification range is wider, and the transaction is quicker and more convenient. Secondly, the image of the commodity shelf is obtained in a mode of scanning the commodity shelf, and a user can drag, rotate and/or zoom so as to conveniently and clearly view commodities in the commodity shelf.
The features mentioned above can be combined in various suitable ways or replaced by equivalent features as long as the object of the invention is achieved.
Drawings
The invention will be described in more detail hereinafter on the basis of embodiments and with reference to the accompanying drawings. Wherein:
fig. 1 shows a flowchart of a smart vending machine control method based on commodity image recognition according to the present application.
Fig. 2 shows a flow chart of smart vending machine article identification based on article image identification according to the present application.
FIG. 3 illustrates a deep convolutional network training process and commodity identification process according to the present application.
In the drawings, like parts are provided with like reference numerals. The drawings are not to scale.
Detailed Description
The invention will be further explained with reference to the drawings.
Fig. 1 shows a flowchart of a smart vending machine control method based on commodity image recognition according to the present application, including the following steps:
the first step is as follows: the method comprises the steps that a commodity shelf image is collected and output and displayed, and the commodity shelf image which is output and displayed can be dragged, rotated and/or zoomed so as to be convenient for viewing commodities in the commodity shelf;
the second step is that: when detecting that the commodities in the commodity list are selected, acquiring images of the selected commodities, and identifying the acquired commodity images.
In this step, identifying the acquired commodity image specifically includes: 1) generating an image of the sold goods; in the step, the image of the sold commodity can be directly input into the intelligent vending machine in a man-machine interaction mode, and certainly, the image of the sold commodity can also be acquired by an image acquisition device of the intelligent vending machine; 2) and training the image of the sold commodity by using a deep convolutional network training method to obtain the trained model parameters. 3) And identifying the acquired commodity image by using the trained model parameters.
In the step, the intelligent vending machine can shoot the image of the commodity selected by the user through the camera.
The fourth step: judging whether the identified commodity is a sold commodity; if yes, displaying and outputting a commodity label of the commodity; if not, displaying prompt information.
The commodity label comprises a commodity name and a commodity price. When the product label of the product is displayed, the product price and the product name of the product may be displayed at the same time, or only the product price may be displayed.
Fig. 2 shows a flowchart of another intelligent vending machine control method based on commodity image recognition according to the present application, when the identified commodity is a vending commodity, the method further includes the steps of: 1) displaying selectable payment modes; 2) executing a payment process when any payment mode is detected to be selected; 3) and when the payment is judged to be successful, controlling the selected commodity to be delivered. The payment mode can adopt any existing mode, such as bank card, WeChat or Paibao and the like.
Preferably, the method further comprises the steps of: and when the identified commodity is not the sold commodity, displaying the prompt information, and then displaying the commodity shelf image again.
The intelligent vending machine control method is realized based on an intelligent vending machine control system. The intelligent vending machine control system comprises a display module, an image acquisition module, an identification module, a judgment module and a control module.
The display module is used for displaying the commodity shelf images and displaying the commodity labels and the prompt messages of the commodities.
The image acquisition device is used for acquiring the commodity shelf image and acquiring the image of the selected commodity; when the image of the sold commodity needs to be collected, the image collecting device can also be used for collecting the image of the sold commodity. Typically, the image capture device is a camera.
The recognition module is used for training the images of the sold commodities by using a deep convolutional network training method to obtain trained model parameters, and recognizing the selected commodity images according to the trained model parameters.
The judging module is used for judging whether the identified commodity is a selling commodity. The control module is used for controlling according to the judgment result of the judgment module; and if the identified commodity is judged to be a sold commodity, displaying the price of the commodity, and if not, displaying prompt information.
In a specific embodiment, the recognition module comprises a storage submodule, and the storage submodule is used for storing the trained model parameters.
The application also provides an intelligent vending machine which is used for vending by the intelligent vending machine control method.
In summary, the intelligent vending machine control method of the application abandons the way of identifying the electronic tag and inputting the goods channel number of the commodity to be purchased in the prior art based on commodity image identification, and obtains the model parameters of the sold commodity by using a deep convolution network training method, stores the model parameters in the intelligent vending machine in advance, and identifies the image of the selected commodity by using the model parameters. The identification mode is more intelligent and automatic, the identification range is wider, and the transaction is quicker and more convenient. Secondly, the image of the commodity shelf is obtained in a mode of scanning the commodity shelf, and a user can drag, rotate and/or zoom so as to conveniently and clearly view commodities in the commodity shelf.
Although the invention herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention. It is therefore to be understood that numerous modifications may be made to the illustrative embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that features described in different dependent claims and herein may be combined in ways different from those described in the original claims. It is also to be understood that features described in connection with individual embodiments may be used in other described embodiments.
Claims (10)
1. An intelligent vending machine control method based on commodity image recognition is characterized by comprising the following steps:
s10: acquiring a commodity shelf image and displaying the acquired commodity shelf image, wherein the commodity shelf image can be dragged and/or rotated and/or zoomed so as to be convenient for viewing commodities in the commodity shelf image;
s20: when detecting that the commodity in the commodity shelf image is selected, acquiring the image of the selected commodity and identifying the acquired commodity image;
s30: judging whether the identified commodity is a sold commodity;
if yes, displaying the commodity label of the selected commodity; if not, displaying prompt information;
in step S20, the recognizing the captured commodity image includes: and generating an image of the sold commodity, training the image of the sold commodity by using a deep convolutional network training method, acquiring a trained model parameter, and identifying the acquired commodity image by using the trained model parameter.
2. The intelligent vending machine control method according to claim 1, wherein in step S10, the commodity shelf images are commodity shelf stereoscopic images and/or commodity shelf front images and/or commodity shelf videos.
3. The smart merchandiser control method of claim 1, in which the merchandise labels include a merchandise name and a merchandise price.
4. The smart vending machine control method according to claim 1, wherein in step S20, the image of the vended merchandise is generated by:
and generating an image of the sold commodity in a man-machine interaction mode, or generating the image of the sold commodity in a scanning mode.
5. The smart vending machine control method according to any one of claims 1-4, further comprising, when the identified item is a vended item, the steps of:
s40: displaying selectable payment modes;
s50: executing a payment process when any payment mode is detected to be selected;
s60: and when the payment is judged to be successful, controlling the selected commodity to be delivered.
6. The smart vending machine control method according to any one of claims 1 to 4, wherein after the identified merchandise is not a vended merchandise and the prompt message is displayed, the merchandise rack image collected in step S10 is displayed.
7. The utility model provides an intelligence vending machine control system based on commodity image recognition which characterized in that includes:
the display module is used for displaying the commodity shelf image, and displaying the commodity label of the commodity and prompt information;
the image acquisition device is used for acquiring the commodity shelf image and acquiring the image of the selected commodity;
the recognition module is used for training the images of the sold commodities by utilizing a deep convolutional network training method to obtain trained model parameters and recognizing the selected commodities according to the trained model parameters;
the judging module is used for judging whether the identified commodity is a selling commodity or not;
the control module is used for controlling according to the judgment result of the judgment module;
and if the identified commodity is judged to be a sold commodity, displaying a commodity label of the commodity, and if not, displaying prompt information.
8. The smart merchandiser control system as recited in claim 7 wherein the identification module further includes a storage sub-module for storing the trained model parameters.
9. The intelligent merchandiser control system of claim 7, wherein the image capture module is further configured to capture images of the vended items.
10. A smart vending machine, characterized in that a smart vending machine control method according to any one of claims 1-6 is used.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910872072.8A CN112507762A (en) | 2019-09-16 | 2019-09-16 | Intelligent vending machine control method and system based on commodity image recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910872072.8A CN112507762A (en) | 2019-09-16 | 2019-09-16 | Intelligent vending machine control method and system based on commodity image recognition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112507762A true CN112507762A (en) | 2021-03-16 |
Family
ID=74923806
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910872072.8A Withdrawn CN112507762A (en) | 2019-09-16 | 2019-09-16 | Intelligent vending machine control method and system based on commodity image recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112507762A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201201145A (en) * | 2010-06-17 | 2012-01-01 | Foshan Cornucopia Digital Electric Nanhai Co Ltd | Animation embedded automatic vending machine |
| CN203059066U (en) * | 2013-03-22 | 2013-07-17 | 程抒一 | Intelligent interactive goods shelf |
| CN206011103U (en) * | 2016-08-29 | 2017-03-15 | 昆山塔米机器人有限公司 | Human emulated robot Vending Machine |
| CN107274557A (en) * | 2017-06-22 | 2017-10-20 | 王滨 | Human-computer interaction method of commerce, self-service decoration machine and storage medium |
| CN108038998A (en) * | 2017-12-22 | 2018-05-15 | 南京工业大学 | A kind of semi-automatic selling system of supermarket |
| CN208225191U (en) * | 2018-05-23 | 2018-12-11 | 胡江礼 | A kind of rotary-type Vending Machine |
| CN109685001A (en) * | 2018-12-24 | 2019-04-26 | 石狮市森科智能科技有限公司 | Human body measurements of the chest, waist and hips data acquisition method and intelligence sell clothing system and Intelligent unattended sells clothing machine |
| CN109961566A (en) * | 2017-12-26 | 2019-07-02 | 阿里巴巴集团控股有限公司 | Automatic vending machine and its information processing method, apparatus and system |
| CN209232033U (en) * | 2018-12-27 | 2019-08-09 | 俞伟栋 | A kind of rotary lifting-type automatic vending device |
| CN110163996A (en) * | 2018-02-13 | 2019-08-23 | 青岛海尔特种电冰柜有限公司 | Automatic vending machine and its control method |
-
2019
- 2019-09-16 CN CN201910872072.8A patent/CN112507762A/en not_active Withdrawn
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201201145A (en) * | 2010-06-17 | 2012-01-01 | Foshan Cornucopia Digital Electric Nanhai Co Ltd | Animation embedded automatic vending machine |
| CN203059066U (en) * | 2013-03-22 | 2013-07-17 | 程抒一 | Intelligent interactive goods shelf |
| CN206011103U (en) * | 2016-08-29 | 2017-03-15 | 昆山塔米机器人有限公司 | Human emulated robot Vending Machine |
| CN107274557A (en) * | 2017-06-22 | 2017-10-20 | 王滨 | Human-computer interaction method of commerce, self-service decoration machine and storage medium |
| CN108038998A (en) * | 2017-12-22 | 2018-05-15 | 南京工业大学 | A kind of semi-automatic selling system of supermarket |
| CN109961566A (en) * | 2017-12-26 | 2019-07-02 | 阿里巴巴集团控股有限公司 | Automatic vending machine and its information processing method, apparatus and system |
| CN110163996A (en) * | 2018-02-13 | 2019-08-23 | 青岛海尔特种电冰柜有限公司 | Automatic vending machine and its control method |
| CN208225191U (en) * | 2018-05-23 | 2018-12-11 | 胡江礼 | A kind of rotary-type Vending Machine |
| CN109685001A (en) * | 2018-12-24 | 2019-04-26 | 石狮市森科智能科技有限公司 | Human body measurements of the chest, waist and hips data acquisition method and intelligence sell clothing system and Intelligent unattended sells clothing machine |
| CN209232033U (en) * | 2018-12-27 | 2019-08-09 | 俞伟栋 | A kind of rotary lifting-type automatic vending device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108320404B (en) | Commodity identification method and device based on neural network and self-service cash register | |
| CN109300263B (en) | Settlement method and device for image recognition technology based on convolutional neural network | |
| CN108335408B (en) | Article identification method, device and system for vending machine and storage medium | |
| CN110866429B (en) | Missing scanning identification method, device, self-service cashing terminal and system | |
| EP2975568A1 (en) | System and method for self-checkout using product images | |
| CN110175590A (en) | A kind of commodity recognition method and device | |
| WO2019165892A1 (en) | Automatic vending method and apparatus, and computer-readable storage medium | |
| JP6549558B2 (en) | Sales registration device, program and sales registration method | |
| CN109726759B (en) | Unmanned vending method, device, system, electronic equipment and computer readable medium | |
| JP2004127013A (en) | Point-of-sale information managing device | |
| CN110119914B (en) | Object replenishment processing method, device and system | |
| US11488126B2 (en) | Cashier fraud detecting system and method and product image selection generation for artificial neural network learning related applications | |
| CN109409218A (en) | Reminding method, apparatus and system | |
| CN111222389B (en) | Analysis method and system for commodity on super commodity shelf | |
| EP3293685A1 (en) | Information processing apparatus that identifies an item based on a captured image thereof | |
| KR20120132179A (en) | Method and apparatus for transmitting intention using photographing image | |
| CN111079478A (en) | Unmanned goods selling shelf monitoring method and device, electronic equipment and system | |
| US11748787B2 (en) | Analysis method and system for the item on the supermarket shelf | |
| CN110119915A (en) | Object enters library processing method, device and system | |
| CN113468914B (en) | Method, device and equipment for determining purity of commodity | |
| CN113706227A (en) | Goods shelf commodity recommendation method and device | |
| CN111222388B (en) | Settlement method and system based on visual recognition | |
| CN107533783A (en) | Product calling mechanism, program and control method | |
| CN109191546A (en) | Map method of formation, information acquisition method, apparatus and system | |
| CN112507762A (en) | Intelligent vending machine control method and system based on commodity image recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210316 |
|
| WW01 | Invention patent application withdrawn after publication |