CN116704666A - Vending method, computer readable storage medium, and vending machine - Google Patents
Vending method, computer readable storage medium, and vending machine Download PDFInfo
- Publication number
- CN116704666A CN116704666A CN202310755656.3A CN202310755656A CN116704666A CN 116704666 A CN116704666 A CN 116704666A CN 202310755656 A CN202310755656 A CN 202310755656A CN 116704666 A CN116704666 A CN 116704666A
- Authority
- CN
- China
- Prior art keywords
- goods
- image
- brand
- feature map
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000001514 detection method Methods 0.000 claims abstract description 69
- 238000007599 discharging Methods 0.000 claims abstract description 3
- 238000000605 extraction Methods 0.000 claims description 22
- 238000010586 diagram Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 9
- 238000011084 recovery Methods 0.000 claims description 7
- 230000003044 adaptive effect Effects 0.000 claims description 6
- 239000013589 supplement Substances 0.000 abstract description 3
- 235000013361 beverage Nutrition 0.000 description 12
- 230000007246 mechanism Effects 0.000 description 10
- 230000008901 benefit Effects 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 6
- 238000012423 maintenance Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 3
- 238000004806 packaging method and process Methods 0.000 description 3
- 238000004064 recycling Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000000295 complement effect Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000001965 increasing effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000010845 search algorithm Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07F—COIN-FREED OR LIKE APPARATUS
- G07F11/00—Coin-freed apparatus for dispensing, or the like, discrete articles
- G07F11/004—Restocking arrangements therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07F—COIN-FREED OR LIKE APPARATUS
- G07F11/00—Coin-freed apparatus for dispensing, or the like, discrete articles
- G07F11/62—Coin-freed apparatus for dispensing, or the like, discrete articles in which the articles are stored in compartments in fixed receptacles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02W—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO WASTEWATER TREATMENT OR WASTE MANAGEMENT
- Y02W90/00—Enabling technologies or technologies with a potential or indirect contribution to greenhouse gas [GHG] emissions mitigation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Control Of Vending Devices And Auxiliary Devices For Vending Devices (AREA)
Abstract
The invention discloses a vending method, a computer readable storage medium and an automatic vending machine. The method comprises the following steps: acquiring an image of a shelf area of the vending machine, and recording the image as a first image; detecting goods on the goods shelves in the first image by using a trained target detection model, and obtaining the shelf loading rate of goods of each brand according to detection results; and determining a target brand and the number thereof according to the stocking rate and the target share, and controlling the vending machine to carry out replenishment or discharging according to the target brand and the number thereof. And detecting images of the goods shelf area of the automatic vending machine by using the trained target detection model to obtain brands corresponding to the goods in the goods shelf area of the automatic vending machine, calculating the loading rate of each brand, determining the target brands and the quantity thereof according to the loading rate and the target share so as to supplement or deliver the goods, thereby maintaining the quantity proportion of the goods corresponding to the share of each brand.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a vending method, a computer readable storage medium, and an automatic vending machine.
Background
Retail as the last ring of providing consumer personal or social power systems with consumer goods and related services, there is an increasing demand for efficiency in the retail industry as the pace of life increases. The intelligent selling not only saves the fund cost of a great deal of labor force, but also reduces the operation cost of investors, and gradually becomes an indispensable selling mode in the life of people. At present, intelligent selling is still in a development stage in China, and the application technology and the operation simplicity are in the continuous improvement process. How to provide a vending scheme for vending machines is a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The present invention aims to solve at least one of the technical problems in the related art to some extent. Therefore, an object of the present invention is to provide a vending method, which can effectively monitor the vending condition of an automatic vending machine, and can automatically load and unload goods according to the vending condition.
A second object of the present invention is to propose a computer readable storage medium.
A third object of the present invention is to propose a vending machine.
To achieve the above object, an embodiment of a first aspect of the present invention provides a vending method, the method including: acquiring an image of a shelf area of the vending machine, and recording the image as a first image; detecting goods on the goods shelves in the first image by using a trained target detection model, and obtaining the shelf loading rate of goods of each brand according to detection results; and determining a target brand and the number thereof according to the stocking rate and the target share, and controlling the vending machine to carry out replenishment or discharging according to the target brand and the number thereof.
According to the automatic vending machine provided by the embodiment of the invention, the trained target detection model is utilized to detect the image of the goods shelf area of the automatic vending machine, the brands corresponding to the goods in the goods shelf area of the automatic vending machine are obtained, the shelf rate of each brand is calculated, the target brands and the quantity thereof are determined according to the shelf rate and the target share, and the goods are replenished or discharged, so that the quantity proportion of the goods corresponding to the share of each brand is maintained. The automatic vending machine provided by the embodiment of the invention can effectively monitor the vending condition of the automatic vending machine, and can realize automatic loading and unloading according to the vending condition, and has the advantages of low maintenance cost and high flexibility.
In addition, the vending machine according to the above embodiment of the present invention may further have the following additional technical features:
according to an embodiment of the present invention, the target detection model includes a position detection module, a feature extraction module and an identification module, and detects goods on a shelf in the first image by using the trained target detection model, and obtains a shelf rate of goods of each brand according to a detection result, including: carrying out cargo detection according to the first image by utilizing the position detection module to obtain the position information of each cargo; cutting the first image according to the position information to obtain one or more second images; performing feature extraction on the second image by using the feature extraction module to obtain feature images of all cargoes; matching the cargo feature map with a preset feature map in a preset database, and obtaining brands corresponding to all cargoes according to the matching result; and determining the loading rate of each brand according to the brand corresponding to each cargo.
According to one embodiment of the present invention, the position detection module includes a feature extraction layer and a spatial attention layer, and performs cargo detection according to the first image by using the position detection module to obtain position information of each cargo, including: performing feature extraction according to the first image by using the feature extraction layer to obtain a first feature map; after the first feature map is convolved, the convolved first feature map is subjected to space self-adaptive weighting by utilizing the space attention layer, so that a second feature map is obtained; and obtaining the position information of each cargo according to the first characteristic diagram and the second characteristic diagram.
According to one embodiment of the invention, the spatial attention layer performs spatial adaptive weighting according to the aspect ratio and the relative rectangular shape of the cargo, wherein the spatial adaptive weighting adopts the following formula expression:
wherein y (P) 0 ) For a certain point value in the second feature map, P n For the input point position in the convolved first feature map, w (P n ) For the weight value corresponding to the input point in the first characteristic diagram after convolution, X (P n +P 0 +ΔP n ) For shifting P based on input points in the convolved first feature map 0 +ΔP n A value corresponding to the position, wherein P 0 Is a fixed offset value for convolution operations, ΔP n Is convolved P 0 +P n Offset of points in x and y directions, Δm n For the proportion of each position characteristic weight, deltam n Between (0, 1).
According to one embodiment of the present invention, the method for constructing the preset database includes: acquiring a cargo image of a preset brand; inputting the cargo image into the trained target detection model to obtain the preset feature map; and obtaining the preset database based on the preset feature map.
According to one embodiment of the invention, the method further comprises: when new goods exist, acquiring an image of the new goods and recording the image as a third image, wherein the new goods comprise newly added brands of goods and/or goods after being replaced and packaged; inputting the third image into the trained target detection model to obtain a newly added feature map of the new goods; and storing the newly added feature map to the preset database.
According to one embodiment of the invention, the determining the target brands and the number thereof according to the stocking rate and the target share comprises: when the fact that the empty preset brands exist is judged according to the loading rate, the target share is updated; and determining the target brands and the quantity thereof according to the updated target shares.
According to one embodiment of the invention, the method further comprises: when a bottle recovery request is received, acquiring an image of the bottle to be recovered, and recording the image as a fourth image; extracting features of the bottle image by using the trained target detection model to obtain a bottle feature map; matching the bottle feature map with a preset feature map in the preset database, and determining the brand of the bottle to be recovered according to a matching result; and conveying the bottle to be recovered to a bottle recovery area of the brand to be recovered, and performing cashback.
To achieve the above object, an embodiment of a second aspect of the present invention proposes a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements a vending method as proposed by the embodiment of the first aspect of the present invention.
To achieve the above object, an embodiment of a third aspect of the present invention provides a vending machine, including a memory, and a processor, where the memory stores a computer program, and when the computer program is executed by the processor, the vending method according to the embodiment of the first aspect of the present invention is implemented.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
FIG. 1 is a flow chart of a vending method according to an embodiment of the invention;
FIG. 2 is a flow chart of deriving the rate of loading of brands of goods using a trained target detection model in accordance with an embodiment of the invention;
FIG. 3 is a flow chart of obtaining location information for each item according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a position detection module according to an embodiment of the invention;
FIG. 5 is a diagram of a spatial attention mechanism correspondence in accordance with an embodiment of the present invention;
FIG. 6 is a flow chart of the construction of a preset database according to an embodiment of the present invention;
FIG. 7 is a flow chart of the construction of a preset database according to yet another embodiment of the present invention;
FIG. 8 is a flow chart of determining a target brand and its quantity, according to an embodiment of the present invention;
FIG. 9 is a flow chart of recycling beverage bottles according to an embodiment of the present invention;
fig. 10 is a schematic view of a vending machine in accordance with an embodiment of the present invention.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present invention and should not be construed as limiting the invention.
The vending method, the computer readable storage medium, and the vending machine according to the embodiments of the present invention are described in detail below with reference to fig. 1 to 10 and the detailed description.
Fig. 1 is a flow chart of a vending method according to an embodiment of the invention. As shown in fig. 1, the vending method may include:
s101, acquiring an image of a shelf area of the vending machine, and marking the image as a first image;
s102, detecting goods on a goods shelf in a first image by using a trained target detection model, and obtaining the rate of loading goods of each brand according to detection results;
and S103, determining the target brands and the number thereof according to the loading rate and the target share, and controlling the vending machine to carry out replenishment or delivery according to the target brands and the number thereof.
It should be noted that, the vending machine is provided with a camera corresponding to the shelf area so as to collect the image of the shelf area of the vending machine.
In order to realize automatic loading and automatic replenishment of the automatic vending machine, the embodiment of the invention acquires an image of a goods shelf area of the automatic vending machine, detects goods in the image by using a trained target detection model, and obtains the loading rate of goods of various brands on the goods shelf of the automatic vending machine according to a detection result. And determining the target brands and the quantity thereof according to the loading rate of the brands of goods, the target share (duty ratio) of the brands of goods and the total quantity of the goods which can be placed in the goods shelf area of the vending machine, so as to control the vending machine to carry out replenishment or unloading according to the target brands and the quantity thereof.
When the vending machine initially loads goods, an image of a goods shelf area of the vending machine is acquired, the image is detected by using a trained target detection model, and the loading rate of goods of each brand is obtained according to a detection result. Because the vending machine initially loads, the loading rate of the obtained goods of each brand is 0, at the moment, each target brand and the quantity thereof can be determined according to the target share (the ratio) of the goods of each brand and the total quantity of the goods which can be placed in the goods shelf area of the vending machine, and the vending machine is controlled to supplement or deliver the goods according to the target brand and the quantity thereof.
When the vending machine initially loads goods, the image of the goods shelf area of the vending machine can be obtained without directly determining each target brand and the quantity thereof according to the target share (the ratio) of each brand of goods and the total quantity of the goods which can be placed in the goods shelf area of the vending machine, and controlling the vending machine to supplement goods or deliver goods according to the target brand and the quantity thereof. The embodiment of the invention does not limit the loading mode of the vending machine during initial loading.
It should be noted that, considering a single brand to monopolize the vending machine, it may cause waste and loss of customer resources, and meanwhile, may not meet diversified consumer demands of customers. Therefore, vending machines will generally sell various brands of goods and different types of goods at the same time, and for brands, the distribution of goods placement space on the shelves in vending machines will affect the sales of brands, and in general, the larger the distribution space, the more the number and variety of goods can be placed, and the greater the purchase probability of consumers. Thus, brands all want to get more on-shelf share. At this time, each brand needs to balance its own loading rate to purchase the corresponding share, so as to obtain the proportional number corresponding to the goods sold by the vending machine.
According to the selling method provided by the embodiment of the invention, the proportion of the number of the goods on the goods shelves of each brand is set according to the share purchased by the brand side, the goods are timely supplemented according to the selling condition, and meanwhile, the proportion of the number of the goods corresponding to each brand share can be maintained. By using the vending method of the embodiment of the invention, the number of goods on the goods shelf can be ensured to be sold in the proportion, so that the benefits of all the brands can be ensured.
In one embodiment of the invention, the saturation and the empty rate of the vending machine can be calculated according to the detection result, namely the discrimination of the corresponding brands of the goods on the goods shelf, so that the situation of the vending machine can be known according to the saturation and the empty rate of the vending machine, the replenishment frequency of the vending machine can be determined, and the brands can be conveniently adjusted to the share purchased by the vending machine. Each brand party can also monitor the rate of listing of the respective brands in the vending machine according to the share ratio purchased by the brands.
In one embodiment of the present invention, as shown in fig. 2, the target detection model may include a position detection module, a feature extraction module, and an identification module, and the detecting the goods on the goods shelf in the first image using the trained target detection model, and obtaining the shelf rate of the goods of each brand according to the detection result may include:
s201, detecting cargoes according to the first image by using a position detection module to obtain position information of each cargoes;
s202, cutting the first image according to the position information to obtain one or more second images;
s203, performing feature extraction on the second image by using a feature extraction module to obtain feature graphs of all cargoes;
s204, matching the cargo feature map with a preset feature map in a preset database, and obtaining brands corresponding to all cargoes according to the matching result;
s205, determining the loading rate of each brand according to the brand corresponding to each cargo.
In order to improve the detection accuracy of the goods on the goods shelf in the first image, namely the image of the goods shelf area of the automatic vending machine, the position detection module of the trained target detection model in the embodiment of the invention increases a spatial attention mechanism, so that the detection accuracy of the goods on the goods shelf is more accurate.
Specifically, the position detection module in the target detection model is utilized to detect cargoes in the first image, and position information of each cargo is obtained. And cutting the first image according to the obtained position information of each cargo, namely cutting out each cargo in the first image to obtain one or more second images. When there is only one cargo in the first image, a second image is obtained. When a plurality of cargoes are in the first image, a plurality of second images are obtained. In the embodiment of the invention, the number of the second images is identical to the number of cargoes in the first image.
And carrying out feature extraction on each cut second image (cargo) by utilizing a feature extraction module to obtain each cargo feature map. And matching each cargo feature map with a preset feature map in a preset database to obtain brand matching rates corresponding to each cargo feature map, wherein the maximum brand matching rate corresponding to each cargo feature map is the brand corresponding to the cargo in the second image. And carrying out statistical calculation on the brands corresponding to the obtained goods to obtain the shelf rate of each brand.
It should be noted that, the network architecture adopted by the target detection model in the embodiment of the present invention includes, but is not limited to, a fast rcnn network, a yolov3 network, a yolov4 network, a yolov5 network, and a yolov7 network.
In one embodiment of the present invention, as shown in fig. 3, the position detection module includes a feature extraction layer and a spatial attention layer, and performs cargo detection according to the first image by using the position detection module to obtain position information of each cargo, including:
s301, carrying out feature extraction according to a first image by utilizing a feature extraction layer to obtain a first feature map;
s302, after the first feature map is convolved, the convolved first feature map is subjected to space self-adaptive weighting by using a space attention layer, so as to obtain a second feature map;
and S303, obtaining the position information of each cargo according to the first characteristic diagram and the second characteristic diagram.
The position detection module according to the embodiment of the invention can increase a spatial attention mechanism for the position detection module aiming at goods, such as beverage sold by the vending machine, relative bottle bodies of beverage bottles and fixed forms thereof, namely the position detection module adopts the spatial attention mechanism when detecting the position information of the goods, such as the beverage bottles. The increased spatial attention aims at enhancing the feature expression of the key region by essentially transforming the spatial information in the first image into another space through the position detection module and preserving the key information, and generating a weight mask for each position, and weighting the output, thereby enhancing the specific target region of interest while weakening the irrelevant background region.
Specifically, the first image is input to a feature extraction layer of the object detection model, and a first feature map of the first image is obtained. The first feature map is input to a convolution layer (conv), referring to fig. 4, after the convolution layer (conv) convolves the first feature map, the convolved first feature map is weighted in a space adaptive manner by using a spatial attention layer, so as to obtain a second feature map. The spatial attention layer adds spatial weighting to complete the realization of spatial attention and improve the feature extraction capability. And multiplying the first characteristic diagram by the second characteristic diagram, and analyzing the acquired characteristics to obtain the position information of each cargo, such as the beverage bottle.
In one embodiment of the invention, the spatial attention layer performs spatial adaptive weighting according to the aspect ratio of the cargo and the relative rectangular shape, wherein the spatial adaptive weighting uses the following formula expression:
wherein y (P) 0 ) For a certain point value in the second feature map, i.e. in the spatial attention layer output feature, P n For the input point position in the convolved first feature map, w (P n ) For the weight value corresponding to the input point in the first characteristic diagram after convolution, X (P n +P 0 +ΔP n ) For shifting P based on input points in the convolved first feature map 0 +ΔP n A value corresponding to the position, wherein P 0 Is a fixed offset value for convolution operations, ΔP n Is convolved P 0 +P n Offset of points in x and y directions, Δm n For the proportion of each position characteristic weight, deltam n Between (0, 1).
Illustratively, the position detection module employs an attention mechanism for the beverage bottle that increases the attention mechanism to the beverage bottle morphology. Among the attention-increasing mechanisms are mainly the aspect ratio of the beverage bottle, and the relatively rectangular morphology.
If the coefficient Δm n A value of 0 indicates that the characteristics of this partial region have no effect on the output. Coefficient Deltam n Obtained by training learning, this increases the output channel at the deformable convolution to 3N.
Fig. 4 is a schematic diagram of a position detection module according to an embodiment of the present invention, where a first feature map is generated by conventional convolution, the first feature map has the same length and width as an input feature, and a channel is a 3N local spatial attention module, where N is a channel value of the input feature, where in 3N, 2N is an offset (offset) of a sampling point, and is an offset in x and y directions, and N is a weight (weight) of the offset sampling point to the point. As shown in fig. 5, the spatial attention mechanism according to the embodiment of the present invention can obtain the attention of the global information to the ≡point in fig. 5, and the attention of the local information amount can be obtained only by convolution with respect to the previous spatial attention. The spatial attention mechanism of the embodiment of the invention is more comprehensive and effective for acquiring attention.
In one embodiment of the present invention, as shown in fig. 6, the method for constructing the preset database may include:
s401, acquiring a goods image of a preset brand;
s402, inputting a cargo image into a trained target detection model to obtain a preset feature map;
s403, obtaining a preset database based on the preset feature map.
Before the loading rate calculation of each brand of goods is performed, the goods images sold by the brand party (preset brand) purchasing the share of the vending machine are required to be acquired. The number of the acquired goods images of each good can be 10-20.
Specifically, the acquired goods image is input into a trained target detection model to obtain a feature map of goods corresponding to a preset brand, and the feature map is recorded as a preset feature map. And storing the preset feature map into a preset database so as to be convenient for calling when detecting the brand corresponding to the corresponding goods.
In one embodiment of the present invention, as shown in fig. 7, the vending method further comprises:
s501, when new goods exist, acquiring an image of the new goods and marking the image as a third image, wherein the new goods comprise newly added brands of goods and/or the packaged goods are replaced;
s502, inputting a third image into a trained target detection model to obtain a newly added feature map of new goods;
s503, storing the newly added feature map into a preset database.
In order to reduce the maintenance cost of an algorithm, when the outer package of a preset brand of goods is replaced or a new brand of goods is added, images of the newly added brand of goods and/or the goods after the replacement and packaging are acquired. The number of the obtained goods images corresponding to the newly added brands of goods and/or the goods after the packaging is replaced is within the range of 10-20.
Specifically, an image of the newly added brand of goods and/or the goods after the packaging is replaced is input into a trained target detection model, and a newly added feature map of the new goods is obtained. And adding the newly added brand goods and/or the newly added feature map of the packaged goods into a preset database, wherein the preset database can form a feature library of a corresponding brand based on the newly added brand goods image and/or the packaged goods image. When the brand of the goods is detected, the detected feature images of the goods are matched with the preset feature images in the preset database, and the identification of the brand of the goods can be completed. Retraining the discriminant algorithm model to match new categories relative to the discriminant algorithm in the recognition algorithm greatly reduces the maintenance cost of the algorithm.
In one embodiment of the invention, as shown in FIG. 8, determining a target brand and its number from the rate of shelves and the target shares includes:
s601, updating the target share when the existence of the empty preset brand is judged according to the shelf rate;
s602, determining the target brands and the quantity thereof according to the updated target shares.
According to the selling method provided by the embodiment of the invention, the loading rate of each brand is dynamically monitored based on the picture recognition algorithm and the search algorithm according to the set share, and the goods source is supplied according to the set share so as to maintain the share proportion of each brand. In order to prevent the resources of the vending machine from being wasted in a brand out-of-stock state, the occupied share is redistributed according to the shares of the rest brands, namely, the target share is updated, and the target brands and the quantity thereof are determined according to the updated target share. And after the shortage is complemented, controlling the vending machine to complement or deliver goods according to the purchased share of each brand again so as to maintain the corresponding quantity proportion of the goods of each brand.
For example, when one of the brands is sold empty, it is assumed that B is sold empty, i.e., the proportion of A and C in the vending machine is adjusted toAccording to->And determining the quantity of the goods corresponding to the A and the C, and putting the goods corresponding to the quantity A and the C on the shelf. After the logistics personnel complement the empty B in the vending machine, the vending machine resets the share ratio A:B:C=5:3:2, and determines a target according to the loading rate and the target shareBrands and the quantity thereof, and controls the vending machine to carry out replenishment or delivery according to the target brands and the quantity thereof. And further, the benefits of all brands can be guaranteed, and meanwhile, the resources are not wasted.
In one embodiment of the present invention, as shown in fig. 9, the vending method further comprises:
s701, when a bottle recovery request is received, acquiring an image of a bottle to be recovered, and recording the image as a fourth image;
s702, extracting features of the bottle image by using a trained target detection model to obtain a bottle feature map;
s703, matching the bottle feature map with a preset feature map in a preset database, and determining the brand of the bottle to be recovered according to the matching result;
and S704, conveying the bottle to be recovered to a bottle recovery area of the brand to be recovered, and performing cashback.
Embodiments of the present invention add a beverage bottle recycling mechanism to a vending machine. When a bottle recovery request is received, the image of the bottle to be recovered, which is acquired by the camera, is utilized to extract the characteristics of the bottle image by utilizing the trained target detection model, and a bottle characteristic diagram is obtained. And matching the bottle body characteristic diagram with a preset characteristic diagram in a preset database, classifying the bottle bodies into beverage bottles of corresponding brands according to the identified brands after the brands matched with the beverage bottles of the brands are identified, and performing cashback to the users. Thereby reducing the cost of recycling the beverage bottle.
According to the vending method, the trained target detection model is utilized to detect images of the goods shelf area of the vending machine, brands corresponding to goods in the goods shelf area of the vending machine are obtained, the loading rate of each brand is calculated, the target brands and the quantity of the target brands are determined according to the loading rate and the target share, and replenishment or unloading is carried out, so that the quantity proportion of the goods corresponding to the share of each brand is maintained. The automatic vending machine provided by the embodiment of the invention can effectively monitor the vending condition of the automatic vending machine, automatically load and unload goods according to the vending condition, and has the advantages of low maintenance cost and high flexibility.
The invention provides a computer readable storage medium.
In this embodiment, a computer program is stored on a computer readable storage medium, which when executed by a processor, implements the vending method as described above.
The invention provides a vending machine.
Fig. 10 is a schematic view of a vending machine in accordance with an embodiment of the present invention. As shown in fig. 10, vending machine 100 may include a memory 10, a processor 20, and a computer program stored on memory 10 that when executed by processor 20 performs the vending method as described above.
The computer readable storage medium and the automatic vending machine realize effective monitoring of the vending condition of the automatic vending machine and automatic loading and unloading according to the vending condition by utilizing the vending method, and have the advantages of low maintenance cost and high flexibility.
It should be noted that the logic and/or steps represented in the flowcharts or otherwise described herein, for example, may be considered as a ordered listing of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the device or element being referred to must have a specific orientation, be configured and operated in a specific orientation, and therefore should not be construed as limiting the present invention.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.
In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured," and the like are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly, through intermediaries, or both, may be in communication with each other or in interaction with each other, unless expressly defined otherwise. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
In the present invention, unless expressly stated or limited otherwise, a first feature "up" or "down" a second feature may be the first and second features in direct contact, or the first and second features in indirect contact via an intervening medium. Moreover, a first feature being "above," "over" and "on" a second feature may be a first feature being directly above or obliquely above the second feature, or simply indicating that the first feature is level higher than the second feature. The first feature being "under", "below" and "beneath" the second feature may be the first feature being directly under or obliquely below the second feature, or simply indicating that the first feature is less level than the second feature.
While embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the invention, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the invention.
Claims (10)
1. A vending method, the method comprising:
acquiring an image of a shelf area of the vending machine, and recording the image as a first image;
detecting goods on the goods shelves in the first image by using a trained target detection model, and obtaining the shelf loading rate of goods of each brand according to detection results;
and determining a target brand and the number thereof according to the stocking rate and the target share, and controlling the vending machine to carry out replenishment or discharging according to the target brand and the number thereof.
2. The vending method of claim 1, wherein the target detection model includes a position detection module, a feature extraction module, and an identification module, the goods on the goods shelf in the first image are detected by using the trained target detection model, and the loading rate of the goods of each brand is obtained according to the detection result, and the vending method includes:
carrying out cargo detection according to the first image by utilizing the position detection module to obtain the position information of each cargo;
cutting the first image according to the position information to obtain one or more second images;
performing feature extraction on the second image by using the feature extraction module to obtain feature images of all cargoes;
matching the cargo feature map with a preset feature map in a preset database, and obtaining brands corresponding to all cargoes according to the matching result;
and determining the loading rate of each brand according to the brand corresponding to each cargo.
3. The vending method of claim 2, wherein the position detection module comprises a feature extraction layer and a spatial attention layer, and wherein the detecting goods by the position detection module according to the first image to obtain the position information of each good comprises:
performing feature extraction according to the first image by using the feature extraction layer to obtain a first feature map;
after the first feature map is convolved, the convolved first feature map is subjected to space self-adaptive weighting by utilizing the space attention layer, so that a second feature map is obtained;
and obtaining the position information of each cargo according to the first characteristic diagram and the second characteristic diagram.
4. A vending method as claimed in claim 3, wherein the spatial attention layer is spatially adaptively weighted according to the aspect ratio and the relative rectangular morphology of the good, wherein the spatially adaptive weighting is expressed by the following formula:
wherein y (P) 0 ) For a certain point value in the second feature map, P n For the input point position in the convolved first feature map, w (P n ) For the weight value corresponding to the input point in the first characteristic diagram after convolution, X (P n +P 0 +ΔP n ) For shifting P based on input points in the convolved first feature map 0 +ΔP n A value corresponding to the position, wherein P 0 Is a fixed offset value for convolution operations, ΔP n Is convolved P 0 +P n Offset of points in x and y directions, Δm n For the proportion of each position characteristic weight, deltam n Between (0, 1).
5. The vending method of claim 2, wherein the method of constructing the preset database comprises:
acquiring a cargo image of a preset brand;
inputting the cargo image into the trained target detection model to obtain the preset feature map;
and obtaining the preset database based on the preset feature map.
6. The vending method of claim 5, wherein the method further comprises:
when new goods exist, acquiring an image of the new goods and recording the image as a third image, wherein the new goods comprise newly added brands of goods and/or goods after being replaced and packaged;
inputting the third image into the trained target detection model to obtain a newly added feature map of the new goods;
and storing the newly added feature map to the preset database.
7. The vending method of claim 1, wherein the determining the target brands and the quantity thereof from the stocking rates and target shares comprises:
when the fact that the empty preset brands exist is judged according to the loading rate, the target share is updated;
and determining the target brands and the quantity thereof according to the updated target shares.
8. The vending method of claim 2, wherein the method further comprises:
when a bottle recovery request is received, acquiring an image of the bottle to be recovered, and recording the image as a fourth image;
extracting features of the bottle image by using the trained target detection model to obtain a bottle feature map;
matching the bottle feature map with a preset feature map in the preset database, and determining the brand of the bottle to be recovered according to a matching result;
and conveying the bottle to be recovered to a bottle recovery area of the brand to be recovered, and performing cashback.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the vending method as claimed in any one of claims 1-8.
10. A vending machine comprising a memory, a processor, said memory having stored thereon a computer program, wherein said computer program, when executed by said processor, implements the vending method of any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310755656.3A CN116704666A (en) | 2023-06-21 | 2023-06-21 | Vending method, computer readable storage medium, and vending machine |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310755656.3A CN116704666A (en) | 2023-06-21 | 2023-06-21 | Vending method, computer readable storage medium, and vending machine |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116704666A true CN116704666A (en) | 2023-09-05 |
Family
ID=87827480
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310755656.3A Pending CN116704666A (en) | 2023-06-21 | 2023-06-21 | Vending method, computer readable storage medium, and vending machine |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116704666A (en) |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102789601A (en) * | 2012-04-12 | 2012-11-21 | 北京京东世纪贸易有限公司 | Goods yard structure control method and device |
| CN109242060A (en) * | 2018-08-30 | 2019-01-18 | 上海扩博智能技术有限公司 | New restocking product fast searching method, system, equipment and storage medium |
| CN110942050A (en) * | 2019-12-20 | 2020-03-31 | 华南理工大学 | Automatic vending machine commodity identification system based on image processing |
| CN111062665A (en) * | 2019-12-17 | 2020-04-24 | 中鑫汇金(广州)科技有限公司 | Automatic vending machine replenishment method and device, computer equipment and storage medium |
| US20200372660A1 (en) * | 2019-05-21 | 2020-11-26 | Beihang University | Image salient object segmentation method and apparatus based on reciprocal attention between foreground and background |
| CN112613441A (en) * | 2020-12-29 | 2021-04-06 | 新疆爱华盈通信息技术有限公司 | Abnormal driving behavior recognition and early warning method and electronic equipment |
| US20210117674A1 (en) * | 2019-09-26 | 2021-04-22 | Shanghai Sensetime Intelligent Technology Co., Ltd. | Image processing method and device and storage medium |
| CN113065492A (en) * | 2021-04-12 | 2021-07-02 | 北京滴普科技有限公司 | Cloud-edge cooperative automatic ordering method, device and system and storage medium thereof |
| WO2021147257A1 (en) * | 2020-01-20 | 2021-07-29 | 上海商汤智能科技有限公司 | Network training method and apparatus, image processing method and apparatus, and electronic device and storage medium |
| CN113287120A (en) * | 2021-04-09 | 2021-08-20 | 深圳市锐明技术股份有限公司 | Vehicle driving environment abnormity monitoring method and device, electronic equipment and storage medium |
| CN114022999A (en) * | 2021-10-27 | 2022-02-08 | 北京云迹科技有限公司 | Method, device, equipment and medium for detecting shortage of goods of vending machine |
| US20220415027A1 (en) * | 2021-06-29 | 2022-12-29 | Shandong Jianzhu University | Method for re-recognizing object image based on multi-feature information capture and correlation analysis |
| CN115705701A (en) * | 2021-08-12 | 2023-02-17 | 上海顺如丰来技术有限公司 | Automatic inventory checking method and device for goods |
| CN116188790A (en) * | 2022-12-29 | 2023-05-30 | 中国电信股份有限公司 | Camera shielding detection method and device, storage medium and electronic equipment |
| CN116246391A (en) * | 2023-03-20 | 2023-06-09 | 广东便捷神科技股份有限公司 | Goods adjustment optimization method between vending machine sites |
| US20230184927A1 (en) * | 2021-12-15 | 2023-06-15 | Anhui University | Contextual visual-based sar target detection method and apparatus, and storage medium |
-
2023
- 2023-06-21 CN CN202310755656.3A patent/CN116704666A/en active Pending
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102789601A (en) * | 2012-04-12 | 2012-11-21 | 北京京东世纪贸易有限公司 | Goods yard structure control method and device |
| CN109242060A (en) * | 2018-08-30 | 2019-01-18 | 上海扩博智能技术有限公司 | New restocking product fast searching method, system, equipment and storage medium |
| US20200372660A1 (en) * | 2019-05-21 | 2020-11-26 | Beihang University | Image salient object segmentation method and apparatus based on reciprocal attention between foreground and background |
| US20210117674A1 (en) * | 2019-09-26 | 2021-04-22 | Shanghai Sensetime Intelligent Technology Co., Ltd. | Image processing method and device and storage medium |
| CN111062665A (en) * | 2019-12-17 | 2020-04-24 | 中鑫汇金(广州)科技有限公司 | Automatic vending machine replenishment method and device, computer equipment and storage medium |
| CN110942050A (en) * | 2019-12-20 | 2020-03-31 | 华南理工大学 | Automatic vending machine commodity identification system based on image processing |
| WO2021147257A1 (en) * | 2020-01-20 | 2021-07-29 | 上海商汤智能科技有限公司 | Network training method and apparatus, image processing method and apparatus, and electronic device and storage medium |
| CN112613441A (en) * | 2020-12-29 | 2021-04-06 | 新疆爱华盈通信息技术有限公司 | Abnormal driving behavior recognition and early warning method and electronic equipment |
| CN113287120A (en) * | 2021-04-09 | 2021-08-20 | 深圳市锐明技术股份有限公司 | Vehicle driving environment abnormity monitoring method and device, electronic equipment and storage medium |
| CN113065492A (en) * | 2021-04-12 | 2021-07-02 | 北京滴普科技有限公司 | Cloud-edge cooperative automatic ordering method, device and system and storage medium thereof |
| US20220415027A1 (en) * | 2021-06-29 | 2022-12-29 | Shandong Jianzhu University | Method for re-recognizing object image based on multi-feature information capture and correlation analysis |
| CN115705701A (en) * | 2021-08-12 | 2023-02-17 | 上海顺如丰来技术有限公司 | Automatic inventory checking method and device for goods |
| CN114022999A (en) * | 2021-10-27 | 2022-02-08 | 北京云迹科技有限公司 | Method, device, equipment and medium for detecting shortage of goods of vending machine |
| US20230184927A1 (en) * | 2021-12-15 | 2023-06-15 | Anhui University | Contextual visual-based sar target detection method and apparatus, and storage medium |
| CN116188790A (en) * | 2022-12-29 | 2023-05-30 | 中国电信股份有限公司 | Camera shielding detection method and device, storage medium and electronic equipment |
| CN116246391A (en) * | 2023-03-20 | 2023-06-09 | 广东便捷神科技股份有限公司 | Goods adjustment optimization method between vending machine sites |
Non-Patent Citations (1)
| Title |
|---|
| 沈文祥;秦品乐;曾建潮;: "基于多级特征和混合注意力机制的室内人群检测网络", 计算机应用, no. 12, 10 December 2019 (2019-12-10), pages 3496 - 3502 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106960301B (en) | Medicine e-commerce mirror image sorting method and system | |
| CN109949095A (en) | Replenishing method and equipment | |
| CN106447470A (en) | Delivery order distribution method and delivery order distribution device | |
| WO2022007643A1 (en) | Warehouse sorting method, apparatus and system, and server, robot and storage medium | |
| CN110097203A (en) | Inventory's dispatching method, inventory's dispatching device and computer readable storage medium | |
| US11301919B1 (en) | Systems and methods for parallelizing customer input with real-time picking for item substitutions | |
| CN110209348A (en) | Date storage method, device, electronic equipment and storage medium | |
| CN111126857B (en) | Intensive storage task management method and device and electronic equipment | |
| CN110390449A (en) | Warehouse replenishing method and device | |
| WO2019099854A1 (en) | Distributed sensor systems and methods for inventory control | |
| CN112801565A (en) | Goods allocation dispatching method, system and storage medium for intelligent storage | |
| CN116402445B (en) | Goods delivery scheduling method, device and computer equipment | |
| CN111723777A (en) | Method and device for judging commodity taking and placing process, intelligent container and readable storage medium | |
| KR102503198B1 (en) | Device and method for providing optimal cargo delivery route based on artificial intelligence | |
| US10713614B1 (en) | Weight and vision based item tracking | |
| CN111145430A (en) | Method and device for detecting commodity placing state and computer storage medium | |
| CN113935528B (en) | Intelligent scheduling method, intelligent scheduling device, computer equipment and storage medium | |
| US20170270545A1 (en) | Context-specific forecasting device | |
| WO2019099796A1 (en) | Distributed sensor system and method for inventory management and predictive replenishment | |
| CN111932185A (en) | Medicine inventory allocation system and method | |
| CN116704666A (en) | Vending method, computer readable storage medium, and vending machine | |
| CN112766868A (en) | Logistics vehicle distribution order matching method and device and computer equipment | |
| CN109493178A (en) | A kind of cloud retail order decomposition method and system | |
| CN114529843A (en) | Cargo congestion identification method and device, electronic equipment and storage medium | |
| CN113525978B (en) | Goods warehouse-in method, device and system for dense warehouse |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |