CN112800804A - Price tag-based out-of-stock detection method and device - Google Patents
Price tag-based out-of-stock detection method and device Download PDFInfo
- Publication number
- CN112800804A CN112800804A CN201911028666.7A CN201911028666A CN112800804A CN 112800804 A CN112800804 A CN 112800804A CN 201911028666 A CN201911028666 A CN 201911028666A CN 112800804 A CN112800804 A CN 112800804A
- Authority
- CN
- China
- Prior art keywords
- shelf
- stock
- detection
- training
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 287
- 238000012549 training Methods 0.000 claims abstract description 98
- 230000004888 barrier function Effects 0.000 claims abstract description 35
- 238000000034 method Methods 0.000 claims abstract description 33
- 238000010586 diagram Methods 0.000 claims description 28
- 238000012360 testing method Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 16
- 238000013135 deep learning Methods 0.000 claims description 16
- 238000012795 verification Methods 0.000 claims description 12
- 238000003860 storage Methods 0.000 claims description 9
- 238000012544 monitoring process Methods 0.000 claims description 7
- 238000012806 monitoring device Methods 0.000 claims description 2
- 230000005484 gravity Effects 0.000 description 13
- 238000012545 processing Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 230000006872 improvement Effects 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000007717 exclusion Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000002360 preparation method Methods 0.000 description 2
- 241000531116 Blitum bonus-henricus Species 0.000 description 1
- 235000008645 Chenopodium bonus henricus Nutrition 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention provides a price tag-based out-of-stock detection method and device, wherein the method comprises the following steps: collecting an image of a shelf to be detected; inputting an image of a shelf to be detected into a barrier detection model generated by pre-training to obtain a shelf lattice image of the shelf which is not shielded by a barrier; the obstacle detection model is generated by pre-training according to a plurality of obstacle detection samples of a preset scene; according to the goods shelf grid image which is not shielded by the barrier, goods shelf grid coordinate information which is predetermined based on the position information of the price tag and a stock shortage detection model generated by pre-training are obtained, and a stock shortage detection result is obtained; the out-of-stock detection model is generated by pre-training a plurality of out-of-stock detection samples according to a preset scene. By the technical scheme, the efficiency and accuracy of the out-of-stock detection are improved, and the cost of the out-of-stock detection is reduced.
Description
Technical Field
The invention relates to the technical field of data detection processing, in particular to a price tag-based out-of-stock detection method and device.
Background
The commodity is the most important part of the supermarket, the supermarket generally specially sends a tally clerk to detect goods on the goods shelf for timely replenishment, the replenishment efficiency of the supermarket can be improved by monitoring the commodity in real time, the attractiveness of the goods shelf is improved, the sales volume of the supermarket is increased, and therefore huge profits are brought to the supermarket. The traditional goods shortage detection method mainly comprises two main types: the first type of manual inspection method is that a supermarket tally clerk manually inspects and completes the counting and arrangement of commodities on a goods shelf, and the method consumes a large amount of manpower and financial resources and is not timely in replenishment; the second type of gravity sensor method detects the goods in short supply through the gravity of the goods shelf, although the method can timely remind the goods in short supply, a gravity sensor needs to be installed below each kind of goods and an early warning threshold value needs to be set, the operation is complex, the setting difficulty of the gravity warning threshold value is high, the detection precision is low, the requirement on the goods shelf is high, the supermarket goods shelf needs to be improved, the arrangement is not facilitated, and a large amount of financial resources are consumed. The two methods have the problems of complex operation, low detection efficiency and accuracy and huge labor or financial consumption.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a price tag-based out-of-stock detection method, which is used for improving the efficiency and accuracy of out-of-stock detection and reducing the cost of out-of-stock detection and comprises the following steps:
collecting an image of a shelf to be detected;
inputting the image of the shelf to be detected into an obstacle detection model generated by pre-training to obtain a shelf lattice image which is not shielded by an obstacle; the obstacle detection model is generated by pre-training according to a plurality of obstacle detection samples of a preset scene;
according to the goods shelf grid image which is not shielded by the barrier, goods shelf grid coordinate information which is predetermined based on the position information of the price tag and a stock shortage detection model generated by pre-training are obtained, and a stock shortage detection result is obtained; the out-of-stock detection model is generated by pre-training a plurality of out-of-stock detection samples according to a preset scene.
The embodiment of the invention also provides a price tag-based out-of-stock detection device, which is used for improving the efficiency and accuracy of out-of-stock detection and reducing the cost of out-of-stock detection, and comprises the following components:
the acquisition unit is used for acquiring an image of the shelf to be detected;
the shielding detection unit is used for inputting the image of the shelf to be detected into an obstacle detection model generated by pre-training to obtain a shelf lattice image which is not shielded by an obstacle; the obstacle detection model is generated by pre-training according to a plurality of obstacle detection samples of a preset scene;
the goods shortage detection unit is used for obtaining goods shortage detection results according to goods shelf grid images which are not shielded by the barrier, goods shelf grid coordinate information which is predetermined based on the position information of the price tags and a goods shortage detection model generated by pre-training; the out-of-stock detection model is generated by pre-training a plurality of out-of-stock detection samples according to a preset scene.
The embodiment of the invention also provides computer equipment which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor realizes the price tag-based out-of-stock detection method when executing the computer program.
The embodiment of the invention also provides a computer readable storage medium, and the computer readable storage medium stores a computer program for executing the price tag-based out-of-stock detection method.
The price tag-based out-of-stock detection scheme provided by the embodiment of the invention has the beneficial technical effects that:
firstly, compared with the existing scheme of lack of goods detection based on a gravity sensor, which has inaccurate detection result, is not beneficial to improvement and deployment and has high cost of manpower and material resources, the lack of goods detection scheme provided by the embodiment of the invention has the following steps: collecting an image of a shelf to be detected; inputting an image of a shelf to be detected into a barrier detection model generated by pre-training to obtain a shelf lattice image of the shelf which is not shielded by a barrier; the obstacle detection model is generated by pre-training according to a plurality of obstacle detection samples of a preset scene; according to the goods shelf grid image which is not shielded by the barrier, goods shelf grid coordinate information which is predetermined based on the position information of the price tag and a stock shortage detection model generated by pre-training are obtained, and a stock shortage detection result is obtained; the out-of-stock detection model is generated by pre-training a plurality of out-of-stock detection samples according to a preset scene, so that out-of-stock detection based on price tags is realized, the accuracy rate of out-of-stock detection is improved, a large amount of improvement on commercial excess goods shelves is not needed, and the cost of out-of-stock detection is reduced.
Secondly, compare with current scheme of patrolling and examining through the manual work and carrying out the out-of-stock and detecting, also improved the efficiency that the out-of-stock detected.
In conclusion, the price tag-based out-of-stock detection scheme provided by the embodiment of the invention improves the efficiency and accuracy of out-of-stock detection and reduces the cost of out-of-stock detection.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a price tag-based out-of-stock detection method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a price tag-based stock out detection method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of obstacle detection model generation in an embodiment of the present invention;
FIG. 4 is a schematic illustration of the pre-determination of shelf grid coordinate information in an embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating the generation of a back-order detection model in an embodiment of the present invention;
FIG. 6 is a schematic diagram illustrating the labeling of out-of-stock detection sample data when an out-of-stock detection model is generated in the embodiment of the present invention;
FIG. 7 is a diagram illustrating the types of grid areas predicted by the out-of-stock detection model in an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a price tag-based out-of-stock detection device in an embodiment of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The inventor finds that the existing shortage detection schemes for the commercial overload mainly comprise the following two types:
the first, manual inspection method: and continuously patrolling in the supermarket by a special tally clerk to count the goods shortage condition of the goods shelf. Although the method has higher detection precision, a supermarket needs to employ a special tally clerk to carry out inspection continuously, particularly for a large supermarket, the workload is large, the labor is consumed, the time from the shortage of goods to the statistics to the actual replenishment cannot be guaranteed, and a certain economic loss is caused to the supermarket.
In the second method, a gravity sensor is arranged on a supermarket shelf, and the lack of goods detection is realized according to the change of the gravity of the shelf. The method can warn of the shortage of goods in time, but the method needs to update the detection shelf, a large number of gravity sensors are required to be arranged on the commodity shelf, the deployment cost is increased, the shelf of a supermarket which is already in operation is greatly changed, the investment is increased, the gravity sensor method needs to set a gravity early warning threshold value, the threshold value needs to be set every time the placement position of the commodity on the shelf is changed, and the adjustment of the goods placed on the shelf at any time is limited, so that the method has great limitation.
The inventor finds the technical problems and provides a price tag-based out-of-stock detection scheme, and the scheme realizes real-time and accurate detection of out-of-stock in a supermarket through supermarket monitoring equipment and a deep learning technology. Firstly, the deep learning technology is continuously developed, the deep learning technology is more and more frequently applied to the fields of image detection and recognition and the like, target detection and recognition based on the deep learning is a very popular research direction in the field of image processing in recent years, and the deep learning is widely applied to various industries. Based on the Internet, enterprises upgrade and reform the production, circulation and sale processes of commodities by applying advanced technical means such as big data, artificial intelligence and the like, further remodel an industrial structure and an ecological circle, and deeply merge online service, offline experience and modern logistics, and a new retail mode is a retail industry development trend. How to construct an intelligent store, the intelligent, efficient and convenient management of supermarket commodities is of great importance to the development of new retail. Secondly, a supermarket is generally provided with monitoring equipment which can obtain detailed information of a shelf, each commodity corresponds to a price tag, under the premise condition, the scheme provided by the embodiment of the invention firstly positions the price tags, then generates a shed lattice image according to the positions of the price tags, then inputs the shed lattice image into a deep learning neural network for goods shortage detection, and finally judges the goods shortage for early warning. The method solves the problems that the prior manual detection and gravity sensor short-of-goods detection method has excessive manpower input, consumes huge financial resources, has low accuracy of short-of-goods detection, especially has poor detection on small objects, has high omission factor, cannot identify small goods and the like. Therefore, the price tag-based out-of-stock detection scheme provided by the embodiment of the invention not only accelerates the rate of replenishment in the supermarket, improves the accuracy of out-of-stock detection and identification, but also greatly saves manpower, material resources and financial resources, and has strong market applicability. The price tag-based out-of-stock detection scheme is described in detail below.
Fig. 1 is a schematic flow chart of a price tag-based out-of-stock detection method in an embodiment of the present invention, and as shown in fig. 1, the method includes the following steps:
step 101: collecting an image of a shelf to be detected;
step 102: inputting the image of the shelf to be detected into an obstacle detection model generated by pre-training to obtain a shelf lattice image which is not shielded by an obstacle; the obstacle detection model is generated by pre-training according to a plurality of obstacle detection samples of a preset scene;
step 103: according to the goods shelf grid image which is not shielded by the barrier, goods shelf grid coordinate information which is predetermined based on the position information of the price tag and a stock shortage detection model generated by pre-training are obtained, and a stock shortage detection result is obtained; the out-of-stock detection model is generated by pre-training a plurality of out-of-stock detection samples according to a preset scene.
The price tag-based out-of-stock detection scheme provided by the embodiment of the invention has the beneficial technical effects that:
firstly, compared with the existing scheme that the lack of goods detection is carried out based on the gravity sensor, the detection result is inaccurate, the improvement and the deployment are not facilitated, and the cost of manpower and material resources is high, the lack of goods detection scheme provided by the embodiment of the invention has the following steps: collecting an image of a shelf to be detected; inputting an image of a shelf to be detected into a barrier detection model generated by pre-training to obtain a shelf lattice image of the shelf which is not shielded by a barrier; the obstacle detection model is generated by pre-training according to a plurality of obstacle detection samples of a preset scene; according to the goods shelf grid image which is not shielded by the barrier, goods shelf grid coordinate information which is predetermined based on the position information of the price tag and a stock shortage detection model generated by pre-training are obtained, and a stock shortage detection result is obtained; the out-of-stock detection model is generated by pre-training a plurality of out-of-stock detection samples according to a preset scene, so that out-of-stock detection based on price tags is realized, the accuracy rate of out-of-stock detection is improved, a large amount of improvement on commercial excess goods shelves is not needed, and the cost of out-of-stock detection is reduced.
Secondly, compare with current scheme of patrolling and examining through the manual work and carrying out the out-of-stock and detecting, also improved the efficiency that the out-of-stock detected.
In conclusion, the price tag-based out-of-stock detection scheme provided by the embodiment of the invention improves the efficiency and accuracy of out-of-stock detection and reduces the cost of out-of-stock detection.
The following describes in detail the steps involved in the price tag-based out-of-stock detection method according to an embodiment of the present invention with reference to fig. 2 to 7.
Firstly, introducing preparation steps: pre-training to generate an obstacle detection model, a backorder detection model and pre-determining goods shelf grid coordinate information.
In an embodiment, the price tag-based out-of-stock detection method may further include: pre-training the generated out-of-stock detection model according to the following method:
acquiring out-of-stock detection sample data; the out-of-stock detection sample data comprises a plurality of shelf grid graphs and grid area type graphs corresponding to the shelf grid graphs;
dividing the out-of-stock detection sample data into a training set, a testing set and a verification set;
training a lack-of-goods detection deep learning network by using the training set to obtain an initial lack-of-goods detection model;
verifying the initial out-of-stock detection model by using the verification set to obtain a verified out-of-stock detection model;
and testing the verified out-of-stock detection model by using the test set to obtain the out-of-stock detection model.
In an embodiment, the price tag-based out-of-stock detection method may further include: pre-training the generated obstacle detection model according to the following method:
acquiring obstacle detection sample data;
dividing the obstacle detection sample data into a training set, a test set and a verification set;
training a barrier detection deep learning network by using the training set to obtain an initial barrier detection model;
verifying the initial obstacle detection model by using the verification set to obtain a verified obstacle detection model;
and testing the verified obstacle detection model by using the test set to obtain the obstacle detection model.
In specific implementation, as shown in fig. 3 to 6, the detailed processes of the obstacle detection model, the out-of-stock detection model and the pre-determination of the shelf grid coordinate information include:
1. firstly, collecting sample data of a training obstacle detection model, an out-of-stock detection model and a price tag identification model used when the goods shelf grid coordinate information is determined in advance, and carrying out tagging work: (1) acquiring obstacle detection data and marking, wherein the obstacle data marking rule is as follows: human occlusion, item occlusion (marking sample data of a training obstacle detection model); (2) collecting price tag data marking rules: the categories are two categories, namely price tags and non-price tags (sample data of a training price tag identification model is labeled); (3) as shown in fig. 6, the out-of-stock data labels are divided into 8 categories (commodity state types), and the labels correspond to the out-of-stock conditions of the commodities respectively (sample data for training the out-of-stock detection model is labeled).
2. And (3) performing model training by using the labeled data: obstacle detection model training, price tag recognition model training and out-of-stock detection model training, this step: (1) inputting the obstacle data (obstacle detection sample data) described in the above "1" into an obstacle detection deep learning network (deep learning neural network), and performing obstacle detection model training to obtain an obstacle detection model; (2) inputting a price tag data set (price tag identification sample data) into a price tag identification deep learning network, and training to obtain a price tag identification model; (3) and inputting the stock shortage data set (stock shortage detection sample data) into the stock shortage detection deep learning network, and training to obtain a stock shortage detection model.
3. The method comprises the following steps of determining the coordinate information of the goods shelf grids in advance: after all goods shelves needing to be detected are photographed, a price tag identification model is input to complete price tag positioning, and according to the price tags and the position information which are identified, the shed grids in the goods shelves are generated, and the generated shed grid coordinate information is temporarily stored, so that the goods shortage detection can be conveniently used for multiple times.
The three steps can be regarded as one-time preparation work, and the adjustment range and frequency of the positions of the goods shelves and the positions of the placed goods are small, so that the model can be used for multiple times after one-time model training and shed lattice generation. The steps of detecting the shortage of goods by applying the obstacle detection model, the shortage detection model and the predetermined shelf coordinate information will be described with reference to fig. 2.
Next, the above step 101 is described.
In one embodiment, capturing an image of a shelf to be inspected may include: and acquiring an image of the shelf to be detected by using supermarket monitoring equipment.
During specific implementation, a supermarket is generally provided with monitoring equipment which can acquire detailed information of a goods shelf, each commodity corresponds to a price tag, an image of the goods shelf to be detected is acquired through the original monitoring equipment of the supermarket, subsequent goods shortage detection is carried out, the supermarket does not need to be greatly modified like a goods shortage detection scheme carried out by a gravity sensor method in the prior art, and cost is saved.
Third, next, the above step 102 is introduced.
In an embodiment, inputting the image of the shelf to be detected into an obstacle detection model generated by pre-training to obtain a shelf lattice image that is not blocked by an obstacle may include:
when the situation that the image of the shelf to be detected is shielded by the obstacle is detected, detecting whether a person exists in the image of the shelf to be detected shielded by the obstacle;
when no person is detected in the image of the shelf to be detected, which is shielded by the obstacle, the shelf lattice image which is not shielded by the article is extracted from the image of the shelf to be detected, so that the shelf lattice image which is not shielded by the obstacle is obtained;
and when people are detected in the image of the shelf to be detected, which is shielded by the barrier, the step of collecting the image of the shelf to be detected is executed again.
In particular implementations, the obstacles may include people and objects. And inputting the detected image into an obstacle detection model to detect the obstacle, if the detection result has shielding, extracting the unshielded grids according to the detected shelter frame to continue the short-order detection, and excluding the shielding grids from participating in the detection. And judging whether the detection result contains the person, if yes, returning to the step 101 that the person is shielded and needs to be photographed again.
Fourth, next, the above step 103 is described.
In one embodiment, obtaining the out-of-stock detection result according to the shelf lattice image which is not blocked by the obstacle, the shelf lattice coordinate information which is predetermined based on the position information of the price tag, and the generated out-of-stock detection model which is trained in advance may include:
inputting a price tag identification model generated by pre-training a Shang-Shen shelf grid image to obtain a price tag corresponding to the Shang-Shen shelf grid and position information of the price tag, and obtaining the predetermined shelf grid coordinate information according to the position information of the price tag (the step is a step of determining the shelf grid coordinate information in advance); the price tag identification model is generated by pre-training according to a plurality of price tag identification samples of a preset scene;
determining position information corresponding to the shelf grids which are not shielded by the barrier according to the predetermined coordinate information of the shelf grids;
and obtaining a shortage detection result according to the position information corresponding to the shelf grids which are not shielded by the barrier and a shortage detection model generated by pre-training.
In one embodiment, obtaining the out-of-stock detection result according to the position information corresponding to the shelf grids which are not blocked by the obstacle and the out-of-stock detection model generated by pre-training may include:
generating a to-be-detected shelf grid graph according to the position information corresponding to the shelf grids which are not shielded by the barrier;
inputting the trellis diagram to be detected into a stock shortage detection model generated by pre-training to obtain a trellis area type diagram (as shown in fig. 7);
identifying the commodity state type (the states of shortage, availability and the like) in the grids according to the grid area type graph;
determining the stock shortage rate (if the stock shortage exists, the specific stock shortage is determined) of each commodity according to the commodity state type and the corresponding area;
and determining the out-of-stock detection result according to the commodity state type and the out-of-stock rate of each commodity.
In specific implementation, grid coordinates of the shelf corresponding to the detection image are read, a grid diagram is generated and input into a stock shortage detection model obtained through pre-training, stock shortage detection is carried out, and through the step, a model prediction result of the stock shortage detection model shown in fig. 7, namely a grid area type diagram, is obtained. As shown in fig. 2, the model prediction result is logically processed, and the logic processing algorithm mainly has the following tasks: calculating the stock shortage rate according to the area of the target category (see the meaning corresponding to the label in fig. 6, that is, the commodity state type corresponding to each shelf area, stock shortage or stock shortage, and the like) identified in the shelf, wherein: the presence of goods, goods in box, shading indicate no stock shortage; the lack of goods, the side edge and the empty box represent lack of goods, but the occupied lack of goods proportion is different; exclusion and advertisement exclusion do not participate in the calculation. And finally, outputting the result of the out-of-stock detection.
The price tag-based out-of-stock detection scheme provided by the invention has the detection precision of over 99 percent. The goods shortage condition of the goods on the goods shelf can be monitored in real time, and a replenishment worker is reminded in time to replenish the goods when the goods shortage occurs, so that the attractiveness of the goods shelf is improved, the cost of a large amount of manpower and financial resources is reduced, and huge profits are brought to the supermarket
Based on the same inventive concept, the embodiment of the invention also provides a price tag-based out-of-stock detection device, which is described in the following embodiment. Because the principle of the price tag-based out-of-stock detection device for solving the problems is similar to the price tag-based out-of-stock detection method, the implementation of the price tag-based out-of-stock detection device can refer to the implementation of the price tag-based out-of-stock detection method, and repeated parts are not described again. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 8 is a schematic structural diagram of a price tag-based out-of-stock detection apparatus in an embodiment of the present invention, and as shown in fig. 8, the apparatus includes:
the acquisition unit 01 is used for acquiring an image of the shelf to be detected;
the shielding detection unit 02 is used for inputting the image of the shelf to be detected into an obstacle detection model generated by pre-training to obtain a shelf lattice image which is not shielded by an obstacle; the obstacle detection model is generated by pre-training according to a plurality of obstacle detection samples of a preset scene;
the out-of-stock detection unit 03 is used for obtaining an out-of-stock detection result according to a shelf grid image which is not shielded by the barrier, shelf grid coordinate information which is predetermined based on the position information of the price tag and an out-of-stock detection model generated by pre-training; the out-of-stock detection model is generated by pre-training a plurality of out-of-stock detection samples according to a preset scene.
In one embodiment, the acquisition unit may be a supermarket monitoring device.
In one embodiment, the out-of-stock detection unit may be specifically configured to:
inputting the Shanghai shelf shed lattice image into a price tag identification model generated by pre-training to obtain a price tag corresponding to the Shanghai shelf shed lattice and position information of the price tag, and obtaining the predetermined shelf shed lattice coordinate information according to the position information of the price tag; the price tag identification model is generated by pre-training according to a plurality of price tag identification samples of a preset scene;
determining position information corresponding to the shelf grids which are not shielded by the barrier according to the predetermined coordinate information of the shelf grids;
and obtaining a shortage detection result according to the position information corresponding to the shelf grids which are not shielded by the barrier and a shortage detection model generated by pre-training.
In one embodiment, obtaining the out-of-stock detection result according to the position information corresponding to the shelf grids which are not shielded by the obstacle and the out-of-stock detection model generated by pre-training comprises:
generating a to-be-detected grid diagram according to the position information corresponding to the shelf grids which are not shielded by the barrier;
inputting the trellis diagram to be detected into a stock shortage detection model generated by pre-training to obtain a trellis area type diagram;
identifying the commodity state type in the grids according to the grid area type graph;
determining the stock out rate of each commodity according to the commodity state type and the corresponding area;
and determining the out-of-stock detection result according to the commodity state type and the out-of-stock rate of each commodity.
In an embodiment, the occlusion detection unit may be specifically configured to:
when the situation that the image of the shelf to be detected is shielded by the obstacle is detected, detecting whether a person exists in the image of the shelf to be detected shielded by the obstacle;
when no person is detected in the image of the shelf to be detected, which is shielded by the obstacle, the shelf lattice image which is not shielded by the article is extracted from the image of the shelf to be detected, so that the shelf lattice image which is not shielded by the obstacle is obtained;
and when people are detected in the image of the shelf to be detected, which is shielded by the barrier, the step of collecting the image of the shelf to be detected is executed again.
In one embodiment, the price tag-based out-of-stock detection apparatus may further include: the training unit is used for pre-training the generated out-of-stock detection model according to the following method:
acquiring out-of-stock detection sample data; the out-of-stock detection sample data comprises a plurality of shelf grid graphs and grid area type graphs corresponding to the shelf grid graphs;
dividing the out-of-stock detection sample data into a training set, a testing set and a verification set;
training a lack-of-goods detection deep learning network by using the training set to obtain an initial lack-of-goods detection model;
verifying the initial out-of-stock detection model by using the verification set to obtain a verified out-of-stock detection model;
and testing the verified out-of-stock detection model by using the test set to obtain the out-of-stock detection model.
In one embodiment, the price tag-based out-of-stock detection apparatus may further include: an obstacle detection model training unit, configured to train the generated obstacle detection model in advance according to the following method:
acquiring obstacle detection sample data;
dividing the obstacle detection sample data into a training set, a test set and a verification set;
training a barrier detection deep learning network by using the training set to obtain an initial barrier detection model;
verifying the initial obstacle detection model by using the verification set to obtain a verified obstacle detection model;
and testing the verified obstacle detection model by using the test set to obtain the obstacle detection model.
The embodiment of the invention also provides computer equipment which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor realizes the price tag-based out-of-stock detection method when executing the computer program.
The embodiment of the invention also provides a computer readable storage medium, and the computer readable storage medium stores a computer program for executing the price tag-based out-of-stock detection method.
The price tag-based out-of-stock detection scheme provided by the invention has the following beneficial technical effects: the method and the device have the advantages that the out-of-stock detection based on the price tags is realized, the accuracy of the out-of-stock detection is improved, the commercial shelf is not required to be improved greatly, and the cost of the out-of-stock detection is reduced.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes may be made to the embodiment of the present invention by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (14)
1. A price tag-based out-of-stock detection method is characterized by comprising the following steps:
collecting an image of a shelf to be detected;
inputting the image of the shelf to be detected into an obstacle detection model generated by pre-training to obtain a shelf lattice image which is not shielded by an obstacle; the obstacle detection model is generated by pre-training according to a plurality of obstacle detection samples of a preset scene;
according to the goods shelf grid image which is not shielded by the barrier, goods shelf grid coordinate information which is predetermined based on the position information of the price tag and a stock shortage detection model generated by pre-training are obtained, and a stock shortage detection result is obtained; the out-of-stock detection model is generated by pre-training a plurality of out-of-stock detection samples according to a preset scene.
2. The price tag-based out-of-stock detection method of claim 1, wherein capturing an image of a shelf to be detected comprises: and acquiring an image of the shelf to be detected by using supermarket monitoring equipment.
3. The method for detecting out-of-stock based on price tags as claimed in claim 1, wherein the step of obtaining out-of-stock detection results according to shelf grid images which are not blocked by obstacles, shelf grid coordinate information which is predetermined based on position information of price tags and an out-of-stock detection model generated by pre-training comprises the following steps:
inputting the Shanghai shelf shed lattice image into a price tag identification model generated by pre-training to obtain a price tag corresponding to the Shanghai shelf shed lattice and position information of the price tag, and obtaining the predetermined shelf shed lattice coordinate information according to the position information of the price tag; the price tag identification model is generated by pre-training according to a plurality of price tag identification samples of a preset scene;
determining position information corresponding to the shelf grids which are not shielded by the barrier according to the predetermined coordinate information of the shelf grids;
and obtaining a shortage detection result according to the position information corresponding to the shelf grids which are not shielded by the barrier and a shortage detection model generated by pre-training.
4. The method for detecting out-of-stock based on price tags as claimed in claim 3, wherein the step of obtaining out-of-stock detection results according to the position information corresponding to the shelf grids which are not shielded by the obstacles and the out-of-stock detection model generated by pre-training comprises the following steps:
generating a to-be-detected grid diagram according to the position information corresponding to the shelf grids which are not shielded by the barrier;
inputting the trellis diagram to be detected into a stock shortage detection model generated by pre-training to obtain a trellis area type diagram;
identifying the commodity state type in the grids according to the grid area type graph;
determining the stock out rate of each commodity according to the commodity state type and the corresponding area;
and determining the out-of-stock detection result according to the commodity state type and the out-of-stock rate of each commodity.
5. The price tag-based out-of-stock detection method of claim 1, wherein the inputting of the image of the shelf to be detected into a pre-trained obstacle detection model to obtain a shelf grid image which is not blocked by an obstacle comprises:
when the situation that the image of the shelf to be detected is shielded by the obstacle is detected, detecting whether a person exists in the image of the shelf to be detected shielded by the obstacle;
when no person is detected in the image of the shelf to be detected, which is shielded by the obstacle, the shelf lattice image which is not shielded by the article is extracted from the image of the shelf to be detected, so that the shelf lattice image which is not shielded by the obstacle is obtained;
and when people are detected in the image of the shelf to be detected, which is shielded by the barrier, the step of collecting the image of the shelf to be detected is executed again.
6. The price tag-based out-of-stock detection method of claim 1, further comprising: pre-training the generated out-of-stock detection model according to the following method:
acquiring out-of-stock detection sample data; the out-of-stock detection sample data comprises a plurality of shelf grid graphs and grid area type graphs corresponding to the shelf grid graphs;
dividing the out-of-stock detection sample data into a training set, a testing set and a verification set;
training a lack-of-goods detection deep learning network by using the training set to obtain an initial lack-of-goods detection model;
verifying the initial out-of-stock detection model by using the verification set to obtain a verified out-of-stock detection model;
and testing the verified out-of-stock detection model by using the test set to obtain the out-of-stock detection model.
7. The utility model provides an out-of-stock detection device based on price tag which characterized in that includes:
the acquisition unit is used for acquiring an image of the shelf to be detected;
the shielding detection unit is used for inputting the image of the shelf to be detected into an obstacle detection model generated by pre-training to obtain a shelf lattice image which is not shielded by an obstacle; the obstacle detection model is generated by pre-training according to a plurality of obstacle detection samples of a preset scene;
the goods shortage detection unit is used for obtaining goods shortage detection results according to goods shelf grid images which are not shielded by the barrier, goods shelf grid coordinate information which is predetermined based on the position information of the price tags and a goods shortage detection model generated by pre-training; the out-of-stock detection model is generated by pre-training a plurality of out-of-stock detection samples according to a preset scene.
8. The price tag-based stock shortage detection apparatus of claim 7, wherein the acquisition unit is a supermarket monitoring device.
9. The price tag-based out-of-stock detection apparatus of claim 7, wherein the out-of-stock detection unit is specifically configured to:
inputting the Shanghai shelf shed lattice image into a price tag identification model generated by pre-training to obtain a price tag corresponding to the Shanghai shelf shed lattice and position information of the price tag, and obtaining the predetermined shelf shed lattice coordinate information according to the position information of the price tag; the price tag identification model is generated by pre-training according to a plurality of price tag identification samples of a preset scene;
determining position information corresponding to the shelf grids which are not shielded by the barrier according to the predetermined coordinate information of the shelf grids;
and obtaining a shortage detection result according to the position information corresponding to the shelf grids which are not shielded by the barrier and a shortage detection model generated by pre-training.
10. The price tag-based out-of-stock detection device of claim 9, wherein obtaining out-of-stock detection results according to the position information corresponding to the shelf grids that are not covered by the obstacle and an out-of-stock detection model generated by pre-training comprises:
generating a to-be-detected grid diagram according to the position information corresponding to the shelf grids which are not shielded by the barrier;
inputting the trellis diagram to be detected into a stock shortage detection model generated by pre-training to obtain a trellis area type diagram;
identifying the commodity state type in the grids according to the grid area type graph;
determining the stock out rate of each commodity according to the commodity state type and the corresponding area;
and determining the out-of-stock detection result according to the commodity state type and the out-of-stock rate of each commodity.
11. The price tag-based out-of-stock detection apparatus of claim 7, wherein the occlusion detection unit is specifically configured to:
when the situation that the image of the shelf to be detected is shielded by the obstacle is detected, detecting whether a person exists in the image of the shelf to be detected shielded by the obstacle;
when no person is detected in the image of the shelf to be detected, which is shielded by the obstacle, the shelf lattice image which is not shielded by the article is extracted from the image of the shelf to be detected, so that the shelf lattice image which is not shielded by the obstacle is obtained;
and when people are detected in the image of the shelf to be detected, which is shielded by the barrier, the step of collecting the image of the shelf to be detected is executed again.
12. The price tag-based stock shortage detection apparatus of claim 7, further comprising: the training unit is used for pre-training the generated out-of-stock detection model according to the following method:
acquiring out-of-stock detection sample data; the out-of-stock detection sample data comprises a plurality of shelf grid graphs and grid area type graphs corresponding to the shelf grid graphs;
dividing the out-of-stock detection sample data into a training set, a testing set and a verification set;
training a lack-of-goods detection deep learning network by using the training set to obtain an initial lack-of-goods detection model;
verifying the initial out-of-stock detection model by using the verification set to obtain a verified out-of-stock detection model;
and testing the verified out-of-stock detection model by using the test set to obtain the out-of-stock detection model.
13. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 6 when executing the computer program.
14. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911028666.7A CN112800804A (en) | 2019-10-28 | 2019-10-28 | Price tag-based out-of-stock detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911028666.7A CN112800804A (en) | 2019-10-28 | 2019-10-28 | Price tag-based out-of-stock detection method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112800804A true CN112800804A (en) | 2021-05-14 |
Family
ID=75803055
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911028666.7A Pending CN112800804A (en) | 2019-10-28 | 2019-10-28 | Price tag-based out-of-stock detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112800804A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113326796A (en) * | 2021-06-17 | 2021-08-31 | 北京百度网讯科技有限公司 | Object detection method, model training method and device and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3032459A1 (en) * | 2014-12-10 | 2016-06-15 | Ricoh Company, Ltd. | Realogram scene analysis of images: shelf and label finding |
| CN105701519A (en) * | 2014-12-10 | 2016-06-22 | 株式会社理光 | Realogram scene analysis of images: superpixel scene analysis |
| US20170103515A1 (en) * | 2015-10-09 | 2017-04-13 | Pricer Ab | Determination of a product position along a fixture or positions of objects in images |
| CN108364005A (en) * | 2018-03-07 | 2018-08-03 | 上海扩博智能技术有限公司 | Automatic identifying method, system, equipment and the storage medium of price tag |
| CN108416403A (en) * | 2018-03-08 | 2018-08-17 | 上海扩博智能技术有限公司 | The automatic correlation method of commodity and label, system, equipment and storage medium |
| US20180260772A1 (en) * | 2017-01-31 | 2018-09-13 | Focal Systems, Inc | Out-of-stock detection based on images |
| CN109308434A (en) * | 2017-07-27 | 2019-02-05 | 浙江汉朔电子科技有限公司 | Electronics price tag, server and system |
| CN110322197A (en) * | 2019-06-24 | 2019-10-11 | 深圳前海微众银行股份有限公司 | Commodity intelligence replenishing method, device, terminal and storage medium |
-
2019
- 2019-10-28 CN CN201911028666.7A patent/CN112800804A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3032459A1 (en) * | 2014-12-10 | 2016-06-15 | Ricoh Company, Ltd. | Realogram scene analysis of images: shelf and label finding |
| CN105701519A (en) * | 2014-12-10 | 2016-06-22 | 株式会社理光 | Realogram scene analysis of images: superpixel scene analysis |
| US20170103515A1 (en) * | 2015-10-09 | 2017-04-13 | Pricer Ab | Determination of a product position along a fixture or positions of objects in images |
| US20180260772A1 (en) * | 2017-01-31 | 2018-09-13 | Focal Systems, Inc | Out-of-stock detection based on images |
| CN109308434A (en) * | 2017-07-27 | 2019-02-05 | 浙江汉朔电子科技有限公司 | Electronics price tag, server and system |
| CN108364005A (en) * | 2018-03-07 | 2018-08-03 | 上海扩博智能技术有限公司 | Automatic identifying method, system, equipment and the storage medium of price tag |
| CN108416403A (en) * | 2018-03-08 | 2018-08-17 | 上海扩博智能技术有限公司 | The automatic correlation method of commodity and label, system, equipment and storage medium |
| US20190279144A1 (en) * | 2018-03-08 | 2019-09-12 | Shanghai Clobotics Technology Co., Ltd. | Method, system and device for association of commodities and price tags |
| CN110322197A (en) * | 2019-06-24 | 2019-10-11 | 深圳前海微众银行股份有限公司 | Commodity intelligence replenishing method, device, terminal and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113326796A (en) * | 2021-06-17 | 2021-08-31 | 北京百度网讯科技有限公司 | Object detection method, model training method and device and electronic equipment |
| CN113326796B (en) * | 2021-06-17 | 2022-11-29 | 北京百度网讯科技有限公司 | Object detection method, model training method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109446883B (en) | Commodity state identification method and device, electronic equipment and readable storage medium | |
| CN109840504B (en) | Article taking and placing behavior identification method and device, storage medium and equipment | |
| CN108898109A (en) | The determination methods, devices and systems of article attention rate | |
| CN116188475B (en) | Intelligent control method, system and medium for automatic optical detection of appearance defects | |
| US20200167715A1 (en) | Methods for real-time skill assessment of multi-step tasks performed by hand movements using a video camera | |
| CN108345912A (en) | Commodity rapid settlement system based on RGBD information and deep learning | |
| CN114066900A (en) | Image segmentation method and device, electronic equipment and storage medium | |
| Milella et al. | Towards intelligent retail: Automated on-shelf availability estimation using a depth camera | |
| Pham et al. | A YOLO-based real-time packaging defect detection system | |
| CN108629318A (en) | A kind of shelf display recognition methods based on image recognition technology, device and system | |
| CN112800804A (en) | Price tag-based out-of-stock detection method and device | |
| CN111091104A (en) | Target object protection detection method, device, equipment and storage medium | |
| CN111985269A (en) | Detection model construction method, detection device, server and medium | |
| WO2021081688A1 (en) | Price tag-based out-of-stock detection method and device | |
| CN111626981A (en) | Method and device for identifying category of goods to be detected | |
| CN112668365A (en) | Material warehousing identification method, device, equipment and storage medium | |
| CN115965122A (en) | Method for obtaining national economy manufacturing industry corresponding to commodity based on machine learning | |
| CN114255377A (en) | Differential commodity detection and classification method for intelligent container | |
| CN111797896A (en) | Commodity identification method and device based on intelligent baking | |
| Kumari et al. | CertiMart: Use Computer Vision to Digitize and Automate Supermarket with Fruit Quality Measuring and Maintaining | |
| CN116740473B (en) | Automatic sorting method and system for fish catch based on machine vision | |
| CN110728310A (en) | Target detection model fusion method and system based on hyper-parameter optimization | |
| Geng et al. | Shelf Product Detection Based on Deep Neural Network | |
| Zhang et al. | Design and Implementation of a Vision-and Grating-Sensor-Based Intelligent Unmanned Settlement System | |
| CN110956761B (en) | Object processing method and system, computer system and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |