CN109543527A - For the commodity detection method of unmanned shelf, device and retail terminal - Google Patents
For the commodity detection method of unmanned shelf, device and retail terminal Download PDFInfo
- Publication number
- CN109543527A CN109543527A CN201811221070.4A CN201811221070A CN109543527A CN 109543527 A CN109543527 A CN 109543527A CN 201811221070 A CN201811221070 A CN 201811221070A CN 109543527 A CN109543527 A CN 109543527A
- Authority
- CN
- China
- Prior art keywords
- commodity
- image
- shelf
- module
- unmanned
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
Abstract
This application discloses a kind of commodity detection methods and device for unmanned shelf, this method comprises: obtaining the commodity image in collected unmanned shelf on every layer；It cuts the commodity image and inputs the object category detection model that training obtains in advance；Position and size of each commodity in the commodity image are exported according to the object category detection model；And the quantity of type of merchandize and each classification in the commodity image is retrieved and exported in commodity image java standard library.Present application addresses the technical problems that the commodity detection method effect on unmanned shelf is poor.Image retrieval technologies are cooperated using target detection technique by the application, the functions such as customer's picking, the return of goods and businessman's stock can be realized on unmanned shelf, to save the manpower and material resources of unmanned shelf and traditional shelf.
Description
Technical field
This application involves computer vision fields, in particular to a kind of commodity detection method for unmanned shelf
And device.
Background technique
Unmanned shelf are a kind of common retail terminals.
Inventors have found that common unmanned shelf are to fall the commodity of user's selection from inside, so that user will fall off
Commodity take out, however this shelf sale commodity basis process is complex.Further, the type sum number sold goods is affected
The upper of amount is newly sold, and customer consumption experience is poor.
For the problem that the commodity detection method effect on unmanned shelf in the related technology is poor, not yet propose at present effective
Solution.
Summary of the invention
The main purpose of the application is to provide a kind of whole for the commodity detection method of unmanned shelf, device and retail
End, to solve the problems, such as that the commodity detection method effect on unmanned shelf is poor.
To achieve the goals above, according to the one aspect of the application, a kind of commodity inspection for unmanned shelf is provided
Survey method.
The commodity detection method for unmanned shelf according to the application includes: to obtain in collected unmanned shelf every layer
On commodity image；It cuts the commodity image and inputs the object category detection model that training obtains in advance；According to the object
Body classification detection model exports position and size of each commodity in the commodity image；And in commodity image standard
The quantity of type of merchandize and each classification in the commodity image is retrieved and exported in library.
Further, cutting the commodity image and inputting the object category detection model that training obtains in advance includes: to cut out
Cut the region in addition to shelf ontology；Determine object category；The commodity for whether having determining classification in the commodity image detected；
And if detecting the commodity for having determining classification in the commodity image, position and the size of the type commodity are exported, wherein
Object category includes at least: any one or more in bottled, canned, box-packed or packed.
To achieve the goals above, according to the another aspect of the application, a kind of retail terminal is provided.
Retail terminal according to the application includes: the unmanned shelf using the commodity detection method, the unmanned goods
Frame further include: picture pick-up device, the top for being mounted on every layer of shelf are used to acquire the commodity image in unmanned shelf on every layer；Sense
Equipment is answered, is installed on every layer of shelf for detecting the commodity weight in unmanned shelf on every layer；And processing equipment, lead to
Network is crossed by the data transmission on the picture pick-up device and sensing apparatus to server；Wherein, the cabinet door of the unmanned shelf is every
When closing one time, starts the picture pick-up device and execute Image Acquisition operation and sensing apparatus execution weighing operation.
Further, the processing equipment includes: the first authority module, for receiving the first behaviour to the unmanned shelf
Make identity authority；It sells goods module, for judging the reduction of commodity on the unmanned shelf according to the first operation identity authority
Classification and quantity；And order confirmation module, for generating purchase order according to the unit price and quantity of commodity.
Further, the processing equipment includes: the second authority module, for receiving the second behaviour to the unmanned shelf
Make identity authority；Return of goods module, for judging the increase of commodity on the unmanned shelf according to the second operation identity authority
Classification and quantity；And reimbursement confirmation module, for confirming reimbursement according to the unit price and quantity of commodity.
Further, the processing equipment includes: third authority module, for receiving the third behaviour to the unmanned shelf
Make identity authority；Stock module, for operating the variation that identity authority judges commodity on the unmanned shelf according to the third
Classification and quantity；And stock checklist module, for generating bill of goods according to the type and quantity of commodity.
Further, retail terminal further include: display equipment is mounted on outside the cabinet door of the unmanned shelf and is used to show
Merchandise news, wherein the merchandise news includes: commodity amount, cargo price, commodity total price, any one in merchandise classification
Kind is a variety of.
To achieve the goals above, according to the another aspect of the application, a kind of commodity inspection for unmanned shelf is provided
Survey device.
According to a kind of commodity detection device for unmanned shelf of the application, comprising: module is obtained, for obtaining acquisition
To unmanned shelf in commodity image on every layer；Module is cut, for cutting the commodity image and inputting trained in advance
The object category detection model arrived；Output module exists for exporting each commodity according to the object category detection model
Position and size in the commodity image；And retrieval module, it is described for retrieving and exporting in commodity image java standard library
The quantity of type of merchandize and each classification in commodity image.
Further, the cutting module includes: cutting unit, for cropping the region in addition to shelf ontology；Really
Order member, for determining object category；Detection unit, for detecting the commodity for whether having determining classification in the commodity image；
And output unit, when for detecting the commodity for having determining classification in the commodity image, export the type commodity position and
Size.
Further, the retrieval module includes: sample unit, is marked for establishing the classification of commodity as image pattern
Quasi- library；Whether judging unit, packaging or type to judge commodity update；And replacement unit, for judging the packet of commodity
When dress or type commodity update, described image sample is replaced.
In the embodiment of the present application, it by the way of obtaining the commodity image in collected unmanned shelf on every layer, cuts out
It cuts the commodity image and inputs the object category detection model that training obtains in advance.It is defeated by the object category detection model
Position and size of each commodity in the commodity image out, has reached and has retrieved and export in commodity image java standard library
The purpose of the quantity of type of merchandize and each classification in the commodity image, to realize combining target detection and image retrieval
Realize the technical effect of unmanned shelf automatic vending, and then it is poor to solve the commodity detection method effect on unmanned shelf
Technical problem.Feel in addition, also the user experience is improved, reduce businessman's O&M cost.
Detailed description of the invention
The attached drawing constituted part of this application is used to provide further understanding of the present application, so that the application's is other
Feature, objects and advantages become more apparent upon.The illustrative examples attached drawing and its explanation of the application is for explaining the application, not
Constitute the improper restriction to the application.In the accompanying drawings:
Fig. 1 is the commodity detection method schematic diagram for unmanned shelf according to the embodiment of the present application；
Fig. 2 is the commodity detection method schematic diagram for unmanned shelf according to the embodiment of the present application；
Fig. 3 is the retail terminal structural schematic diagram according to the embodiment of the present application；
Fig. 4 is the retail terminal structural schematic diagram according to the embodiment of the present application；
Fig. 5 is the retail terminal structural schematic diagram according to the embodiment of the present application；
Fig. 6 is the retail terminal structural schematic diagram according to the embodiment of the present application；
Fig. 7 is the retail terminal structural schematic diagram according to the embodiment of the present application；
Fig. 8 is the commodity detection device schematic diagram for unmanned shelf according to the embodiment of the present application；
Fig. 9 is the commodity detection device schematic diagram for unmanned shelf according to the embodiment of the present application；
Figure 10 is the commodity detection device schematic diagram for unmanned shelf according to the embodiment of the present application；
Figure 11 is the commodity image schematic diagram in the unmanned shelf according to the embodiment of the present application on every layer；And
Figure 12 is that the quantity of type of merchandize and each classification is shown in the output commodity image according to the embodiment of the present application
It is intended to.
Specific embodiment
In order to make those skilled in the art more fully understand application scheme, below in conjunction in the embodiment of the present application
Attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is only
The embodiment of the application a part, instead of all the embodiments.Based on the embodiment in the application, ordinary skill people
Member's every other embodiment obtained without making creative work, all should belong to the model of the application protection
It encloses.
It should be noted that the description and claims of this application and term " first " in above-mentioned attached drawing, "
Two " etc. be to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should be understood that using in this way
Data be interchangeable under appropriate circumstances, so as to embodiments herein described herein.In addition, term " includes " and " tool
Have " and their any deformation, it is intended that cover it is non-exclusive include, for example, containing a series of steps or units
Process, method, system, product or equipment those of are not necessarily limited to be clearly listed step or unit, but may include without clear
Other step or units listing to Chu or intrinsic for these process, methods, product or equipment.
In this application, term " on ", "lower", "left", "right", "front", "rear", "top", "bottom", "inner", "outside",
" in ", "vertical", "horizontal", " transverse direction ", the orientation or positional relationship of the instructions such as " longitudinal direction " be orientation based on the figure or
Positional relationship.These terms are not intended to limit indicated dress primarily to better describe the application and embodiment
Set, element or component must have particular orientation, or constructed and operated with particular orientation.
Also, above-mentioned part term is other than it can be used to indicate that orientation or positional relationship, it is also possible to for indicating it
His meaning, such as term " on " also are likely used for indicating certain relations of dependence or connection relationship in some cases.For ability
For the those of ordinary skill of domain, the concrete meaning of these terms in this application can be understood as the case may be.
In addition, term " installation ", " setting ", " being equipped with ", " connection ", " connected ", " socket " shall be understood in a broad sense.For example,
It may be a fixed connection, be detachably connected or monolithic construction；It can be mechanical connection, or electrical connection；It can be direct phase
It even, or indirectly connected through an intermediary, or is two connections internal between device, element or component.
For those of ordinary skills, the concrete meaning of above-mentioned term in this application can be understood as the case may be.
It should be noted that in the absence of conflict, the features in the embodiments and the embodiments of the present application can phase
Mutually combination.The application is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
Method in the application can realize customer on unmanned shelf by target detection technique and image retrieval technologies
The functions such as picking, the return of goods and businessman's stock, to save the manpower and material resources of unmanned shelf and traditional shelf, functionally user experience compared with
It is good.
As shown in Figure 1, this method includes the following steps, namely S102 to step S108:
Step S102 obtains the commodity image in collected unmanned shelf on every layer；
Detect the image obtained, the available type of goods for including and quantity.Specifically, picture is sent into image procossing
Model, and the quantity of the type of goods and each classification in statistical picture.
Step S104 cuts the commodity image and inputs the object category detection model that training obtains in advance；
Cut captured image, due to consideration that unmanned shelf can put being limited in scope for commodity, will in addition to shelf its
He cuts in region.Because since the mirror image of cabinet door glass, shelf external goods are for example, customer itself has phase inside shelf
With when commodity etc. situation and counter inside the disturbing factors such as other information, can have the case where misrecognition.
Step S106 exports position of each commodity in the commodity image according to the object category detection model
It sets and size；
The image that previous step was cut is as the input of object category detection model.Category carries out object detection mould
Type.
Specifically, merchandise classification can refer to bottled, canned, box-packed, packed etc..In view of increase every time new commodity or
Supplier replace the reasons such as commodity packaging will re -training model, and category attribute is relatively fixed, is not susceptible to change, and passes through
Above-mentioned steps only need to find position and size of each commodity in image.
It should be noted that bottled, canned, box-packed, packed specific category is not defined in this application, ability
Field technique personnel can select different classifications according to actual scene and determine the position in commodity image and size.
Step S108 is retrieved in commodity image java standard library and is exported type of merchandize and each classification in the commodity image
Quantity.
It is further comprised before being retrieved in commodity image java standard library according to the position of the commodity obtained in above-mentioned steps and big
Small information is cut out from the original image for obtaining the commodity image in collected unmanned shelf on every layer to obtain the figure of commodity
Picture.
Specifically, the image of each commodity is sent into image encrypting algorithm, the result of image encrypting algorithm output constitutes quotient
Product graphics standard library.By commodity image java standard library, businessman only needs to safeguard quotient when the packaging or type of commodity change
Sample data in product java standard library does not need re -training model.
Then further include the quantity for exporting type of merchandize and each classification in picture, pass through type of merchandize and each classification
Quantity needs be compared with the weighing results on unmanned shelf.
It can be seen from the above description that the application realizes following technical effect:
In the embodiment of the present application, it by the way of obtaining the commodity image in collected unmanned shelf on every layer, cuts out
It cuts the commodity image and inputs the object category detection model that training obtains in advance.It is defeated by the object category detection model
Position and size of each commodity in the commodity image out, has reached and has retrieved and export in commodity image java standard library
The purpose of the quantity of type of merchandize and each classification in the commodity image, to realize combining target detection and image retrieval
Realize the technical effect of unmanned shelf automatic vending, and then it is poor to solve the commodity detection method effect on unmanned shelf
Technical problem.Feel in addition, also the user experience is improved, reduce businessman's O&M cost.
According to the embodiment of the present application, as preferred in the present embodiment, as shown in Fig. 2, cutting the commodity image and defeated
Entering the object category detection model that training obtains in advance includes:
Step S202 crops the region in addition to shelf ontology；
Need to remove extraneous interference or noise in training object category detection model.
Step S204, determines object category；
Object category includes at least: any one or more in bottled, canned, box-packed or packed, determines that object belongs to
Which kind of classification.
Step S206 detects the commodity for whether having determining classification in the commodity image；
The commodity that detection judges whether there is determining classification are carried out in commodity image to occur.
Step S208 exports the position of the type commodity if detecting the commodity for having determining classification in the commodity image
It sets and size,
It since category attribute is relatively fixed, is not susceptible to change, only needs to find each commodity through the above steps and scheming
As inner position and size.For example, the position of the position A and Boxed beverage B of bottled drink.
It should be noted that step shown in the flowchart of the accompanying drawings can be in such as a group of computer-executable instructions
It is executed in computer system, although also, logical order is shown in flow charts, and it in some cases, can be with not
The sequence being same as herein executes shown or described step.
According to the embodiment of the present application, additionally provide it is a kind of for implementing the retail terminal of above-mentioned commodity detection method, such as
Shown in Fig. 3, which includes: the unmanned shelf using above-mentioned commodity detection method, and the unmanned shelf 100 also wrap
It includes:
Picture pick-up device 10, the top for being mounted on every layer of shelf are used to acquire the commodity image in unmanned shelf on every layer；
Sensing apparatus 20 is installed on every layer of shelf for detecting the commodity weight in unmanned shelf on every layer；And processing equipment
30, pass through network for the data transmission on the picture pick-up device 10 and sensing apparatus 20 to server；Wherein, the unmanned goods
At the every closing of the cabinet door of frame one time, starts the picture pick-up device execution Image Acquisition operation and the sensing apparatus is executed to weigh and be grasped
Make.
Specifically, terminal cabinet door is closed every time, and retail terminal is automatically locked.And it is taken pictures once by picture pick-up device 10,
And it is weighed by sensing apparatus 20.Captured photo is as shown in Figure 4.It takes pictures due to only closing Men Shicai starting camera, so goods
Cabinet Internal camera head without the moment it is in the open state can save power consumption and camera fever bring other influences, and
And also ensure the safety of commodity.
It should be noted that not to the specific position of picture pick-up device 10 and sensing apparatus 20 in the retail terminal of the application
It sets and is defined, for example, a fish-eye camera, bottom installation one can be installed with every layer of retail terminal inside shelf top center
A gravity sensor.
It is also to be noted that the data connection obtained on retail terminal is to internal processor 30, internal processor 30
Pass through network connection background server.Above-mentioned commodity detection method can be used as computer object detection relevant software programs into
Commodity target detection is executed in background server after row encapsulation.
According to the embodiment of the present application, as preferred in the present embodiment, as shown in figure 4, the processing equipment 30 includes:
One authority module 301 operates identity authority to the first of the unmanned shelf for receiving；Module of selling goods 302, for according to institute
State reduction classification and quantity that the first operation identity authority judges commodity on the unmanned shelf；And order confirmation module 303,
For generating purchase order according to the unit price and quantity of commodity.
It is received in first authority module 301 of the embodiment of the present application and operates identity authority to the first of the unmanned shelf
Refer to the permission of selling goods of unmanned shelf, that is, the operation identity authority of the user to open the door is commodity purchaser.
For example, the two dimensional code on the display screen of retail terminal can be scanned, and record the subscriber identity information of door opener with
Permission, and receive the picking operational order that user is inputted by display screen.
According to the first operation identity authority to commodity on unmanned shelf in the module 302 of selling goods of the embodiment of the present application
Operation judges go out the reduction classification and quantity of commodity on the unmanned shelf.
For example, commodity purchasing user has taken out 2 bottled drinks from shelf, judged according to the permission of commodity purchasing user
The quantity of the reduction classification of commodity and corresponding classification reduction on the unmanned shelf.
It is fixed according to the unit price to commodity stored in background data base in the order confirmation module 303 of the embodiment of the present application
Valence and commodity stocks quantity generate purchase order.
According to the embodiment of the present application, as preferred in the present embodiment, as shown in figure 5, the processing equipment 30 includes:
Two authority modules 304 operate identity authority to the second of the unmanned shelf for receiving；Return of goods module 305, for according to institute
State increase classification and quantity that the second operation identity authority judges commodity on the unmanned shelf；And reimbursement confirmation module 306,
For confirming reimbursement according to the unit price and quantity of commodity.
Receive in second authority module 304 of the embodiment of the present application is to the first operation identity authority of the unmanned shelf
Refer to the return of goods permission of unmanned shelf, that is, the operation identity authority of the user to open the door is merchandise return person.
For example, the two dimensional code on the display screen of retail terminal can be scanned, and record the subscriber identity information of door opener with
Permission, and receive the return of goods that user is inputted by display screen and (put back to) operational order.
According to the second operation identity authority to commodity on unmanned shelf in the return of goods module 305 of the embodiment of the present application
Operation judges go out the increase classification and quantity of commodity on the unmanned shelf.
For example, commodity purchasing user has put back to 2 box-packed Yoghourts from shelf, judged according to the permission of commodity purchasing user
The increase classification of commodity and the corresponding increased quantity of classification on the unmanned shelf.
It is fixed a price according to what is stored in background data base to the unit price of commodity in the reimbursement confirmation module 306 of the embodiment of the present application
Reimbursement is confirmed with commodity stocks quantity.
According to the embodiment of the present application, as preferred in the present embodiment, as shown in fig. 6, the processing equipment includes: third
Authority module 307, for receiving the third operation identity authority to the unmanned shelf；Stock module 308, for according to
Third operation identity authority judges the variation classification and quantity of commodity on the unmanned shelf；And stock checklist module 309, it uses
According to the type and quantity of commodity generation bill of goods.
The third operation identity authority to the unmanned shelf is received in the third authority module 307 of the embodiment of the present application is
Refer to that the permission of selling goods of unmanned shelf, that is, the operation identity authority of the user to open the door are the commodity persons of getting in stocks.
For example, the two dimensional code on the display screen of retail terminal can be scanned, and record the subscriber identity information of door opener with
Permission, and receive the operational order of getting in stocks that user is inputted by display screen.
According to the first operation identity authority to commodity on unmanned shelf in the stock module 308 of the embodiment of the present application
Operation judges go out the variation classification and quantity of commodity on the unmanned shelf.
For example, commodity are got in stocks, personnel have taken out 2 bottled expired beverages from shelf, have been put into 5 bottles of Yoghourts, have been purchased according to commodity
The permission for buying user judges the quantity of the variation classification of commodity and corresponding classification reduction on the unmanned shelf.
Life is generated according to the type and quantity stored in background data base in the stock checklist module 309 of the embodiment of the present application
At bill of goods.
According to the embodiment of the present application, as preferred in the present embodiment, as shown in fig. 7, retail terminal further include: also wrap
Include: display equipment 40 is mounted on outside the cabinet door of the unmanned shelf and is used for display of commodity information, wherein the merchandise news
It include: commodity amount, cargo price, commodity total price, any one or more in merchandise classification.
The operational order for being also used to that user is supported to input in the display equipment 40 of the embodiment of the present application, for example, three permissions:
It sells goods, return goods and gets ready the goods.By two-dimensional code display on the display apparatus 40, recorded on retail terminal after scanning the two dimensional code
The information and permission of door opener.Picking can be selected on the screen according to different permission users, return goods or exit.And businessman uses
Family can choose stock.
It should be noted that step shown in the flowchart of the accompanying drawings can be in such as a group of computer-executable instructions
It is executed in computer system, although also, logical order is shown in flow charts, and it in some cases, can be with not
The sequence being same as herein executes shown or described step.
According to the embodiment of the present application, as shown in figure 8, additionally providing a kind of commodity detection device for unmanned shelf, wrap
It includes: module 1001 is obtained, for obtaining the commodity image in collected unmanned shelf on every layer；Module 1002 is cut, is used for
It cuts the commodity image and inputs the object category detection model that training obtains in advance；Output module 1003, for according to institute
It states object category detection model and exports position and size of each commodity in the commodity image；And retrieval module
1004, for the quantity of type of merchandize and each classification in the commodity image to be retrieved and exported in commodity image java standard library.
According to the embodiment of the present application, as preferred in the present embodiment, as shown in figure 9, the cutting module includes: to cut
Unit 10021, for cropping the region in addition to shelf ontology；Determination unit 10022, for determining object category；Detection
Unit 10023, for detecting the commodity for whether having determining classification in the commodity image；And output unit 10024, for examining
When surveying the commodity for having determining classification in the commodity image, position and the size of the type commodity are exported.
Need to remove the external world when cutting in unit 10021 in training object category detection model in the embodiment of the present application
Interference or noise.
Object category includes at least in the determination unit 10022 of the embodiment of the present application: bottled, canned, box-packed or packed
In any one or more, determine which kind of classification object belongs to.
Detection, which is carried out, in commodity image in the detection unit 10023 of the embodiment of the present application judges whether there is determining classification
Commodity occur.
Since category attribute is relatively fixed in the output unit 10024 of the embodiment of the present application, it is not susceptible to change, passes through
Above-mentioned steps only need to find position and size of each commodity in image.For example, the position A and Boxed beverage of bottled drink
The position of B.
Specifically, using the image cut as the input of object category detection model.Category carries out object detection mould
Type.
Specifically, classification can refer to bottled, canned, box-packed, packed etc..In view of increasing new commodity or supply every time
Quotient replace the reasons such as commodity packaging will re -training model, and category attribute is relatively fixed, is not susceptible to change, by above-mentioned
Step only needs to find position and size of each commodity in image.
It should be noted that bottled, canned, box-packed, packed specific category is not defined in this application, ability
Field technique personnel can select different classifications according to actual scene and determine the position in commodity image and size.
According to the embodiment of the present application, as preferred in the present embodiment, as shown in Figure 10, the retrieval module includes: sample
This unit 10041, for the classification of commodity to be established java standard library as image pattern；Judging unit 10042, to judge commodity
Packaging or type whether update；And replacement unit 10043 replaces when packaging or type commodity for judging commodity update
Change described image sample.
The classification of commodity is established into java standard library as image pattern in the sample unit 10041 of the embodiment of the present application.Tool
Body, merchandise classification can refer to it is bottled, it is canned, it is box-packed, it is packed etc..It is established by image pattern and contains bottled, canned, box
Dress, the merchandise news of the classifications such as packed.
The packaging or type of commodity are judged whether there is in background data base in the judging unit 10042 of the embodiment of the present application
Update.
When judging that the packaging of commodity or type commodity update in the replacement unit 10043 of the embodiment of the present application, institute is replaced
State image pattern.
By commodity image java standard library, businessman only needs to safeguard commercial standards when the packaging or type of commodity change
Sample data in library does not need re -training model.
Figure 11 to Figure 12 is please referred to, by taking refrigerator as an example, the real-time mode of the application is described in detail.The application is logical
Target detection technique and image retrieval technologies are crossed, the functions such as customer's picking, the return of goods and businessman's stock are realized, to save unmanned shelf
It is functionally relatively friendly with the manpower and material resources of traditional shelf.
Preparation stage: refrigerator, multiple fish-eye cameras, multiple gravity sensors, microprocessor and electronical display
Screen；Internal every layer of shelf top center installs the fish-eye camera with wide angle shot effect, bottom peace in refrigerator
Fill a gravity sensor；Outside installation display screen.Three permissions of counter: picking, the return of goods and stock are set, for example, customer
There are picking, return of goods permission, stock personnel have stock permission, and record the information and permission of door opener.
The commodity purchasing stage: after having configured counter, the two dimensional code on display screen is scanned.Customer users can be on the screen
Selection picking is returned goods or is exited, while businessman user selects stock.In each closing refrigerator door, it is automatically locked and carries out
It takes pictures primary and weighs.
The target detection stage: by data connection to processor, processor passes through network connection background server.It is taken on backstage
The image obtained in business device, and obtain the type of goods for including and quantity.And according to the type of goods and quantity of acquisition, comparison is answered
The actual weight for having weight and being obtained by gravity sensor.
It is specifically included in the above-mentioned target detection stage: detecting the image of acquisition, and obtain the type of goods for including and quantity.
Picture is sent into image processing model.The quantity of type of goods and each classification in statistical picture.
Specifically, cut captured image, shelf can put being limited in scope for commodity, will in addition to shelf other regions into
Row is cut, because since the mirror image of cabinet door glass, shelf external goods are for example, customer users itself has identical quotient inside shelf
The disturbing factors such as other information inside product and counter can have the case where misrecognition.
Secondly, the image cut, which is sent into category, carries out object detection model, merchandise classification can be bottled, tank
Dress, it is box-packed, it is packed etc., it will re -training mould due to increasing new commodity or supplier every time and replacing commodity packaging etc.
Type, and category attribute is relatively fixed, is not susceptible to change, this step only need to find position of each commodity in image and big
It is small.
Finally, being cut from original image according to the position of previous step commodity and size information.Then, by each quotient
The image of product is sent into image encrypting algorithm, and the advantage of image encrypting algorithm is that businessman user only needs the packaging or kind in commodity
The sample data in backstage commercial standards library is safeguarded when class changes, and does not need re -training model；It exports in picture
Type of goods and each classification quantity.
There should be weight according to the type of goods of acquisition and quantity and calculating, comparison there should be weight and obtain by gravity sensor
Take actual weight.The record that comparing result and last time close the door.
For normal condition, for customer users, type of merchandize, the list of quantity variation are shown on the electronic display screen
Valence, quantity and total price.For businessman user, the type of merchandize and quantity of quantity variation are shown on the electronic display screen.
Abnormal conditions refer to: if actual weight and should have weight to have error, illustrating there are incomplete commodity, give on screen
Related backstage personnel prompt.
The Error processing stage: the record of the result of comparison and last time shutdown saves result and simultaneously exports the quotient for having quantity to change
Kind class, unit price, quantity and total price.User is then given if any weight error accordingly to prompt；Customer users confirm and pay or move back
The corresponding amount of money of money, stock personnel confirm merchandise news.And relevant information is automatically recorded in background server.
Obviously, those skilled in the art should be understood that each module of above-mentioned the application or each step can be with general
Computing device realize that they can be concentrated on a single computing device, or be distributed in multiple computing devices and formed
Network on, optionally, they can be realized with the program code that computing device can perform, it is thus possible to which they are stored
Be performed by computing device in the storage device, perhaps they are fabricated to each integrated circuit modules or by they
In multiple modules or step be fabricated to single integrated circuit module to realize.In this way, the application be not limited to it is any specific
Hardware and software combines.
The foregoing is merely preferred embodiment of the present application, are not intended to limit this application, for the skill of this field
For art personnel, various changes and changes are possible in this application.Within the spirit and principles of this application, made any to repair
Change, equivalent replacement, improvement etc., should be included within the scope of protection of this application.
Claims (10)
1. a kind of commodity detection method for unmanned shelf characterized by comprising
Obtain the commodity image in collected unmanned shelf on every layer；
It cuts the commodity image and inputs the object category detection model that training obtains in advance；
Position and size of each commodity in the commodity image are exported according to the object category detection model；And
The quantity of type of merchandize and each classification in the commodity image is retrieved and exported in commodity image java standard library.
2. commodity detection method according to claim 1, which is characterized in that cut the commodity image and input instruction in advance
The object category detection model got includes:
Crop the region in addition to shelf ontology；
Determine object category；
The commodity for whether having determining classification in the commodity image detected；And
If detecting the commodity for having determining classification in the commodity image, position and the size of the type commodity are exported,
Wherein, object category includes at least: any one or more in bottled, canned, box-packed or packed.
3. a kind of retail terminal characterized by comprising using the unmanned goods of commodity detection method as described in claim 1
Frame, the unmanned shelf further include:
Picture pick-up device, the top for being mounted on every layer of shelf are used to acquire the commodity image in unmanned shelf on every layer；
Sensing apparatus is installed on every layer of shelf for detecting the commodity weight in unmanned shelf on every layer；And
Processing equipment passes through network for the data transmission on the picture pick-up device and sensing apparatus to server；
Wherein,
At the every closing of the cabinet door of the unmanned shelf one time, starts the picture pick-up device and execute Image Acquisition operation and the induction
Equipment executes weighing operation.
4. the retail terminal stated according to claim 3, which is characterized in that the processing equipment includes:
First authority module operates identity authority to the first of the unmanned shelf for receiving；
It sells goods module, for judging the reduction classification sum number of commodity on the unmanned shelf according to the first operation identity authority
Amount；And
Order confirmation module, for generating purchase order according to the unit price and quantity of commodity.
5. the retail terminal stated according to claim 3, which is characterized in that the processing equipment includes:
Second authority module operates identity authority to the second of the unmanned shelf for receiving；
Return of goods module, for judging the increase classification sum number of commodity on the unmanned shelf according to the second operation identity authority
Amount；And
Reimbursement confirmation module, for confirming reimbursement according to the unit price and quantity of commodity.
6. the retail terminal stated according to claim 3, which is characterized in that the processing equipment includes:
Third authority module, for receiving the third operation identity authority to the unmanned shelf；
Stock module, for operating the variation classification sum number that identity authority judges commodity on the unmanned shelf according to the third
Amount；And
Stock checklist module, for generating bill of goods according to the type and quantity of commodity.
7. the retail terminal stated according to claim 3, which is characterized in that further include:
It shows equipment, is mounted on outside the cabinet door of the unmanned shelf and is used for display of commodity information,
Wherein,
The merchandise news includes: commodity amount, cargo price, commodity total price, any one or more in merchandise classification.
8. a kind of commodity detection device for unmanned shelf characterized by comprising
Module is obtained, for obtaining the commodity image in collected unmanned shelf on every layer；
Module is cut, for cutting the commodity image and inputting the object category detection model that training obtains in advance；
Output module, for exporting position of each commodity in the commodity image according to the object category detection model
It sets and size；And
Retrieval module, for type of merchandize and each classification in the commodity image to be retrieved and exported in commodity image java standard library
Quantity.
9. commodity detection device according to claim 8, which is characterized in that the cutting module includes:
Unit is cut, for cropping the region in addition to shelf ontology；
Determination unit, for determining object category；
Detection unit, for detecting the commodity for whether having determining classification in the commodity image；And
Output unit, when for detecting the commodity for having determining classification in the commodity image, export the type commodity position and
Size.
10. commodity detection device according to claim 8, which is characterized in that the retrieval module includes:
Sample unit, for the classification of commodity to be established java standard library as image pattern；
Whether judging unit, packaging or type to judge commodity update；And
Replacement unit replaces described image sample when packaging or type commodity for judging commodity update.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811221070.4A CN109543527A (en) | 2018-10-19 | 2018-10-19 | For the commodity detection method of unmanned shelf, device and retail terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811221070.4A CN109543527A (en) | 2018-10-19 | 2018-10-19 | For the commodity detection method of unmanned shelf, device and retail terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109543527A true CN109543527A (en) | 2019-03-29 |
Family
ID=65844305
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811221070.4A Pending CN109543527A (en) | 2018-10-19 | 2018-10-19 | For the commodity detection method of unmanned shelf, device and retail terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109543527A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110110652A (en) * | 2019-05-05 | 2019-08-09 | 达闼科技（北京）有限公司 | A kind of object detection method, electronic equipment and storage medium |
| CN110287203A (en) * | 2019-05-24 | 2019-09-27 | 北京百度网讯科技有限公司 | For the update method of vending machine, updating device and vending machine |
| CN110321797A (en) * | 2019-05-31 | 2019-10-11 | 苏宁云计算有限公司 | Commodity recognition method and device |
| CN110543839A (en) * | 2019-08-20 | 2019-12-06 | 南京掌控网络科技有限公司 | commodity goods laying rate acquisition method based on computer vision |
| CN110705371A (en) * | 2019-09-09 | 2020-01-17 | 上海零眸智能科技有限公司 | Refrigerator structure and commodity arrangement detection method thereof |
| CN110929668A (en) * | 2019-11-29 | 2020-03-27 | 珠海大横琴科技发展有限公司 | Commodity detection method and device based on unmanned goods shelf |
| CN110987140A (en) * | 2019-11-28 | 2020-04-10 | 浙江由由科技有限公司 | Commodity weighing auxiliary equipment and weighing equipment |
| CN111191551A (en) * | 2019-12-23 | 2020-05-22 | 深圳前海达闼云端智能科技有限公司 | Commodity detection method and device |
| CN111291834A (en) * | 2020-03-27 | 2020-06-16 | 华士磐典科技(上海)有限公司 | Method for rapidly generating shelf digital display chart |
| CN111507253A (en) * | 2020-04-16 | 2020-08-07 | 腾讯科技（深圳）有限公司 | Method and device for auditing displayed articles based on artificial intelligence |
| CN111881894A (en) * | 2019-12-09 | 2020-11-03 | 上海扩博智能技术有限公司 | Method, system, equipment and storage medium for collecting goods selling information of container |
| CN111882606A (en) * | 2020-07-01 | 2020-11-03 | 上海品览数据科技有限公司 | Goods shelf commodity layering method based on deep learning |
| CN112270664A (en) * | 2020-04-16 | 2021-01-26 | 上海扩博智能技术有限公司 | Transparent packaging product detection method, system, equipment and storage medium |
| CN112446437A (en) * | 2020-12-11 | 2021-03-05 | 上海品览数据科技有限公司 | Goods shelf commodity specification identification method based on machine vision |
| CN113139768A (en) * | 2021-03-24 | 2021-07-20 | 广东便捷神科技股份有限公司 | Goods shortage monitoring method based on unmanned vending machine |
| WO2022199078A1 (en) * | 2021-03-26 | 2022-09-29 | 苏宁易购集团股份有限公司 | Method and apparatus for generating item dynamic information, device, and computer-readable medium |
| WO2023130450A1 (en) * | 2022-01-10 | 2023-07-13 | 烟台创迹软件有限公司 | Standard image acquisition method, standard image acquisition apparatus, and shelf allocation estimation method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106570510A (en) * | 2016-11-07 | 2017-04-19 | 南京航空航天大学 | Supermarket commodity identification method |
| JP2017117281A (en) * | 2015-12-25 | 2017-06-29 | ザ コカ・コーラ カンパニーThe Coca‐Cola Company | Vending machine and mobile terminal |
| CN107730168A (en) * | 2017-09-28 | 2018-02-23 | 中南大学 | A kind of automatic vending machine automated stock control system and method based on image recognition |
| CN108320379A (en) * | 2018-02-28 | 2018-07-24 | 成都果小美网络科技有限公司 | Good selling method, device and the self-service machine compared based on image |
| CN108416901A (en) * | 2018-03-27 | 2018-08-17 | 合肥美的智能科技有限公司 | Method and device for identifying goods in intelligent container and intelligent container |
-
2018
- 2018-10-19 CN CN201811221070.4A patent/CN109543527A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2017117281A (en) * | 2015-12-25 | 2017-06-29 | ザ コカ・コーラ カンパニーThe Coca‐Cola Company | Vending machine and mobile terminal |
| CN106570510A (en) * | 2016-11-07 | 2017-04-19 | 南京航空航天大学 | Supermarket commodity identification method |
| CN107730168A (en) * | 2017-09-28 | 2018-02-23 | 中南大学 | A kind of automatic vending machine automated stock control system and method based on image recognition |
| CN108320379A (en) * | 2018-02-28 | 2018-07-24 | 成都果小美网络科技有限公司 | Good selling method, device and the self-service machine compared based on image |
| CN108416901A (en) * | 2018-03-27 | 2018-08-17 | 合肥美的智能科技有限公司 | Method and device for identifying goods in intelligent container and intelligent container |
Non-Patent Citations (1)
| Title |
|---|
| 刘永豪: "基于深度学习的货架商品检测技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110110652A (en) * | 2019-05-05 | 2019-08-09 | 达闼科技（北京）有限公司 | A kind of object detection method, electronic equipment and storage medium |
| CN110287203A (en) * | 2019-05-24 | 2019-09-27 | 北京百度网讯科技有限公司 | For the update method of vending machine, updating device and vending machine |
| CN110287203B (en) * | 2019-05-24 | 2021-03-26 | 北京百度网讯科技有限公司 | Updating method and updating device for vending machine and vending machine |
| CN110321797A (en) * | 2019-05-31 | 2019-10-11 | 苏宁云计算有限公司 | Commodity recognition method and device |
| CN110543839A (en) * | 2019-08-20 | 2019-12-06 | 南京掌控网络科技有限公司 | commodity goods laying rate acquisition method based on computer vision |
| CN110705371A (en) * | 2019-09-09 | 2020-01-17 | 上海零眸智能科技有限公司 | Refrigerator structure and commodity arrangement detection method thereof |
| CN110705371B (en) * | 2019-09-09 | 2023-06-06 | 上海零眸智能科技有限公司 | Refrigerator structure and commodity surface arrangement detection method thereof |
| CN110987140A (en) * | 2019-11-28 | 2020-04-10 | 浙江由由科技有限公司 | Commodity weighing auxiliary equipment and weighing equipment |
| CN110929668A (en) * | 2019-11-29 | 2020-03-27 | 珠海大横琴科技发展有限公司 | Commodity detection method and device based on unmanned goods shelf |
| CN111881894A (en) * | 2019-12-09 | 2020-11-03 | 上海扩博智能技术有限公司 | Method, system, equipment and storage medium for collecting goods selling information of container |
| CN111191551A (en) * | 2019-12-23 | 2020-05-22 | 深圳前海达闼云端智能科技有限公司 | Commodity detection method and device |
| CN111291834A (en) * | 2020-03-27 | 2020-06-16 | 华士磐典科技(上海)有限公司 | Method for rapidly generating shelf digital display chart |
| CN111291834B (en) * | 2020-03-27 | 2022-06-10 | 华士磐典科技(上海)有限公司 | Method for rapidly generating shelf digital display chart |
| CN111507253A (en) * | 2020-04-16 | 2020-08-07 | 腾讯科技（深圳）有限公司 | Method and device for auditing displayed articles based on artificial intelligence |
| CN112270664A (en) * | 2020-04-16 | 2021-01-26 | 上海扩博智能技术有限公司 | Transparent packaging product detection method, system, equipment and storage medium |
| CN111882606A (en) * | 2020-07-01 | 2020-11-03 | 上海品览数据科技有限公司 | Goods shelf commodity layering method based on deep learning |
| CN112446437A (en) * | 2020-12-11 | 2021-03-05 | 上海品览数据科技有限公司 | Goods shelf commodity specification identification method based on machine vision |
| CN113139768A (en) * | 2021-03-24 | 2021-07-20 | 广东便捷神科技股份有限公司 | Goods shortage monitoring method based on unmanned vending machine |
| CN113139768B (en) * | 2021-03-24 | 2023-01-13 | 广东便捷神科技股份有限公司 | Goods shortage monitoring method based on unmanned vending machine |
| WO2022199078A1 (en) * | 2021-03-26 | 2022-09-29 | 苏宁易购集团股份有限公司 | Method and apparatus for generating item dynamic information, device, and computer-readable medium |
| WO2023130450A1 (en) * | 2022-01-10 | 2023-07-13 | 烟台创迹软件有限公司 | Standard image acquisition method, standard image acquisition apparatus, and shelf allocation estimation method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109543527A (en) | For the commodity detection method of unmanned shelf, device and retail terminal | |
| CN108875664B (en) | Method and device for identifying purchased goods and vending machine | |
| US10679174B2 (en) | Multifunctional self-service shipping and mail processing system | |
| CN109214751B (en) | Intelligent inventory management system based on inventory position change | |
| CN207783158U (en) | Object positioning system | |
| CN207752560U (en) | Unmanned retail cabinet with monitoring function | |
| CN105049528B (en) | Safe and controllable smart network system based on self service and constructing method | |
| CN102973116B (en) | Intelligent mailbox system and control method | |
| CN107374177A (en) | It is a kind of have sell goods, share and the Multifunctional demonstration system and method for transfer | |
| CN110717769B (en) | Goods returning method, device and equipment for intelligent cabinet articles and storage medium | |
| CN109166000A (en) | A kind of Intelligent cargo cabinet and the method for creating order | |
| JP7264401B2 (en) | Accounting methods, devices and systems | |
| CN110197240A (en) | The control method of unmanned Weighing type sales counter based on image recognition | |
| WO2019165895A1 (en) | Automatic vending method and system, and vending device and vending machine | |
| CN109377657A (en) | A kind of Vending Machine and its operation method | |
| JP2023038200A (en) | Sales device, sales system and sales method | |
| CN114387735A (en) | Method, device and system for picking up goods | |
| CN108171286B (en) | Unmanned selling method and system | |
| US11651329B2 (en) | Machine readable technologies for the smart shipping of multiple products | |
| CN110675098A (en) | Control method and information processing apparatus | |
| CN111951486A (en) | Alarm method and device for distribution cabinet, storage medium and electronic device | |
| WO2014196130A1 (en) | Information processing system for identifying used commodities in domestic electrical appliances, and security system | |
| CN208798940U (en) | It is a kind of with selling goods, share and the showcase of middle rotating function | |
| CN209103410U (en) | Vending equipment | |
| US20190156317A1 (en) | Container, shopping assistance system for use with same, shopping assistance method for use with same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |