CN112395918B - Goods shelf identification method, device and system - Google Patents
Goods shelf identification method, device and system Download PDFInfo
- Publication number
- CN112395918B CN112395918B CN201910756178.1A CN201910756178A CN112395918B CN 112395918 B CN112395918 B CN 112395918B CN 201910756178 A CN201910756178 A CN 201910756178A CN 112395918 B CN112395918 B CN 112395918B
- Authority
- CN
- China
- Prior art keywords
- shelf
- electronic price
- price tag
- image
- goods shelf
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000013136 deep learning model Methods 0.000 claims description 33
- 238000001514 detection method Methods 0.000 claims description 28
- 238000010586 diagram Methods 0.000 claims description 27
- 239000013598 vector Substances 0.000 claims description 23
- 238000012549 training Methods 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 13
- 238000012937 correction Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 description 11
- 230000006870 function Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 230000004927 fusion Effects 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 2
- 230000004397 blinking Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a goods shelf identification method, a device and a system, wherein the method comprises the following steps: obtaining an image of a shelf; detecting an electronic price tag on a shelf from an image of the shelf; identifying the identification on the electronic price tag; and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identification on the electronic price tag and the article. The invention can accurately identify goods on shelves, and has high efficiency and strong expandability.
Description
Technical Field
The present invention relates to the field of the internet, and in particular, to a method, apparatus and system for identifying shelf items.
Background
Identification of shelf items (e.g., merchandise) has been a significant concern in commercial communities. The traditional article identification thought is marked by collecting article pictures, and is given to a deep learning algorithm for training to generate a model, and the articles are identified through the model. However, the method has two unsolvable problems, firstly, the quantity of articles is very large, and the training data of massive articles which need to be acquired are used for training, so that the efficiency is low. And secondly, the types of the articles are continuously increased and updated, and training data are required to be continuously updated for identifying new articles, so that the expandability is poor. These two challenges pose a significant hurdle to the wide range of applications for item identification.
Disclosure of Invention
The embodiment of the invention provides a goods shelf object identification method which is used for identifying goods shelf objects and has high efficiency and strong expandability, and the method comprises the following steps:
obtaining an image of a shelf;
detecting an electronic price tag on a shelf from an image of the shelf;
identifying the identification on the electronic price tag;
and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identification on the electronic price tag and the article.
The embodiment of the invention provides a goods shelf object identification device which is used for identifying goods shelf objects and has high efficiency and strong expandability, and the device comprises:
obtaining an image of a shelf;
detecting an electronic price tag on a shelf from an image of the shelf;
identifying the identification on the electronic price tag;
and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identification on the electronic price tag and the article.
The embodiment of the invention provides a goods shelf object identification system which is used for identifying goods shelf objects and has high efficiency and strong expandability, and the system comprises: the above-mentioned goods identification device, camera, wherein,
the camera is used for shooting images of the goods shelf and sending the images of the goods shelf to the goods shelf identification device.
The embodiment of the invention also provides computer equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the goods shelf object identification method when executing the computer program.
The embodiment of the invention also provides a computer readable storage medium, which stores a computer program for executing the goods shelf identification method.
In the embodiment of the invention, an image of a goods shelf is obtained; detecting an electronic price tag on a shelf from an image of the shelf; identifying the identification on the electronic price tag; and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identification on the electronic price tag and the article. In the process, only the electronic price tag on the goods shelf is detected according to the images of the goods shelf, each article is not required to be detected, the detection of the electronic price tag is small in detection workload compared with the detection of the articles, the efficiency is high, when the types of the articles are increased, only the electronic price tag is required to be identified again, a large number of goods shelf images are not required to be used for identifying new articles, therefore, the identification efficiency is high, and the expandability is high.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. In the drawings:
FIG. 1 is a flowchart of a method for identifying a shelf object according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a shelf image in an embodiment of the invention;
FIG. 3 is a schematic diagram of electronic price tag detection in an embodiment of the present invention;
FIG. 4 is a schematic diagram of a shelf object according to an embodiment of the present invention;
FIG. 5 is a detailed flowchart of a method for identifying a shelf object according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a shelf object identifying apparatus according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a shelf object identification system according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in further detail with reference to the accompanying drawings. The exemplary embodiments of the present invention and their descriptions herein are for the purpose of explaining the present invention, but are not to be construed as limiting the invention.
In the description of the present specification, the terms "comprising," "including," "having," "containing," and the like are open-ended terms, meaning including, but not limited to. Reference to the terms "one embodiment," "a particular embodiment," "some embodiments," "for example," etc., means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. The sequence of steps involved in the embodiments is used to schematically illustrate the practice of the present application, and is not limited thereto and may be appropriately adjusted as desired.
Fig. 1 is a flowchart of a method for identifying a shelf object according to an embodiment of the present invention, as shown in fig. 1, the method includes:
step 101, obtaining an image of a goods shelf;
102, detecting an electronic price tag on a shelf from an image of the shelf;
step 103, identifying the identification on the electronic price tag;
step 104, determining the corresponding article of the electronic price tag by inquiring the corresponding relation between the identification on the electronic price tag and the article.
In the embodiment of the invention, an image of a goods shelf is obtained; detecting an electronic price tag on a shelf from an image of the shelf; identifying the identification on the electronic price tag; and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identification on the electronic price tag and the article. In the process, only the electronic price tag on the goods shelf is detected according to the images of the goods shelf, each article is not required to be detected, the detection of the electronic price tag is small in detection workload compared with the detection of the articles, the efficiency is high, when the types of the articles are increased, only the electronic price tag is required to be identified again, a large number of goods shelf images are not required to be used for identifying new articles, therefore, the identification efficiency is high, and the expandability is high.
In specific implementation, in step 101, a plurality of devices such as a camera, a video camera, a high-speed camera, etc. may be used to capture an image of a shelf, and obtain the image of the shelf, and fig. 2 is a schematic diagram of an image of the shelf in an embodiment of the present invention, where the image includes an article and an electronic price tag, and each article has a corresponding electronic price tag. In step 102, an electronic price tag is searched from an image corresponding to fig. 2, fig. 3 is a schematic diagram of electronic price tag detection in the embodiment of the present invention, and the electronic price tag can be detected, including information such as a position of the electronic price tag, in the step, the electronic price tag is detected from the image instead of an article, and compared with the article, the detection process of the electronic price tag with a fixed shape and position has smaller workload, high efficiency and higher accuracy; in step 103, the identification of the electronic price tag can be obtained from the database of the electronic price tag when the identification of the electronic price tag is identified, and in step 104, the corresponding article of the electronic price tag is determined by querying the corresponding relationship between the identification of the electronic price tag and the article, for example, the corresponding relationship between the electronic price tag and the SKU of the article can be queried from the database, so as to determine the article corresponding to the electronic price tag. When the types of the articles are increased, only the electronic price tags are required to be detected again, complicated images of the articles are not required to be identified from a large number of images, and the efficiency and the accuracy are higher.
In practice, there are various methods for detecting electronic price tags on a shelf from an image of the shelf, and one example thereof is given below.
In one embodiment, detecting an electronic price tag on a shelf from an image of the shelf includes:
and inputting the image of the goods shelf into a deep learning model, and detecting the electronic price tag on the goods shelf.
In one embodiment, the deep learning model is trained by the following method:
obtaining a historical image of a goods shelf;
extracting a feature vector of the historical image;
and training a deep learning model by using the feature vector.
In specific implementation, the common deep learning model includes an automatic encoder, a sparse encoder, a convolutional neural network, RNN and LSTM, and the deep learning model training method is substantially similar, that is, the three steps in the above embodiment, and in addition, when the deep learning model is trained by using the feature vector, the method specifically includes: and adjusting parameters of the deep learning model in the training process until the loss function of the deep learning model meets the preset convergence condition, so as to obtain the trained deep learning model.
In another embodiment, the deep learning model is trained by the following method:
dividing historical images of the goods shelf into training set data and test set data;
extracting feature vectors of training set data;
training a deep learning model by using the feature vectors of the training set data,
adjusting parameters of the deep learning model in the training process until a loss function of the deep learning model meets a preset convergence condition, and obtaining a trained deep learning model;
testing the trained deep learning model by using the test set data;
when the accuracy of the trained deep learning model is smaller than the preset accuracy, the quantity of the training set data is increased or the proportion of the training set data and the testing set data is adjusted to retrain the training until the accuracy of the trained deep learning model is not smaller than the preset accuracy.
The above embodiment describes a process of deep learning a model, that is, when the deep learning model is used to identify a history image of a shelf when an electronic price tag on the shelf is needed, and in specific application, there are various methods for training the deep learning model according to the history image of the shelf, and two embodiments are given below.
In one embodiment, after obtaining the historical image of the shelf, further comprising:
searching an image for displaying the identification of the electronic price tag from the historical image of the goods shelf;
extracting feature vectors of the historical image, including:
and extracting the characteristic vector from the image displaying the identification of the electronic price tag.
In the above embodiment, there are many images of electronic price tags in the historical image of the shelf, and the display of the electronic price tags is updated continuously along with time, at a certain moment, the identifications of the electronic price tags, such as the specific codes of the electronic price tags, can be displayed, the images showing the identifications of the electronic price tags are found, and the feature vectors are extracted for training the deep learning model.
In another embodiment, after obtaining the historical image of the shelf, further comprising:
obtaining a lighting sequence image of the electronic price tag from the historical image of the goods shelf;
extracting feature vectors of the historical image, including:
and extracting the characteristic vector from the lighting sequence image of the electronic price tag.
In the above embodiment, the electronic price tag periodically displays blinking of different colors, for example, white is used as an initial color, { white, red, blue, green } may be a lighting sequence of one electronic price tag, { white, green, blue, red } may be a lighting sequence of another electronic price tag, so that the electronic price tag may be distinguished by the lighting sequence, and therefore, a lighting sequence image of the electronic price tag may be obtained from a history image of a shelf, and feature vectors may be extracted to train the deep learning model.
Of course, it is understood that there may be other ways of training the deep learning model, and the related variations should fall within the protection scope of the present invention.
In an embodiment, the method for identifying a shelf object further comprises:
acquiring wireless signal frequency when heartbeat data of the electronic price tag are received by the base station;
according to the wireless signal frequency when the heartbeat data of the electronic price tag is received by the base station, determining a goods shelf where the electronic price tag is located;
if the determined goods shelf where the electronic price tag is located is not consistent with the image of the goods shelf where the electronic price tag is detected, repeating the following steps until the goods shelf where the electronic price tag is located is consistent with the image of the goods shelf where the electronic price tag is detected: obtaining a new image of the shelf; from the new image of the shelf, the electronic price tag on the shelf is detected.
In the above embodiment, the wireless signal frequencies when the heartbeat data of different electronic price tags are received by the base station are different, and the shelf where the electronic price tag is located may be determined from the correspondence between the wireless signal frequencies already stored and the electronic price tag, for example, the location of the shelf where the electronic price tag is located is determined, if the determined shelf where the electronic price tag is located is not consistent with the image of the shelf where the electronic price tag is detected, the following steps are repeatedly executed until the two match: obtaining a new image of the shelf; from the new image of the shelf, the electronic price tag on the shelf is detected. The process can achieve the effect of joint measurement through fusion of the deep learning model and the wireless signal frequency based on the base station, so that the accuracy of goods shelf identification is improved.
In one embodiment, the identification on the electronic price tag comprises: encoding on an electronic price tag.
Of course, it should be understood that the identification of the electronic price tag, such as the name, may be represented in other manners, and the related variations should fall within the protection scope of the present invention.
In one embodiment, detecting an electronic price tag on a shelf from an image of the shelf includes: detecting the position of the electronic price tag on the goods shelf from the image of the goods shelf;
the method further comprises the steps of:
generating a shelf diagram of goods shelf according to the detected position of the electronic price tag on the goods shelf;
and detecting the out-of-stock state of the goods shelf according to the grid diagram.
Fig. 4 is a schematic diagram of a shelf diagram of goods in an embodiment of the present invention, that is, each type of goods and the corresponding electronic price tag can be placed in one shelf, so that whether there is a good in each shelf can be easily detected.
In one embodiment, according to the grid chart, detecting the out-of-stock condition of the shelf includes:
for each grid in the grid graph, if the grid has no article, the grid is in a stock-out state; otherwise, the out-of-stock condition of the grid is not out-of-stock.
In the above embodiment, the article corresponding to the electronic price tag in each grid may be searched from the database or other data storage location, and if the grid has no article, the out-of-stock state of the grid is out-of-stock; otherwise, the out-of-stock condition of the grid is not out-of-stock.
Based on the above embodiments, the present invention proposes the following embodiment to describe the detailed flow of the method for identifying a shelf object, and fig. 5 is a detailed flow chart of the method for identifying a shelf object according to the embodiment of the present invention, as shown in fig. 5, in one embodiment, the detailed flow of the method for identifying a shelf object includes:
step 501, obtaining an image of a shelf;
step 502, inputting an image of the goods shelf into a deep learning model, and detecting an electronic price tag on the goods shelf;
detecting the position of an electronic price tag on a shelf from an image of the shelf;
step 503, identifying the identification on the electronic price tag;
step 504, determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identification on the electronic price tag and the article;
step 505, obtaining the wireless signal frequency when the heartbeat data of the electronic price tag is received by the base station;
step 506, determining the shelf where the electronic price tag is located according to the wireless signal frequency when the heartbeat data of the electronic price tag is received by the base station;
step 507, if the determined goods shelf where the electronic price tag is located is not consistent with the image of the goods shelf where the electronic price tag is detected, repeating the following steps until the two match: obtaining a new image of the shelf; detecting an electronic price tag on the goods shelf from the new image of the goods shelf;
step 508, generating a grid chart of goods shelf according to the detected position of the electronic price tag on the goods shelf;
step 509, detecting the out-of-stock state of the shelf according to the grid diagram.
Of course, it is understood that other variations of the detailed flow of the method for identifying the shelf items are also possible, and all related variations should fall within the protection scope of the present invention.
In summary, in the method provided by the embodiment of the invention, the image of the goods shelf is obtained; detecting an electronic price tag on a shelf from an image of the shelf; identifying the identification on the electronic price tag; and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identification on the electronic price tag and the article. In the process, only the electronic price tag on the goods shelf is detected according to the images of the goods shelf, each article is not required to be detected, the detection of the electronic price tag is small in detection workload compared with the detection of the articles, the efficiency is high, when the types of the articles are increased, only the electronic price tag is required to be identified again, a large number of goods shelf images are not required to be used for identifying new articles, therefore, the identification efficiency is high, and the expandability is high. In addition, through the fusion of the deep learning model and the wireless signal frequency based on the base station, the effect of combined measurement can be achieved, and therefore the accuracy of goods shelf identification is improved.
Based on the same inventive concept, the embodiment of the invention also provides a goods shelf object identification device, as described in the following embodiment. Since the principles of solving the problems are similar to those of the shelf object identifying method, the implementation of the device can be referred to the implementation of the method, and the repetition is omitted.
Fig. 6 is a schematic diagram of a shelf object identifying apparatus according to an embodiment of the present invention, as shown in fig. 6, the apparatus includes:
an image obtaining module 601, configured to obtain an image of a shelf;
a detection module 602, configured to detect an electronic price tag on a shelf from an image of the shelf;
a first identifying module 603, configured to identify an identifier on the electronic price tag;
the second identifying module 604 is configured to determine an item corresponding to the electronic price tag by querying a correspondence between the identifier on the electronic price tag and the item.
In an embodiment, the apparatus further comprises a correction module 605 for:
acquiring wireless signal frequency when heartbeat data of the electronic price tag are received by the base station;
according to the wireless signal frequency when the heartbeat data of the electronic price tag is received by the base station, determining a goods shelf where the electronic price tag is located;
if the determined goods shelf where the electronic price tag is located is not consistent with the image of the goods shelf where the electronic price tag is detected, repeating the following steps until the goods shelf where the electronic price tag is located is consistent with the image of the goods shelf where the electronic price tag is detected: obtaining a new image of the shelf using the image obtaining module 601; the electronic price tag on the shelf is detected from the new image of the shelf using the detection module 602.
In an embodiment, the detection module 602 is further configured to: detecting the position of the electronic price tag on the goods shelf from the image of the goods shelf;
the apparatus further comprises an out-of-stock condition detection module 606 for:
generating a shelf diagram of goods shelf according to the detected position of the electronic price tag on the goods shelf;
and detecting the out-of-stock state of the goods shelf according to the grid diagram.
In one embodiment, the out-of-stock condition detection module 606 is specifically configured to:
for each grid in the grid graph, if the grid has no article, the grid is in a stock-out state; otherwise, the out-of-stock condition of the grid is not out-of-stock.
In one embodiment, the detection module 602 is specifically configured to:
and inputting the image of the goods shelf into a deep learning model, and detecting the electronic price tag on the goods shelf.
In one embodiment, the deep learning model is trained by the following method:
obtaining a historical image of a goods shelf;
extracting a feature vector of the historical image;
and training a deep learning model by using the feature vector.
In one embodiment, after obtaining the historical image of the shelf, further comprising:
searching an image for displaying the identification of the electronic price tag from the historical image of the goods shelf;
extracting feature vectors of the historical image, including:
and extracting the characteristic vector from the image displaying the identification of the electronic price tag.
In one embodiment, after obtaining the historical image of the shelf, further comprising:
obtaining a lighting sequence image of the electronic price tag from the historical image of the goods shelf;
extracting feature vectors of the historical image, including:
and extracting the characteristic vector from the lighting sequence image of the electronic price tag.
In one embodiment, the identification on the electronic price tag comprises: encoding on an electronic price tag.
In summary, in the device provided by the embodiment of the invention, the image of the goods shelf is obtained; detecting an electronic price tag on a shelf from an image of the shelf; identifying the identification on the electronic price tag; and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identification on the electronic price tag and the article. In the process, only the electronic price tag on the goods shelf is detected according to the images of the goods shelf, each article is not required to be detected, the detection of the electronic price tag is small in detection workload compared with the detection of the articles, the efficiency is high, when the types of the articles are increased, only the electronic price tag is required to be identified again, a large number of goods shelf images are not required to be used for identifying new articles, therefore, the identification efficiency is high, and the expandability is high. In addition, through the fusion of the deep learning model and the wireless signal frequency based on the base station, the effect of combined measurement can be achieved, and therefore the accuracy of goods shelf identification is improved.
The embodiment of the present invention further provides a system for identifying a shelf object, and fig. 7 is a schematic diagram of a system for identifying a shelf object according to the embodiment of the present invention, where the system includes the above-mentioned device 701 for identifying a shelf object, a camera 702,
the camera 702 is used to capture an image of the shelf and send the image of the shelf to the shelf item identification device.
In an embodiment, the shelf item identification system further comprises a base station 703 for:
the electronic heartbeat data is received, and the wireless signal frequency at the time when the electronic price tag heartbeat data is received by the base station is transmitted to the shelf product identification device 701.
In summary, in the system provided by the embodiment of the invention, the image of the goods shelf is obtained; detecting an electronic price tag on a shelf from an image of the shelf; identifying the identification on the electronic price tag; and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identification on the electronic price tag and the article. In the process, only the electronic price tag on the goods shelf is detected according to the images of the goods shelf, each article is not required to be detected, the detection of the electronic price tag is small in detection workload compared with the detection of the articles, the efficiency is high, when the types of the articles are increased, only the electronic price tag is required to be identified again, a large number of goods shelf images are not required to be used for identifying new articles, therefore, the identification efficiency is high, and the expandability is high. In addition, through the fusion of the deep learning model and the wireless signal frequency based on the base station, the effect of combined measurement can be achieved, and therefore the accuracy of goods shelf identification is improved.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (14)
1. A method of identifying a shelf item, comprising:
obtaining an image of a shelf;
detecting an electronic price tag on a shelf from an image of the shelf;
identifying the identification on the electronic price tag;
determining the corresponding article of the electronic price tag by inquiring the corresponding relation between the identification on the electronic price tag and the article;
further comprises:
acquiring wireless signal frequency when heartbeat data of the electronic price tag are received by the base station;
according to the wireless signal frequency when the heartbeat data of the electronic price tag is received by the base station, determining a goods shelf where the electronic price tag is located;
if the determined goods shelf where the electronic price tag is located is not consistent with the image of the goods shelf where the electronic price tag is detected, repeating the following steps until the goods shelf where the electronic price tag is located is consistent with the image of the goods shelf where the electronic price tag is detected: obtaining a new image of the shelf; from the new image of the shelf, the electronic price tag on the shelf is detected.
2. The method of claim 1, wherein detecting the electronic price tag on the shelf from the image of the shelf comprises:
and inputting the image of the goods shelf into a deep learning model, and detecting the electronic price tag on the goods shelf.
3. The method of claim 2, wherein the deep learning model is trained by:
obtaining a historical image of a goods shelf;
extracting a feature vector of the historical image;
and training a deep learning model by using the feature vector.
4. The method of identifying a shelf item of claim 3, further comprising, after obtaining the historical image of the shelf:
searching an image for displaying the identification of the electronic price tag from the historical image of the goods shelf;
extracting feature vectors of the historical image, including:
and extracting the characteristic vector from the image displaying the identification of the electronic price tag.
5. The method of identifying a shelf item of claim 3, further comprising, after obtaining the historical image of the shelf:
obtaining a lighting sequence image of the electronic price tag from the historical image of the goods shelf;
extracting feature vectors of the historical image, including:
and extracting the characteristic vector from the lighting sequence image of the electronic price tag.
6. The method of claim 1, wherein the identification on the electronic price label comprises: encoding on an electronic price tag.
7. The method of claim 1, wherein detecting the electronic price tag on the shelf from the image of the shelf comprises: detecting the position of the electronic price tag on the goods shelf from the image of the goods shelf;
the method further comprises the steps of:
generating a shelf diagram of goods shelf according to the detected position of the electronic price tag on the goods shelf;
and detecting the out-of-stock state of the goods shelf according to the grid diagram.
8. The method of claim 7, wherein detecting the absence of a shelf based on the trellis diagram comprises:
for each grid in the grid graph, if the grid has no article, the grid is in a stock-out state; otherwise, the out-of-stock condition of the grid is not out-of-stock.
9. A shelf-item identification apparatus, comprising:
the image acquisition module is used for acquiring images of the goods shelves;
the detection module is used for detecting the electronic price tag on the goods shelf from the image of the goods shelf;
the first identification module is used for identifying the identification on the electronic price tag;
the second identification module is used for determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identification on the electronic price tag and the article;
the system further comprises a correction module for:
acquiring wireless signal frequency when heartbeat data of the electronic price tag are received by the base station;
according to the wireless signal frequency when the heartbeat data of the electronic price tag is received by the base station, determining a goods shelf where the electronic price tag is located;
if the determined goods shelf where the electronic price tag is located is not consistent with the image of the goods shelf where the electronic price tag is detected, repeating the following steps until the goods shelf where the electronic price tag is located is consistent with the image of the goods shelf where the electronic price tag is detected: obtaining a new image of the shelf by using the image obtaining module; the electronic price tag on the shelf is detected from the new image of the shelf by the detection module.
10. The shelf-item identification device of claim 9, wherein the detection module is further configured to: detecting the position of the electronic price tag on the goods shelf from the image of the goods shelf;
the apparatus further comprises an out-of-stock condition detection module for:
generating a shelf diagram of goods shelf according to the detected position of the electronic price tag on the goods shelf;
and detecting the out-of-stock state of the goods shelf according to the grid diagram.
11. A shelf item identification system comprising a shelf item identification device as claimed in any one of claims 9 to 10 and a camera, wherein,
the camera is used for shooting images of the goods shelf and sending the images of the goods shelf to the goods shelf identification device.
12. The shelf-item identification system of claim 11, further comprising a base station for:
and receiving the heartbeat data of the electronic price tag, and sending the wireless signal frequency of the heartbeat data of the electronic price tag when the heartbeat data is received by the base station to the goods shelf object identification device.
13. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 8 when executing the computer program.
14. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program for executing the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910756178.1A CN112395918B (en) | 2019-08-16 | 2019-08-16 | Goods shelf identification method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910756178.1A CN112395918B (en) | 2019-08-16 | 2019-08-16 | Goods shelf identification method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112395918A CN112395918A (en) | 2021-02-23 |
| CN112395918B true CN112395918B (en) | 2024-03-01 |
Family
ID=74601921
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910756178.1A Active CN112395918B (en) | 2019-08-16 | 2019-08-16 | Goods shelf identification method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112395918B (en) |
Citations (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102150411A (en) * | 2008-09-08 | 2011-08-10 | 阿尔卡特朗讯公司 | System for resolving a service to be provisioned to a terminal device a related terminal device and a related service resolving server |
| CN102542420A (en) * | 2012-01-16 | 2012-07-04 | 北京汉朔科技有限公司 | Electronic shelf label system |
| CN202422155U (en) * | 2011-08-17 | 2012-09-05 | 周欣炜 | System for realizing real-time tracking of precise cargo placement in cargo-carrying process via RFID |
| JP2013182506A (en) * | 2012-03-02 | 2013-09-12 | Nagoya Electrical Educational Foundation | Lesson operation management system using communication display device |
| KR20150143249A (en) * | 2014-06-11 | 2015-12-23 | 삼성전기주식회사 | Product price decision apparatus and method for an electronic information label system |
| CN105380632A (en) * | 2015-10-23 | 2016-03-09 | 江苏久祥汽车电器集团有限公司 | Biological identification robot system with high safety performance |
| CN105913299A (en) * | 2015-06-15 | 2016-08-31 | 金荣德 | Travel destination one stop shopping system based on 3D panoramic image and control method thereof |
| CN106526534A (en) * | 2016-10-17 | 2017-03-22 | 南京理工大学 | Device and method for automatic sorting carrying of articles based on radio navigation through moving trolley |
| CN107105020A (en) * | 2017-04-06 | 2017-08-29 | 苏州易泰勒电子科技有限公司 | A kind of communication means of electronic bin tag system |
| CN107615315A (en) * | 2015-04-27 | 2018-01-19 | 哌咖熟食公司 | Monitoring and control system for food arrangement and the food arrangement with such system |
| CN107784332A (en) * | 2016-08-24 | 2018-03-09 | 浙江汉朔电子科技有限公司 | Electronics price tag information processing system and method |
| CN107886022A (en) * | 2017-11-01 | 2018-04-06 | 阳光凯讯(北京)科技有限公司 | Intelligent repository goods and materials orientation management system based on ultra wide band and radio frequency identification |
| CN108012325A (en) * | 2017-10-30 | 2018-05-08 | 上海神添实业有限公司 | A kind of navigation locating method based on UWB and binocular vision |
| CN108334912A (en) * | 2017-01-19 | 2018-07-27 | 浙江汉朔电子科技有限公司 | Article display aided management system |
| CN108364005A (en) * | 2018-03-07 | 2018-08-03 | 上海扩博智能技术有限公司 | Automatic identifying method, system, equipment and the storage medium of price tag |
| CN108447172A (en) * | 2013-03-12 | 2018-08-24 | 洲际大品牌有限责任公司 | Automatic vending device based on display and method |
| CN108647553A (en) * | 2018-05-10 | 2018-10-12 | 上海扩博智能技术有限公司 | Rapid expansion method, system, equipment and the storage medium of model training image |
| CN207965932U (en) * | 2018-03-27 | 2018-10-12 | 北京京东方显示技术有限公司 | A kind of intelligent commodity shelf, commodity chip and smart shopper system |
| CN108694421A (en) * | 2017-04-11 | 2018-10-23 | 浙江汉朔电子科技有限公司 | Electronics price tag system and working method |
| CN109141451A (en) * | 2018-07-13 | 2019-01-04 | 京东方科技集团股份有限公司 | Positioning system of doing shopping and method, intelligent shopping trolley, electronic equipment |
| CN109242060A (en) * | 2018-08-30 | 2019-01-18 | 上海扩博智能技术有限公司 | New restocking product fast searching method, system, equipment and storage medium |
| CN109271935A (en) * | 2018-09-17 | 2019-01-25 | 北京旷视科技有限公司 | The matching method of article and electronic tag, apparatus and system |
| CN109308434A (en) * | 2017-07-27 | 2019-02-05 | 浙江汉朔电子科技有限公司 | Electronics price tag, server and system |
| WO2019073063A1 (en) * | 2017-10-13 | 2019-04-18 | Ses-Imagotag | Method for initializing or updating a realogram database for a shelf space, using optical signals emitted by electronic shelf labels |
| CN109961447A (en) * | 2019-03-29 | 2019-07-02 | 京东方科技集团股份有限公司 | Price tag detection method and device, electronic equipment, storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5194052B2 (en) * | 2010-05-18 | 2013-05-08 | 東芝テック株式会社 | Electronic shelf label system |
| US9442958B2 (en) * | 2014-12-24 | 2016-09-13 | International Business Machines Corporation | Product identification via image analysis |
| US10327082B2 (en) * | 2016-03-02 | 2019-06-18 | Bragi GmbH | Location based tracking using a wireless earpiece device, system, and method |
-
2019
- 2019-08-16 CN CN201910756178.1A patent/CN112395918B/en active Active
Patent Citations (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102150411A (en) * | 2008-09-08 | 2011-08-10 | 阿尔卡特朗讯公司 | System for resolving a service to be provisioned to a terminal device a related terminal device and a related service resolving server |
| CN202422155U (en) * | 2011-08-17 | 2012-09-05 | 周欣炜 | System for realizing real-time tracking of precise cargo placement in cargo-carrying process via RFID |
| CN102542420A (en) * | 2012-01-16 | 2012-07-04 | 北京汉朔科技有限公司 | Electronic shelf label system |
| JP2013182506A (en) * | 2012-03-02 | 2013-09-12 | Nagoya Electrical Educational Foundation | Lesson operation management system using communication display device |
| CN108447172A (en) * | 2013-03-12 | 2018-08-24 | 洲际大品牌有限责任公司 | Automatic vending device based on display and method |
| KR20150143249A (en) * | 2014-06-11 | 2015-12-23 | 삼성전기주식회사 | Product price decision apparatus and method for an electronic information label system |
| CN107615315A (en) * | 2015-04-27 | 2018-01-19 | 哌咖熟食公司 | Monitoring and control system for food arrangement and the food arrangement with such system |
| CN105913299A (en) * | 2015-06-15 | 2016-08-31 | 金荣德 | Travel destination one stop shopping system based on 3D panoramic image and control method thereof |
| CN105380632A (en) * | 2015-10-23 | 2016-03-09 | 江苏久祥汽车电器集团有限公司 | Biological identification robot system with high safety performance |
| CN107784332A (en) * | 2016-08-24 | 2018-03-09 | 浙江汉朔电子科技有限公司 | Electronics price tag information processing system and method |
| CN106526534A (en) * | 2016-10-17 | 2017-03-22 | 南京理工大学 | Device and method for automatic sorting carrying of articles based on radio navigation through moving trolley |
| CN108334912A (en) * | 2017-01-19 | 2018-07-27 | 浙江汉朔电子科技有限公司 | Article display aided management system |
| CN107105020A (en) * | 2017-04-06 | 2017-08-29 | 苏州易泰勒电子科技有限公司 | A kind of communication means of electronic bin tag system |
| CN108694421A (en) * | 2017-04-11 | 2018-10-23 | 浙江汉朔电子科技有限公司 | Electronics price tag system and working method |
| CN109308434A (en) * | 2017-07-27 | 2019-02-05 | 浙江汉朔电子科技有限公司 | Electronics price tag, server and system |
| WO2019073063A1 (en) * | 2017-10-13 | 2019-04-18 | Ses-Imagotag | Method for initializing or updating a realogram database for a shelf space, using optical signals emitted by electronic shelf labels |
| CN108012325A (en) * | 2017-10-30 | 2018-05-08 | 上海神添实业有限公司 | A kind of navigation locating method based on UWB and binocular vision |
| CN107886022A (en) * | 2017-11-01 | 2018-04-06 | 阳光凯讯(北京)科技有限公司 | Intelligent repository goods and materials orientation management system based on ultra wide band and radio frequency identification |
| CN108364005A (en) * | 2018-03-07 | 2018-08-03 | 上海扩博智能技术有限公司 | Automatic identifying method, system, equipment and the storage medium of price tag |
| CN207965932U (en) * | 2018-03-27 | 2018-10-12 | 北京京东方显示技术有限公司 | A kind of intelligent commodity shelf, commodity chip and smart shopper system |
| CN108647553A (en) * | 2018-05-10 | 2018-10-12 | 上海扩博智能技术有限公司 | Rapid expansion method, system, equipment and the storage medium of model training image |
| CN109141451A (en) * | 2018-07-13 | 2019-01-04 | 京东方科技集团股份有限公司 | Positioning system of doing shopping and method, intelligent shopping trolley, electronic equipment |
| CN109242060A (en) * | 2018-08-30 | 2019-01-18 | 上海扩博智能技术有限公司 | New restocking product fast searching method, system, equipment and storage medium |
| CN109271935A (en) * | 2018-09-17 | 2019-01-25 | 北京旷视科技有限公司 | The matching method of article and electronic tag, apparatus and system |
| CN109961447A (en) * | 2019-03-29 | 2019-07-02 | 京东方科技集团股份有限公司 | Price tag detection method and device, electronic equipment, storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112395918A (en) | 2021-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2019192231A1 (en) | Product positioning method and device, apparatus, and storage medium | |
| CN109409218A (en) | Reminding method, apparatus and system | |
| CN108345912A (en) | Commodity rapid settlement system based on RGBD information and deep learning | |
| CN109271935A (en) | The matching method of article and electronic tag, apparatus and system | |
| CN110119914A (en) | Object replenishes processing method, device and system | |
| CN112508132B (en) | Training method and device for identifying SKU | |
| CN110705424A (en) | Method and device for positioning commodity display position and storage medium | |
| US11748787B2 (en) | Analysis method and system for the item on the supermarket shelf | |
| WO2021003595A1 (en) | Method for determining moving position of electronic price tag, device and system | |
| CN109191546B (en) | Map generation method, information acquisition method, device and system | |
| CN106203225A (en) | Pictorial element based on the degree of depth is deleted | |
| JP2019174959A (en) | Commodity shelf position registration program and information processing apparatus | |
| CN112395918B (en) | Goods shelf identification method, device and system | |
| CN113706227A (en) | Goods shelf commodity recommendation method and device | |
| WO2021030924A1 (en) | Method, apparatus and system for identifying item on shelf | |
| EP3671410B1 (en) | Method and device to control a virtual reality display unit | |
| CN112560563A (en) | Article layout identification method and device and electronic equipment | |
| CN115909335A (en) | Commodity labeling method and device | |
| CN110096946A (en) | A kind of self-service system merged based on pressure sensitivity and vision | |
| CN115294525A (en) | Market commodity supervisory systems | |
| CN112261582B (en) | Electronic price tag moving position determining method, device and system | |
| CN111160330B (en) | Training method for improving image recognition accuracy with assistance of electronic tag recognition | |
| CN109697445A (en) | Image processing method and device | |
| CN104867026B (en) | Method and system for providing commodity image and terminal device for outputting commodity image | |
| CN112699865B (en) | Electronic price tag identification system and method and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 314031 podium Building 1 / F, 4 / F and 7 / F, building 5 / F, No.1 building, Jiaxing photovoltaic technology innovation park, 1288 Kanghe Road, Xiuzhou District, Jiaxing City, Zhejiang Province Applicant after: Hanshuo Technology Co.,Ltd. Address before: 314000 33 buildings in Zhifu center, Xiuzhou District, Jiaxing, Zhejiang. Applicant before: ZHEJIANG HANSHOW TECHNOLOGY Co.,Ltd. |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |