CN112395918A - Goods shelf identification method, device and system - Google Patents

Goods shelf identification method, device and system Download PDF

Info

Publication number
CN112395918A
CN112395918A CN201910756178.1A CN201910756178A CN112395918A CN 112395918 A CN112395918 A CN 112395918A CN 201910756178 A CN201910756178 A CN 201910756178A CN 112395918 A CN112395918 A CN 112395918A
Authority
CN
China
Prior art keywords
shelf
electronic price
price tag
image
goods
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910756178.1A
Other languages
Chinese (zh)
Other versions
CN112395918B (en
Inventor
梁敏
侯世国
庄艺唐
金小平
张彩蝶
苏汛沅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Hanshow Technology Co ltd
Original Assignee
Zhejiang Hanshow Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Hanshow Technology Co ltd filed Critical Zhejiang Hanshow Technology Co ltd
Priority to CN201910756178.1A priority Critical patent/CN112395918B/en
Publication of CN112395918A publication Critical patent/CN112395918A/en
Application granted granted Critical
Publication of CN112395918B publication Critical patent/CN112395918B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/20Scenes; Scene-specific elements in augmented reality scenes

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a method, a device and a system for identifying shelf articles, wherein the method comprises the following steps: obtaining an image of the shelf; detecting an electronic price tag on the shelf from the image of the shelf; identifying the mark on the electronic price tag; and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identifier on the electronic price tag and the article. The invention can accurately identify goods on the goods shelf, and has high efficiency and strong expandability.

Description

Goods shelf identification method, device and system
Technical Field
The invention relates to the field of internet, in particular to a shelf article identification method, device and system.
Background
The identification of shelf items (e.g., merchandise) has been a significant concern for businesses. The traditional article identification thought is that an article picture is collected for marking, a deep learning algorithm is given for training to generate a model, and the article is identified through the model. However, two problems which cannot be solved exist in this way, the first problem is that the number of articles is very large, and the training data of a large number of articles needs to be acquired for training, so that the efficiency is low. Secondly, the types of the articles are continuously increased and updated, training data are required to be continuously updated for identifying new articles, and the expandability is poor. These two challenges pose significant obstacles to the widespread use of item identification.
Disclosure of Invention
The embodiment of the invention provides a shelf article identification method, which is used for identifying shelf articles and has high efficiency and strong expandability, and comprises the following steps:
obtaining an image of the shelf;
detecting an electronic price tag on the shelf from the image of the shelf;
identifying the mark on the electronic price tag;
and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identifier on the electronic price tag and the article.
The embodiment of the invention provides a shelf article identification device, which is used for identifying shelf articles and has high efficiency and strong expandability, and comprises the following components:
obtaining an image of the shelf;
detecting an electronic price tag on the shelf from the image of the shelf;
identifying the mark on the electronic price tag;
and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identifier on the electronic price tag and the article.
The embodiment of the invention provides a goods shelf identification system, which is used for identifying goods shelves, has high efficiency and strong expandability and comprises the following components: the above-mentioned shelf item recognition apparatus, the camera, wherein,
the camera is used for shooting the image of the goods shelf and sending the image of the goods shelf to the goods shelf identification device.
The embodiment of the invention also provides computer equipment which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor realizes the shelf item identification method when executing the computer program.
The embodiment of the invention also provides a computer readable storage medium, and the computer readable storage medium stores a computer program for executing the shelf item identification method.
In an embodiment of the present invention, an image of a shelf is obtained; detecting an electronic price tag on the shelf from the image of the shelf; identifying the mark on the electronic price tag; and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identifier on the electronic price tag and the article. In the process, the electronic price tags on the goods shelf are detected only according to the images of the goods shelf, each kind of goods does not need to be detected, the detection of the electronic price tags is less in workload compared with the detection of the goods, the efficiency is high, when the kinds of the goods are increased, only one electronic price tag needs to be identified again, and a large number of goods shelf images are not needed to identify new goods, so that the identification efficiency is high, and the expandability is high.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts. In the drawings:
FIG. 1 is a flow chart of a method of shelf item identification in an embodiment of the present invention;
FIG. 2 is a schematic illustration of a shelf image in an embodiment of the invention;
FIG. 3 is a schematic diagram of electronic price tag detection according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a trellis diagram for a shelf item in an embodiment of the present invention;
FIG. 5 is a detailed flowchart of a method for identifying shelf items according to an embodiment of the present invention;
FIG. 6 is a schematic view of a shelf item identification device according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a shelf item identification system in an embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
In the description of the present specification, the terms "comprising," "including," "having," "containing," and the like are used in an open-ended fashion, i.e., to mean including, but not limited to. Reference to the description of the terms "one embodiment," "a particular embodiment," "some embodiments," "for example," etc., means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. The sequence of steps involved in the embodiments is for illustrative purposes to illustrate the implementation of the present application, and the sequence of steps is not limited and can be adjusted as needed.
Fig. 1 is a flowchart of a shelf item identification method according to an embodiment of the present invention, and as shown in fig. 1, the method includes:
step 101, obtaining an image of a shelf;
102, detecting an electronic price tag on a shelf from an image of the shelf;
103, identifying the mark on the electronic price tag;
and 104, determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identifier on the electronic price tag and the article.
In an embodiment of the present invention, an image of a shelf is obtained; detecting an electronic price tag on the shelf from the image of the shelf; identifying the mark on the electronic price tag; and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identifier on the electronic price tag and the article. In the process, the electronic price tags on the goods shelf are detected only according to the images of the goods shelf, each kind of goods does not need to be detected, the detection of the electronic price tags is less in workload compared with the detection of the goods, the efficiency is high, when the kinds of the goods are increased, only one electronic price tag needs to be identified again, and a large number of goods shelf images are not needed to identify new goods, so that the identification efficiency is high, and the expandability is high.
In step 101, images of the shelf may be captured by using various devices such as a camera, a video camera, a high-speed camera, and the like, and the images of the shelf are obtained, where fig. 2 is a schematic diagram of the images of the shelf in an embodiment of the present invention, where the diagram includes articles and electronic price tags, and each type of article has a corresponding electronic price tag. In step 102, an electronic price tag is searched from the image corresponding to fig. 2, fig. 3 is a schematic diagram of detecting the electronic price tag in the embodiment of the present invention, and the electronic price tag can be detected, including information such as a position of the electronic price tag, in this step, the electronic price tag is detected from the image, but not the article, and compared with the article, the detection process of the electronic price tag with a fixed shape and position has smaller workload, high efficiency, and higher accuracy; in step 103, the identification on the electronic price tag can be identified from the database of the electronic price tag, and in step 104, the item corresponding to the electronic price tag is determined by querying the corresponding relationship between the identification on the electronic price tag and the item, for example, the corresponding relationship between the electronic price tag and the item SKU can be queried from the database, so as to determine the item corresponding to the electronic price tag. When the variety of the object is added, only the electronic price tag needs to be detected again, the complex object image does not need to be identified from a large number of images, the efficiency is higher, and the accuracy is high.
In practice, there are many methods for detecting the electronic price tag on the shelf from the image of the shelf, and one example is given below.
In one embodiment, detecting an electronic price tag on a shelf from an image of the shelf includes:
and inputting the image of the shelf into a deep learning model, and detecting the electronic price tag on the shelf.
In one embodiment, the deep learning model is obtained by training using the following method:
obtaining a historical image of the shelf;
extracting a feature vector of the historical image;
and training a deep learning model by using the feature vector.
In specific implementation, the commonly used deep learning model includes an automatic encoder, a sparse encoder, a convolutional neural network, RNN, LSTM, and the like, and the deep learning model training method is substantially similar, that is, the three steps in the above embodiment, and in addition, when the deep learning model is trained by using the feature vector, the method specifically includes: and adjusting parameters of the deep learning model in the training process until the loss function of the deep learning model meets the preset convergence condition, and obtaining the trained deep learning model.
In another embodiment, the deep learning model is obtained by training with the following method:
dividing historical images of the shelf into training set data and test set data;
extracting a feature vector of the training set data;
training the deep learning model by using the feature vectors of the training set data,
adjusting parameters of the deep learning model in the training process until a loss function of the deep learning model meets a preset convergence condition, and obtaining the trained deep learning model;
testing the trained deep learning model by using the test set data;
and when the accuracy of the trained deep learning model is less than the preset accuracy, increasing the number of the training set data or adjusting the proportion of the training set data and the test set data for retraining until the accuracy of the trained deep learning model is not less than the preset accuracy.
The above embodiment introduces a process of a deep learning model, that is, a historical image of a shelf is needed when the deep learning model is used to identify an electronic price tag on the shelf, and when the deep learning model is applied specifically, there are various methods for training the deep learning model according to the historical image of the shelf, and two embodiments are given below.
In one embodiment, after obtaining the historical image of the shelf, the method further comprises:
searching an image for displaying the identifier of the electronic price tag from the historical image of the shelf;
extracting a feature vector of the historical image, wherein the feature vector comprises the following steps:
feature vectors are extracted from the image showing the identity of the electronic price tag.
In the above embodiment, there are many images of electronic price tags in the history image of the shelf, and the display of the electronic price tags is updated continuously with time, at a certain time, the identifiers of the electronic price tags, such as specific codes of the electronic price tags, can be displayed, the images of the identifiers of the electronic price tags are found, and the feature vectors are extracted for training the deep learning model.
In another embodiment, after obtaining the historical image of the shelf, the method further comprises:
obtaining a lighting sequence image of the electronic price tag from a historical image of the shelf;
extracting a feature vector of the historical image, wherein the feature vector comprises the following steps:
and extracting a characteristic vector from the lighting sequence image of the electronic price tag.
In the above embodiment, the electronic price tag cycle displays different colors of flickers, for example, white is used as a starting color, { white, red, blue, green } may be a lighting sequence of one electronic price tag, and { white, green, blue, red } may be a lighting sequence of another electronic price tag, so that the electronic price tags can be distinguished by the lighting sequences, and therefore, a lighting sequence image of the electronic price tag can be obtained from a history image of a shelf, and a feature vector can be extracted to train the deep learning model.
Of course, it is understood that there may be other ways to train the deep learning model, and all the related variations should fall within the scope of the present invention.
In one embodiment, the shelf item identification method further comprises:
acquiring the frequency of a wireless signal when heartbeat data of the electronic price tag is received by a base station;
determining a shelf where the electronic price tag is located according to the frequency of the wireless signal when the heartbeat data of the electronic price tag is received by the base station;
if the determined shelf where the electronic price tag is located does not accord with the detected image of the shelf of the electronic price tag, the following steps are repeatedly executed until the two images accord with each other: obtaining a new image of the shelf; from the new image of the shelf, the electronic price tag on the shelf is detected.
In the above embodiment, the radio signal frequencies when the heartbeat data of different electronic price tags are received by the base station are different, and the shelf where the electronic price tag is located may be determined from the correspondence between the stored radio signal frequencies and the electronic price tags, for example, the position of the shelf where the electronic price tag is located may be determined, and if the determined shelf where the electronic price tag is located does not match the image of the shelf where the electronic price tag is detected, the following steps are repeatedly performed until the two correspond to each other: obtaining a new image of the shelf; from the new image of the shelf, the electronic price tag on the shelf is detected. The process can achieve the effect of joint measurement through the combination of the deep learning model and the base station-based wireless signal frequency, so that the accuracy of shelf article identification is improved.
In one embodiment, the indicia on the electronic price tag comprises: a code on the electronic price tag.
Of course, it is understood that other ways of representing the identity of the electronic price tag, such as name, may be used, and all such modifications are intended to fall within the scope of the present invention.
In one embodiment, detecting an electronic price tag on a shelf from an image of the shelf includes: detecting the position of an electronic price tag on a shelf from an image of the shelf;
the method further comprises the following steps:
generating a shelf diagram of the shelf items according to the detected positions of the electronic price tags on the shelf;
and detecting the stock shortage state of the goods shelf according to the grid diagram.
Fig. 4 is a schematic diagram of a shelf diagram of shelf items according to an embodiment of the present invention, i.e., each type of item and corresponding electronic price tag can be placed in a shelf, so that the presence or absence of items in each shelf can be easily detected.
In one embodiment, the detecting of the out-of-stock condition of the shelf according to the grid diagram comprises:
for each grid in the grid graph, if the grid has no article, the shortage state of the grid is shortage; otherwise, the goods shortage state of the shed lattice is no goods shortage.
In the above embodiment, the article corresponding to the electronic price tag in each shelf can be searched from the database or other data storage locations, and if the shelf has no article, the out-of-stock state of the shelf is out-of-stock; otherwise, the goods shortage state of the shed lattice is no goods shortage.
Based on the above embodiment, the present invention provides the following embodiment to explain a detailed flow of the method for identifying a shelf item, fig. 5 is a detailed flow chart of the method for identifying a shelf item according to the embodiment of the present invention, as shown in fig. 5, in an embodiment, the detailed flow of the method for identifying a shelf item includes:
step 501, obtaining an image of a shelf;
step 502, inputting the image of the shelf into a deep learning model, and detecting an electronic price tag on the shelf;
including detecting a location of an electronic price tag on a shelf from an image of the shelf;
step 503, identifying the mark on the electronic price tag;
step 504, determining an article corresponding to the electronic price tag by inquiring the corresponding relation between the identifier on the electronic price tag and the article;
step 505, obtaining the frequency of the wireless signal when the heartbeat data of the electronic price tag is received by the base station;
step 506, determining a shelf where the electronic price tag is located according to the frequency of the wireless signal when the heartbeat data of the electronic price tag is received by the base station;
step 507, if the determined shelf where the electronic price tag is located does not accord with the image of the shelf where the electronic price tag is detected, repeating the following steps until the two are consistent: obtaining a new image of the shelf; detecting an electronic price tag on the shelf from a new image of the shelf;
step 508, generating a shelf diagram of the shelf items according to the detected positions of the electronic price tags on the shelves;
and 509, detecting the stock shortage state of the shelf according to the shed graph.
Of course, it is understood that there may be other variations to the detailed flow of the above-mentioned method for identifying shelf items, and all such variations should fall within the scope of the present invention.
In summary, in the method provided in the embodiment of the present invention, an image of a shelf is obtained; detecting an electronic price tag on the shelf from the image of the shelf; identifying the mark on the electronic price tag; and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identifier on the electronic price tag and the article. In the process, the electronic price tags on the goods shelf are detected only according to the images of the goods shelf, each kind of goods does not need to be detected, the detection of the electronic price tags is less in workload compared with the detection of the goods, the efficiency is high, when the kinds of the goods are increased, only one electronic price tag needs to be identified again, and a large number of goods shelf images are not needed to identify new goods, so that the identification efficiency is high, and the expandability is high. In addition, the effect of joint measurement can be achieved through the fusion of the deep learning model and the base station-based wireless signal frequency, so that the accuracy of shelf article identification is improved.
Based on the same inventive concept, the embodiment of the invention also provides a shelf item identification device, as described in the following embodiments. Since the principles for solving the problems are similar to the shelf item identification method, the implementation of the device can be referred to the implementation of the method, and repeated details are not repeated.
Fig. 6 is a schematic view of a shelf item identification apparatus according to an embodiment of the present invention, and as shown in fig. 6, the apparatus includes:
an image obtaining module 601, configured to obtain an image of a shelf;
a detection module 602, configured to detect an electronic price tag on a shelf from an image of the shelf;
a first recognition module 603, configured to recognize an identifier on the electronic price tag;
the second identifying module 604 is configured to determine an item corresponding to the electronic price tag by querying a corresponding relationship between the identifier on the electronic price tag and the item.
In one embodiment, the apparatus further comprises a correction module 605 for:
acquiring the frequency of a wireless signal when heartbeat data of the electronic price tag is received by a base station;
determining a shelf where the electronic price tag is located according to the frequency of the wireless signal when the heartbeat data of the electronic price tag is received by the base station;
if the determined shelf where the electronic price tag is located does not accord with the detected image of the shelf of the electronic price tag, the following steps are repeatedly executed until the two images accord with each other: acquiring a new image of the shelf by using the image acquisition module 601; the electronic price tags on the shelves are detected from the new image of the shelf by the detection module 602.
In an embodiment, the detection module 602 is further configured to: detecting the position of an electronic price tag on a shelf from an image of the shelf;
the apparatus further includes an out-of-stock status detection module 606 to:
generating a shelf diagram of the shelf items according to the detected positions of the electronic price tags on the shelf;
and detecting the stock shortage state of the goods shelf according to the grid diagram.
In one embodiment, the out-of-stock status detection module 606 is specifically configured to:
for each grid in the grid graph, if the grid has no article, the shortage state of the grid is shortage; otherwise, the goods shortage state of the shed lattice is no goods shortage.
In an embodiment, the detection module 602 is specifically configured to:
and inputting the image of the shelf into a deep learning model, and detecting the electronic price tag on the shelf.
In one embodiment, the deep learning model is obtained by training using the following method:
obtaining a historical image of the shelf;
extracting a feature vector of the historical image;
and training a deep learning model by using the feature vector.
In one embodiment, after obtaining the historical image of the shelf, the method further comprises:
searching an image for displaying the identifier of the electronic price tag from the historical image of the shelf;
extracting a feature vector of the historical image, wherein the feature vector comprises the following steps:
feature vectors are extracted from the image showing the identity of the electronic price tag.
In one embodiment, after obtaining the historical image of the shelf, the method further comprises:
obtaining a lighting sequence image of the electronic price tag from a historical image of the shelf;
extracting a feature vector of the historical image, wherein the feature vector comprises the following steps:
and extracting a characteristic vector from the lighting sequence image of the electronic price tag.
In one embodiment, the indicia on the electronic price tag comprises: a code on the electronic price tag.
In summary, in the apparatus provided in the embodiment of the present invention, an image of a shelf is obtained; detecting an electronic price tag on the shelf from the image of the shelf; identifying the mark on the electronic price tag; and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identifier on the electronic price tag and the article. In the process, the electronic price tags on the goods shelf are detected only according to the images of the goods shelf, each kind of goods does not need to be detected, the detection of the electronic price tags is less in workload compared with the detection of the goods, the efficiency is high, when the kinds of the goods are increased, only one electronic price tag needs to be identified again, and a large number of goods shelf images are not needed to identify new goods, so that the identification efficiency is high, and the expandability is high. In addition, the effect of joint measurement can be achieved through the fusion of the deep learning model and the base station-based wireless signal frequency, so that the accuracy of shelf article identification is improved.
Fig. 7 is a schematic diagram of a shelf item identification system according to an embodiment of the present invention, which includes the above shelf item identification device 701 and a camera 702, wherein,
the camera 702 is used to take an image of the shelf and send the image of the shelf to the shelf item identification device.
In one embodiment, the shelf item identification system further comprises a base station 703 for:
the electronic heartbeat data is received, and the wireless signal frequency at which the heartbeat data of the electronic price tag is received by the base station is transmitted to the shelf item recognition device 701.
In summary, in the system provided in the embodiment of the present invention, an image of the shelf is obtained; detecting an electronic price tag on the shelf from the image of the shelf; identifying the mark on the electronic price tag; and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identifier on the electronic price tag and the article. In the process, the electronic price tags on the goods shelf are detected only according to the images of the goods shelf, each kind of goods does not need to be detected, the detection of the electronic price tags is less in workload compared with the detection of the goods, the efficiency is high, when the kinds of the goods are increased, only one electronic price tag needs to be identified again, and a large number of goods shelf images are not needed to identify new goods, so that the identification efficiency is high, and the expandability is high. In addition, the effect of joint measurement can be achieved through the fusion of the deep learning model and the base station-based wireless signal frequency, so that the accuracy of shelf article identification is improved.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (16)

1. A method of identifying a shelf item, comprising:
obtaining an image of the shelf;
detecting an electronic price tag on the shelf from the image of the shelf;
identifying the mark on the electronic price tag;
and determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identifier on the electronic price tag and the article.
2. The shelf item identification method of claim 1, wherein detecting an electronic price tag on the shelf from the image of the shelf comprises:
and inputting the image of the shelf into a deep learning model, and detecting the electronic price tag on the shelf.
3. The shelf item identification method of claim 2, wherein the deep learning model is obtained by training using:
obtaining a historical image of the shelf;
extracting a feature vector of the historical image;
and training a deep learning model by using the feature vector.
4. The shelf item identification method as claimed in claim 3, further comprising, after obtaining the historical image of the shelf:
searching an image for displaying the identifier of the electronic price tag from the historical image of the shelf;
extracting a feature vector of the historical image, wherein the feature vector comprises the following steps:
feature vectors are extracted from the image showing the identity of the electronic price tag.
5. The shelf item identification method as claimed in claim 3, further comprising, after obtaining the historical image of the shelf:
obtaining a lighting sequence image of the electronic price tag from a historical image of the shelf;
extracting a feature vector of the historical image, wherein the feature vector comprises the following steps:
and extracting a characteristic vector from the lighting sequence image of the electronic price tag.
6. The shelf item identification method as recited in claim 1, further comprising:
acquiring the frequency of a wireless signal when heartbeat data of the electronic price tag is received by a base station;
determining a shelf where the electronic price tag is located according to the frequency of the wireless signal when the heartbeat data of the electronic price tag is received by the base station;
if the determined shelf where the electronic price tag is located does not accord with the detected image of the shelf of the electronic price tag, the following steps are repeatedly executed until the two images accord with each other: obtaining a new image of the shelf; from the new image of the shelf, the electronic price tag on the shelf is detected.
7. The shelf item identification method as recited in claim 1, wherein the indicia on the electronic price tag comprises: a code on the electronic price tag.
8. The shelf item identification method of claim 1, wherein detecting an electronic price tag on the shelf from the image of the shelf comprises: detecting the position of an electronic price tag on a shelf from an image of the shelf;
the method further comprises the following steps:
generating a shelf diagram of the shelf items according to the detected positions of the electronic price tags on the shelf;
and detecting the stock shortage state of the goods shelf according to the grid diagram.
9. The method for identifying shelf items according to claim 8, wherein detecting the lack of stock of the shelf on the basis of the trellis diagram comprises:
for each grid in the grid graph, if the grid has no article, the shortage state of the grid is shortage; otherwise, the goods shortage state of the shed lattice is no goods shortage.
10. A shelf item identification device, comprising:
the image acquisition module is used for acquiring an image of the goods shelf;
the detection module is used for detecting the electronic price tags on the goods shelves from the images of the goods shelves;
the first identification module is used for identifying the mark on the electronic price tag;
and the second identification module is used for determining the article corresponding to the electronic price tag by inquiring the corresponding relation between the identifier on the electronic price tag and the article.
11. The shelf item identification device as recited in claim 10 further comprising a calibration module for:
acquiring the frequency of a wireless signal when heartbeat data of the electronic price tag is received by a base station;
determining a shelf where the electronic price tag is located according to the frequency of the wireless signal when the heartbeat data of the electronic price tag is received by the base station;
if the determined shelf where the electronic price tag is located does not accord with the detected image of the shelf of the electronic price tag, the following steps are repeatedly executed until the two images accord with each other: acquiring a new image of the shelf by using an image acquisition module; and detecting the electronic price tags on the goods shelf from the new image of the goods shelf by using the detection module.
12. The shelf item identification device of claim 10, wherein the detection module is further configured to: detecting the position of an electronic price tag on a shelf from an image of the shelf;
the device also comprises an out-of-stock state detection module used for:
generating a shelf diagram of the shelf items according to the detected positions of the electronic price tags on the shelf;
and detecting the stock shortage state of the goods shelf according to the grid diagram.
13. A shelf item identification system comprising the shelf item identification apparatus according to any one of claims 10 to 12, a camera, wherein,
the camera is used for shooting the image of the goods shelf and sending the image of the goods shelf to the goods shelf identification device.
14. The shelf item identification system of claim 13, further comprising a base station for:
and receiving the electronic heartbeat data, and transmitting the wireless signal frequency of the electronic price tag when the heartbeat data is received by the base station to the goods shelf identification device.
15. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 9 when executing the computer program.
16. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing the method of any one of claims 1 to 9.
CN201910756178.1A 2019-08-16 2019-08-16 Goods shelf identification method, device and system Active CN112395918B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910756178.1A CN112395918B (en) 2019-08-16 2019-08-16 Goods shelf identification method, device and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910756178.1A CN112395918B (en) 2019-08-16 2019-08-16 Goods shelf identification method, device and system

Publications (2)

Publication Number Publication Date
CN112395918A true CN112395918A (en) 2021-02-23
CN112395918B CN112395918B (en) 2024-03-01

Family

ID=74601921

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910756178.1A Active CN112395918B (en) 2019-08-16 2019-08-16 Goods shelf identification method, device and system

Country Status (1)

Country Link
CN (1) CN112395918B (en)

Citations (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102150411A (en) * 2008-09-08 2011-08-10 阿尔卡特朗讯公司 System for resolving a service to be provisioned to a terminal device a related terminal device and a related service resolving server
US20110285505A1 (en) * 2010-05-18 2011-11-24 Toshiba Tec Kabushiki Kaisha Electronic shelf label control apparatus and method
CN102542420A (en) * 2012-01-16 2012-07-04 北京汉朔科技有限公司 Electronic shelf label system
CN202422155U (en) * 2011-08-17 2012-09-05 周欣炜 System for realizing real-time tracking of precise cargo placement in cargo-carrying process via RFID
JP2013182506A (en) * 2012-03-02 2013-09-12 Nagoya Electrical Educational Foundation Lesson operation management system using communication display device
KR20150143249A (en) * 2014-06-11 2015-12-23 삼성전기주식회사 Product price decision apparatus and method for an electronic information label system
CN105380632A (en) * 2015-10-23 2016-03-09 江苏久祥汽车电器集团有限公司 Biological identification robot system with high safety performance
US20160188634A1 (en) * 2014-12-24 2016-06-30 International Business Machines Corporation Product identification via image analysis
CN105913299A (en) * 2015-06-15 2016-08-31 金荣德 Travel destination one stop shopping system based on 3D panoramic image and control method thereof
CN106526534A (en) * 2016-10-17 2017-03-22 南京理工大学 Device and method for automatic sorting carrying of articles based on radio navigation through moving trolley
CN107105020A (en) * 2017-04-06 2017-08-29 苏州易泰勒电子科技有限公司 A kind of communication means of electronic bin tag system
US20170257717A1 (en) * 2016-03-02 2017-09-07 Bragi GmbH Location Based Tracking Using a Wireless Earpiece Device, System, and Method
CN107615315A (en) * 2015-04-27 2018-01-19 哌咖熟食公司 Monitoring and control system for food arrangement and the food arrangement with such system
CN107784332A (en) * 2016-08-24 2018-03-09 浙江汉朔电子科技有限公司 Electronics price tag information processing system and method
CN107886022A (en) * 2017-11-01 2018-04-06 阳光凯讯(北京)科技有限公司 Intelligent repository goods and materials orientation management system based on ultra wide band and radio frequency identification
CN108012325A (en) * 2017-10-30 2018-05-08 上海神添实业有限公司 A kind of navigation locating method based on UWB and binocular vision
CN108334912A (en) * 2017-01-19 2018-07-27 浙江汉朔电子科技有限公司 Article display aided management system
CN108364005A (en) * 2018-03-07 2018-08-03 上海扩博智能技术有限公司 Automatic identifying method, system, equipment and the storage medium of price tag
CN108447172A (en) * 2013-03-12 2018-08-24 洲际大品牌有限责任公司 Automatic vending device based on display and method
CN108647553A (en) * 2018-05-10 2018-10-12 上海扩博智能技术有限公司 Rapid expansion method, system, equipment and the storage medium of model training image
CN207965932U (en) * 2018-03-27 2018-10-12 北京京东方显示技术有限公司 A kind of intelligent commodity shelf, commodity chip and smart shopper system
CN108694421A (en) * 2017-04-11 2018-10-23 浙江汉朔电子科技有限公司 Electronics price tag system and working method
CN109141451A (en) * 2018-07-13 2019-01-04 京东方科技集团股份有限公司 Positioning system of doing shopping and method, intelligent shopping trolley, electronic equipment
CN109242060A (en) * 2018-08-30 2019-01-18 上海扩博智能技术有限公司 New restocking product fast searching method, system, equipment and storage medium
CN109271935A (en) * 2018-09-17 2019-01-25 北京旷视科技有限公司 The matching method of article and electronic tag, apparatus and system
CN109308434A (en) * 2017-07-27 2019-02-05 浙江汉朔电子科技有限公司 Electronics price tag, server and system
WO2019073063A1 (en) * 2017-10-13 2019-04-18 Ses-Imagotag Method for initializing or updating a realogram database for a shelf space, using optical signals emitted by electronic shelf labels
CN109961447A (en) * 2019-03-29 2019-07-02 京东方科技集团股份有限公司 Price tag detection method and device, electronic equipment, storage medium

Patent Citations (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102150411A (en) * 2008-09-08 2011-08-10 阿尔卡特朗讯公司 System for resolving a service to be provisioned to a terminal device a related terminal device and a related service resolving server
US20110285505A1 (en) * 2010-05-18 2011-11-24 Toshiba Tec Kabushiki Kaisha Electronic shelf label control apparatus and method
CN202422155U (en) * 2011-08-17 2012-09-05 周欣炜 System for realizing real-time tracking of precise cargo placement in cargo-carrying process via RFID
CN102542420A (en) * 2012-01-16 2012-07-04 北京汉朔科技有限公司 Electronic shelf label system
JP2013182506A (en) * 2012-03-02 2013-09-12 Nagoya Electrical Educational Foundation Lesson operation management system using communication display device
CN108447172A (en) * 2013-03-12 2018-08-24 洲际大品牌有限责任公司 Automatic vending device based on display and method
KR20150143249A (en) * 2014-06-11 2015-12-23 삼성전기주식회사 Product price decision apparatus and method for an electronic information label system
US20160188634A1 (en) * 2014-12-24 2016-06-30 International Business Machines Corporation Product identification via image analysis
CN107615315A (en) * 2015-04-27 2018-01-19 哌咖熟食公司 Monitoring and control system for food arrangement and the food arrangement with such system
CN105913299A (en) * 2015-06-15 2016-08-31 金荣德 Travel destination one stop shopping system based on 3D panoramic image and control method thereof
CN105380632A (en) * 2015-10-23 2016-03-09 江苏久祥汽车电器集团有限公司 Biological identification robot system with high safety performance
US20170257717A1 (en) * 2016-03-02 2017-09-07 Bragi GmbH Location Based Tracking Using a Wireless Earpiece Device, System, and Method
CN107784332A (en) * 2016-08-24 2018-03-09 浙江汉朔电子科技有限公司 Electronics price tag information processing system and method
CN106526534A (en) * 2016-10-17 2017-03-22 南京理工大学 Device and method for automatic sorting carrying of articles based on radio navigation through moving trolley
CN108334912A (en) * 2017-01-19 2018-07-27 浙江汉朔电子科技有限公司 Article display aided management system
CN107105020A (en) * 2017-04-06 2017-08-29 苏州易泰勒电子科技有限公司 A kind of communication means of electronic bin tag system
CN108694421A (en) * 2017-04-11 2018-10-23 浙江汉朔电子科技有限公司 Electronics price tag system and working method
CN109308434A (en) * 2017-07-27 2019-02-05 浙江汉朔电子科技有限公司 Electronics price tag, server and system
WO2019073063A1 (en) * 2017-10-13 2019-04-18 Ses-Imagotag Method for initializing or updating a realogram database for a shelf space, using optical signals emitted by electronic shelf labels
CN108012325A (en) * 2017-10-30 2018-05-08 上海神添实业有限公司 A kind of navigation locating method based on UWB and binocular vision
CN107886022A (en) * 2017-11-01 2018-04-06 阳光凯讯(北京)科技有限公司 Intelligent repository goods and materials orientation management system based on ultra wide band and radio frequency identification
CN108364005A (en) * 2018-03-07 2018-08-03 上海扩博智能技术有限公司 Automatic identifying method, system, equipment and the storage medium of price tag
CN207965932U (en) * 2018-03-27 2018-10-12 北京京东方显示技术有限公司 A kind of intelligent commodity shelf, commodity chip and smart shopper system
CN108647553A (en) * 2018-05-10 2018-10-12 上海扩博智能技术有限公司 Rapid expansion method, system, equipment and the storage medium of model training image
CN109141451A (en) * 2018-07-13 2019-01-04 京东方科技集团股份有限公司 Positioning system of doing shopping and method, intelligent shopping trolley, electronic equipment
CN109242060A (en) * 2018-08-30 2019-01-18 上海扩博智能技术有限公司 New restocking product fast searching method, system, equipment and storage medium
CN109271935A (en) * 2018-09-17 2019-01-25 北京旷视科技有限公司 The matching method of article and electronic tag, apparatus and system
CN109961447A (en) * 2019-03-29 2019-07-02 京东方科技集团股份有限公司 Price tag detection method and device, electronic equipment, storage medium

Also Published As

Publication number Publication date
CN112395918B (en) 2024-03-01

Similar Documents

Publication Publication Date Title
CN110383274B (en) Method, device, system, storage medium, processor and terminal for identifying equipment
WO2019192231A1 (en) Product positioning method and device, apparatus, and storage medium
US8280764B2 (en) System and method for delivery of electronic coupons
EP3438577A1 (en) Intelligent refrigerator and control method and control system thereof
CN109271935A (en) The matching method of article and electronic tag, apparatus and system
CN109308434A (en) Electronics price tag, server and system
CN110110652B (en) Target detection method, electronic device and storage medium
CN108345912A (en) Commodity rapid settlement system based on RGBD information and deep learning
CN110119914A (en) Object replenishes processing method, device and system
CN110866515A (en) Method and device for identifying object behaviors in plant and electronic equipment
JPWO2016163564A1 (en) Information processing apparatus, information processing system, position notification method, and program
US20210279784A1 (en) Analysis method and system for the item on the supermarket shelf
CN110673717A (en) Method and apparatus for controlling output device
CN109743497B (en) Data set acquisition method and system and electronic device
CN113938674B (en) Video quality detection method, device, electronic equipment and readable storage medium
CN112395918B (en) Goods shelf identification method, device and system
WO2021030924A1 (en) Method, apparatus and system for identifying item on shelf
CN109191546A (en) Map method of formation, information acquisition method, apparatus and system
EP3671410A1 (en) Method and device to control a virtual reality display unit
CN115909335A (en) Commodity labeling method and device
CN110413869A (en) Method and apparatus for pushed information
CN111435511B (en) Order processing method, device, equipment, system and readable storage medium
CN112261582B (en) Electronic price tag moving position determining method, device and system
CN104867026B (en) Method and system for providing commodity image and terminal device for outputting commodity image
CN112699865B (en) Electronic price tag identification system and method and server

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
CB02 Change of applicant information
CB02 Change of applicant information

Address after: 314031 podium Building 1 / F, 4 / F and 7 / F, building 5 / F, No.1 building, Jiaxing photovoltaic technology innovation park, 1288 Kanghe Road, Xiuzhou District, Jiaxing City, Zhejiang Province

Applicant after: Hanshuo Technology Co.,Ltd.

Address before: 314000 33 buildings in Zhifu center, Xiuzhou District, Jiaxing, Zhejiang.

Applicant before: ZHEJIANG HANSHOW TECHNOLOGY Co.,Ltd.

SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant