CN111310706B - Commodity price tag identification method and device, electronic equipment and storage medium - Google Patents

Commodity price tag identification method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN111310706B
CN111310706B CN202010128049.0A CN202010128049A CN111310706B CN 111310706 B CN111310706 B CN 111310706B CN 202010128049 A CN202010128049 A CN 202010128049A CN 111310706 B CN111310706 B CN 111310706B
Authority
CN
China
Prior art keywords
position information
commodity
price tag
depth
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010128049.0A
Other languages
Chinese (zh)
Other versions
CN111310706A (en
Inventor
秦永强
高达辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Innovation Wisdom Shanghai Technology Co ltd
Original Assignee
Innovation Wisdom Shanghai Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Innovation Wisdom Shanghai Technology Co ltd filed Critical Innovation Wisdom Shanghai Technology Co ltd
Priority to CN202010128049.0A priority Critical patent/CN111310706B/en
Publication of CN111310706A publication Critical patent/CN111310706A/en
Application granted granted Critical
Publication of CN111310706B publication Critical patent/CN111310706B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application provides a commodity price tag identification method and device, electronic equipment and a computer readable storage medium, wherein the method comprises the following steps: extracting commodity position information and price tag position information from the image to be identified; according to the depth value of each pixel point of the image to be recognized, commodity depth information corresponding to the commodity position information and price tag depth information corresponding to the price tag position information are respectively calculated; performing correlation calculation of the commodity position information and the price tag position information, and correlation calculation of the commodity depth information and the price tag depth information to obtain price tag position information correlated with the commodity position information; and associating and outputting the commodity category information corresponding to the commodity position information and the price information of the price tag position information associated with the commodity position information. According to the technical scheme, depth dimension information can be introduced in the matching process of the commodity position information and the price tag position information, and failure of commodity price tag identification caused by correlation errors is avoided.

Description

Commodity price tag identification method and device, electronic equipment and storage medium
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for identifying a commodity price tag, an electronic device, and a computer-readable storage medium.
Background
The commodity price label (called price label for short) based on the image recognition technology has important value for retail commodity channel monitoring, and is beneficial to brand merchants to obtain the price information of commodities at distribution terminals in time. In a channel display scenario, items are usually associated with price tags one to one, and each price tag close to an item is labeled with the price of the item. When the image of the display scene is identified, the background feature and the character pose of the price tag on the image are very complicated due to flexible and changeable shooting angles.
Currently, price tags and commodities in an input image (generally, a two-dimensional plane image shot by a device such as a mobile phone) can be detected through an image recognition technology, price information in the price tags is recognized through a character recognition technology, finally, the price tag closest to each commodity is found from the two-dimensional plane image, and the price on the price tag is used as the price of the commodity. In this scheme, the product in the two-dimensional plane image may be a product on a far shelf, and the price tag is a price tag near, and the product and the price tag may be associated incorrectly, thereby causing a failure in identification.
Disclosure of Invention
An embodiment of the present application provides a method and an apparatus for identifying a price tag of a product, an electronic device, and a computer-readable storage medium, so as to solve a problem that identification of the price tag of the product fails due to a correlation error between the price tag and the product.
In one aspect, the present application provides a method for identifying a price tag of a commodity, including:
extracting commodity position information and price tag position information from the image to be identified;
according to the depth value of each pixel point of the image to be recognized, commodity depth information corresponding to the commodity position information and price tag depth information corresponding to the price tag position information are respectively calculated;
performing correlation calculation of the commodity position information and the price tag position information, and performing correlation calculation of the commodity depth information and the price tag depth information to obtain price tag position information correlated with the commodity position information;
and associating and outputting the commodity category information corresponding to the commodity position information and the price information of the price tag position information associated with the commodity position information.
In an embodiment, before the associating and outputting the item category information corresponding to the item position information and the price information of the price tag position information associated with the item position information, the method further includes:
cutting a commodity sub-image corresponding to each commodity position information and a price tag sub-image corresponding to each price tag position information on the image to be identified;
and identifying the commodity category information of the commodity subimage and the price information of the price tag subimage to obtain commodity category information corresponding to each commodity position information and price information corresponding to each price tag position information.
In an embodiment, the calculating, according to the depth value of each pixel point of the image to be recognized, the commodity depth information corresponding to the commodity position information and the price tag depth information corresponding to the price tag position information respectively includes:
calculating the depth value of each pixel point of the image to be recognized based on a monocular depth estimation algorithm;
and calculating the commodity depth information of each commodity position information and the price tag depth information of each price tag position information according to the depth value of each pixel point of the image to be identified.
In one embodiment, before the performing the correlation calculation of the commodity position information and the price tag position information, the method further includes:
judging whether the depth value of each pixel point is lower than a preset depth value threshold, if so, dividing the pixel point into a foreground area, and if not, dividing the pixel point into a background area;
judging whether the ratio of the image area corresponding to the price tag position information to the background area reaches a preset ratio threshold value or not;
and if so, filtering the price tag position information.
In an embodiment, the performing correlation calculation between the commodity position information and the price tag position information, and performing correlation calculation between the commodity depth information and the price tag depth information to obtain the price tag position information associated with the commodity position information includes:
screening price tag position information with a plane distance smaller than a preset distance threshold value for each commodity position information, and taking the price tag position information as candidate price tag position information corresponding to the commodity position information;
calculating a depth distance according to commodity depth information of each commodity position information and price tag depth information of candidate price tag position information corresponding to the commodity position information;
and for each commodity position information and each candidate price tag position information corresponding to the commodity position information, calculating a correlation parameter based on the plane distance and the depth distance, and establishing a correlation relation between the candidate price tag position information with the highest correlation parameter and the commodity position information.
In an embodiment, said calculating a correlation parameter based on said planar distance and said depth distance includes:
calculating a plane correlation value based on the plane distance, and calculating a depth correlation value based on the depth distance;
generating the association parameter based on the plane association value and the depth association value.
In one embodiment, after screening out candidate price tag position information for the commodity position information, the method further includes:
determining the relative position of the price tag corresponding to each candidate price tag position information and the commodity corresponding to the commodity position information based on each commodity position information and the corresponding candidate price tag position information;
the calculating of the correlation parameter based on the plane distance and the depth distance includes:
calculating the correlation parameter based on the planar distance, the depth distance, and the relative orientation.
In an embodiment, said calculating said correlation parameter based on said planar distance, said depth distance and said relative orientation comprises:
calculating a plane correlation value based on the plane distance, and calculating a depth correlation value based on the depth distance;
selecting an orientation correlation value corresponding to the relative orientation;
generating a correlation parameter based on the plane correlation value, the depth correlation value and the orientation correlation value.
On the other hand, this application still provides a commodity price tag recognition device, includes:
the extraction module is used for extracting commodity position information and price tag position information from the image to be identified;
the first calculation module is used for respectively calculating commodity depth information corresponding to the commodity position information and price tag depth information corresponding to the price tag position information according to the depth value of each pixel point of the image to be identified;
the second calculation module is used for performing correlation calculation of the commodity position information and the price tag position information, performing correlation calculation of the commodity depth information and the price tag depth information, and obtaining price tag position information correlated with the commodity position information;
and the association module is used for associating and outputting the commodity category information corresponding to the commodity position information and the price information of the price tag position information associated with the commodity position information.
Further, the present application also provides an electronic device, including:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to execute the above item price tag identification method.
In addition, the present application also provides a computer-readable storage medium, which stores a computer program executable by a processor to perform the above-mentioned commodity price tag identification method.
According to the technical scheme, the commodity depth information corresponding to the commodity position information and the price tag depth information corresponding to the price tag position information are calculated, depth dimension information can be introduced in the matching process of the commodity position information and the price tag position information, the association degree of the commodity position information and the corresponding price tag position information is improved, and failure of commodity price tag identification due to association errors is avoided.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly described below.
Fig. 1 is a schematic view of an application scenario of a method for identifying a commodity price tag according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
fig. 3 is a schematic flow chart illustrating a method for identifying a price tag of a commodity according to an embodiment of the present application;
fig. 4 is a block diagram of a product price tag identification apparatus according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
Like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Fig. 1 is a schematic view of an application scenario of a commodity price tag identification method according to an embodiment of the present application. As shown in fig. 1, the application scenario includes a server 30 and a client 20, the server 30 may be a server, a server cluster, or a cloud computing center, and the server 30 may perform a commodity price tag identification service on an image of a display scenario acquired by the client 20. The client 20 may be a smart device such as a camera, a smart phone, and a tablet computer.
As shown in fig. 2, the present embodiment provides an electronic apparatus 1 including: at least one processor 11 and a memory 12, one processor 11 being exemplified in fig. 2. The processor 11 and the memory 12 are connected by a bus 10, and the memory 12 stores instructions executable by the processor 11, and the instructions are executed by the processor 11, so that the electronic device 1 can execute all or part of the flow of the method in the embodiments described below. In an embodiment, the electronic device 1 may be the server 30.
The Memory 12 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically Erasable Programmable Read-Only Memory (EEPROM), erasable Programmable Read-Only Memory (EPROM), programmable Read-Only Memory (PROM), read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk or optical disk.
The present application also provides a computer-readable storage medium storing a computer program executable by the processor 11 to perform the method for identifying price tags for goods provided by the present application.
Referring to fig. 3, a flow chart of a method for identifying price tags of goods according to an embodiment of the present application is shown, and as shown in fig. 3, the method may include the following steps 310 to 340.
Step 310: and extracting commodity position information and price tag position information from the image to be identified.
The image to be recognized is an image of a display scene, which may include one or more merchandise items, one or more price tags.
The commodity position information represents the position of the commodity in the image to be identified, and the price tag position information represents the position of the price tag in the image to be identified.
The commodity position information may be represented by a circumscribed rectangular frame of the commodity pattern, and the price tag position information may be represented by a circumscribed rectangular frame of the price tag pattern. After a two-dimensional coordinate system is established on the image to be recognized, the coordinates (x) of the upper left corner of the circumscribed rectangular frame can be used 1 ，y 1 ) And the coordinates of the lower right corner (x) 2 ，y 2 ) Indicating the position information of the product or the position information of the price tag, and is marked as (x) 1 ，y 1 ，x 2 ，y 2 )。
In an embodiment, the server may extract the commodity position information and the price tag position information in the image to be recognized through the target detection model.
The target detection model may be any one of network models such as YOLO (You Only Look one), fast R-CNN (Fast regional Convolutional Neural Networks), fast R-CNN (Faster regional Convolutional Neural Networks), and the like.
The target detection model can be trained through a first sample image set, the first sample image set comprises a large number of labeled sample images, and labeled labels comprise position information and category information of objects to be identified in the sample images. Here, the category information may be a commodity and a price tag.
For example, the category information corresponding to the article location information may be 1, and the category information corresponding to the tag location information may be 0. The label is represented as (x) 1 ，y 1 ，x 2 ，y 2 1) indicating the coordinates (x) of the position in the image to be recognized at the upper left corner 1 ，y 1 ) And the coordinates of the lower right corner (x) 2 ，y 2 ) The rectangular frame of (2) is commodity position information.
The target detection model is trained to obtain model parameters which can be applied, so that target detection service can be executed.
The server side can input the image to be recognized into the trained target detection model, and the position information of each commodity and the position information of each price tag are recognized through the target detection model.
Step 320: and respectively calculating commodity depth information corresponding to the commodity position information and price tag depth information corresponding to the price tag position information according to the depth value of each pixel point of the image to be recognized. The commodity depth information is used for representing the depth of the commodity in the image to be identified, and the price tag depth information is used for representing the depth of the price tag in the image to be identified.
In an embodiment, the server may calculate a depth value of each pixel point of the image to be recognized based on a monocular depth estimation algorithm. The Monocular Depth Estimation algorithm may be any one of FastDepth (Fast singular Depth Estimation on Embedded Systems, fast Estimation algorithm of Monocular Depth in Embedded Systems), multi-Scale Deep Network (Multi-Scale Depth Network), FCRN (full Convolutional Residual Network), and the like.
In an embodiment, after the image to be recognized is calculated by the server, a gray scale map with the same size as the image to be recognized is obtained, and the gray scale value of each pixel point in the gray scale map represents the depth value of the pixel point at the same position in the image to be recognized. Thus, the depth value may be a numerical value between 0 and 255.
The server side can calculate the commodity depth information of each commodity position information and the price tag depth information of each price tag position information according to the depth value of each pixel point of the image to be recognized.
In an embodiment, the server may calculate an average depth value of depth values of all pixel points corresponding to each piece of product location information, and then use the average depth value as the product depth information corresponding to the product location information.
The server side can calculate the average depth value of the depth values of all the pixel points corresponding to each price tag position information, and then the average depth value is used as the price tag depth information corresponding to the price tag position information.
In another embodiment, the server may select a median of the depth values of all pixel points corresponding to each piece of commodity location information, and then use the median of the depth values as the commodity depth information corresponding to the commodity location information.
The server side can select the median of the depth values of all the pixel points corresponding to each price tag position information, and then the median of the depth values is used as the price tag depth information corresponding to the price tag position information.
Step 330: and performing correlation calculation of the commodity position information and the price tag position information, and performing correlation calculation of the commodity depth information and the price tag depth information to obtain price tag position information correlated with the commodity position information.
In an embodiment, price tag position information with a plane distance smaller than a preset distance threshold may be screened for each commodity position information as candidate price tag position information corresponding to the commodity position information. The distance threshold may be set according to the actual situation of the exhibition scene, and may be 10 centimeters, for example. The planar distance may include a horizontal distance and a vertical distance, and in this case, the distance threshold is divided into a horizontal distance threshold and a vertical distance threshold, and the horizontal distance threshold may be 8 centimeters and the vertical distance threshold may be 10 centimeters.
In one embodiment, the server may determine coordinates (x 3, y 3) of a center point of a rectangular frame indicated by the location information of each commodity; and determines coordinates (x 4, y 4) of the center point of the rectangular frame indicated by each price tag position information. And the server calculates the distance in the horizontal direction and the distance in the vertical direction according to the central point corresponding to the commodity position information and the central point corresponding to the price tag position information.
The server can screen price tag position information with the plane distance smaller than a preset distance threshold value for each commodity position information to serve as candidate price tag position information of the commodity position information.
The product and price tag are not too far apart either in the horizontal or vertical direction. Therefore, for each commodity position information, the server can screen out the price tag position information of which the horizontal direction distance is smaller than the horizontal distance threshold value and the vertical direction distance is smaller than the vertical distance threshold value as the candidate price tag position information corresponding to the commodity position information.
The server side can calculate the depth distance according to the commodity depth information of each commodity position information and the price tag depth information of the candidate price tag position information corresponding to the commodity position information.
For each commodity position information and each candidate price tag position information corresponding to the commodity position information, the server side can calculate association parameters based on the plane distance and the depth distance, and establish an association relation between the candidate price tag position information with the highest association parameters and the commodity position information.
In an embodiment, the server may calculate a plane association value based on the plane distance and calculate a depth association value based on the depth distance. Wherein the plane distance includes a horizontal direction distance and a vertical direction distance, and thus, the plane correlation value includes a horizontal correlation value and a vertical correlation value.
The calculation process of the horizontal correlation value can be expressed by the following formula (1):
Figure BDA0002395114600000101
wherein S is horizon Represents a horizontal correlation value, w 1 Width, w, of a circumscribed rectangular frame corresponding to the commodity position information 2 Width of circumscribed rectangle frame corresponding to candidate price tag position information, d horizon And e represents a natural constant, and represents the horizontal direction distance between the center point of the circumscribed rectangular frame corresponding to the commodity position information and the center point of the circumscribed rectangular frame corresponding to the candidate price tag position information.
The calculation process of the vertical correlation value can be expressed by the following formula (2):
Figure BDA0002395114600000102
wherein S is vertical Represents a vertical correlation value, h 1 Height, h, of a circumscribed rectangular frame corresponding to the commodity position information 2 Height of circumscribed rectangle frame corresponding to candidate price tag position information, d vertical And e represents a natural constant, and the distance in the vertical direction between the center point of the circumscribed rectangular frame corresponding to the commodity position information and the center point of the circumscribed rectangular frame corresponding to the candidate price tag position information is represented.
The calculation process of the depth correlation value can be expressed by the following formula (3):
Figure BDA0002395114600000111
wherein S is depth Indicates a depth-related value, d 1 Product depth information indicating correspondence of product position information, d 2 And e represents a natural constant.
The server side can generate the association parameters based on the plane association value and the depth association value. The calculation process of the correlation parameter can be expressed by the following formula (4):
S＝a*S depth +b*S horizon +c*S vertical (4)
wherein S represents a correlation parameter, S depth Indicates depth correlation value, a indicates pre-depthSet a weight coefficient, S, corresponding to the depth-related value horizon Representing a horizontal correlation value, b representing a preset weight coefficient corresponding to the horizontal correlation value, S vertical Represents a vertical correlation value, and c represents a preset weight coefficient corresponding to the vertical correlation value.
In one embodiment, according to the relative position of the commodity and the price tag in the display scene, the position association parameter S may be pre-allocated to the commodity position information and the candidate price tag position information orient . The candidate price tags and the commodities have approximately eight orientation relations: the candidate price tags are located above, below, to the left, to the right, above left, above right, below left, and below right of the commodity.
The server side can determine the relative position of the price tag corresponding to each candidate price tag position information and the commodity corresponding to the commodity position information based on each commodity position information and the corresponding candidate price tag position information.
Illustratively, the server may indicate the center point (x) of the circumscribed rectangle frame through the commodity position information 5 ，y 5 ) The center point (x) of the circumscribed rectangular frame indicated by the candidate price tag position information 6 ，y 6 ) And determining the relative orientation.
If x 5 Is equal to x 6 、y 5 Is equal to y 6 The two center points coincide.
If x 5 Is equal to x 6 、y 5 Less than y 6 The price tag is arranged above the commodity.
If x 5 Is equal to x 6 、y 5 Greater than y 6 And the price tag is arranged below the commodity.
If x 5 Less than x 6 、y 5 Is equal to y 6 The price tag is to the right of the item.
If x 5 Greater than x 6 、y 5 Is equal to y 6 The price tag is on the left side of the item.
If x 5 Greater than x 6 、y 5 Greater than y 6 The price tag is arranged at the lower left of the commodity.
If x 5 Greater than x 6 、y 5 Less than y 6 And the price tag is arranged at the upper left part of the commodity.
If x 5 Less than x 6 、y 5 Greater than y 6 The price tag is arranged at the lower right part of the commodity.
If x 5 Less than x 6 、y 5 Less than y 6 The price tag is arranged on the upper right of the commodity.
When calculating the above-mentioned association parameters, the server may calculate the association parameters based on the plane distance, the depth distance, and the relative orientation.
In an embodiment, the server may calculate a plane association value based on the plane distance and calculate a depth association value based on the depth distance.
The server may select a bearing association value corresponding to the relative bearing.
For each commodity position information, the server side selects a direction correlation value corresponding to the relative direction for each candidate price tag position information.
For one exhibition scenario, four orientation-related values may be set: lower correlation value S t Upper correlation value S b Left side correlation value S j Right side correlation value S r . Such as: in a display scene of a convenience store, the price tag is generally positioned below the product, but the case where the price tag is positioned on the left or right of the product is not excluded, and therefore, in the case of the convenience store, the lower related value S selected when the price tag is positioned below the product is the lower related value S t Maximum, left associated value S selected when price tag is to the left of the item j A right side correlation value S selected when the price tag is on the right side of the commodity r Next, the upper correlation value S selected when the price tag is above the merchandise b And is minimal.
If the price tag corresponding to the candidate price tag position information is above the commodity corresponding to the commodity position information, the upper correlation value can be selected as the direction correlation value. And so on.
If the price tag corresponding to the candidate price tag position information is positioned at the upper left of the commodity corresponding to the commodity position information, the left correlation value and the upper correlation value can be selected, an average correlation value is calculated, and the average correlation value is used as the orientation correlation value. And so on.
The server side can generate the association parameters based on the plane association value, the depth association value and the azimuth association value. The calculation process of the correlation parameter can be expressed by the following formula (5):
S＝a*S depth +b*S horizon +c*S vertical +m*S orient (5)
wherein S represents a correlation parameter, S depth Representing depth-related values, a representing a preset weight coefficient corresponding to the depth-related values, S horizon Representing a horizontal correlation value, b representing a preset weight coefficient corresponding to the horizontal correlation value, S vertical Representing a vertical correlation value, c representing a preset weight coefficient corresponding to the vertical correlation value, S orient Representing the orientation-related value, and m represents a preset weight coefficient corresponding to the orientation-related value.
After the association parameters of each commodity position information and each candidate price tag position information corresponding to the commodity position information are obtained through calculation, the candidate price tag position information with the highest association parameters and the commodity position information can be associated.
Step 340: and associating and outputting the commodity category information corresponding to the commodity position information and the price information of the price tag position information associated with the commodity position information.
Before step 340 is executed, the server may cut out a commodity sub-image corresponding to each commodity position information and a price tag sub-image corresponding to each price tag position information on the image to be identified.
The commodity sub-image is a partial image containing commodities in the to-be-identified image indicated by the commodity position information, and the price tag sub-image is a partial image containing price tags in the to-be-identified image indicated by the price tag position information.
The service end can identify the commodity category information of the commodity subimage and the price information of the price tag subimage to obtain commodity category information corresponding to each commodity position information and price information corresponding to each price tag position information.
In an embodiment, the service end may identify the commodity category information of the commodity sub-image through the image classification model. The product type information here is specific type information of the product in the actual display scene. Such as: the display scene is a convenience store, and the category information can comprise various types of snacks, daily necessities and the like of each brand.
The image classification model may be any one of CNN (Convolutional Neural Networks), SVM (Support Vector Machine), BPNN (Back Propagation Neural Network), and the like.
And training the image classification model through a second sample image set, wherein the second sample image set comprises a large number of labeled sample images, and the labeled labels are commodity category information to be identified in the sample images.
The image classification model is trained to obtain model parameters which can be applied, so that the image classification business can be executed.
The service end can identify the text information of the price tag image through the text identification model.
The character recognition model may be any one of CTPN (Connectionist Text forward Network), DMPNet (Deep Matching Prior Network), EAST (Efficient and accessible Scene Text detector), and the like.
There may be a variety of textual information in the price tag, such as the name of the item, the model number of the item, promotional information, etc. In view of this, the server may extract price information from the text information according to a price extraction policy. Wherein, the price extraction strategy can be set according to the actual display scene. Such as: the character string only containing the numeric characters can be screened out, then whether characters such as 'price' or 'rah' exist before the character string is checked, and if yes, the character string is indicated as price information. Or, if only the character string containing only the numeric characters is screened from the character information, the character string is the price information.
After the commodity category information and the price information are identified, the server can output the commodity category information corresponding to the commodity position information and the price information of the price tag position information related to the commodity position information in a related mode.
Here, the association output is output by associating the product type information with the price information, and it can be determined that the product type information corresponds to the price information by the association.
In an embodiment, before performing step 330, the server may determine whether the depth value of each pixel point is lower than a preset depth value threshold.
On one hand, if the depth value is lower than the threshold value of the depth value, the server may partition the pixel point into the foreground area.
On the other hand, if the depth value is not lower than the depth value threshold, the server may partition the pixel point into the background area.
In an embodiment, the server may generate a mask (mask) for the foreground region, where the mask is a single-channel image with a value of 0 or 1 and has the same size as the image to be recognized. Each pixel point in the mask represents the area where the pixel point at the same position in the image to be identified is located; if the value of the pixel point in the mask is 1, the corresponding pixel point is in the foreground area, and if the value of the pixel point in the mask is 0, the corresponding pixel point is in the background area.
The server can judge the part of the image area corresponding to the price tag position information in the background area, and the ratio of the image area to the image area reaches a preset ratio threshold. Here, the ratio threshold may be an empirical value, and may be 0.5, for example.
The server can calculate the ratio of the number of the pixels of which the value of the price tag position information in the mask is 0 to the total number of the pixels corresponding to the price tag position information, so that the ratio of the part of the image area corresponding to the price tag position information in the background area to the image area is determined, and whether the ratio reaches a ratio threshold value is judged.
On one hand, if the ratio threshold value is not reached, the price tag position information is in the foreground area, and then correlation calculation of the commodity position information and the price tag position information can be carried out subsequently.
On the other hand, if the ratio threshold is reached, it is indicated that the price tag position information is in the background area, and the service end needs to filter the price tag position information during subsequent correlation calculation.
Through the measure of the embodiment, price tag position information in a background area can be filtered, and subsequent calculation amount is reduced.
Fig. 4 is a block diagram of a product price tag identification apparatus according to an embodiment of the present invention. As shown in fig. 4, the apparatus may include: an extraction module 410, a first calculation module 420, a second calculation module 430, and an association module 440.
An extraction module 410, configured to extract commodity position information and price tag position information from an image to be identified;
a first calculating module 420, configured to calculate, according to a depth value of each pixel point of the image to be recognized, commodity depth information corresponding to the commodity position information and price tag depth information corresponding to the price tag position information respectively;
a second calculating module 430, configured to perform correlation calculation between the commodity position information and the price tag position information, perform correlation calculation between the commodity depth information and the price tag depth information, and obtain price tag position information associated with the commodity position information;
the associating module 440 is configured to associate and output the commodity category information corresponding to the commodity position information and the price information of the price tag position information associated with the commodity position information.
In one embodiment, the apparatus further comprises:
the cutting module is used for cutting a commodity subimage corresponding to each commodity position information and a price tag subimage corresponding to each price tag position information on the image to be identified;
and the identification module is used for identifying the commodity category information of the commodity subimage and the price information of the price tag subimage to obtain commodity category information corresponding to each commodity position information and price information corresponding to each price tag position information.
In an embodiment, the first calculating module 420 is further configured to:
calculating the depth value of each pixel point of the image to be recognized based on a monocular depth estimation algorithm;
and calculating the commodity depth information of each commodity position information and the price tag depth information of each price tag position information according to the depth value of each pixel point of the image to be identified.
In one embodiment, the apparatus further comprises a filtering module for:
judging whether the depth value of each pixel point is lower than a preset depth value threshold, if so, dividing the pixel point into a foreground area, and if not, dividing the pixel point into a background area;
judging whether the ratio of the image area corresponding to the price tag position information to the background area reaches a preset ratio threshold value or not;
and if so, filtering the price tag position information.
In an embodiment, the second calculating module 430 is further configured to:
screening price tag position information with a plane distance smaller than a preset distance threshold value for each commodity position information, and taking the price tag position information as candidate price tag position information corresponding to the commodity position information;
calculating a depth distance according to commodity depth information of each commodity position information and price tag depth information of candidate price tag position information corresponding to the commodity position information;
and for each commodity position information and each candidate price tag position information corresponding to the commodity position information, calculating association parameters based on the plane distance and the depth distance, and establishing an association relation between the candidate price tag position information with the highest association parameter and the commodity position information.
In an embodiment, the second calculating module 430 is further configured to:
calculating a plane correlation value based on the plane distance, and calculating a depth correlation value based on the depth distance;
generating the association parameter based on the plane association value and the depth association value.
In an embodiment, the second calculating module 430 is further configured to:
determining the relative position of the price tag corresponding to each candidate price tag position information and the commodity corresponding to the commodity position information based on each commodity position information and the corresponding candidate price tag position information;
calculating the correlation parameter based on the planar distance, the depth distance, and the relative orientation.
In an embodiment, the second calculating module 430 is further configured to:
calculating a plane correlation value based on the plane distance, and calculating a depth correlation value based on the depth distance;
selecting an orientation correlation value corresponding to the relative orientation;
generating a correlation parameter based on the plane correlation value, the depth correlation value and the orientation correlation value.
The implementation processes of the functions and actions of the modules in the device are specifically described in the implementation processes of the corresponding steps in the commodity price tag identification method, and are not described again here.
In the embodiments provided in the present application, the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.

Claims (9)

1. A method for identifying price tags of commodities is characterized by comprising the following steps:
extracting commodity position information and price tag position information from the image to be identified;
according to the depth value of each pixel point of the image to be recognized, commodity depth information corresponding to the commodity position information and price tag depth information corresponding to the price tag position information are respectively calculated;
screening price tag position information of which the plane distance is smaller than a preset distance threshold value for each commodity position information as candidate price tag position information corresponding to the commodity position information; wherein the plane distance comprises a horizontal direction distance and a vertical direction distance; the distance in the horizontal direction and the distance in the vertical direction are obtained by calculation according to the coordinates of the center point of the rectangular frame indicated by the commodity position information and the coordinates of the center point of the rectangular frame indicated by the price tag position information;
calculating a depth distance according to commodity depth information of each commodity position information and price tag depth information of candidate price tag position information corresponding to the commodity position information;
calculating a plane correlation value based on the plane distance and a depth correlation value based on the depth distance for each commodity position information and each candidate price tag position information corresponding to the commodity position information; generating association parameters based on the plane association value and the depth association value, and establishing an association relation between candidate price tag position information with the highest association parameters and the commodity position information;
and associating and outputting the commodity category information corresponding to the commodity position information and the price information of the price tag position information associated with the commodity position information.
2. The method according to claim 1, wherein before the associating and outputting the item category information corresponding to the item position information and the price information of the price tag position information associated with the item position information, the method further comprises:
cutting a commodity sub-image corresponding to each commodity position information and a price tag sub-image corresponding to each price tag position information on the image to be identified;
and identifying the commodity category information of the commodity subimage and the price information of the price tag subimage to obtain commodity category information corresponding to each commodity position information and price information corresponding to each price tag position information.
3. The method according to claim 1, wherein the calculating, according to the depth value of each pixel point of the image to be recognized, the commodity depth information corresponding to the commodity position information and the price tag depth information corresponding to the price tag position information respectively includes:
calculating the depth value of each pixel point of the image to be identified based on a monocular depth estimation algorithm;
and calculating the commodity depth information of each commodity position information and the price tag depth information of each price tag position information according to the depth value of each pixel point of the image to be recognized.
4. The method of claim 1, wherein prior to said performing the correlation calculation of the merchandise location information and the price tag location information, the method further comprises:
judging whether the depth value of each pixel point is lower than a preset depth value threshold, if so, dividing the pixel point into a foreground area, and if not, dividing the pixel point into a background area;
judging whether the ratio of the image area corresponding to the price tag position information to the background area reaches a preset ratio threshold value or not;
and if so, filtering the price tag position information.
5. The method of claim 1, wherein after screening candidate price tag location information for the merchandise location information, the method further comprises:
determining the relative position of the price tag corresponding to each candidate price tag position information and the commodity corresponding to the commodity position information based on each commodity position information and the corresponding candidate price tag position information;
the calculating of the correlation parameter based on the plane distance and the depth distance includes:
calculating the correlation parameter based on the planar distance, the depth distance, and the relative orientation.
6. The method of claim 5, wherein said calculating the correlation parameter based on the planar distance, the depth distance, and the relative orientation comprises:
calculating a plane correlation value based on the plane distance, and calculating a depth correlation value based on the depth distance;
selecting an orientation correlation value corresponding to the relative orientation;
generating a correlation parameter based on the plane correlation value, the depth correlation value and the orientation correlation value.
7. An article price tag identification device, comprising:
the extraction module is used for extracting commodity position information and price tag position information from the image to be identified;
the first calculation module is used for respectively calculating commodity depth information corresponding to the commodity position information and price tag depth information corresponding to the price tag position information according to the depth value of each pixel point of the image to be identified;
the second calculation module is used for screening price tag position information of which the plane distance is smaller than a preset distance threshold value for each commodity position information as candidate price tag position information corresponding to the commodity position information; wherein the plane distance comprises a horizontal direction distance and a vertical direction distance; the distance in the horizontal direction and the distance in the vertical direction are obtained by calculation according to the coordinates of the center point of the rectangular frame indicated by the commodity position information and the coordinates of the center point of the rectangular frame indicated by the price tag position information; calculating a depth distance according to commodity depth information of each commodity position information and price tag depth information of candidate price tag position information corresponding to the commodity position information; calculating a plane correlation value based on the plane distance and a depth correlation value based on the depth distance for each commodity position information and each candidate price tag position information corresponding to the commodity position information; generating a correlation parameter based on the plane correlation value and the depth correlation value, and establishing a correlation relation between candidate price tag position information with the highest correlation parameter and the commodity position information;
and the association module is used for associating and outputting the commodity category information corresponding to the commodity position information and the price information of the price tag position information associated with the commodity position information.
8. An electronic device, characterized in that the electronic device comprises:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to perform the article price tag identification method of any one of claims 1-6.
9. A computer-readable storage medium, characterized in that the storage medium stores a computer program executable by a processor to perform the method of item price tag identification according to any one of claims 1-6.
CN202010128049.0A 2020-02-28 2020-02-28 Commodity price tag identification method and device, electronic equipment and storage medium Active CN111310706B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010128049.0A CN111310706B (en) 2020-02-28 2020-02-28 Commodity price tag identification method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010128049.0A CN111310706B (en) 2020-02-28 2020-02-28 Commodity price tag identification method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN111310706A CN111310706A (en) 2020-06-19
CN111310706B true CN111310706B (en) 2022-10-21

Family

ID=71147832

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010128049.0A Active CN111310706B (en) 2020-02-28 2020-02-28 Commodity price tag identification method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN111310706B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112215142B (en) * 2020-10-12 2021-08-13 上海汉时信息科技有限公司 Method, device and equipment for detecting goods shelf stock shortage rate based on depth image information
CN113269052A (en) * 2021-04-30 2021-08-17 广州图匠数据科技有限公司 Price tag identification method, terminal and storage device
CN113627411A (en) * 2021-10-14 2021-11-09 广州市玄武无线科技股份有限公司 Super-resolution-based commodity identification and price matching method and system
CN114202761B (en) * 2022-02-16 2022-06-21 广东数源智汇科技有限公司 Information batch extraction method based on picture information clustering
CN117275011B (en) * 2023-10-11 2024-06-14 广州市玄武无线科技股份有限公司 Commodity identification and commodity price tag matching method, system, terminal and medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006337190A (en) * 2005-06-02 2006-12-14 Tokyo Electric Power Co Inc:The Depth value calculation method and depth value calculation device
CN108447061A (en) * 2018-01-31 2018-08-24 深圳市阿西莫夫科技有限公司 Merchandise information processing method, device, computer equipment and storage medium
CN109214348A (en) * 2018-09-19 2019-01-15 北京极智嘉科技有限公司 A kind of obstacle detection method, device, equipment and storage medium
CN109784323A (en) * 2019-01-21 2019-05-21 北京旷视科技有限公司 Method, apparatus, electronic equipment and the computer storage medium of image recognition

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106203227B (en) * 2016-06-28 2018-09-18 无锡威峰科技股份有限公司 The method that positioning refreshing is carried out to electronic price label by graphic code

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006337190A (en) * 2005-06-02 2006-12-14 Tokyo Electric Power Co Inc:The Depth value calculation method and depth value calculation device
CN108447061A (en) * 2018-01-31 2018-08-24 深圳市阿西莫夫科技有限公司 Merchandise information processing method, device, computer equipment and storage medium
CN109214348A (en) * 2018-09-19 2019-01-15 北京极智嘉科技有限公司 A kind of obstacle detection method, device, equipment and storage medium
CN109784323A (en) * 2019-01-21 2019-05-21 北京旷视科技有限公司 Method, apparatus, electronic equipment and the computer storage medium of image recognition

Also Published As

Publication number Publication date
CN111310706A (en) 2020-06-19

Similar Documents

Publication Publication Date Title
CN111310706B (en) Commodity price tag identification method and device, electronic equipment and storage medium
CN108416403B (en) Method, system, equipment and storage medium for automatically associating commodity with label
CN107292248B (en) Commodity management method and system based on image recognition technology
CN108734162B (en) Method, system, equipment and storage medium for identifying target in commodity image
CN111680688B (en) Character recognition method and device, electronic equipment and storage medium
CN111340126A (en) Article identification method and device, computer equipment and storage medium
CN114494892B (en) Goods shelf commodity display information identification method, device, equipment and storage medium
CN110717366A (en) Text information identification method, device, equipment and storage medium
CN110796141A (en) Target detection method and related equipment
US10460191B1 (en) Dynamically optimizing photo capture for multiple subjects
CN112036400B (en) Method for constructing network for target detection and target detection method and system
CN112906794A (en) Target detection method, device, storage medium and terminal
CN113627411A (en) Super-resolution-based commodity identification and price matching method and system
CN113935774A (en) Image processing method, image processing device, electronic equipment and computer storage medium
CN111428743B (en) Commodity identification method, commodity processing device and electronic equipment
CN110991201B (en) Bar code detection method and related device
CN113657370B (en) Character recognition method and related equipment thereof
CN109741380B (en) Textile picture fast matching method and device
CN109035558B (en) Commodity recognition algorithm online learning system for unmanned sales counter
CN117275011B (en) Commodity identification and commodity price tag matching method, system, terminal and medium
CN112150471A (en) Semantic segmentation method and device based on few samples, electronic equipment and storage medium
CN110705363A (en) Commodity specification identification method and device
CN115860026A (en) Bar code detection method and device, bar code detection equipment and readable storage medium
CN115587769A (en) Method and device for detecting commodity out-of-stock state, computer equipment and storage medium
CN113743382A (en) Shelf display detection method, device and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant