CN112215142B - Method, device and equipment for detecting goods shelf stock shortage rate based on depth image information - Google Patents
Method, device and equipment for detecting goods shelf stock shortage rate based on depth image information Download PDFInfo
- Publication number
- CN112215142B CN112215142B CN202011083902.8A CN202011083902A CN112215142B CN 112215142 B CN112215142 B CN 112215142B CN 202011083902 A CN202011083902 A CN 202011083902A CN 112215142 B CN112215142 B CN 112215142B
- Authority
- CN
- China
- Prior art keywords
- shelf
- depth
- commodity
- stock
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
- G06V20/36—Indoor scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
Abstract
The invention relates to a method, a device and equipment for detecting the stock shortage of a goods shelf based on depth image information, wherein the method comprises the following steps: acquiring a shelf depth image and shelf display information of a target shelf; the shelf display information comprises price tag information and shelf information of a plurality of commodities; according to the depth value of each pixel point in the goods shelf depth image, the price tag information and the shelf information, obtaining the depth value of each pixel point in the price tag boundary frame and the depth value of each pixel point in the shelf boundary frame; calculating the depth mean value of pixel points in the price tag bounding box, and recording the depth mean value as a price tag depth value; obtaining an out-of-stock depth threshold value corresponding to the commodity according to the minimum coefficient of the preset out-of-stock depth range corresponding to the commodity and the price tag depth value; and detecting the stock shortage rate of each commodity in the target goods shelf based on the number of the target pixel points in the shed lattice boundary frame corresponding to the commodity and the total number of the pixel points in the shed lattice boundary frame corresponding to the commodity. Compared with the prior art, the method improves the accuracy and efficiency of goods shelf out-of-stock detection.
Description
Technical Field
The invention belongs to the technical field of intelligent business overload, and particularly relates to a method, a device and equipment for detecting the stock shortage rate of a goods shelf based on depth image information.
Background
With the rapid development of artificial intelligence technology and retail economy, intelligent management technology for store shops, supermarkets and convenience stores is continuously emerging. Digital shelves are an important link in intelligent retail technology, and the out-of-stock detection technology for the digital shelves is an important factor influencing the sales volume of commodities.
In the prior art, the goods shelf shortage detection is realized mainly by depending on manual checking and sensing of the weight of goods by a gravity sensor. On one hand, with the expansion of the scale of the business super scale, the manual disk inspection not only consumes time and labor, but also has low accuracy and poor real-time performance; on the other hand, retail commodities are various in variety, and a detection mode based on the gravity sensor is difficult to ensure that different commodities are matched with one or more gravity sensors, so that the price is high and the management is inconvenient.
Disclosure of Invention
The invention provides a method, a device and equipment for detecting the goods shelf shortage rate based on depth image information, which can solve the problems of low accuracy and poor real-time performance of goods shelf shortage detection, and the technical scheme of the invention has the following implementation mode:
in a first aspect, an embodiment of the present application provides a method for detecting a shelf stock shortage rate based on depth image information, including:
acquiring a shelf depth image and shelf display information of a target shelf; the shelf display information comprises price tag information and shelf lattice information of a plurality of commodities in the target shelf, the price tag information comprises positions and boundary frame sizes of price tags corresponding to the commodities in the shelf depth image, and the shelf lattice information comprises positions and boundary frame sizes of shelf lattices corresponding to the commodities in the shelf depth image;
according to the depth value of each pixel point in the goods shelf depth image, the price tag information and the shelf information, obtaining the depth value of each pixel point in the price tag boundary box and the depth value of each pixel point in the shelf boundary box;
calculating the depth mean value of the pixel points in the price tag bounding box according to the depth value of each pixel point in the price tag bounding box, and recording the depth mean value as a price tag depth value;
obtaining an out-of-stock depth threshold value corresponding to the commodity according to the preset out-of-stock depth range minimum coefficient corresponding to the commodity and the price tag depth value corresponding to the commodity;
detecting the stock shortage rate of each commodity in the target shelf based on the number of target pixel points in the shed lattice boundary frame corresponding to the commodity and the total number of pixel points in the shed lattice boundary frame corresponding to the commodity; and the target pixel points are pixel points of which the depth values in the shed lattice bounding boxes corresponding to the commodities are greater than the backorder depth threshold value.
Optionally, the acquiring the shelf depth image and the shelf display information of the target shelf includes:
acquiring a position of a depth camera and a shelf depth image taken by the depth camera at the position;
determining an identification of the target shelf captured by the depth camera based on the position of the depth camera;
and obtaining shelf display information of the target shelf according to the identification of the target shelf.
Optionally, the acquiring the shelf depth image and the shelf display information of the target shelf includes:
acquiring a position of a depth camera and an initial shelf depth image taken by the depth camera at the position;
determining the identifications of a plurality of target shelves shot by the depth camera according to the position of the depth camera;
intercepting the initial shelf depth image according to the position and the size of the target shelf in the initial shelf depth image to obtain a shelf depth image of each target shelf;
and obtaining shelf display information of each target shelf according to the identification of the target shelf.
Optionally, the acquiring the position of the depth camera includes:
acquiring an identifier of the depth camera and map information of the depth camera; the identification of the depth camera in the map information of the depth camera corresponds to the position of the depth camera one by one;
and acquiring the position of the depth camera according to the identifier of the depth camera.
Optionally, the depth camera is mounted on a mobile device, wherein,
the method for acquiring the position of the depth camera comprises the following steps:
acquiring a navigation point where the mobile equipment is located; the navigation point is a preset photographing point of the depth camera in the business surpass;
and acquiring the position of the depth camera according to the position of the navigation point.
Optionally, the obtaining of the out-of-stock depth threshold corresponding to the commodity according to the preset minimum coefficient of the out-of-stock depth range corresponding to the commodity and the price tag depth value corresponding to the commodity includes:
acquiring the accommodating depth of the target shelf;
acquiring the shortage depth range corresponding to the commodity according to the preset shortage depth range minimum coefficient corresponding to the commodity and the accommodating depth;
and acquiring the goods shortage depth threshold corresponding to the goods according to the price tag depth value corresponding to the goods and the goods shortage depth range.
Optionally, before the obtaining of the out-of-stock depth threshold corresponding to the commodity according to the price tag depth value corresponding to the commodity and the out-of-stock depth range, the method further includes:
acquiring the adjacent pixel point of each pixel point in the price tag bounding box and the mean value of the depth values of the adjacent pixel points;
and if the difference value between the depth value of any one pixel point and the mean value of the depth values of the adjacent pixel points meets the judgment condition of a preset noise pixel point, removing the pixel point from the price tag bounding box.
Optionally, the method for detecting the shelf stock out rate based on the depth image information further includes the steps of:
according to the out-of-stock rate of each commodity in the target shelf, obtaining the commodity to be replenished, wherein the out-of-stock rate is greater than a preset replenishment threshold value;
acquiring the position information of the goods to be restocked; the position information of the goods to be replenished comprises the position information of the target shelf in the business surpassing and the position information of the goods to be replenished in the target shelf;
and displaying the goods to be replenished and the position information and the stock shortage rate of the goods to be replenished.
Optionally, the method for detecting the shelf stock out rate based on the depth image information further includes the steps of:
and when the stock shortage rate of the goods to be replenished is not greater than a preset replenishment threshold value, canceling to display the goods to be replenished and the position information and the stock shortage rate of the goods to be replenished.
In a second aspect, an embodiment of the present application provides an apparatus for detecting a shelf stock out rate based on depth image information, including:
the first acquisition unit is used for acquiring a shelf depth image and shelf display information of a target shelf; the shelf display information comprises price tag information and shelf lattice information of each of a plurality of commodities in the target shelf, the price tag information comprises the position and the size of a boundary frame of a price tag corresponding to the commodity in the shelf depth image, and the shelf lattice information comprises the position and the size of a boundary frame of a shelf lattice corresponding to the commodity in the shelf depth image;
the second obtaining unit is used for obtaining the depth value of each pixel point in the price tag boundary frame and the depth value of each pixel point in the shelf grid boundary frame according to the depth value of each pixel point in the shelf depth image, the price tag information and the shelf grid information;
the first operation unit is used for calculating the depth mean value of the pixel points in the price tag bounding box according to the depth value of each pixel point in the price tag bounding box and recording the depth value of the price tag;
the second operation unit is used for obtaining the stock shortage depth threshold value corresponding to the commodity according to the preset stock shortage depth range minimum coefficient corresponding to the commodity and the price tag depth value corresponding to the commodity;
the detection unit is used for detecting the stock shortage of each commodity in the target goods shelf based on the number of target pixel points in the shed boundary frame corresponding to the commodity and the total number of pixel points in the shed boundary frame corresponding to the commodity; and the target pixel points are pixel points of which the depth values in the shed lattice bounding boxes corresponding to the commodities are greater than the backorder depth threshold value.
Optionally, the apparatus for detecting a shelf stock shortage rate based on depth image information further includes:
the display early warning unit is used for displaying and early warning the position information and the stock shortage rate of the goods to be replenished in the target shelf; and the stock shortage rate of the goods to be replenished in the target shelf is greater than a preset replenishment threshold value.
In a third aspect, an embodiment of the present application provides an apparatus, including: a processor, a memory and a computer program stored in the memory and executable on the processor, the processor when executing the computer program implementing the steps of the method of detecting a shelf out-of-stock rate based on depth image information as described in the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps of the method for detecting a shelf out-of-stock rate based on depth image information as described in the first aspect.
Compared with the prior art, the invention has the following beneficial effects: according to the method, the depth value of each pixel point in the price tag bounding box corresponding to each commodity and the depth value of each pixel point in the shelf bounding box are obtained by obtaining the shelf depth image and the shelf display information of the target shelf, the depth mean value of the pixel points in the price tag bounding box is calculated and recorded as the price tag depth value, the shortage depth threshold value corresponding to each commodity is obtained according to the minimum coefficient of the preset shortage depth range of each commodity and the price tag depth value corresponding to the commodity, so that the shortage depth threshold value not only can reflect the shortage characteristics of different commodities, but also can meet the differentiated requirements of different customers on shortage detection, and finally the shortage rate of each commodity in the target shelf is quickly detected by obtaining the number of target pixel points in the shelf bounding box corresponding to each commodity and the total number of pixel points in the shelf bounding box corresponding to the commodity, the method for detecting the out-of-stock rate based on the shelf depth image information not only improves the accuracy of shelf out-of-stock detection, but also effectively improves the real-time performance and efficiency of detection, has low cost and is more beneficial to shelf management.
For a better understanding and implementation, the technical solutions of the present application are described in detail below with reference to the accompanying drawings.
Drawings
FIG. 1 is a schematic flowchart of a method for detecting shelf out-of-stock rate based on depth image information according to an embodiment of the present disclosure;
FIG. 2 is a schematic illustration of a display of a shelf provided in accordance with an embodiment of the present application;
fig. 3 is a schematic flowchart of S101 in a method for detecting a shelf stock out rate based on depth image information according to an embodiment of the present application;
fig. 4 is a schematic flowchart of S101 in a method for detecting a shelf out-of-stock rate based on depth image information according to another embodiment of the present application;
fig. 5 is a schematic flowchart of S104 in a method for detecting a shelf out-of-stock rate based on depth image information according to an embodiment of the present application;
FIG. 6 is a schematic flowchart of a method for detecting a shelf out-of-stock rate based on depth image information according to another embodiment of the present disclosure;
FIG. 7 is a schematic structural diagram of an apparatus for detecting a shelf out-of-stock rate based on depth image information according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an apparatus for detecting a shelf stock out rate based on depth image information according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if/if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
A large number of shelves are placed in a store for goods. The goods shelves are reasonably arranged and the goods condition of the goods shelves is detected in time, so that the storefront management efficiency can be improved, and the turnover is increased.
To this end, referring to fig. 1, a flowchart of a method for detecting a shelf out-of-stock rate based on depth image information according to an embodiment of the present application is shown, where the method includes the following steps:
s101: acquiring a shelf depth image and shelf display information of a target shelf; the shelf display information comprises price tag information and shelf lattice information of a plurality of commodities in the target shelf, the price tag information comprises positions and boundary frame sizes of price tags corresponding to the commodities in the shelf depth image, and the shelf lattice information comprises positions and boundary frame sizes of shelf lattices corresponding to the commodities in the shelf depth image.
The execution main body of the method for detecting the shelf stock shortage based on the depth image information is equipment (hereinafter referred to as detection equipment) for detecting the shelf stock shortage based on the depth image information. The detection device may be an independent device, such as a PC host, a server, or an intelligent interaction device, or may be a component such as a controller, a processor, or a processing chip, which is attached to a hardware component of the independent device.
The detection device acquires a shelf depth image and shelf display information of a target shelf.
The target shelf is a shelf to be subjected to out-of-stock detection, and a plurality of price tags and commodities corresponding to the price tags are placed on the target shelf.
The shelf depth image of the target shelf refers to a shelf depth image captured by a depth camera. In the embodiment of the present application, the type of the depth camera may be a TOF type, an RGB binocular type, or a structured light type, and a specific type thereof is not limited herein.
In a shelf depth image shot by a depth camera, each pixel point corresponds to a depth value and represents the distance from an object point corresponding to the pixel point to the depth camera.
In an alternative embodiment, the depth camera may be integrated in the detection device, and in another alternative embodiment, the depth camera may be a separate device.
Specifically, the detection device establishes data connection with the depth camera, sends a shooting instruction to the depth camera, controls the depth camera to execute a shooting action, and receives a shelf depth image returned by the depth camera.
The following explains shelf display information of a target shelf, and how to acquire the shelf display information of the target shelf:
referring to fig. 2, fig. 2 is a schematic display diagram of a shelf according to an embodiment of the present application, in fig. 2, a plurality of commodities are displayed on the shelf, each commodity corresponds to a price tag a and a shelf B, and the price tag is used for displaying information such as a commodity selling price, a commodity name, and a manufacturer, and is generally fixed at the front side of the shelf and faces a customer. The shed lattices are used for displaying commodities, the sizes of the shed lattices corresponding to each commodity may be different, and only one commodity can be placed in one shed lattice.
In this embodiment of the application, if the shelf needs to be short of stock for detection, the area occupied by the price tag and the shelf lattice corresponding to each commodity in the whole shelf depth image needs to be acquired, that is, the shelf display information of the target shelf needs to be acquired.
The shelf display information comprises price tag information and shelf lattice information of a plurality of commodities in the target shelf, the price tag information comprises positions and boundary frame sizes of price tags corresponding to the commodities in the shelf depth image, and the shelf lattice information comprises positions and boundary frame sizes of shelf lattices corresponding to the commodities in the shelf depth image.
In an alternative embodiment, both the price tag information and the shelf information are stored in the form of a Rect-type parameter, wherein the Rect-type parameter format is (x, y, width, height), wherein x and y represent the position of the price tag image or shelf image corresponding to the item in the shelf depth image, and wherein width and height represent the length and width of the price tag bounding box or shelf bounding box corresponding to the item. Specifically, in the present embodiment, x and y refer to coordinates of the upper left corner of the price tag image in the shelf depth image.
In the embodiment of the present application, shelf display information of each target shelf is stored in advance, and the storage form may be a structured data relation table, or an unstructured xml file, or the like.
Because each target shelf corresponds to one shelf display information, the detection equipment can inquire the corresponding shelf display information according to the identification of the target shelf. Specifically, referring to fig. 3, in order to obtain the shelf depth image and the shelf display information of the target shelf, step S101 includes steps S1011 to S1013:
s1011: a position of a depth camera and a shelf depth image taken by the depth camera at the position are acquired.
The detection device first acquires the position of the depth camera, namely specific position information of the depth camera in the business surpass, and receives a shelf depth image shot by the depth camera at the position.
S1012: determining an identification of the target shelf captured by the depth camera based on the position of the depth camera.
In this embodiment, the positions of the depth cameras are in one-to-one correspondence with the identifications of the target shelves, and thus, the detection device determines the identifications of the target shelves photographed by the depth cameras according to the positions of the depth cameras.
S1013: and obtaining shelf display information of the target shelf according to the identification of the target shelf.
And traversing the shelf display information of all the shelves by the detection equipment according to the identification of the target shelf to obtain the shelf display information of the target shelf.
Specifically, the shelf display information may be stored in the detection device or in an external device that establishes a data connection with the detection device, and when stored in the external device, the detection device transmits the identification of the target shelf to the external device and receives the shelf display information returned from the external device.
In this embodiment, the shooting angle of the depth camera can only cover one target shelf, but in practical situations, due to the difference of the depth camera angles, some shooting angles of the depth camera can cover a plurality of target shelves.
Therefore, to improve the overall efficiency of the out-of-stock detection, please refer to fig. 4, fig. 4 is a schematic flowchart of S101 in a method for detecting a shelf out-of-stock rate based on depth image information according to another embodiment of the present application, and specifically, step S101 includes steps S1014 to S1017:
s1014: a position of a depth camera and an initial shelf depth image taken by the depth camera at the position are acquired.
The detection device first acquires the position of the depth camera, namely specific position information of the depth camera in the business surpass, and receives an initial shelf depth image taken by the depth camera at the position.
Wherein the initial shelf depth image comprises shelf depth images of a plurality of target shelves.
S1015: and determining the identifications of a plurality of target shelves shot by the depth camera according to the position of the depth camera.
The position of the depth camera corresponds to the identification of the target shelf, and according to the position of the depth camera, the shooting visual angle of the depth camera can be determined, so that the identification of a plurality of target shelves shot by the depth camera is determined.
S1016: and intercepting the initial shelf depth image according to the position and the size of the target shelf in the initial shelf depth image to obtain a shelf depth image of each target shelf.
And performing image recognition on the initial shelf depth image, determining the position and the size of the image of each target shelf in the initial shelf depth image, and intercepting the initial shelf depth image according to the position and the size to obtain the shelf depth image of each target shelf.
S1017: and obtaining shelf display information of each target shelf according to the identification of the target shelf.
In this embodiment, by capturing the initial shelf depth image, the shelf depth image and the shelf display information of more target shelves can be acquired while using fewer depth cameras, thereby improving the detection efficiency.
In steps S1011 to S1013 or steps S1014 to S1017, the position of the shelf depth camera is required. When the actual lack of goods is detected, the depth camera may be fixed in the business surpass, or may be mounted on the mobile device, and the following explains the process of acquiring the depth camera under these two conditions:
(1) if the depth camera is fixed in the business surpass, the detection equipment acquires the identifier of the depth camera and the map information of the depth camera; the identification of the depth camera in the map information of the depth camera corresponds to the position of the depth camera one by one. And acquiring the position of the depth camera according to the identification of the depth camera.
(2) If the depth camera is mounted on the mobile device, the depth camera moves accordingly due to the movement of the mobile device. Wherein, the mobile equipment can be a robot or the like.
The mobile equipment moves in the business surpass according to a preset moving track, and a plurality of navigation points in the moving track patrol in the business surpass according to the positions of the navigation points and the sequence of the navigation points.
In this process, if the detection device wants to acquire the position of the depth camera, the detection device acquires the navigation point where the mobile device is located. In the embodiment of the application, the navigation point is also a preset photographing point of the depth camera in the business surpass.
And then, the detection equipment acquires the position of the depth camera according to the position of the navigation point.
S102: and acquiring the depth value of each pixel point in the price tag boundary frame and the depth value of each pixel point in the shelf grid boundary frame according to the depth value of each pixel point in the shelf depth image, the price tag information and the shelf grid information.
The price tag information comprises the position and the size of the boundary frame of the price tag corresponding to the commodity in the shelf depth image, and the shelf information comprises the position and the size of the boundary frame of the shelf corresponding to the commodity in the shelf depth image, so that the detection equipment can obtain the price tag boundary frame and the shelf boundary frame corresponding to the commodity from the shelf depth image according to the price tag information and the shelf information, and then obtain the depth value of each pixel point in the price tag boundary frame and the depth value of each pixel point in the shelf boundary frame according to the depth value of each pixel point in the shelf depth image.
S103: and calculating the depth mean value of the pixel points in the price tag bounding box according to the depth value of each pixel point in the price tag bounding box, and recording the depth mean value as the price tag depth value.
The detection equipment obtains the depth value of each pixel point in the price tag bounding box and the number of the pixel points in the price tag bounding box, calculates the depth mean value of the pixel points in the price tag bounding box, and records the depth mean value as the price tag depth value.
The price tag depth value can reflect the distance between the price tag corresponding to the commodity and the depth camera.
S104: and obtaining the stock shortage depth threshold value corresponding to the commodity according to the preset minimum coefficient of the stock shortage depth range corresponding to the commodity and the price tag depth value corresponding to the commodity.
The shapes and sizes of the outer packages of different commodities are different, for example, for the commodities 1 and 2, the depth of the compartment actually occupied by the commodity 1 is 10cm, and the depth of the compartment actually occupied by the commodity 2 is 50 cm. If the goods 1 are short of goods, the goods 1 needs to be differentiated, for example, because the depth of the goods 1 actually occupying the grids is 10cm, if the positions 10cm away from the grids are all empty, the goods 1 can be determined to be short of goods, but because the depth of the goods 2 actually occupying the grids is 50cm, if the positions 10cm away from the grids are all empty, the goods are not necessarily short of goods, and the goods may be only misplaced in the placing process.
Based on the above reasons, considering the characteristics of different commodities and the requirements of customers, in the embodiment of the present application, the minimum coefficient of the out-of-stock depth range corresponding to the commodity may be preset.
Specifically, the detection device may obtain an accommodation depth1 of the target shelf and a depth2 of a shelf actually occupied by different commodities, preset a stock shortage depth range minimum coefficient of the commodities according to a ratio of depth2 to depth1, and store each commodity and the stock shortage depth range minimum coefficient corresponding to each commodity in the detection device.
Each product referred to herein is a product corresponding to only one price label, and does not refer to a product having the same name.
In order to obtain the shortage depth threshold corresponding to each commodity and improve the accuracy of detecting the shortage rate of the shelves, referring to fig. 5, step S104 includes steps S1041 to S1043, which are as follows:
s1041: and acquiring the accommodating depth of the target shelf.
The accommodating depth of the target shelf refers to the straight-line distance between the price tag of the shelf and the back plate of the shelf.
S1042: and acquiring the stock shortage depth range corresponding to the commodity according to the preset stock shortage depth range minimum coefficient corresponding to the commodity and the accommodating depth.
And the detection equipment acquires the stock shortage depth range corresponding to the commodity according to the preset stock shortage depth range minimum coefficient corresponding to the commodity and the accommodating depth. For example, the minimum coefficient of the preset stock out depth range corresponding to the commodity 1 is 0.1, the accommodating depth is 1m, and then the stock out depth range corresponding to the commodity 1 is 0.1m to 1 m.
S1043: and acquiring the goods shortage depth threshold corresponding to the goods according to the price tag depth value corresponding to the goods and the goods shortage depth range.
And the detection equipment acquires the goods shortage depth threshold corresponding to the goods according to the price tag depth value corresponding to the goods and the goods shortage depth range. For example: the out-of-stock depth range corresponding to the commodity 1 is 0.1m to 1m, the mean value of the depth values of all pixel points in the price tag image of the commodity 1 is 0.8m, and the out-of-stock depth threshold value corresponding to the commodity 1 acquired by the detection device is 0.9 m.
It should be noted that the above example is only one example provided, and the specific values and depth units thereof have no limiting effect.
In an optional embodiment, since the depth values of all the pixel points in the price tag bounding box obtained by the depth camera may be affected by the conditions of light, sight shielding, and the like, the depth values may have noise, and in order to accurately obtain the out-of-stock depth threshold, before executing step S1033, the detection device may perform denoising processing on all the pixel points in the price tag bounding box, specifically:
and the detection equipment acquires the adjacent pixel point of each pixel point in the price tag boundary frame and the mean value of the depth values of the adjacent pixel points. In this embodiment, the neighboring pixel point may be a pixel point directly adjacent to the pixel point, and in other optional embodiments, the neighboring pixel point may also be a pixel point whose center is the pixel point and whose distance from the pixel point does not exceed a preset distance.
And if the difference value between the depth value of any one pixel point and the mean value of the depth values of the adjacent pixel points meets the judgment condition of a preset noise pixel point, removing the pixel point from the price tag bounding box.
S105: detecting the stock shortage rate of each commodity in the target shelf based on the number of target pixel points in the shed lattice boundary frame corresponding to the commodity and the total number of pixel points in the shed lattice boundary frame corresponding to the commodity; and the target pixel points are pixel points of which the depth values in the shed lattice bounding boxes corresponding to the commodities are greater than the backorder depth threshold value.
The target pixel points are pixel points of which the depth values in the shed lattice bounding boxes corresponding to the commodities are larger than the backorder depth threshold value.
And detecting the stock shortage rate of each commodity in the target shelf by the detection equipment based on the number of the target pixel points and the total number of the pixel points in the shed lattice boundary frame corresponding to the commodity.
Specifically, the total number of pixels in the grid bounding box corresponding to a certain commodity in the target shelf is amount1, the number of target pixels in the grid bounding box corresponding to the commodity is amount2, and the stock shortage rate of the commodity in the target shelf, which is obtained by the detection device, is amount2/amount 1.
In this embodiment, it should be further noted that, when the detection device executes the method for detecting the stock shortage of the shelf based on the depth image information to obtain the stock shortage of each commodity in the target shelf, the stock shortage of each commodity in the target shelf may be obtained step by step in a loop traversal manner, or the stock shortage of a plurality of commodities in the target shelf may be obtained simultaneously in a parallel calculation manner, and a specific execution manner in the detection device is not limited herein.
According to the method, the depth value of each pixel point in the price tag bounding box corresponding to each commodity and the depth value of each pixel point in the shelf bounding box are obtained by obtaining the shelf depth image and the shelf display information of the target shelf, the depth mean value of the pixel points in the price tag bounding box is calculated and recorded as the price tag depth value, the shortage depth threshold value corresponding to each commodity is obtained according to the minimum coefficient of the preset shortage depth range of each commodity and the price tag depth value corresponding to the commodity, so that the shortage depth threshold value not only can reflect the shortage characteristics of different commodities, but also can meet the differentiated requirements of different customers on shortage detection, and finally the shortage rate of each commodity in the target shelf is quickly detected by obtaining the number of target pixel points in the shelf bounding box corresponding to each commodity and the total number of pixel points in the shelf bounding box corresponding to the commodity, the method for detecting the out-of-stock rate based on the shelf depth image information not only improves the accuracy of shelf out-of-stock detection, but also effectively improves the real-time performance and efficiency of detection, has low cost and is more beneficial to shelf management.
To improve the replenishment efficiency of the commodity and achieve more efficient management of the shelves, please refer to fig. 6, which is a flowchart illustrating a method for detecting the stock shortage of the shelves based on depth image information according to another embodiment of the present application, the method includes steps S201 to S208, where steps S201 to S205 are the same as steps S101 to S105, and specifically as follows:
s201: acquiring a shelf depth image and shelf display information of a target shelf; the shelf display information comprises price tag information and shelf lattice information of a plurality of commodities in the target shelf, the price tag information comprises positions and boundary frame sizes of price tags corresponding to the commodities in the shelf depth image, and the shelf lattice information comprises positions and boundary frame sizes of shelf lattices corresponding to the commodities in the shelf depth image.
S202: and acquiring the depth value of each pixel point in the price tag boundary frame and the depth value of each pixel point in the shelf grid boundary frame according to the depth value of each pixel point in the shelf depth image, the price tag information and the shelf grid information.
S203: and calculating the depth mean value of the pixel points in the price tag bounding box according to the depth value of each pixel point in the price tag bounding box, and recording the depth mean value as the price tag depth value.
S204: and obtaining the stock shortage depth threshold value corresponding to the commodity according to the preset minimum coefficient of the stock shortage depth range corresponding to the commodity and the price tag depth value corresponding to the commodity.
S205: detecting the stock shortage rate of each commodity in the target shelf based on the number of target pixel points in the shed lattice boundary frame corresponding to the commodity and the total number of pixel points in the shed lattice boundary frame corresponding to the commodity; and the target pixel points are pixel points of which the depth values in the shed lattice bounding boxes corresponding to the commodities are greater than the backorder depth threshold value.
S206: and obtaining the commodities to be replenished, of which the shortage rate is greater than a preset replenishment threshold value, according to the shortage rate of each commodity in the target shelf.
And presetting a replenishment threshold value in the detection equipment, and when the goods taking rate of the goods is greater than the preset replenishment threshold value, indicating that the goods needs to be replenished.
Specifically, the detection device obtains the stock shortage rate of each commodity in the target shelf in real time, and obtains the commodity to be replenished, wherein the stock shortage rate is larger than a preset replenishment threshold value.
S207: acquiring the position information of the goods to be restocked; the position information of the goods to be restocked comprises position information of the target shelf in the business surpassing area and position information of the goods to be restocked in the target shelf.
The position information of the target shelf in the business surpassing can be acquired by the detection equipment through map information of the shelf based on the identification of the target shelf. In the embodiment of the present application, the map information of the shelves includes the position or the floor and the position of each shelf.
The position information of the goods to be replenished in the target shelf can be acquired by the detection equipment based on the actual position of the grids of each goods in the target shelf and the grids corresponding to the goods to be replenished. In the embodiment of the application, the actual position of the grid represents that the goods are on the rows and columns of the shelf.
S208: and displaying the goods to be replenished and the position information and the stock shortage rate of the goods to be replenished.
And the detection equipment displays the goods to be replenished and the position information and the stock shortage rate of the goods to be replenished.
Specifically, in an optional embodiment, the detection device is provided with a display screen, and the detection device displays the goods to be restocked and the position information and the stock shortage rate of the goods to be restocked through the display screen. In another optional embodiment, the detection device establishes data connection with an independent display device, and displays the goods to be replenished and the position information and the stock shortage rate of the goods to be replenished through the independent display device. The independent display device may be an electronic large screen, a touch display screen, or a projection device, which is not limited herein.
After the goods to be replenished, the position information and the stock shortage rate of the goods to be replenished are displayed, the staff can know the commodity sales dynamics and the condition of the goods to be replenished in real time, so that the corresponding replenishment operation can be adopted in time.
In an optional embodiment, after the staff or the robot completes commodity replenishment, the detection device judges whether the stock shortage rate of the commodity to be replenished is not greater than a preset replenishment threshold value, and if so, the position information and the stock shortage rate of the commodity to be replenished and the commodity to be replenished are cancelled and displayed, so that the operation of the staff is reduced, and the checking is more convenient.
In another optional embodiment, the detection device may also remind the staff of the current goods to be restocked, the position information of the goods to be restocked, and the out-of-stock rate through a voice broadcaster, an alarm bell, or other devices.
Fig. 7 is a schematic structural diagram of an apparatus for detecting a shelf out-of-stock rate based on depth image information according to an embodiment of the present disclosure. The apparatus may be implemented as all or part of a device for detecting shelf out-of-stock based on depth image information by software, hardware, or a combination of both. The device 7 comprises: a first acquisition unit 71, a second acquisition unit 72, a first arithmetic unit 73, a second arithmetic unit 74, and a detection unit 75.
A first acquiring unit 71 configured to acquire a shelf depth image and shelf display information of a target shelf; the shelf display information comprises price tag information and shelf lattice information of each of a plurality of commodities in the target shelf, the price tag information comprises the position and the size of a boundary frame of a price tag corresponding to the commodity in the shelf depth image, and the shelf lattice information comprises the position and the size of a boundary frame of a shelf lattice corresponding to the commodity in the shelf depth image;
the second obtaining unit 72 is configured to obtain a depth value of each pixel point in the price tag bounding box and a depth value of each pixel point in the shelf grid bounding box according to the depth value of each pixel point in the shelf depth image, the price tag information, and the shelf grid information;
the first operation unit 73 is configured to calculate a depth mean value of the pixels in the price tag bounding box according to the depth value of each pixel in the price tag bounding box, and record the depth mean value as a price tag depth value;
the second operation unit 74 is configured to obtain an out-of-stock depth threshold corresponding to the commodity according to the preset minimum coefficient of the out-of-stock depth range corresponding to the commodity and the price tag depth value corresponding to the commodity;
the detecting unit 75 is configured to detect the stock shortage of each commodity in the target shelf based on the number of target pixel points in the shelf boundary frame corresponding to the commodity and the total number of pixel points in the shelf boundary frame corresponding to the commodity; and the target pixel points are pixel points of which the depth values in the shed lattice bounding boxes corresponding to the commodities are greater than the backorder depth threshold value.
According to the method, the depth value of each pixel point in the price tag bounding box corresponding to each commodity and the depth value of each pixel point in the shelf bounding box are obtained by obtaining the shelf depth image and the shelf display information of the target shelf, the depth mean value of the pixel points in the price tag bounding box is calculated and recorded as the price tag depth value, the shortage depth threshold value corresponding to each commodity is obtained according to the minimum coefficient of the preset shortage depth range of each commodity and the price tag depth value corresponding to the commodity, so that the shortage depth threshold value not only can reflect the shortage characteristics of different commodities, but also can meet the differentiated requirements of different customers on shortage detection, and finally the shortage rate of each commodity in the target shelf is quickly detected by obtaining the number of target pixel points in the shelf bounding box corresponding to each commodity and the total number of pixel points in the shelf bounding box corresponding to the commodity, the method for detecting the out-of-stock rate based on the shelf depth image information not only improves the accuracy of shelf out-of-stock detection, but also effectively improves the real-time performance and efficiency of detection, has low cost and is more beneficial to shelf management.
Optionally, the apparatus 7 comprises:
the display early warning unit is used for displaying and early warning the position information and the stock shortage rate of the goods to be replenished in the target shelf; and the stock shortage rate of the goods to be replenished in the target shelf is greater than a preset replenishment threshold value.
Optionally, the first obtaining unit 71 includes:
a third acquisition unit for acquiring a position of a depth camera and a shelf depth image taken by the depth camera at the position;
the first identification acquisition unit is used for determining the identification of the target shelf shot by the depth camera according to the position of the depth camera;
and the first display information acquisition unit is used for acquiring the shelf display information of the target shelf according to the identification of the target shelf.
Optionally, the first obtaining unit 71 includes:
a fourth acquisition unit for acquiring a position of a depth camera and an initial shelf depth image taken by the depth camera at the position;
the second identification acquisition unit is used for determining the identifications of a plurality of target shelves shot by the depth camera according to the position of the depth camera;
the first image acquisition unit is used for intercepting the initial shelf depth image according to the position and the size of the target shelf in the initial shelf depth image to obtain a shelf depth image of each target shelf;
and the second display information acquisition unit is used for acquiring shelf display information of each target shelf according to the identification of the target shelf.
Optionally, the third obtaining unit includes:
an identification and map acquisition unit for acquiring an identification of the depth camera and map information of the depth camera; the identification of the depth camera in the map information of the depth camera corresponds to the position of the depth camera one by one;
and the first position acquisition unit is used for acquiring the position of the depth camera according to the identifier of the depth camera.
The depth camera is mounted on the mobile device, and the third acquiring unit includes:
a navigation point obtaining unit, configured to obtain a navigation point where the mobile device is located; the navigation point is a preset photographing point of the depth camera in the business surpass;
and the second position acquisition unit is used for acquiring the position of the depth camera according to the position of the navigation point.
Optionally, the second operation unit 74 includes:
an accommodation depth acquiring unit for acquiring an accommodation depth of the target shelf;
the out-of-stock depth range acquiring unit is used for acquiring an out-of-stock depth range corresponding to the commodity according to a preset minimum coefficient of the out-of-stock depth range corresponding to the commodity and the accommodating depth;
and the second operation unit is used for acquiring the stock shortage depth threshold value corresponding to the commodity according to the price tag depth value corresponding to the commodity and the stock shortage depth range.
Optionally, the second operation unit 74 further includes:
the noise reduction operation unit is used for acquiring the adjacent pixel point of each pixel point in the price tag boundary frame and the mean value of the depth values of the adjacent pixel points;
and the pixel point removing unit is used for removing the pixel point from the price tag bounding box if the difference value between the depth value of any one pixel point and the mean value of the depth values of the adjacent pixel points meets the preset noise pixel point judgment condition.
Optionally, the apparatus 7 further includes:
and the cancellation display unit is used for canceling and displaying the goods to be replenished and the position information and the stock shortage rate of the goods to be replenished when the stock shortage rate of the goods to be replenished is not greater than a preset stock replenishment threshold value.
Fig. 8 is a schematic structural diagram of an apparatus for detecting a shelf stock out rate based on depth image information according to an embodiment of the present disclosure. As shown in fig. 8, the apparatus 8 for detecting a shelf out-of-stock rate based on depth image information may include: a processor 80, a memory 81 and a computer program 82 stored in said memory 81 and executable on said processor 80, such as: a program for detecting the stock shortage of the shelf based on the depth image information; the processor 80, when executing the computer program 82, implements the steps in the above-described method embodiments, such as the steps S101 to S105 shown in fig. 1. Alternatively, the processor 80, when executing the computer program 82, implements the functions of the modules/units in the above-described device embodiments, such as the functions of the modules 71 to 75 shown in fig. 7.
The processor 80 may include one or more processing cores, among others. The processor 80 is connected to various parts in the control device 8 by various interfaces and lines, and executes various functions of the control device 8 and processes data by operating or executing instructions, programs, code sets or instruction sets stored in the memory 81 and calling data in the memory 81, and optionally, the processor 80 may be implemented in at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), Programmable Logic Array (PLA). The processor 80 may integrate one or a combination of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing contents required to be displayed by the touch display screen; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 80, but may be implemented by a single chip.
The Memory 81 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). Optionally, the memory 81 includes a non-transitory computer-readable medium. The memory 81 may be used to store instructions, programs, code sets or instruction sets. The memory 81 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions, etc.), instructions for implementing the above-mentioned method embodiments, and the like; the storage data area may store data and the like referred to in the above respective method embodiments. The memory 81 may optionally be at least one memory device located remotely from the processor 80.
An embodiment of the present application further provides a computer storage medium, where the computer storage medium may store a plurality of instructions, where the instructions are suitable for being loaded by a processor and executed by the method steps in the embodiments shown in fig. 1 and fig. 3 to fig. 6, and a specific execution process may refer to specific descriptions of the embodiments shown in fig. 1 and fig. 3 to fig. 6, which is not described herein again.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc.
The present invention is not limited to the above-described embodiments, and various modifications and variations of the present invention are intended to be included within the scope of the claims and the equivalent technology of the present invention if they do not depart from the spirit and scope of the present invention.
Claims (12)
1. A method for detecting the stock shortage rate of a goods shelf based on depth image information is characterized by comprising the following steps:
acquiring a shelf depth image and shelf display information of a target shelf; the shelf display information comprises price tag information and shelf lattice information of a plurality of commodities in the target shelf, the price tag information comprises positions and boundary frame sizes of price tags corresponding to the commodities in the shelf depth image, and the shelf lattice information comprises positions and boundary frame sizes of shelf lattices corresponding to the commodities in the shelf depth image;
according to the depth value of each pixel point in the goods shelf depth image, the price tag information and the shelf information, obtaining the depth value of each pixel point in the price tag boundary box and the depth value of each pixel point in the shelf boundary box;
calculating the depth mean value of the pixel points in the price tag bounding box according to the depth value of each pixel point in the price tag bounding box, and recording the depth mean value as a price tag depth value;
obtaining an out-of-stock depth threshold value corresponding to the commodity according to the preset out-of-stock depth range minimum coefficient corresponding to the commodity and the price tag depth value corresponding to the commodity;
detecting the stock shortage rate of each commodity in the target shelf based on the number of target pixel points in the shed lattice boundary frame corresponding to the commodity and the total number of pixel points in the shed lattice boundary frame corresponding to the commodity; the target pixel points are pixel points of which the depth values in the shed lattice bounding boxes corresponding to the commodities are greater than the backorder depth threshold value;
wherein the obtaining of the out-of-stock depth threshold value corresponding to the commodity according to the preset out-of-stock depth range minimum coefficient corresponding to the commodity and the price tag depth value corresponding to the commodity comprises,
acquiring the accommodating depth of the target shelf;
acquiring the shortage depth range corresponding to the commodity according to the preset shortage depth range minimum coefficient corresponding to the commodity and the accommodating depth;
and acquiring the goods shortage depth threshold corresponding to the goods according to the price tag depth value corresponding to the goods and the goods shortage depth range.
2. The method for detecting the shelf out-of-stock rate based on depth image information as claimed in claim 1, wherein the obtaining of the shelf depth image and the shelf display information of the target shelf comprises the steps of:
acquiring a position of a depth camera and a shelf depth image taken by the depth camera at the position;
determining an identification of the target shelf captured by the depth camera based on the position of the depth camera;
and obtaining shelf display information of the target shelf according to the identification of the target shelf.
3. The method for detecting the shelf out-of-stock rate based on depth image information as claimed in claim 1, wherein the obtaining of the shelf depth image and the shelf display information of the target shelf comprises the steps of:
acquiring a position of a depth camera and an initial shelf depth image taken by the depth camera at the position;
determining the identifications of a plurality of target shelves shot by the depth camera according to the position of the depth camera;
intercepting the initial shelf depth image according to the position and the size of the target shelf in the initial shelf depth image to obtain a shelf depth image of each target shelf;
and obtaining shelf display information of each target shelf according to the identification of the target shelf.
4. The method for detecting the shelf out-of-stock rate based on the depth image information as claimed in claim 2 or 3, wherein the obtaining the position of the depth camera comprises the steps of:
acquiring an identifier of the depth camera and map information of the depth camera; the identification of the depth camera in the map information of the depth camera corresponds to the position of the depth camera one by one;
and acquiring the position of the depth camera according to the identifier of the depth camera.
5. The method for detecting a shelf out-of-stock rate based on depth image information as claimed in claim 2 or 3, wherein the depth camera is mounted on a mobile device,
the method for acquiring the position of the depth camera comprises the following steps:
acquiring a navigation point where the mobile equipment is located; the navigation point is a preset photographing point of the depth camera in the business surpass;
and acquiring the position of the depth camera according to the position of the navigation point.
6. The method for detecting the shelf out-of-stock rate based on the depth image information as claimed in claim 1, wherein before the out-of-stock depth threshold corresponding to the commodity is obtained according to the price tag depth value corresponding to the commodity and the out-of-stock depth range, the method further comprises:
acquiring the adjacent pixel point of each pixel point in the price tag bounding box and the mean value of the depth values of the adjacent pixel points;
and if the difference value between the depth value of any one pixel point and the mean value of the depth values of the adjacent pixel points meets the judgment condition of a preset noise pixel point, removing the pixel point from the price tag bounding box.
7. The method for detecting shelf out-of-stock rate based on depth image information as claimed in any one of claims 1 to 3, further comprising the steps of:
according to the out-of-stock rate of each commodity in the target shelf, obtaining the commodity to be replenished, wherein the out-of-stock rate is greater than a preset replenishment threshold value;
acquiring the position information of the goods to be restocked; the position information of the goods to be replenished comprises the position information of the target shelf in the business surpassing and the position information of the goods to be replenished in the target shelf;
and displaying the goods to be replenished and the position information and the stock shortage rate of the goods to be replenished.
8. The method for detecting shelf out-of-stock rate based on depth image information as claimed in claim 7, further comprising the steps of:
and when the stock shortage rate of the goods to be replenished is not greater than a preset replenishment threshold value, canceling to display the goods to be replenished and the position information and the stock shortage rate of the goods to be replenished.
9. An apparatus for detecting a shelf out-of-stock rate based on depth image information, comprising:
the first acquisition unit is used for acquiring a shelf depth image and shelf display information of a target shelf; the shelf display information comprises price tag information and shelf lattice information of each of a plurality of commodities in the target shelf, the price tag information comprises the position and the size of a boundary frame of a price tag corresponding to the commodity in the shelf depth image, and the shelf lattice information comprises the position and the size of a boundary frame of a shelf lattice corresponding to the commodity in the shelf depth image;
the second obtaining unit is used for obtaining the depth value of each pixel point in the price tag boundary frame and the depth value of each pixel point in the shelf grid boundary frame according to the depth value of each pixel point in the shelf depth image, the price tag information and the shelf grid information;
the first operation unit is used for calculating the depth mean value of the pixel points in the price tag bounding box according to the depth value of each pixel point in the price tag bounding box and recording the depth value of the price tag;
the second operation unit is used for obtaining the stock shortage depth threshold value corresponding to the commodity according to the preset stock shortage depth range minimum coefficient corresponding to the commodity and the price tag depth value corresponding to the commodity, and comprises the step of obtaining the accommodating depth of the target shelf; acquiring the shortage depth range corresponding to the commodity according to the preset shortage depth range minimum coefficient corresponding to the commodity and the accommodating depth; acquiring an out-of-stock depth threshold corresponding to the commodity according to the price tag depth value corresponding to the commodity and the out-of-stock depth range;
the detection unit is used for detecting the stock shortage of each commodity in the target goods shelf based on the number of target pixel points in the shed boundary frame corresponding to the commodity and the total number of pixel points in the shed boundary frame corresponding to the commodity; and the target pixel points are pixel points of which the depth values in the shed lattice bounding boxes corresponding to the commodities are greater than the backorder depth threshold value.
10. The apparatus for detecting a shelf out-of-stock rate based on depth image information as claimed in claim 9, further comprising:
the display early warning unit is used for displaying and early warning the position information and the stock shortage rate of the goods to be replenished in the target shelf; and the stock shortage rate of the goods to be replenished in the target shelf is greater than a preset replenishment threshold value.
11. An apparatus for detecting a shelf out-of-stock rate based on depth image information, comprising: processor, memory and a computer program stored in the memory and executable on the processor, characterized in that the steps of the method according to any of claims 1 to 8 are implemented when the processor executes the computer program.
12. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011083902.8A CN112215142B (en) | 2020-10-12 | 2020-10-12 | Method, device and equipment for detecting goods shelf stock shortage rate based on depth image information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011083902.8A CN112215142B (en) | 2020-10-12 | 2020-10-12 | Method, device and equipment for detecting goods shelf stock shortage rate based on depth image information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112215142A CN112215142A (en) | 2021-01-12 |
| CN112215142B true CN112215142B (en) | 2021-08-13 |
Family
ID=74053244
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011083902.8A Active CN112215142B (en) | 2020-10-12 | 2020-10-12 | Method, device and equipment for detecting goods shelf stock shortage rate based on depth image information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112215142B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112883955B (en) * | 2021-03-10 | 2024-02-02 | 洛伦兹（北京）科技有限公司 | Shelf layout detection method, device and computer readable storage medium |
| CN113674336A (en) * | 2021-07-27 | 2021-11-19 | 浙江大华技术股份有限公司 | Shelf empty information determining method, computer equipment and storage device |
| CN116935198B (en) * | 2023-09-12 | 2023-11-24 | 汉朔科技股份有限公司 | Method and device for detecting shortage of goods on goods shelf and robot |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109328359A (en) * | 2016-06-30 | 2019-02-12 | 波萨诺瓦机器人知识产权有限公司 | Multi-camera system for inventory tracking |
| CN110533028A (en) * | 2019-07-31 | 2019-12-03 | 北京三快在线科技有限公司 | Detection method, device, electronic equipment and the storage medium of commodity display state |
| CN111310706A (en) * | 2020-02-28 | 2020-06-19 | 创新奇智(上海)科技有限公司 | Commodity price tag identification method and device, electronic equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9811754B2 (en) * | 2014-12-10 | 2017-11-07 | Ricoh Co., Ltd. | Realogram scene analysis of images: shelf and label finding |
| JP2019200697A (en) * | 2018-05-18 | 2019-11-21 | 東芝テック株式会社 | Shelf management system and program |
| CN108833857B (en) * | 2018-07-10 | 2021-01-15 | 京东方科技集团股份有限公司 | Goods shelf monitoring method and system and goods shelf |
| CN109726759B (en) * | 2018-12-28 | 2021-08-17 | 北京旷视科技有限公司 | Unmanned vending method, device, system, electronic equipment and computer readable medium |
-
2020
- 2020-10-12 CN CN202011083902.8A patent/CN112215142B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109328359A (en) * | 2016-06-30 | 2019-02-12 | 波萨诺瓦机器人知识产权有限公司 | Multi-camera system for inventory tracking |
| CN110533028A (en) * | 2019-07-31 | 2019-12-03 | 北京三快在线科技有限公司 | Detection method, device, electronic equipment and the storage medium of commodity display state |
| CN111310706A (en) * | 2020-02-28 | 2020-06-19 | 创新奇智(上海)科技有限公司 | Commodity price tag identification method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112215142A (en) | 2021-01-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112215142B (en) | Method, device and equipment for detecting goods shelf stock shortage rate based on depth image information | |
| US10909667B1 (en) | Image rectification using transformation data | |
| US10902280B1 (en) | System to search three-dimensional object data | |
| US10699421B1 (en) | Tracking objects in three-dimensional space using calibrated visual cameras and depth cameras | |
| US10713803B1 (en) | Testing validity of items using image data analysis | |
| US20190333039A1 (en) | Produce and bulk good management within an automated shopping environment | |
| US10388019B1 (en) | Associating an agent with an event based on multiple inputs | |
| EP3434626A1 (en) | Projection instruction device, parcel sorting system, and projection instruction method | |
| CN107403332B (en) | Goods shelf fetching detection system and method | |
| JP2009048430A (en) | Customer behavior analysis device, customer behavior determination system, and customer buying behavior analysis system | |
| US10671856B1 (en) | Detecting item actions and inventory changes at an inventory location | |
| US20090195388A1 (en) | Flow line recognition system | |
| US11301984B1 (en) | System to determine user interaction with fixture | |
| US20190042847A1 (en) | Instruction projecting device, package sorting system and instruction projecting method | |
| CN112883955B (en) | Shelf layout detection method, device and computer readable storage medium | |
| US10675659B2 (en) | Instruction projecting device, package sorting system and instruction projecting method | |
| EP3434623B1 (en) | Projection indicator, cargo assortment system, and projection indicating method | |
| CN112074842A (en) | Price tag information display method and device and shelf system | |
| Milella et al. | Towards intelligent retail: Automated on-shelf availability estimation using a depth camera | |
| CN110785362B (en) | Projection indicating device, cargo sorting system and projection indicating method | |
| CN111507792A (en) | Self-service shopping method, computer readable storage medium and system | |
| US10372753B2 (en) | System for verifying physical object absences from assigned regions using video analytics | |
| CN110781780B (en) | Vacancy detection method and related device | |
| US11117744B1 (en) | Determination of untidy item return to an inventory location | |
| EP3434625B1 (en) | Projection instruction device, parcel sorting system, and projection instruction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |