CN110766646A - Display rack shielding detection method and device and storage medium - Google Patents
Display rack shielding detection method and device and storage medium Download PDFInfo
- Publication number
- CN110766646A CN110766646A CN201810835774.4A CN201810835774A CN110766646A CN 110766646 A CN110766646 A CN 110766646A CN 201810835774 A CN201810835774 A CN 201810835774A CN 110766646 A CN110766646 A CN 110766646A
- Authority
- CN
- China
- Prior art keywords
- area
- image
- detection
- depth
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Abstract
The utility model provides a method, a device and a storage medium for detecting the shielding of a display rack, which relate to the technical field of image detection, wherein the method comprises the following steps: acquiring a two-dimensional reference image which is acquired by a three-dimensional camera device and contains a display shelf in an unobstructed state and reference depth information corresponding to the two-dimensional reference image; acquiring a two-dimensional detection image containing a display rack and detection depth information corresponding to the two-dimensional detection image through a three-dimensional camera device; and comparing the detection depth information with the reference depth information, and determining whether the display shelf is shielded or not based on the comparison result. The method, the device and the storage medium can improve the accuracy of judging the occurrence of the shielding of the display rack, obtain the accurate shielding part, provide reliable guarantee for subsequent analysis and decision-making work, improve the sensitivity of customers, improve the working efficiency and enhance the competitiveness.
Description
Technical Field
The present disclosure relates to the field of image detection technologies, and in particular, to a method and an apparatus for detecting display rack occlusion, and a storage medium.
Background
In a physical shop for selling commodities, store clerks usually need to periodically check and patrol commodities on a display rack so as to ensure that enough commodities can be sold on the display rack, the commodities are orderly arranged and are arranged in a standard way, and the shopping experience and sales volume of customers are further improved. The display stand may be a counter or the like. With the development of image recognition technology, the commodity image recognition is carried out by shooting the display rack through the camera so as to replace part of the traditional manual method for checking and patrolling regularly. When the display shelf is shot by the camera, the display shelf is possibly shielded (completely or partially) by objects such as the body of a customer, so that the accuracy of checking and patrolling is influenced.
For the occlusion problem, the existing solutions are generally: selecting a picture with the best effect (such as the largest number of identified commodities) as a final identification analysis result by adopting a multi-time shooting method; and detecting human bodies in the effective range of the picture (the picture area where the display shelf is located). The existing scheme has the defects of poor accuracy of detection results, high implementation difficulty and the like. Therefore, a new solution for display rack occlusion detection is needed.
Disclosure of Invention
In view of the above, one technical problem to be solved by the present disclosure is to provide a display rack occlusion detection method, device and storage medium.
According to one aspect of the present disclosure, a display rack occlusion detection method is provided, including: acquiring a two-dimensional reference image which is acquired by a three-dimensional camera device and contains a display shelf in an unobstructed state and reference depth information corresponding to the two-dimensional reference image; acquiring a two-dimensional detection image containing the display rack and detection depth information corresponding to the two-dimensional detection image through the three-dimensional camera device; and comparing the detection depth information with the reference depth information, and determining whether the display rack is shielded or not based on a comparison result.
Optionally, the comparing the detection depth information with the reference depth information, and determining whether the display shelf is blocked based on a comparison result further include: determining a display shelf image area in the two-dimensional reference image and the two-dimensional inspection image; dividing the image area of the display frame based on an area division rule to obtain a plurality of detection areas; obtaining first region depth information of the detection region in the two-dimensional reference image based on the reference depth information; obtaining second region depth information of the detection region in the two-dimensional detection image based on the detection depth information; and comparing the first region depth information and the second region depth information corresponding to the same detection region, and determining whether the detection region is an occlusion image region or not based on the comparison result.
Optionally, the dividing the display rack image area based on the area division rule to obtain a plurality of detection areas includes: acquiring an equal division of image length and an equal division of image width; and gridding the image area of the display frame based on the image length equant number and the image width equant number to obtain a plurality of detection areas.
Optionally, the reference depth information includes: a first depth value of a pixel point in the two-dimensional reference image; the obtaining first region depth information of the detection region in the two-dimensional reference image based on the reference depth information includes: obtaining a first area depth value of the detection area according to a first depth value of a pixel point located in the display shelf image area; the detecting depth information includes: detecting a second depth value of a pixel point in the two-dimensional detection image; the obtaining second region depth information of the detection region in the two-dimensional detection image includes: and obtaining a second area depth value of the detection area according to the second depth value of the pixel point positioned in the display frame image area.
Optionally, the obtaining the first area depth value of the detection area according to the first depth values of the pixel points located in the image area of the display shelf comprises obtaining an average value of the first depth values of all the pixel points located in the detection area as the first area depth value of the detection area; the obtaining a second region depth value of the detection region according to the second depth value of the pixel point located in the display shelf image region comprises: and obtaining the average value of the second depth values of all the pixel points in the detection area, and taking the average value as the second area depth value of the detection area.
Optionally, the determining whether the detection area is an occlusion image area based on the comparison result includes: and determining whether the difference between the first area depth value and the second area depth value of the same detection area is larger than a difference threshold value, if so, determining that the detection area is an occluded image area, and if not, determining that the detection area is not the occluded image area.
Optionally, the determining whether the display rack is occluded based on the comparison result comprises: obtaining a display rack shielding area corresponding to all the shielding image areas; and determining the area occupation ratio and the position of the occluded area of the display frame based on the display frame occlusion area.
Optionally, the reference depth information includes: a first depth image corresponding to the two-dimensional reference image acquired by the three-dimensional image pickup device, wherein the first depth value includes: a first gray value of a pixel point in the first depth image; the detecting depth information includes: a second depth image corresponding to the two-dimensional detection image acquired by the three-dimensional camera device, wherein the second depth value includes: a second gray value of a pixel point in the second depth image.
According to another aspect of the present disclosure, there is provided a display rack blocking detection apparatus, including: the device comprises a reference depth acquisition module, a display frame acquisition module and a display control module, wherein the reference depth acquisition module is used for acquiring a two-dimensional reference image which is acquired by a three-dimensional camera device and contains the display frame in an unobstructed state and reference depth information corresponding to the two-dimensional reference image; the detection depth acquisition module is used for acquiring a two-dimensional detection image containing the display rack and detection depth information corresponding to the two-dimensional detection image through the three-dimensional camera device; and the display rack shielding determining module is used for comparing the detection depth information with the reference depth information and determining whether the display rack is shielded or not based on a comparison result.
Optionally, the display rack occlusion determining module includes: an area dividing unit for determining a display shelf image area in the two-dimensional reference image and the two-dimensional detection image; dividing the image area of the display frame based on an area division rule to obtain a plurality of detection areas; a region depth obtaining unit configured to obtain first region depth information of the detection region in the two-dimensional reference image based on the reference depth information; obtaining second region depth information of the detection region in the two-dimensional detection image based on the detection depth information; and the area occlusion determining unit is used for comparing the first area depth information and the second area depth information corresponding to the same detection area and determining whether the detection area is an occlusion image area or not based on the comparison result.
Optionally, the region dividing unit is configured to obtain an equal number of image lengths and an equal number of image widths; and gridding the image area of the display frame based on the image length equant number and the image width equant number to obtain a plurality of detection areas.
Optionally, the reference depth information includes: a first depth value of a pixel point in the two-dimensional reference image; the area depth obtaining unit is used for obtaining a first area depth value of the detection area according to a first depth value of a pixel point in the display frame image area; the detecting depth information includes: detecting a second depth value of a pixel point in the two-dimensional detection image; and the area depth obtaining unit is used for obtaining a second area depth value of the detection area according to the second depth value of the pixel point positioned in the display frame image area.
Optionally, the area depth obtaining unit is configured to obtain an average value of the first depth values of all the pixel points located in the detection area, as the first area depth value of the detection area; and obtaining the average value of the second depth values of all the pixel points in the detection area, and taking the average value as the second area depth value of the detection area.
Optionally, the area occlusion determining unit is configured to determine whether a difference between a first area depth value and a second area depth value of the same detection area is greater than a difference threshold, if so, determine that the detection area is an occlusion image area, and if not, determine that the detection area is not an occlusion image area.
Optionally, the area occlusion determining unit is configured to obtain an occlusion area of the display rack corresponding to all of the occlusion image areas; and determining the area occupation ratio and the position of the occluded area of the display frame based on the display frame occlusion area.
Optionally, the reference depth information includes: a first depth image corresponding to the two-dimensional reference image acquired by the three-dimensional image pickup device, wherein the first depth value includes: a first gray value of a pixel point in the first depth image; the detecting depth information includes: a second depth image corresponding to the two-dimensional detection image acquired by the three-dimensional camera device, wherein the second depth value includes: a second gray value of a pixel point in the second depth image.
According to still another aspect of the present disclosure, there is provided a display stand occlusion detecting device, including: a memory; and a processor coupled to the memory, the processor configured to perform the method as described above based on instructions stored in the memory.
According to yet another aspect of the disclosure, a computer-readable storage medium stores computer instructions for execution by a processor to perform a method as described above.
According to the display rack shielding detection method, the display rack shielding detection device and the storage medium, whether shielding exists between the display rack and the camera or not, and information such as shielding area size and shielding occurrence position is judged by utilizing image depth information collected by the three-dimensional camera device; the accuracy of judging the appearance of the shielding of the display rack can be improved, the accurate shielding part is obtained, reliable guarantee can be provided for follow-up analysis and decision-making work, the feeling of a customer is improved, the working efficiency is improved, and the competitiveness is enhanced.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without inventive exercise.
FIG. 1 is a schematic flow chart diagram illustrating one embodiment of a method for display rack occlusion detection according to the present disclosure;
FIG. 2 is a schematic flow chart diagram illustrating another embodiment of a method for display rack occlusion detection according to the present disclosure;
FIG. 3 is a schematic view of a process for determining occlusion in an embodiment of a method for occlusion detection for a display shelf according to the present disclosure;
FIG. 4 is a schematic diagram of a 3D camera depth change caused by an occluding object;
FIG. 5 is a block schematic diagram of one embodiment of a display rack occlusion detection apparatus according to the present disclosure;
FIG. 6 is a block diagram of a display rack occlusion determination module in an embodiment of a display rack occlusion detection apparatus according to the present disclosure;
fig. 7 is a block schematic diagram of a further embodiment of a display stand occlusion detection device according to the present disclosure.
Detailed Description
The present disclosure now will be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments of the disclosure are shown. The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure. The technical solution of the present disclosure is described in various aspects below with reference to various figures and embodiments.
Hereinafter, "first", "second", etc. are only descriptive differences and have no other special meanings.
Fig. 1 is a schematic flow chart of an embodiment of a display rack occlusion detection method according to the present disclosure, and as shown in fig. 1, the display rack occlusion detection method includes steps 101-104.
Step 101, acquiring a two-dimensional reference image which is acquired by a three-dimensional camera device and contains a display shelf in an unobstructed state and reference depth information corresponding to the two-dimensional reference image. When the showing stand is in the non-shielding state, no shielding object is arranged in front of the showing stand, namely, no shielding object is arranged between the three-dimensional camera device and the showing stand. The three-dimensional camera device can be various, and can be a 3D camera and the like. The display rack can be a rack or a showcase for placing and displaying various articles, such as a shelf in a shopping mall, a shelf in an unmanned supermarket, an automatic vending counter, a bookshelf in a bookstore, and the like.
And 102, acquiring a two-dimensional detection image containing a display rack and detection depth information corresponding to the two-dimensional detection image through a three-dimensional camera device.
And 103, comparing the detection depth information with the reference depth information, and determining whether the display shelf is shielded or not based on the comparison result.
In the method for detecting shielding of the display rack in the embodiment, the image depth information acquired by the three-dimensional camera device is used for judging whether shielding exists between the display rack and the camera at the image acquisition time, and information such as the shielding area size and the shielding occurrence position exists between the display rack and the camera, so that subsequent analysis and decision of a system are supported.
The 2D images collected by the 3D camera can be utilized to complete the inspection tour of the goods on the showing stand and the like. As shown in fig. 4, when occlusion occurs, the depth from the object to the 3D camera becomes short, and therefore, by using this change, it can be easily determined whether occlusion exists between the display shelf and the 3D camera. When there is an occluding object between the target object (display shelf) and the camera, the depth value between the two (i.e., the length of the dashed line in fig. 4) will be significantly smaller relative to when there is no occluding object.
Fig. 2 is a schematic flow chart of a display rack occlusion detection method according to another embodiment of the disclosure, as shown in fig. 2, the display rack occlusion detection method includes steps 201 and 205.
In step 201, the display shelf image area in the two-dimensional reference image and the two-dimensional inspection image is determined. The cargo image area in the image may be determined using a variety of methods known in the art.
And step 204, obtaining second region depth information of the detection region in the two-dimensional detection image based on the detection depth information.
In one embodiment, the region division rule may be various. For example, the number of equally divided image lengths and the number of equally divided image widths are acquired, and the display shelf image area is subjected to gridding processing based on the number of equally divided image lengths and the number of equally divided image widths, thereby obtaining a plurality of detection areas. For example, the resolution of the display frame image area is 720 × 480, the obtained image has a length equal to 4 and a width equal to 3, and the display frame image area is gridded to obtain 12 detection areas, each of which is the same.
Whether shielding exists between the display rack and the 3D camera or not is detected, meanwhile, in order to determine the specific position where shielding occurs, the image area of the display rack can be divided into detection areas with finer granularity, the detection areas are used as the minimum detection unit for subsequent depth detection, and finally the whole shielding condition of the display rack is obtained according to the shielding condition of each detection area.
The detection area can be divided into areas with the same size or areas with different sizes, and the corresponding coordinate ranges of the areas on the image are recorded. For example, the horizontal direction is used to take a shelf laminate as a boundary, the vertical direction is used to divide the shelf image area into k areas by 4 equal parts, the number of the k areas can be 1 to k from top to bottom in sequence from left to right, k is 4, the number of the shelf layers is counted, and the positions of the rectangular frames of the detection areas in the image, namely the (x, y) coordinates of the upper left corner and the lower right corner, are recorded.
In one embodiment, a three-dimensional imaging device such as a 3D camera can acquire depth information of a subject, i.e., three-dimensional position and size information, in addition to a two-dimensional planar image, and generally includes a plurality of cameras + depth sensors. In a computer vision system, three-dimensional scene information provides more possibilities for various computer vision applications such as image segmentation, target detection, object tracking and the like, and a Depth image (Depth map) is widely applied as a general three-dimensional scene information expression mode. A depth image of an object may be obtained by a three-dimensional camera device, where a pixel value in the depth image reflects a distance from the object to a camera in a scene, for example, a gray value of each pixel point of the depth image may be used to represent a distance from a certain point in the scene to the camera.
The reference depth information comprises a first depth value of a pixel point in the two-dimensional reference image, and a first area depth value of the detection area is obtained according to the first depth value of the pixel point in the display frame image area. The detection depth information comprises a second depth value of a pixel point in the two-dimensional detection image, and a second area depth value of the detection area is obtained according to the second depth value of the pixel point in the display frame image area.
The reference depth information comprises a first depth image which is acquired by the three-dimensional camera device and corresponds to the two-dimensional reference image, wherein the first depth value comprises a first gray value of a pixel point in the first depth image, and the first gray value is used for representing the distance between the three-dimensional camera device and a point on a container corresponding to the pixel point. The detection depth information comprises a second depth image which is acquired by the three-dimensional camera device and corresponds to the two-dimensional detection image, wherein the second depth value comprises a second gray value of a pixel point in the second depth image, and the second gray value is used for representing the distance between the three-dimensional camera device and a point on a container corresponding to the pixel point.
Fig. 3 is a schematic flow chart of occlusion determination in an embodiment of a method for occlusion detection of a display shelf according to the present disclosure, as shown in fig. 3:
In one embodiment, the areas of the display shelf are filled with the commodities according to a predefined placing mode, and meanwhile, it is ensured that no shielding exists between the camera and the display shelf when the image is acquired, and the average depth value of each detection area in the image acquired by the camera at the moment is used as the average depth value when the area of the display shelf is not shielded, namely the first area depth value of the detection area.
Assuming that the coordinates of a pixel point on an image are (x, y), the coordinates of the upper left corner of a detection region k are (x _ k _1, y _ k _1), and the coordinates of the lower right corner of the detection region k are (x _ k _2, y _ k _2), if x > x _ k _1, y > y _ k _1, x < x _ k _2, and y < y _ k _2 are satisfied at the same time, the pixel point belongs to the detection region k.
The average depth value of the detection area is calculated as:
dk is the average depth of the detection area k, namely the depth value of the first area, di is the distance from each pixel point in the detection area k to the camera, namely the first depth value (which can be obtained by the 3D camera), n is the total number of the pixel points in the detection area, and i is the number of 1-n pixel points.
Keeping the relative position between the showing stand and the camera fixed, when the shielding condition of the showing stand needs to be judged each time, immediately acquiring the current showing stand picture, and calculating the average depth of each detection area, namely the depth value of the second area, wherein the calculation method is as shown in a formula (1-1).
If the obtained second area depth value of a certain detection area i is assumed to be DRi, the first depth value when the display shelf corresponding to the detection area is not shielded is obtained to be DSi, if DRi < DSi-T, the display shelf area corresponding to the detection area i is shielded, otherwise, the display shelf area corresponding to the detection area i is not shielded, wherein T is a difference threshold value used for stabilizing errors possibly brought by the depth camera.
And obtaining a display rack shielding area corresponding to the whole shielding image area, and determining the shielded area occupation ratio and position of the display rack based on the display rack shielding area. And the display rack area corresponding to the shielding image area is the shielding area of the display rack and is determined as the position where shielding occurs. The total area and position of the shielding area of the display shelf can be counted, and the ratio of the total area and the position to the area of the display shelf is the shielding area ratio.
In one embodiment, as shown in fig. 5, the present disclosure provides a display stand occlusion detection device 50 comprising: a reference depth acquisition module 51, a detection depth acquisition module 52 and a display shelf occlusion determination module 53. The reference depth acquiring module 51 acquires a two-dimensional reference image including a display shelf in an unobstructed state acquired by the three-dimensional imaging device and reference depth information corresponding to the two-dimensional reference image. The detection depth acquiring module 52 acquires a two-dimensional detection image including the display shelf and detection depth information corresponding to the two-dimensional detection image through the three-dimensional camera device. The display shelf occlusion determining module 53 compares the detected depth information with the reference depth information, and determines whether the display shelf is occluded based on the comparison result.
As shown in fig. 6, the display shelf occlusion determining module 53 includes: an area dividing unit 531, an area depth obtaining unit 532, and an area occlusion determining unit 533. The region dividing unit 531 determines the display frame image regions in the two-dimensional reference image and the two-dimensional detection image, and divides the display frame image regions based on a region division rule to obtain a plurality of detection regions. The region depth obtaining unit 532 obtains first region depth information of the detection region in the two-dimensional reference image based on the reference depth information, and obtains second region depth information of the detection region in the two-dimensional detection image based on the detection depth information. The region occlusion determining unit 533 compares the first region depth information and the second region depth information corresponding to the same detection region, and determines whether the detection region is an occlusion image region based on the comparison result.
In one embodiment, the region dividing unit 531 obtains the image length equal division number and the image width equal division number, and performs gridding processing on the display frame image region based on the image length equal division number and the image width equal division number to obtain a plurality of detection regions.
The reference depth information includes: the region depth obtaining unit 532 obtains a first region depth value of the detection region according to the first depth value of the pixel point located in the display frame image region. Detecting the depth information includes: the area depth obtaining unit 532 obtains a second area depth value of the detection area according to the second depth value of the pixel point located in the area of the display frame image.
The area depth obtaining unit 532 obtains an average value of first depth values for all pixel points located in the detection area as a first area depth value of the detection area, and obtains an average value of second depth values for all pixel points located in the detection area as a second area depth value of the detection area.
The area occlusion determining unit 533 determines whether a difference between a first area depth value and a second area depth value of the same detection area is greater than a difference threshold, and if so, determines that the detection area is an occluded image area, and if not, determines that the detection area is not an occluded image area. The region occlusion determining unit 533 obtains a display shelf occlusion region corresponding to the all-occluded image region, and determines the occluded area ratio and position of the display shelf based on the display shelf occlusion region.
Fig. 7 is a block diagram of another embodiment of a display rack occlusion detection device according to the present disclosure. As shown in fig. 7, the apparatus may include a memory 71, a processor 72, a communication interface 73, and a bus 74. The memory 71 is used for storing instructions, the processor 72 is coupled to the memory 71, and the processor 72 is configured to execute the display shelf occlusion detection method described above based on the instructions stored in the memory 71.
The memory 71 may be a high-speed RAM memory, a non-volatile memory (non-volatile memory), or the like, and the memory 71 may be a memory array. The storage 71 may also be partitioned and the blocks may be combined into virtual volumes according to certain rules. Processor 72 may be a central processing unit CPU, or an application specific integrated circuit asic, or one or more integrated circuits configured to implement the display rack occlusion detection methods of the present disclosure.
In one embodiment, the present disclosure provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement a display rack occlusion detection method as in any of the above embodiments.
In the method, the device and the storage medium for detecting the shielding of the display rack in the embodiment, whether shielding exists between the display rack and the camera or not, and information such as shielding area, shielding occurrence position and the like is judged by utilizing the image depth information acquired by the three-dimensional camera device; the accuracy of judging that the display shelf is shielded can be improved, the accurate shielding part is obtained, the realization is simple and convenient, reliable guarantee can be provided for follow-up analysis and decision-making work, the sensitivity of a client is improved, the working efficiency is improved, and the competitiveness is enhanced.
The method and system of the present disclosure may be implemented in a number of ways. For example, the methods and systems of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Further, in some embodiments, the present disclosure may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.
The description of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to practitioners skilled in this art. The embodiment was chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated.
Claims (18)
1. A display rack shielding detection method comprises the following steps:
acquiring a two-dimensional reference image which is acquired by a three-dimensional camera device and contains a display shelf in an unobstructed state and reference depth information corresponding to the two-dimensional reference image;
acquiring a two-dimensional detection image containing the display rack and detection depth information corresponding to the two-dimensional detection image through the three-dimensional camera device;
and comparing the detection depth information with the reference depth information, and determining whether the display rack is shielded or not based on a comparison result.
2. The method of claim 1, the comparing the detected depth information to the reference depth information, determining whether the display shelf is occluded based on the comparison result, further comprising:
determining a display shelf image area in the two-dimensional reference image and the two-dimensional inspection image;
dividing the image area of the display frame based on an area division rule to obtain a plurality of detection areas;
obtaining first region depth information of the detection region in the two-dimensional reference image based on the reference depth information;
obtaining second region depth information of the detection region in the two-dimensional detection image based on the detection depth information;
and comparing the first region depth information and the second region depth information corresponding to the same detection region, and determining whether the detection region is an occlusion image region or not based on the comparison result.
3. The method of claim 2, wherein the dividing the display frame image area based on the area division rule to obtain a plurality of detection areas comprises:
acquiring an equal division of image length and an equal division of image width;
and gridding the image area of the display frame based on the image length equant number and the image width equant number to obtain a plurality of detection areas.
4. The method of claim 2, the reference depth information comprising: a first depth value of a pixel point in the two-dimensional reference image; the obtaining first region depth information of the detection region in the two-dimensional reference image based on the reference depth information includes:
obtaining a first area depth value of the detection area according to a first depth value of a pixel point located in the display shelf image area;
the detecting depth information includes: detecting a second depth value of a pixel point in the two-dimensional detection image; the obtaining second region depth information of the detection region in the two-dimensional detection image includes:
and obtaining a second area depth value of the detection area according to the second depth value of the pixel point positioned in the display frame image area.
5. The method of claim 4, wherein obtaining the first area depth value of the detection area according to the first depth value of the pixel point located in the display shelf image area comprises:
obtaining the average value of the first depth values of all the pixel points in the detection area, and taking the average value as the first area depth value of the detection area;
the obtaining a second region depth value of the detection region according to the second depth value of the pixel point located in the display shelf image region comprises:
and obtaining the average value of the second depth values of all the pixel points in the detection area, and taking the average value as the second area depth value of the detection area.
6. The method of claim 5, wherein determining whether the detection region is an occlusion image region based on the comparison comprises:
and determining whether the difference between the first area depth value and the second area depth value of the same detection area is larger than a difference threshold value, if so, determining that the detection area is an occluded image area, and if not, determining that the detection area is not the occluded image area.
7. The method of claim 6, the determining whether the display shelf is occluded based on the comparison comprises:
obtaining a display rack shielding area corresponding to all the shielding image areas;
and determining the area occupation ratio and the position of the occluded area of the display frame based on the display frame occlusion area.
8. The method of claim 4, the reference depth information comprising: a first depth image corresponding to the two-dimensional reference image acquired by the three-dimensional image pickup device, wherein the first depth value includes: a first gray value of a pixel point in the first depth image;
the detecting depth information includes: a second depth image corresponding to the two-dimensional detection image acquired by the three-dimensional camera device, wherein the second depth value includes: a second gray value of a pixel point in the second depth image.
9. A display rack occlusion detection device, comprising:
the device comprises a reference depth acquisition module, a display frame acquisition module and a display control module, wherein the reference depth acquisition module is used for acquiring a two-dimensional reference image which is acquired by a three-dimensional camera device and contains the display frame in an unobstructed state and reference depth information corresponding to the two-dimensional reference image;
the detection depth acquisition module is used for acquiring a two-dimensional detection image containing the display rack and detection depth information corresponding to the two-dimensional detection image through the three-dimensional camera device;
and the display rack shielding determining module is used for comparing the detection depth information with the reference depth information and determining whether the display rack is shielded or not based on a comparison result.
10. The apparatus of claim 9, wherein,
the display rack occlusion determining module comprises:
an area dividing unit for determining a display shelf image area in the two-dimensional reference image and the two-dimensional detection image; dividing the image area of the display frame based on an area division rule to obtain a plurality of detection areas;
a region depth obtaining unit configured to obtain first region depth information of the detection region in the two-dimensional reference image based on the reference depth information; obtaining second region depth information of the detection region in the two-dimensional detection image based on the detection depth information;
and the area occlusion determining unit is used for comparing the first area depth information and the second area depth information corresponding to the same detection area and determining whether the detection area is an occlusion image area or not based on the comparison result.
11. The apparatus of claim 10, wherein,
the region dividing unit is used for acquiring the image length halving number and the image width halving number; and gridding the image area of the display frame based on the image length equant number and the image width equant number to obtain a plurality of detection areas.
12. The apparatus of claim 10, the reference depth information comprising: a first depth value of a pixel point in the two-dimensional reference image;
the area depth obtaining unit is used for obtaining a first area depth value of the detection area according to a first depth value of a pixel point in the display frame image area;
the detecting depth information includes: detecting a second depth value of a pixel point in the two-dimensional detection image;
and the area depth obtaining unit is used for obtaining a second area depth value of the detection area according to the second depth value of the pixel point positioned in the display frame image area.
13. The apparatus of claim 12, wherein,
the area depth obtaining unit is used for obtaining an average value of first depth values of all pixel points located in the detection area, and the average value is used as the first area depth value of the detection area; and obtaining the average value of the second depth values of all the pixel points in the detection area, and taking the average value as the second area depth value of the detection area.
14. The apparatus of claim 13, wherein,
the area occlusion determining unit is configured to determine whether a difference between a first area depth value and a second area depth value of the same detection area is greater than a difference threshold, if so, determine that the detection area is an occluded image area, and if not, determine that the detection area is not an occluded image area.
15. The apparatus of claim 14, wherein,
the area shielding determining unit is used for obtaining a shielding area of the display rack corresponding to all the shielding image areas; and determining the area occupation ratio and the position of the occluded area of the display frame based on the display frame occlusion area.
16. The apparatus of claim 12, the reference depth information comprising: a first depth image corresponding to the two-dimensional reference image acquired by the three-dimensional image pickup device, wherein the first depth value includes: a first gray value of a pixel point in the first depth image;
the detecting depth information includes: a second depth image corresponding to the two-dimensional detection image acquired by the three-dimensional camera device, wherein the second depth value includes: a second gray value of a pixel point in the second depth image.
17. A display rack occlusion detection device, comprising:
a memory; and a processor coupled to the memory, the processor configured to perform the method of any of claims 1-8 based on instructions stored in the memory.
18. A computer-readable storage medium having stored thereon computer instructions for execution by a processor of the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810835774.4A CN110766646A (en) | 2018-07-26 | 2018-07-26 | Display rack shielding detection method and device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810835774.4A CN110766646A (en) | 2018-07-26 | 2018-07-26 | Display rack shielding detection method and device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110766646A true CN110766646A (en) | 2020-02-07 |
Family
ID=69326981
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810835774.4A Pending CN110766646A (en) | 2018-07-26 | 2018-07-26 | Display rack shielding detection method and device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110766646A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111447365A (en) * | 2020-04-16 | 2020-07-24 | 维沃移动通信有限公司 | Shooting method and electronic equipment |
| CN113099210A (en) * | 2021-03-31 | 2021-07-09 | 东莞中国科学院云计算产业技术创新与育成中心 | Three-dimensional image restoration method and device, computer equipment and storage medium |
| CN113962868A (en) * | 2021-09-10 | 2022-01-21 | 数坤(北京)网络科技股份有限公司 | Medical virtual reality image display method and device and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007079659A (en) * | 2005-09-12 | 2007-03-29 | Nec Engineering Ltd | Monitoring device |
| CN104240235A (en) * | 2014-08-26 | 2014-12-24 | 北京君正集成电路股份有限公司 | Method and system for detecting whether camera is covered or not |
| CN104850219A (en) * | 2014-02-19 | 2015-08-19 | 北京三星通信技术研究有限公司 | Equipment and method for estimating posture of human body attached with object |
| CN105303549A (en) * | 2015-06-29 | 2016-02-03 | 北京格灵深瞳信息技术有限公司 | Method of determining position relation between detected objects in video image and device |
| US20160065990A1 (en) * | 2013-04-11 | 2016-03-03 | Nippon Telegraph And Telephone Corporation | Image encoding method, image decoding method, image encoding apparatus, image decoding apparatus, image encoding program, image decoding program, and recording media |
| CN105513112A (en) * | 2014-10-16 | 2016-04-20 | 北京畅游天下网络技术有限公司 | Image processing method and device |
-
2018
- 2018-07-26 CN CN201810835774.4A patent/CN110766646A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007079659A (en) * | 2005-09-12 | 2007-03-29 | Nec Engineering Ltd | Monitoring device |
| US20160065990A1 (en) * | 2013-04-11 | 2016-03-03 | Nippon Telegraph And Telephone Corporation | Image encoding method, image decoding method, image encoding apparatus, image decoding apparatus, image encoding program, image decoding program, and recording media |
| CN104850219A (en) * | 2014-02-19 | 2015-08-19 | 北京三星通信技术研究有限公司 | Equipment and method for estimating posture of human body attached with object |
| CN104240235A (en) * | 2014-08-26 | 2014-12-24 | 北京君正集成电路股份有限公司 | Method and system for detecting whether camera is covered or not |
| CN105513112A (en) * | 2014-10-16 | 2016-04-20 | 北京畅游天下网络技术有限公司 | Image processing method and device |
| CN105303549A (en) * | 2015-06-29 | 2016-02-03 | 北京格灵深瞳信息技术有限公司 | Method of determining position relation between detected objects in video image and device |
Non-Patent Citations (1)
| Title |
|---|
| 李红波;代文海;吴渝;赵永耀;: "动态变换背景帧的虚实遮挡处理方法", 计算机工程与设计 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111447365A (en) * | 2020-04-16 | 2020-07-24 | 维沃移动通信有限公司 | Shooting method and electronic equipment |
| CN111447365B (en) * | 2020-04-16 | 2021-11-30 | 维沃移动通信有限公司 | Shooting method and electronic equipment |
| CN113099210A (en) * | 2021-03-31 | 2021-07-09 | 东莞中国科学院云计算产业技术创新与育成中心 | Three-dimensional image restoration method and device, computer equipment and storage medium |
| CN113962868A (en) * | 2021-09-10 | 2022-01-21 | 数坤(北京)网络科技股份有限公司 | Medical virtual reality image display method and device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107358149B (en) | Human body posture detection method and device | |
| US9025875B2 (en) | People counting device, people counting method and people counting program | |
| US10212324B2 (en) | Position detection device, position detection method, and storage medium | |
| WO2021012644A1 (en) | Shelf commodity detection method and system | |
| US10713610B2 (en) | Methods and systems for occlusion detection and data correction for container-fullness estimation | |
| US10311595B2 (en) | Image processing device and its control method, imaging apparatus, and storage medium | |
| US10984228B2 (en) | Interaction behavior detection method, apparatus, system, and device | |
| US9672634B2 (en) | System and a method for tracking objects | |
| CN108960238B (en) | Multi-camera-based commodity identification processing method and device | |
| CN113498530A (en) | Object size marking system and method based on local visual information | |
| EP3563345B1 (en) | Automatic detection, counting, and measurement of lumber boards using a handheld device | |
| CN108921098B (en) | Human motion analysis method, device, equipment and storage medium | |
| US8395659B2 (en) | Moving obstacle detection using images | |
| CN110766646A (en) | Display rack shielding detection method and device and storage medium | |
| EP3593322B1 (en) | Method of detecting moving objects from a temporal sequence of images | |
| CN113034574B (en) | Commodity ground heap area calculation method and system based on target detection | |
| CA3005452A1 (en) | Methods and systems for container fullness estimation | |
| CN111161346A (en) | Method and device for layering commodities in goods shelf and electronic equipment | |
| CN115249356B (en) | Identification method, device, equipment and storage medium | |
| US20070206093A1 (en) | Image processing apparatus, method for image processing, computer readable medium, and computer data signal | |
| US20130331145A1 (en) | Measuring system for mobile three dimensional imaging system | |
| US11244465B2 (en) | Point cloud data processing device | |
| CN111428743A (en) | Commodity identification method, commodity processing device and electronic equipment | |
| CN112666550A (en) | Moving object detection method and apparatus, fusion processing unit, and medium | |
| US20180357784A1 (en) | Method for characterising a scene by computing 3d orientation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |