WO2022266996A1 - 物体检知方法及物体检知装置 - Google Patents

物体检知方法及物体检知装置 Download PDF

Info

Publication number
WO2022266996A1
WO2022266996A1 PCT/CN2021/102347 CN2021102347W WO2022266996A1 WO 2022266996 A1 WO2022266996 A1 WO 2022266996A1 CN 2021102347 W CN2021102347 W CN 2021102347W WO 2022266996 A1 WO2022266996 A1 WO 2022266996A1
Authority
WO
WIPO (PCT)
Prior art keywords
detection
detection frame
rectangular
frame
image
Prior art date
Application number
PCT/CN2021/102347
Other languages
English (en)
French (fr)
Inventor
贾书军
李想
曙光
王迎春
张烨
Original Assignee
烟台创迹软件有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 烟台创迹软件有限公司 filed Critical 烟台创迹软件有限公司
Priority to CN202180099742.2A priority Critical patent/CN117616468A/zh
Priority to JP2023579455A priority patent/JP2024522881A/ja
Priority to PCT/CN2021/102347 priority patent/WO2022266996A1/zh
Publication of WO2022266996A1 publication Critical patent/WO2022266996A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/255—Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components

Definitions

  • the invention relates to an object detection method and an object detection device for detecting objects from images.
  • Patent Document 1 discloses an example of a product detection method. In Patent Document 1, it is disclosed that images of displayed products are acquired, products in the images are detected, images of each product are intercepted and products are classified according to the spatial positional relationship of each product.
  • Patent Document 1 Specification of Chinese Patent Application Publication No. 110738123
  • Patent Document 1 an image of each product is cut out using a rectangular frame such as a rectangle or a square.
  • the outer shape of the product in the image may be distorted so that it is not a rectangle depending on the shooting angle when the image of the product is acquired or the like.
  • the cropped image may not include a part of the product to be detected or may include images other than the product to be detected, and the detection accuracy of the product may decrease.
  • the present invention solves the above problems, and an object of the present invention is to obtain an object detection method and an object detection device capable of improving the detection accuracy of an object.
  • the object detection method involved in the present invention includes: an image acquisition step, acquiring an image containing an object; a first detection step, using a rectangular first detection frame to detect the object in the image; a detection frame setting step, Setting a non-rectangular second detection frame corresponding to the detected object; and a second detection step, using the second detection frame to detect the object.
  • the object detection device includes: an image acquisition unit, which acquires an image containing an object; a first detection unit, which uses a rectangular first detection frame to detect the object in the image; a detection frame setting unit, A non-rectangular second detection frame corresponding to the detected object is set; and the second detection unit detects the object using the second detection frame.
  • the detection accuracy of the object can be improved.
  • FIG. 1 is a schematic configuration diagram of an object information acquisition system according to Embodiment 1. As shown in FIG. 1
  • FIG. 2 is a control block diagram of the object information acquisition system according to Embodiment 1.
  • FIG. 2 is a control block diagram of the object information acquisition system according to Embodiment 1.
  • FIG. 3 is a flowchart of object information acquisition processing according to Embodiment 1.
  • FIG. 3 is a flowchart of object information acquisition processing according to Embodiment 1.
  • FIG. 4 is an example of a front image of a shelf captured by the imaging device 2 .
  • FIG. 5 is a flowchart of object detection processing according to Embodiment 1.
  • FIG. 5 is a flowchart of object detection processing according to Embodiment 1.
  • FIG. 6 is an example of detection results by the first detection unit according to Embodiment 1.
  • FIG. 6 is an example of detection results by the first detection unit according to Embodiment 1.
  • FIG. 7 is an example of detection results by the second detection unit according to Embodiment 1.
  • FIG. 7 is an example of detection results by the second detection unit according to Embodiment 1.
  • FIG. 8 is a flowchart of object information acquisition processing according to Embodiment 2.
  • FIG. 1 is a schematic configuration diagram of an object information acquisition system 100 according to the first embodiment.
  • the object information acquisition system 100 of this embodiment is a system that is used in a retail store such as a supermarket, and automatically detects and recognizes an object P that is a commodity stored on a shelf S in the store and acquires the object stored on the shelf S. P's information.
  • the object information acquisition system 100 is composed of a processing device 1 and a photographing device 2 .
  • the processing device 1 is a PC including a CPU and a memory, a server on the cloud, or the like.
  • the imaging device 2 is a camera that is installed on the ceiling or wall of the store and captures frontal images of the shelves S. As shown in FIG.
  • the processing device 1 and the imaging device 2 are connected to be communicable by wire or wirelessly. Images captured by the photographing device 2 are sent to the processing device 1 .
  • FIG. 2 is a control block diagram of the object information acquisition system 100 according to the first embodiment.
  • the processing device 1 includes: an object detection unit 10 that detects an object P from an image; an object recognition unit 20 that recognizes the detected object P; and a storage unit 30 .
  • the object detection unit 10 and the object recognition unit 20 are functional units realized by executing programs by the CPU. Alternatively, the object detection unit 10 and the object recognition unit 20 may also be realized by a dedicated processing circuit.
  • the object detection unit 10 includes an image acquisition unit 11 , a first detection unit 12 , a detection frame setting unit 13 , and a second detection unit 14 .
  • the image acquisition unit 11 acquires an image captured by the photographing device 2 and sends it to the first detection unit 12 .
  • the first detection unit 12 detects the object P in the acquired image using an algorithm such as SSD (Single Shot Multibox Detector: object detection algorithm) using deep learning. In the first detection unit 12 , the detection of the object P is performed using the rectangular first detection frame F1 .
  • the detection frame setting unit 13 sets a non-rectangular second detection frame F2 corresponding to the object P detected by the first detection unit 12 .
  • the second detection unit 14 detects the object P using the second detection frame F2 set by the detection frame setting unit 13 , and sends the detection result to the object recognition unit 20 .
  • the object recognition unit 20 recognizes the object P included in the image detected by the second detection unit 14 of the object detection unit 10 based on the shelf information and product information.
  • the type and product name of the object P are recognized by using a known machine learning algorithm.
  • the storage unit 30 is, for example, a volatile or nonvolatile memory such as RAM, ROM, or flash memory.
  • the storage unit 30 stores programs executed by the object detection unit 10 and the object recognition unit 20 , various parameters used in the programs, shelf information, product information, detection frame data, detection history, and the like.
  • the shelf information includes the position of each shelf S in the store, the category of products stored on each shelf S, the number and size of each shelf S, and the number of detection frames on each shelf S.
  • the product information includes identification information such as the type and name of the product.
  • the detection frame data is data of a plurality of non-rectangular detection frames that are candidates for the second detection frame F2 set by the detection frame setting unit 13.
  • FIG. 3 is a flowchart of object information acquisition processing according to Embodiment 1.
  • FIG. This process is periodically executed by the processing device 1 .
  • system initialization is performed (S1).
  • initial values are set for each parameter of the object information acquisition process.
  • the parameters are the number of detection frames, the maximum number of detection frames that can be detected by each shelf, the maximum number of sections of the shelf, the type of detection frames, etc.
  • the front image of the shelf S is photographed by the imaging device 2 and acquired by the image acquisition unit 11 of the processing device 1 (S2).
  • the object detection unit 10 executes object detection processing based on the acquired image (S3). Thereby, a plurality of objects P included in the image are detected.
  • the object recognition part 20 executes the object recognition process (S4). Thereby, the detected object P can be recognized, and the information of the object P accommodated on the shelf S can be acquired.
  • the acquired information on the object P is sent to a management server or the like, and used for grasping sales data, product management, or the like.
  • FIG. 4 is an example of the front image of the shelf S captured by the imaging device 2 .
  • object detection is performed using a rectangular or square detection frame.
  • the imaging device 2 is installed above the ceiling or the wall and takes an image of the shelf S from above, as shown in FIG. 4 , the outer shape of the object P in the image is deformed from a rectangle. Therefore, in the object detection process of the present embodiment, the detection of the image is performed after setting the detection frame for detecting the object as a detection frame suitable for the deformation of the image.
  • FIG. 5 is a flowchart of object detection processing according to Embodiment 1.
  • the first detection is performed by the first detection unit 12 based on the acquired image (S31).
  • the object P is detected using the rectangular first detection frame F1.
  • FIG. 6 is an example of detection results by the first detection unit 12 according to the first embodiment.
  • a plurality of non-rectangular detection frames stored in the storage unit 30 are applied to each detected object P by the detection frame setting unit 13, and the reliability of each detection frame is obtained (S32).
  • the shapes of the plurality of non-rectangular detection frames are parallelogram, trapezoid, circle or ellipse. Also, each shape contains boxes of multiple sizes. Then, make the center coordinates of the non-rectangular detection frame consistent with the center coordinates of the rectangular first detection frame F1 including the object P detected by the first detection, and obtain the reliability of the non-rectangular detection frame .
  • the size of the area of the object P within the detection frame (the sharing of the object P with the non-rectangular detection frame The size of the area of the part) or the size of the area in the detection frame other than the object P (the size of the area of the non-shared part between the object P and the non-rectangular detection frame).
  • the larger the area of the object P within the detection frame the higher the reliability
  • the smaller the area of the detection frame other than the object P the higher the reliability.
  • the higher the ratio of the area of the object P to the area of the detection frame the higher the reliability.
  • the detection frame setting unit 13 filters a plurality of non-rectangular detection frames ( S33 ). Among the multiple non-rectangular detection frames, detection frames whose reliability is lower than a preset threshold are excluded from the candidates. Then, the detection frame setting unit 13 acquires, for each object P, the center coordinates of a plurality of non-rectangular detection frames whose reliability is equal to or greater than the threshold ( S34 ). The detection frame setting unit 13 performs contour detection of each object P in the image using the acquired center coordinates, and acquires position information of the contour of the object P ( S35 ). For contour detection of the object P, a known contour detection algorithm such as edge detection can be used.
  • the contour detection of the object P is performed on each of the center coordinates of a plurality of non-rectangular detection frames.
  • the detection frames with low reliability in step S33 it is possible to suppress an abnormality in which the center coordinates of the non-rectangular detection frames are located outside the object P and the outline of the object P cannot be detected.
  • the detection frame setting unit 13 compares the non-rectangular detection frame with the outline of the object P (S36). Then, a non-rectangular detection frame including all contours of the object P is set as the second detection frame F2 ( S37 ). Steps S36 and S37 are performed for each object P to set the second detection frame F2 corresponding to each object P.
  • the set shape and position of the second detection frame F2 are stored in the storage unit 30 . Among them, when there are multiple non-rectangular detection frames including all contours of the object P, the detection frame with the highest reliability is taken as the second detection frame F2. As described above, the reliability is obtained from the ratio of the area of the object P to the area of the detection frame.
  • any one is selected as the second detection frame F2.
  • a duplication detection algorithm that calculates overlapping areas, if the ratio of overlapping areas is equal to or greater than a threshold value, it is determined that they are the same, and only one can be left. Thus, the best detection frame is selected for each object P.
  • FIG. 7 is an example of detection results by the second detection unit 14 according to the first embodiment.
  • the object P can be detected using the second detection frame F2 along the outer shape of the object P by performing the object detection processing of the present embodiment.
  • the object detection processing of the present embodiment Even when the object P in the image is deformed due to the influence of the imaging angle, etc., it is possible to suppress the detection of a part of the object P and the detection of objects other than the object P, thereby improving the performance of the image.
  • the detection accuracy of the object P In particular, for an object P that is densely arranged such as commodities stored on the shelf S, by using a detection frame along the outer shape of the object, not only the accuracy is improved, but also the detection speed is improved.
  • the detection frame setting unit 13 can gather the multiple non-rectangular detection frames filtered in step S33 at the position of the object P on the shelf S, and compare the inclination with other detection frames in the same cluster Different detection boxes are detected as errors and excluded from the candidates.
  • the detection frame setting unit 13 may estimate the inclination of the object P from the position information of the imaging device 2, detect a non-rectangular detection frame having a slope different from the estimated inclination as an error, and select excluded. Furthermore, the detection frame setting unit 13 may detect a non-rectangular detection frame larger than the size of the shelf as an error based on the shelf information, and exclude it from the candidates.
  • FIG. 8 is a flowchart of object information acquisition processing according to Embodiment 2.
  • FIG. The structure of the object information acquisition system 100 in this embodiment is the same as that in the first embodiment.
  • initialization ( S1 ) and image acquisition ( S2 ) are performed in the same manner as in the first embodiment.
  • the object detection unit 10 judges whether the current detection is a re-detection based on the detection history ( S11 ).
  • the re-detection refers to a case where object detection has been performed on the shelf S in the past and the second detection frame F2 of the object on the shelf S is stored.
  • the first detection (S11: No) object detection processing (S3) and object recognition processing (S4) are executed in the same manner as in Embodiment 1. ).
  • the current detection is a re-detection (S11: Yes)
  • the image captured by the imaging device 2 during the previous detection and the image captured by the imaging device 2 this time are acquired. difference (S12).
  • object detection processing (S3) and object recognition processing (S4) are performed on the difference area. That is, in the present embodiment, detection of the object P and recognition of the object P are performed only for the area where there was a change from the previous time, and the information of the object P in other areas is set to be the same as last time.
  • the present embodiment by performing object detection processing and object recognition processing only on the area where there is a change, it is possible to reduce the processing load at the time of re-detection and improve the processing speed.
  • the second detection frame F2 is set by performing the processing of steps S32 to S36 in FIG. 5 by the detection frame setting unit 13 , but the present invention is not limited thereto.
  • the detection frame setting unit 13 may estimate the inclination of the object P from the position information of the imaging device 2, and set a non-rectangular detection frame having the estimated inclination as the second detection frame F2.
  • the detection frame setting unit 13 may set the detection frame with the highest reliability obtained in step S32 of FIG. 5 as the second detection frame F2 .
  • the detection frame setting unit 13 can detect the contour of the object P detected by the first detection unit 12, and compare the multiple non-rectangular detection frames stored in the storage unit 30 with the contour of the object P. Compare and set the detection frame including the outline of the object P as the second detection frame F2.
  • the object detection unit 10 may select the rectangular first detection frame F1 as one of the candidates for the second detection frame F2, and perform the processing of steps S32 to S36 in FIG. 5 .
  • the object detection unit 10 uses the first detection frame F1 as the second detection frame F2.
  • the object detection unit 10 may select a rectangular detection frame different in size from the first detection frame F1 as a candidate for the second detection frame F2 and perform the processing of steps S32 to S36 in FIG. 5 .
  • the above embodiment detects an object P as a product from an image of a shelf S of a retail store, but is not limited thereto, and can be applied to a method of detecting an object from an image including a plurality of objects.
  • the processing device 1 is configured to have the object detection unit 10 and the object recognition unit 20, but the object detection device having the object detection unit 10 and the object recognition unit 20 may be configured separately. object recognition device.
  • 1-processing device 2-photography device, 10-object detection unit, 11-image acquisition unit, 12-first detection unit, 13-detection frame setting unit, 14-second detection unit, 20- Object recognition unit, 30-storage unit, 100-object information acquisition system, F1-first detection frame, F2-second detection frame, P-object, S-shelf.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

一种物体检知方法和装置，该方法包括：图像获取步骤，获取包含物体的图像；第1检知步骤，使用矩形的第1检知框来检知图像中的物体；检知框设定步骤，设定与所检知的物体对应的非矩形的第2检知框；及第2检知步骤，使用第2检知框来检知物体。

Description

物体检知方法及物体检知装置 技术领域
本发明涉及一种从图像检知物体的物体检知方法及物体检知装置。
背景技术
随着智能零售时代的到来，货架的位置、商品的数量或商品的放置位置等信息对零售企业来说非常重要。并且，掌握货架上的商品的动向在确定销售活动和管理政策上具有重要意义。为了使人们不去货架上就能够掌握该货架上的商品的动向，需要检知及识别货架上的商品，并准确地掌握货架上的商品陈列信息。
在专利文献1中，公开有商品的检知方法的一例。在专利文献1中，公开有获取陈列商品的图像并检知图像中的商品，根据各商品的空间位置关系，截取各商品的图像并进行商品的分类。
专利文献1：中国专利申请公开第110738123号说明书
在专利文献1中，成为使用长方形或正方形等的矩形的框来截取各商品的图像的结构。但是，有时根据获取商品的图像时的摄影角度等，图像内的商品的外形变形而变得不是矩形。此时，有时在所截取的图像中不包含应检知的商品的一部分或包含除了应检知的商品以外的图像，从而导致商品的检知精度降低。
发明内容
本发明用于解决如上所述的课题，其目的在于获得能够提高物体的检知精度的物体检知方法及物体检知装置。
本发明所涉及的物体检知方法包括：图像获取步骤，获取包含物体的图像；第1检知 步骤，使用矩形的第1检知框来检知图像中的物体；检知框设定步骤，设定与所检知的物体对应的非矩形的第2检知框；及第2检知步骤，使用第2检知框来检知物体。
本发明所涉及的物体检知装置具备：图像获取部，获取包含物体的图像；第1检知部，使用矩形的第1检知框来检知图像中的物体；检知框设定部，设定与所检知的物体对应的非矩形的第2检知框；及第2检知部，使用第2检知框来检知物体。
发明效果
根据本发明的物体检知方法及物体检知装置，通过使用与所检知的物体对应的非矩形的第2检知框来检知物体，能够提高物体的检知精度。
附图说明
图1是实施方式1所涉及的物体信息获取系统的概略结构图。
图2是实施方式1所涉及的物体信息获取系统的控制框图。
图3是实施方式1所涉及的物体信息获取处理的流程图。
图4是由摄影装置2拍摄的货架的正面图像的一例。
图5是实施方式1所涉及的物体检知处理的流程图。
图6是基于实施方式1所涉及的第1检知部的检知结果的一例。
图7是基于实施方式1所涉及的第2检知部的检知结果的一例。
图8是实施方式2所涉及的物体信息获取处理的流程图。
具体实施方式
以下，参考附图对本发明的实施方式的物体检知方法及物体检知装置进行说明。另外，在各图中，对相同或相等的部分标注相同符号，并适当省略或简化其说明。并且，关于各图中所记载的结构，其形状、大小及配置等能够在本发明的范围内适当变更。
实施方式1.
图1是实施方式1所涉及的物体信息获取系统100的概略结构图。本实施方式的物体信 息获取系统100为如下系统：在超市等零售商店中使用，并且自动检知及识别作为容纳于商店内的货架S上的商品的物体P并获取容纳于货架S上的物体P的信息。物体信息获取系统100由处理装置1和摄影装置2构成。处理装置1为具备CPU及存储器的PC或云上的服务器等。摄影装置2为设置于商店的天花板或墙壁上且拍摄货架S的正面图像的照相机。处理装置1和摄影装置2通过有线或无线连接为可通信。由摄影装置2拍摄的图像发送至处理装置1。
图2是实施方式1所涉及的物体信息获取系统100的控制框图。处理装置1具有：物体检知部10，从图像检知物体P；物体识别部20，识别所检知的物体P；及存储部30。物体检知部10及物体识别部20为通过由CPU执行程序来实现的功能部。或者，物体检知部10及物体识别部20也可以通过专用处理电路来实现。
物体检知部10具有图像获取部11、第1检知部12、检知框设定部13及第2检知部14。图像获取部11获取由摄影装置2拍摄的图像并发送至第1检知部12。第1检知部12利用使用了深度学习的SSD(Single Shot Multibox Detector：目标检知算法)等算法来检知所获取的图像中的物体P。在第1检知部12中，使用矩形的第1检知框F1来进行物体P的检知。
检知框设定部13设定与由第1检知部12检知的物体P对应的非矩形的第2检知框F2。第2检知部14使用由检知框设定部13设定的第2检知框F2来检知物体P，并将检知结果发送至物体识别部20。
物体识别部20根据货架信息及商品信息来识别由物体检知部10的第2检知部14检知的图像中所包含的物体P。在物体识别部20中，通过使用了已知的机器学习的算法来识别物体P的种类及商品名称。
存储部30例如为RAM、ROM或闪存等易失性或非易失性的存储器。存储部30存储由物体检知部10及物体识别部20执行的程序及在程序中所使用的各种参数以及货架信息、商品信息、检知框数据及检知历史等。货架信息包含商店内的各货架S的位置、容纳于各货架S上的商品的分类、各货架S的节数及尺寸、各货架S上的检知框的数量。商品信息包含商品的种类及名称等识别信息。检知框数据为成为由检知框设定部13设定 的第2检知框F2的候选的多个非矩形的检知框的数据。
图3是实施方式1所涉及的物体信息获取处理的流程图。本处理由处理装置1定期执行。首先，进行系统的初始化(S1)。其中，对物体信息获取处理的各参数设定初始值。参数为检知框的数量、每一节货架能够检知的最大检知框数、货架的最大节数、检知框的种类等。然后，由摄影装置2拍摄货架S的正面图像，并由处理装置1的图像获取部11获取(S2)。
然后，物体检知部10根据所获取的图像执行物体检知处理(S3)。由此，检知图像中所包含的多个物体P。若通过物体检知处理检知到物体P，则由物体识别部20执行物体识别处理(S4)。由此，能够识别所检知的物体P，并获取容纳于货架S上的物体P的信息。所获取的物体P的信息被发送至管理服务器等，并用于掌握销售数据或商品管理等中。
接着，对本实施方式的物体检知处理进行说明。图4是由摄影装置2拍摄的货架S的正面图像的一例。其中，在通常的物体检知的算法中，使用长方形或正方形的矩形的检知框来进行物体检知。但是，在将摄影装置2设置于天花板或墙壁的上方且从上方拍摄货架S的图像的情况下，如图4所示，图像内的物体P的外形从矩形变形。因此，在本实施方式的物体检知处理中，在将检知物体的检知框设定为适合图像的变形的检知框之后，进行图像的检知。
图5是实施方式1所涉及的物体检知处理的流程图。在本处理中，首先根据所获取的图像，由第1检知部12进行第1检知(S31)。其中，使用矩形的第1检知框F1来检知物体P。图6是基于实施方式1所涉及的第1检知部12的检知结果的一例。
接着，通过检知框设定部13将存储于存储部30中的多个非矩形的检知框应用于所检知的各物体P，并获取每个检知框的可靠度(S32)。多个非矩形的检知框的形状为平行四边形、梯形、圆形或椭圆形等。并且，每个形状包含多个尺寸的框。然后，使非矩形的检知框的中心坐标与包含通过第1检知所检知的物体P的矩形的第1检知框F1的中心坐标一致，并获取非矩形的检知框的可靠度。关于每个检知框的可靠度，根据将多个非矩 形的检知框应用于物体P时的、检知框内的物体P的面积的大小(物体P与非矩形的检知框的共用部位的面积的大小)或检知框内的除了物体P以外的面积的大小(物体P与非矩形的检知框的不共用部位的面积的大小)来获取。具体而言，检知框内的物体P的面积越大，可靠度越高，并且检知框内的除了物体P以外的面积越小，可靠度越高。换言之，物体P的面积与检知框的面积的比例越高，可靠度越高。
接着，检知框设定部13进行多个非矩形的检知框的过滤(S33)。其中，从候选中排除多个非矩形的检知框中的可靠度低于预先设定的阈值的检知框。然后，检知框设定部13针对每个物体P获取可靠度为阈值以上的多个非矩形的检知框的中心坐标(S34)。检知框设定部13使用所获取的中心坐标来进行图像中的每个物体P的轮廓检知，并获取物体P的轮廓的位置信息(S35)。关于物体P的轮廓检知，可以使用边缘检测等已知的轮廓检知算法。其中，对多个非矩形的检知框的中心坐标中的每一个进行物体P的轮廓检知。另外，通过在步骤S33中过滤可靠度低的检知框，能够抑制发生非矩形的检知框的中心坐标位于物体P的外部而无法检知物体P的轮廓的异常。
检知框设定部13将非矩形的检知框与物体P的轮廓进行比较(S36)。然后，将包含物体P的所有轮廓的非矩形的检知框设定为第2检知框F2(S37)。对每个物体P进行步骤S36及步骤S37，从而设定与各物体P对应的第2检知框F2。所设定的第2检知框F2的形状及位置存储于存储部30中。其中，在包含物体P的所有轮廓的非矩形的检知框存在多个的情况下，将可靠度最高的检知框作为第2检知框F2。如上所述，可靠度根据物体P的面积与检知框的面积的比例来求出。并且，在可靠度最高的检知框存在多个的情况下，选择任意一个作为第2检知框F2。此时，使用计算重叠面积的重复检知算法，若彼此重叠面积的比例为阈值以上，则判断为相同，从而可以仅留下一个。由此，针对每个物体P选择最佳的检知框。
然后，由第2检知部14进行第2检知(S38)。在第2检知中，使用所设定的第2检知框F2来检知物体P。图7是基于实施方式1所涉及的第2检知部14的检知结果的一例。
如图7所示，通过进行本实施方式的物体检知处理，能够使用沿物体P的外形的第2检知框F2来进行物体P的检知。由此，即使在由于摄影角度的影响等而图像中的物体P 变形的情况下，也能够抑制检知不到物体P的一部分的情况及检知除了物体P以外的物体的情况，从而能够提高物体P的检知精度。尤其，对于容纳于货架S上的商品等密集配置的物体P，通过使用沿物体的外形的检知框，不仅提高精度，并且还提高检知速度。
另外，在物体检知处理中，为了进一步提高检知精度，可以进行下述处理。如图4所示，根据货架S上的物体P的位置，物体P的变形即物体P的倾斜度不同。例如，配置于货架S的右侧的物体P与配置于左侧的物体P的倾斜度不同。因此，检知框设定部13可以将在步骤S33中进行了过滤的多个非矩形的检知框聚集在物体P在货架S的位置上，将倾斜度与相同集群内的其他检知框不同的检知框检测为错误，并从候选中排除。或者，检知框设定部13可以从摄影装置2的位置信息估计物体P的倾斜度，将具有与所估计的倾斜度不同的倾斜度的非矩形的检知框检测为错误，并从候选中排除。而且，检知框设定部13可以根据货架信息将大于货架尺寸的非矩形的检知框检测为错误，并从候选中排除。
实施方式2.
图8是实施方式2所涉及的物体信息获取处理的流程图。本实施方式中的物体信息获取系统100的结构与实施方式1相同。
在本实施方式的物体信息获取处理中，与实施方式1同样地实施初始化(S1)及图像的获取(S2)。然后，物体检知部10根据检知历史来判断本次的检知是否为再检知(S11)。再检知是指如下情况：过去对货架S进行了物体检知并存储有对货架S的物体的第2检知框F2。在本次的检知不是再检知的情况即为第一次检知的情况下(S11：“否”)，与实施方式1同样地执行物体检知处理(S3)及物体识别处理(S4)。
另一方面，在本次的检知为再检知的情况下(S11：“是”)，获取在上次进行检知时由摄影装置2获取的图像与本次由摄影装置2拍摄的图像的差分(S12)。然后，对差分区域执行物体检知处理(S3)及物体识别处理(S4)。即，在本实施方式中，仅对自上次存在变更的区域进行物体P的检知及物体P的识别，并且其他区域中的物体P的信息被设为与上次相同。
根据本实施方式，通过仅对存在变化的区域进行物体检知处理及物体识别处理，能够实现再检知时的处理负担的减轻及处理速度的提高。
以上为实施方式的说明，但是上述实施方式能够变形及组合。例如，在实施方式1中，由检知框设定部13进行图5的步骤S32～S36的处理来设定了第2检知框F2，但是并不限定于此。例如，检知框设定部13可以从摄影装置2的位置信息估计物体P的倾斜度，并将具有所估计的倾斜度的非矩形的检知框设定为第2检知框F2。或者，检知框设定部13可以将在图5的步骤S32中所获取的可靠度最高的检知框设定为第2检知框F2。或者，检知框设定部13可以进行由第1检知部12检知的物体P的轮廓检知，将存储于存储部30中的多个非矩形的检知框与物体P的轮廓进行比较，并将包含物体P的轮廓的检知框设定为第2检知框F2。
并且，物体检知部10可以将矩形的第1检知框F1作为第2检知框F2的候选之一，并进行图5的步骤S32～S36的处理。在第1检知框F1包含物体P的所有轮廓且可靠度高于非矩形的检知框的情况下，物体检知部10将第1检知框F1作为第2检知框F2。或者，物体检知部10可以将与第1检知框F1的大小不同的矩形的检知框作为第2检知框F2的候选，并进行图5的步骤S32～S36的处理。
并且，上述实施方式从零售店的商店的货架S的图像检知作为商品的物体P，但是并不限定于此，能够应用于从包含多个物体的图像检知物体的方法。
而且，在上述实施方式中，处理装置1被设为具有物体检知部10和物体识别部20的结构，但是可以分体构成具有物体检知部10的物体检知装置与具有物体识别部20的物体识别装置。
符号说明
1-处理装置，2-摄影装置，10-物体检知部，11-图像获取部，12-第1检知部，13-检知框设定部，14-第2检知部，20-物体识别部，30-存储部，100-物体信息获取系统，F1-第1检知框，F2-第2检知框，P-物体，S-货架。

Claims (11)

  1. 一种物体检知方法，其包括：
    图像获取步骤，获取包含物体的图像；
    第1检知步骤，使用矩形的第1检知框来检知所述图像中的所述物体；
    检知框设定步骤，设定与所检知的所述物体对应的非矩形的第2检知框；及
    第2检知步骤，使用所述第2检知框来检知所述物体。
  2. 根据权利要求1所述的物体检知方法，其中，
    所述检知框设定步骤包括：
    将多个非矩形的检知框应用于在所述第1检知步骤中所检知的所述物体的步骤；
    获取所述多个非矩形的检知框中的每一个的可靠度的步骤；及
    从所述第2检知框的候选中排除所述多个非矩形的检知框中的所述可靠度小于预先设定的阈值的检知框的步骤。
  3. 根据权利要求2所述的物体检知方法，其中，
    所述检知框设定步骤包括：
    获取所述可靠度为所述阈值以上的检知框的中心坐标的步骤；
    使用所述中心坐标来进行所述物体的轮廓的检知的步骤；及
    将所述检知框与所述物体的所述轮廓进行比较，并将包含所述物体的所有所述轮廓的检知框设定为所述第2检知框的步骤。
  4. 根据权利要求2所述的物体检知方法，其中，
    所述检知框设定步骤包括将应用于所述物体的所述多个非矩形的检知框聚集在所述物体的位置上并从所述第2检知框的候选中排除倾斜度与相同集群内的其他检知框不同的检知框的步骤。
  5. 根据权利要求2所述的物体检知方法，其中，
    所述检知框设定步骤包括从拍摄所述图像的摄影装置的位置信息估计所述物体的倾斜度并从所述第2检知框的候选中排除具有与所估计的倾斜度不同的倾斜度的非矩形的检知框的步骤。
  6. 根据权利要求1所述的物体检知方法，其中，
    所述检知框设定步骤包括：
    进行所述物体的轮廓的检知的步骤；及
    将多个非矩形的检知框与所述物体的所述轮廓进行比较，并将包含所述物体的所有所述轮廓的检知框设定为所述第2检知框的步骤。
  7. 根据权利要求1所述的物体检知方法，其中，
    所述检知框设定步骤包括从拍摄所述图像的摄影装置的位置信息估计所述物体的倾斜度并将具有所估计的倾斜度的非矩形的检知框设定为所述第2检知框的步骤。
  8. 根据权利要求1所述的物体检知方法，其中，
    所述检知框设定步骤包括：
    将多个非矩形的检知框应用于在所述第1检知步骤中所检知的所述物体的步骤；
    获取所述多个非矩形的检知框中的每一个的可靠度的步骤；及
    将所述可靠度最高的检知框设定为所述第2检知框的步骤。
  9. 根据权利要求1至8中任一项所述的物体检知方法，其还具备判定是否为再检知的步骤，
    在为所述再检知的情况下，对上次的图像与本次的图像的差分区域实施所述第1检知步骤、所述检知框设定步骤及所述第2检知步骤。
  10. 根据权利要求1至9中任一项所述的物体检知方法，其中，
    所述第2检知框为平行四边形或梯形。
  11. 一种物体检知装置，其具备：
    图像获取部，获取包含物体的图像；
    第1检知部，使用矩形的第1检知框来检知所述图像中的所述物体；
    检知框设定部，设定与所检知的所述物体对应的非矩形的第2检知框；及
    第2检知部，使用所述第2检知框来检知所述物体。
PCT/CN2021/102347 2021-06-25 2021-06-25 物体检知方法及物体检知装置 WO2022266996A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202180099742.2A CN117616468A (zh) 2021-06-25 2021-06-25 物体检知方法及物体检知装置
JP2023579455A JP2024522881A (ja) 2021-06-25 2021-06-25 物体検知方法及び物体検知装置
PCT/CN2021/102347 WO2022266996A1 (zh) 2021-06-25 2021-06-25 物体检知方法及物体检知装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/102347 WO2022266996A1 (zh) 2021-06-25 2021-06-25 物体检知方法及物体检知装置

Publications (1)

Publication Number Publication Date
WO2022266996A1 true WO2022266996A1 (zh) 2022-12-29

Family

ID=84544000

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/102347 WO2022266996A1 (zh) 2021-06-25 2021-06-25 物体检知方法及物体检知装置

Country Status (3)

Country Link
JP (1) JP2024522881A (zh)
CN (1) CN117616468A (zh)
WO (1) WO2022266996A1 (zh)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1794267A (zh) * 2005-12-29 2006-06-28 兆日科技（深圳）有限公司 用于防伪信息识别过程中矩形框定位的方法
CN103065163A (zh) * 2013-02-04 2013-04-24 成都索贝数码科技股份有限公司 一种基于静态图片的快速目标检测识别系统及方法
JP2016157258A (ja) * 2015-02-24 2016-09-01 Kddi株式会社 人物領域検出装置、方法およびプログラム
CN108960174A (zh) * 2018-07-12 2018-12-07 广东工业大学 一种目标检测结果优化方法及装置
CN109657681A (zh) * 2018-12-28 2019-04-19 北京旷视科技有限公司 图片的标注方法、装置、电子设备及计算机可读存储介质
CN110334752A (zh) * 2019-06-26 2019-10-15 电子科技大学 一种基于梯形卷积的不规则形状物体检测方法
CN111598091A (zh) * 2020-05-20 2020-08-28 北京字节跳动网络技术有限公司 图像识别方法、装置、电子设备及计算可读存储介质
CN112183529A (zh) * 2020-09-23 2021-01-05 创新奇智(北京)科技有限公司 四边形物体检测、模型训练方法、装置、设备及存储介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1794267A (zh) * 2005-12-29 2006-06-28 兆日科技（深圳）有限公司 用于防伪信息识别过程中矩形框定位的方法
CN103065163A (zh) * 2013-02-04 2013-04-24 成都索贝数码科技股份有限公司 一种基于静态图片的快速目标检测识别系统及方法
JP2016157258A (ja) * 2015-02-24 2016-09-01 Kddi株式会社 人物領域検出装置、方法およびプログラム
CN108960174A (zh) * 2018-07-12 2018-12-07 广东工业大学 一种目标检测结果优化方法及装置
CN109657681A (zh) * 2018-12-28 2019-04-19 北京旷视科技有限公司 图片的标注方法、装置、电子设备及计算机可读存储介质
CN110334752A (zh) * 2019-06-26 2019-10-15 电子科技大学 一种基于梯形卷积的不规则形状物体检测方法
CN111598091A (zh) * 2020-05-20 2020-08-28 北京字节跳动网络技术有限公司 图像识别方法、装置、电子设备及计算可读存储介质
CN112183529A (zh) * 2020-09-23 2021-01-05 创新奇智(北京)科技有限公司 四边形物体检测、模型训练方法、装置、设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
XU YONGCHAO; FU MINGTAO; WANG QIMENG; WANG YUKANG; CHEN KAI; XIA GUI-SONG; BAI XIANG: "Gliding Vertex on the Horizontal Bounding Box for Multi-Oriented Object Detection", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, IEEE COMPUTER SOCIETY., USA, vol. 43, no. 4, 18 February 2020 (2020-02-18), USA , pages 1452 - 1459, XP011843410, ISSN: 0162-8828, DOI: 10.1109/TPAMI.2020.2974745 *

Also Published As

Publication number Publication date
CN117616468A (zh) 2024-02-27
JP2024522881A (ja) 2024-06-21

Similar Documents

Publication Publication Date Title
CN108985199B (zh) 商品取放操作的检测方法、装置及存储介质
US10212324B2 (en) Position detection device, position detection method, and storage medium
CN108549870B (zh) 一种对物品陈列进行鉴别的方法及装置
US20170053409A1 (en) Information processing apparatus, information processing method and program
US10217083B2 (en) Apparatus, method, and program for managing articles
EP3563345B1 (en) Automatic detection, counting, and measurement of lumber boards using a handheld device
US20070076922A1 (en) Object detection
US10074029B2 (en) Image processing system, image processing method, and storage medium for correcting color
JP2018048024A (ja) 物品管理装置
GB2430736A (en) Image processing
JP2017004505A (ja) 視覚分析に基づくプラノグラム・コンプライアンス照合のための方法及びシステム
Rosado et al. Supervised learning for Out-of-Stock detection in panoramas of retail shelves
JP2015041164A (ja) 画像処理装置、画像処理方法およびプログラム
EP3214604B1 (en) Orientation estimation method and orientation estimation device
JP2016201105A (ja) 情報処理装置及び情報処理方法
JPWO2018179361A1 (ja) 画像処理装置、画像処理方法、およびプログラム
EP3404513A1 (en) Information processing apparatus, method, and program
Saran et al. Robust visual analysis for planogram compliance problem
US20150116543A1 (en) Information processing apparatus, information processing method, and storage medium
CN113221617A (zh) 在图像中区分人群中的人
US20200005492A1 (en) Image processing device, image processing method, and recording medium
JP6769554B2 (ja) 物体識別装置、物体識別方法、計算装置、システムおよび記録媒体
WO2015079054A1 (en) Estimating gaze from un-calibrated eye measurement points
JP6244960B2 (ja) 物体認識装置、物体認識方法及び物体認識プログラム
CN111832381B (zh) 物体信息登记装置以及物体信息登记方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21946490

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2023579455

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 202180099742.2

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21946490

Country of ref document: EP

Kind code of ref document: A1