WO2023109151A1 - 冰箱内物品信息识别方法和冰箱 - Google Patents

冰箱内物品信息识别方法和冰箱 Download PDF

Info

Publication number
WO2023109151A1
WO2023109151A1 PCT/CN2022/110226 CN2022110226W WO2023109151A1 WO 2023109151 A1 WO2023109151 A1 WO 2023109151A1 CN 2022110226 W CN2022110226 W CN 2022110226W WO 2023109151 A1 WO2023109151 A1 WO 2023109151A1
Authority
WO
WIPO (PCT)
Prior art keywords
line
contour
item
detection
contour line
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/110226
Other languages
English (en)
French (fr)
Inventor
高洪波
孔令磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao Haier Refrigerator Co Ltd
Haier Smart Home Co Ltd
Original Assignee
Qingdao Haier Refrigerator Co Ltd
Haier Smart Home Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qingdao Haier Refrigerator Co Ltd, Haier Smart Home Co Ltd filed Critical Qingdao Haier Refrigerator Co Ltd
Publication of WO2023109151A1 publication Critical patent/WO2023109151A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the invention relates to the field of refrigeration devices, in particular to a method for identifying item information in a refrigerator and a refrigerator.
  • the identification of internal items has become a necessary function of smart refrigerators.
  • one or more cameras are installed inside the refrigerator to collect photos of the items in the refrigerator, and the collected photos are identified and processed and then sent to the user terminal for viewing by the user. Setting the camera above the items can avoid the problem of items occluding each other, and improve the accuracy of item information identification.
  • the wall surface of the bottle seat where the items are placed is generally made of frosted material to eliminate the influence of the mirror image of the items, but this method not only increases the production process, but also affects the appearance of the whole machine.
  • the object of the present invention is to provide a method for identifying item information in a refrigerator and a refrigerator.
  • an embodiment of the present invention provides a method for identifying information on items in a refrigerator, including steps:
  • the orientation of the detection line in the detection frame is opposite to the orientation of the contour line in the contour frame;
  • the corresponding item is a mirror image item
  • the corresponding item is a real item.
  • "obtaining a bird's-eye view image of the space in the bottle holder of the refrigerator” specifically includes:
  • a bird's-eye view image of the space inside the bottle seat is obtained by controlling the camera arranged in the middle of the top of the bottle seat.
  • identifying a contour box composed of intersecting contour lines in the bird's-eye view image specifically includes:
  • the contour lines intersect to form a contour box.
  • “segmenting a plurality of detection frames in the bird's-eye view image, each of which corresponds to an item" specifically includes:
  • the items in the bird's-eye view image include real items placed in the bottle holder and mirrored items corresponding to the real items.
  • the orientation of the detection line located in the detection frame is opposite to the orientation of the contour line located in the outline frame.
  • the method specifically includes:
  • the contour line is located at the upper edge of the contour box
  • the detection line is located at the lower line of the detection frame
  • the corresponding item is a mirror image item
  • the corresponding item is a real item.
  • the method specifically includes:
  • the contour line is located at the lower edge of the contour box
  • the detection line is located on the upper line of the detection frame
  • the corresponding item is a real item
  • the corresponding item is a mirror image item.
  • the method specifically includes:
  • the contour line is located at the left line of the contour box;
  • the detection line is located on the right line of the detection frame
  • the corresponding item is a mirror image item
  • the corresponding item is a real item.
  • the method also includes the steps of:
  • An embodiment of the present invention also provides a refrigerator, including: a bottle holder, a camera, a memory, and a processor, wherein,
  • the bottle holder includes a bottle holder top, a bottom and a bottle holder wall, and the camera is arranged at the middle position of the bottle holder top and is arranged vertically downward;
  • the bottom of the bottle seat is used for placing articles
  • At least one side of the bottle seat wall is a glass surface
  • the memory stores a computer program that can run on the processor, and the processor implements the steps of the method for identifying item information in the refrigerator as described in any one of the above implementation manners when executing the program.
  • the present invention adopts the edge detection algorithm and the Yolo V3 target detection algorithm to calibrate the contour frame and the detection frame corresponding to the item, obtain the contour line and the detection line, the detection line is located in the direction of the detection frame and the contour line is located in the contour frame
  • the orientation is opposite.
  • Fig. 1 is a schematic flowchart of a method for identifying information on items in a refrigerator in an embodiment of the present invention.
  • Fig. 2(a) is a schematic diagram of the space inside the refrigerator bottle holder in Embodiment 1 of the present invention (except for the outline for illustration and the items on adjacent sides, the rest of the content in the figure is omitted).
  • FIG. 2( b ) is a schematic flow chart of a method for identifying item information in a refrigerator in Embodiment 1 of the present invention.
  • Fig. 3(a) is a schematic diagram of the space inside the bottle holder of the refrigerator in the second embodiment of the present invention (except for the outline for illustration and the items on the adjacent sides, the rest of the content in the figure is omitted).
  • Fig. 3(b) is a schematic flowchart of a method for identifying item information in a refrigerator in Embodiment 2 of the present invention.
  • Fig. 4(a) is a schematic diagram of the space inside the bottle holder of the refrigerator in the third embodiment of the present invention (except for the outline for illustration and the items on the adjacent sides, the rest of the content in the figure is omitted).
  • FIG. 4( b ) is a schematic flow chart of a method for identifying item information in a refrigerator in Embodiment 3 of the present invention.
  • Fig. 5(a) is a schematic diagram of the space inside the refrigerator bottle holder in Embodiment 4 of the present invention (except for the outline for illustration and the items on adjacent sides, the rest of the content in the figure is omitted).
  • FIG. 5( b ) is a schematic flowchart of a method for identifying item information in a refrigerator according to Embodiment 4 of the present invention.
  • the present invention provides a method for identifying item information in a refrigerator, comprising steps:
  • the camera is set on the top of the bottle seat, and the object is photographed from above to obtain a bird's-eye view image.
  • the present invention preferably uses the camera It is located in the middle of the top of the bottle seat.
  • two or more cameras may also be arranged in the bottle holder to acquire two or more bird's-eye-view images of the space inside the refrigerator bottle holder taken from different angles.
  • the areas of the bottom of the bottle base and the wall of the bottle base captured by the camera usually have obvious pixel grayscale differences in the image, and obvious edge contours can be detected and identified.
  • an embodiment of the present invention utilizes an edge detection algorithm to identify the contour line of the edge of the bottle seat, and marks the contour lines intersecting in the middle region of the bird's-eye view image, and the contour lines intersect to form a contour frame 1, as shown in Figure 2(a)- 5(a).
  • the edge detection algorithm can adopt currently common algorithms such as Roberts, Sobel, Prewitt, Canny, Log, etc., and the specific algorithm content is the prior art, and will not be repeated here.
  • S3 Segment multiple detection frames in the bird's-eye view image, each of which corresponds to an item.
  • the items in the bottle seat will often cast corresponding shadows on the wall of the bottle seat, that is, its mirror image.
  • the Yolo V3 target detection algorithm uses the Yolo V3 target detection algorithm to calibrate the detection frame 2 corresponding to each item in the bird's-eye view image.
  • the specific content of the Yolo V3 target detection algorithm here is the existing technology, so I will not repeat it here.
  • the outline frame 1 is a square structure, which includes a contour line 11 positioned above the contour frame, a contour line 12 positioned below the contour frame, a contour line 13 positioned on the left side of the contour frame, and a contour line positioned on the right side of the contour frame. Azimuth of the contour line 14 .
  • selecting a contour line may be contour line 11 , contour line 12 , contour line 13 or contour line 14 .
  • the detection frame 2 is a square structure, which includes a detection line 21 located above the detection frame, a detection line 22 located below the detection frame, a detection line 23 located on the left side of the detection frame, And the detection line 24 located on the right side of the detection frame.
  • the selected detection line may be the detection line 21 , the detection line 22 , the detection line 23 or the detection line 24 .
  • the orientation of the selected detection line in the detection frame 2 is opposite to the orientation of the contour line in the contour frame 1.
  • the contour line 11 is selected in step S4
  • the detection line is selected in step S5.
  • a step is also included: acquiring a detection frame adjacent to the contour line.
  • the system only acquires the object detection frames existing on two surfaces adjacent to the contour line 11, and does not use this as a judgment standard for object detection frames on other surfaces.
  • the relative orientation of the selected detection line and the contour line is the same as the orientation of the contour line in the outline frame 1, it is judged whether the item identified in the bird's-eye view image is a real item or a mirror image item.
  • its detection frame 2a includes 21a located in the upper position, 22a located in the lower position, 23a located in the left position and 24a located in the right position .
  • the detection line 22a is selected as the judgment standard in the embodiment of the present invention in the step S5, because the relative orientation of the detection line 22a and the contour line 11 is that the detection line 22a is located at the contour line 11, and the contour line 11 is located at the upper position of the contour frame 1, so for the item a, the relative orientation of the detection line 22a and the contour line 11 is the same as that of the contour line 11 in the contour frame 1, that is, step S61 is executed, and the system Determine that item a is a mirror image item.
  • the detection frame 2b includes 21b at the upper position, 22b at the lower position, 23b at the left position and 24b at the right position.
  • the detection line 22b is selected as the judgment standard in the embodiment of the present invention in the step S5, because the relative orientation of the detection line 22b and the contour line 11 is that the detection line 22b is located at the contour line 11 , and the contour line 11 is located at the upper position of the contour frame 1, so for the item b, the relative orientation between the detection line 22b and the contour line 11 is opposite to the orientation of the contour line 11 in the contour frame 1, that is, step S62 is executed to determine the object b is the real item.
  • the present invention provides four specific methods for identifying items in refrigerator bottle holders, which are described in detail in the following four embodiments.
  • select contour line 11 as shown in Figure 2 (a), described identification method ( Figure 2 (b)) comprises steps:
  • S4A1 Select a contour line 11 , the contour line 11 is located on the upper edge of the contour frame 1 .
  • S5A1 Select a detection line 22 , the detection line 22 is located at the lower line of the detection frame 2 .
  • a step is further included: obtaining the detection frames on both sides adjacent to the contour line 11 .
  • the detection line 22a is above the contour line 11, it is judged that the object a is a mirror image;
  • the detection line 22b since the detection line 22b is located below the contour line 11, it is judged that the object b is a real item;
  • the detection line 22c since the detection line 22c is located above the contour line 11, it is judged that the item c is a mirror image item;
  • the detection frame 2d corresponding to the item d straddles the adjacent two sides of the contour line 11, but since the detection line 22d is located below the contour line 11, the object can also be judged according to the above method d is the real item.
  • the object detection frames on both sides of the contour line 11 acquired in the bird's-eye view can use the object information identification method in FIG. 2( b ) to determine whether the object is a real object or a mirror image.
  • S4B1 Select a contour line 12, the contour line 12 is located at the lower edge of the contour box 1.
  • S5B1 Select a detection line 21 , the detection line 21 is located on the upper line of the detection frame 2 .
  • a step is also included: obtaining the detection frames on both sides adjacent to the contour line 12. As shown in Figure 3(a), for the item e in the figure, since the detection line 21e is located above the contour line 12, it is judged that the item e is a real item; for the item f in the figure, since the detection line 21f is located below the contour line 12, it is judged that Item f is a mirrored item.
  • the object detection frames on both sides of the contour line 12 obtained in the bird's-eye view can use the object information identification method in FIG. 3( b ) to determine whether the object is a real object or a mirror image.
  • S4C1 Select a contour line 13, the contour line 13 is located at the left line of the contour box 1.
  • S5C1 Select a detection line 24 , the detection line 24 is located on the right line of the detection frame 2 .
  • a step is further included: obtaining the detection frames on both sides adjacent to the contour line 13 .
  • the detection line 24g is located on the left side of the contour line 13
  • it is judged that the item g is a mirror image item
  • the detection line 24h is located on the right side of the contour line 13 , that is, it is judged that the item h is a real item.
  • the object detection frames on both sides of the contour line 13 obtained in the bird's-eye view can use the object information identification method in FIG. 4( b ) to determine whether the object is a real object or a mirror image.
  • S4D1 Select a contour line 14 , the contour line 14 is located on the right line of the contour box 1 .
  • S5D1 Select a detection line 23 , the detection line 23 is located on the left line of the detection frame 2 .
  • a step is further included: obtaining the detection frames on both sides adjacent to the contour line 14 .
  • Fig. 5 (a) for the item i in the figure, since the detection line 23i is located on the left side of the contour line 14, it is judged that the item i is a real item; for the item j in the figure, since the detection line 23j is located on the contour line On the right side of 14, it is judged that item j is a mirror image item.
  • the object detection frames on both sides of the contour line 14 acquired in the bird's-eye view can use the object information identification method in FIG. 5( b ) to determine whether the object is a real object or a mirror image.
  • the method further includes steps after identifying the item in the bird's-eye view as a real item or a mirror image item:
  • S7 Exclude mirrored items, and count the number of real items in the bird's-eye view image.
  • the present invention also provides a refrigerator, which includes a bottle holder, a camera, a memory and a processor.
  • the bottle seat includes a top, a bottom, and a wall of the bottle seat, and the camera is set at the middle of the top of the bottle seat and is set vertically downward.
  • the bottom of the bottle seat is used for placing articles, and at least one side of the wall of the bottle seat is a glass surface.
  • the memory stores a computer program that can run on the processor, and when the processor executes the program, the steps of the above-mentioned method for identifying item information in the refrigerator are realized.
  • the present invention uses the edge detection algorithm and the Yolo V3 target detection algorithm to calibrate the contour frame and the detection frame corresponding to the item, and obtain the contour line and the detection line. , by comparing whether the relative orientation of the detection line and the contour line is the same as the orientation of the contour line in the contour box, it is judged whether the object detected in the bird's-eye view image is a real object or a mirror image, so as to eliminate the interference of the mirror image and improve the recognition algorithm. Accuracy.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

一种冰箱内物品信息识别方法和冰箱，获取鸟瞰图像并识别其内的轮廓框，在鸟瞰图像中分割出多个检测框；选取一轮廓线，并识别其位于轮廓框的方位；选取一检测线，其位于检测框的方位与轮廓线位于轮廓框的方位相反；根据检测线与轮廓线相对方位与轮廓线位于轮廓框的方位是否相同，判断对应物品为镜像物品或真实物品。本方法可排除镜像物品干扰，提高识别算法准确率。

Description

冰箱内物品信息识别方法和冰箱
本申请要求了申请日为2021年12月14日，申请号为202111529525.0，发明名称为“冰箱内物品信息识别方法、冰箱和计算机存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及制冷装置领域，尤其涉及一种冰箱内物品信息识别方法和冰箱。
背景技术
随着电器智能化的发展，对内部物品进行识别已经是智能冰箱所必需的功能。通常为了实现对冰箱内存放物品的识别，冰箱内部都安装有一个或多个摄像头用于采集冰箱内物品照片，并对采集到的照片进行识别处理后发送到使用终端，以供用户查看。将摄像头设于物品上方可以避免物品互相遮挡的问题，而提高物品信息识别的准确率。
然而，放置于瓶座内部的物品会在瓶座玻璃柜上产生影子，现有摄像头安装在瓶座中间位置方案采集的物品照片往往会出现物品镜像问题，容易将一件物品识别成两件物品，严重影响识别效果。
现有的方法一般将放置物品的瓶座壁面采用磨砂材质以消除物品镜像的影响，但是该方法不但增加了生产工艺，还会影响整机的外观效果。
发明内容
本发明的目的在于提供一种冰箱内物品信息识别方法和冰箱。
为实现上述发明目的之一，本发明一实施方式提供一种冰箱内物品信息识别方法，包括步骤：
获取冰箱瓶座内空间的鸟瞰图像；
在所述鸟瞰图像中识别出相交轮廓线组成的轮廓框；
在所述鸟瞰图像中分割出多个检测框，每个所述检测框对应于一个物品；
选取一轮廓线，并识别所述轮廓线位于轮廓框的方位；
选取一检测线，所述检测线位于检测框的方位与所述轮廓线位于轮廓框的方位相反；
若所述检测线与所述轮廓线相对方位与所述轮廓线位于轮廓框的方位相同，则对应的物品为镜像物品；
若所述检测线与所述轮廓线相对方位与所述轮廓线位于轮廓框的方位相反，则对应的物品为真实物品。
作为本发明一实施方式的进一步改进，“获取冰箱瓶座内空间的鸟瞰图像”具体包 括：
控制设于所述瓶座顶部中间的摄像机拍摄得到所述瓶座内空间的鸟瞰图像。
作为本发明一实施方式的进一步改进，“在所述鸟瞰图像中识别出相交轮廓线组成的轮廓框”具体包括：
利用边缘检测算法，标定出所述鸟瞰图像中部区域相交的轮廓线；
所述轮廓线相交组成轮廓框。
作为本发明一实施方式的进一步改进，“在所述鸟瞰图像中分割出多个检测框，每个所述检测框对应于一个物品”具体包括：
利用Yolo V3目标检测算法，标定出所述鸟瞰图像中每个物品所对应的检测框；
所述鸟瞰图像中的物品包括放置于所述瓶座内的真实物品以及对应于所述真实物品的镜像物品。
作为本发明一实施方式的进一步改进，在“选取一检测线，所述检测线位于检测框的方位与所述轮廓线位于轮廓框的方位相反”之前还包括步骤：
获取与所述轮廓线相邻两侧的检测框。
作为本发明一实施方式的进一步改进，所述方法具体包括：
选取一轮廓线，所述轮廓线位于所述轮廓框的上边线；
选取一检测线，所述检测线位于所述检测框的下边线；
若所述检测线位于所述轮廓线上方，则对应的物品为镜像物品；
若所述检测线位于所述轮廓线下方，则对应的物品为真实物品。
作为本发明一实施方式的进一步改进，所述方法具体包括：
选取一轮廓线，所述轮廓线位于所述轮廓框的下边线；
选取一检测线，所述检测线位于所述检测框的上边线；
若所述检测线位于所述轮廓线上方，则对应的物品为真实物品；
若所述检测线位于所述轮廓线下方，则对应的物品为镜像物品。
作为本发明一实施方式的进一步改进，所述方法具体包括：
选取一轮廓线，所述轮廓线位于所述轮廓框的左边线；
选取一检测线，所述检测线位于所述检测框的右边线；
若所述检测线位于所述轮廓线左方，则对应的物品为镜像物品；
若所述检测线位于所述轮廓线右方，则对应的物品为真实物品。
作为本发明一实施方式的进一步改进，所述方法还包括步骤：
排除镜像物品，统计所述鸟瞰图像中真实物品的数量。
本发明一实施方式还提供一种冰箱，包括：瓶座，摄像头以及存储器和处理器，其中，
所述瓶座包括瓶座顶部、底部和瓶座壁面，所述摄像头设于所述瓶座顶部中间位置，竖直向下设置；
所述瓶座底部用于放置物品；
所述瓶座壁面至少一面为玻璃面；
所述存储器存储可在所述处理器上运行的计算机程序，所述处理器执行所述程序时实现如上任意一种实施方式中所述的冰箱内物品信息识别方法的步骤。
与现有技术相比，本发明采用边缘检测算法和Yolo V3目标检测算法标定轮廓框和物品对应的检测框，获取轮廓线和检测线，检测线位于检测框的方位与轮廓线位于轮廓框的方位相反，通过比较检测线与轮廓线相对方位与轮廓线位于轮廓框的方位是否相同，来判断鸟瞰图像中检测到的物品是真实物品还是镜像物品，以此可排除镜像物品的干扰，提高识别算法的准确率。
附图说明
图1是本发明一实施方式中一种冰箱内物品信息识别方法的流程示意图。
图2(a)是本发明实施例一中拍摄冰箱瓶座内空间的示意简图(除用以说明的轮廓线及相邻两侧的物品外，省略图中其余内容)。
图2(b)是本发明实施例一中冰箱内物品信息识别方法的流程示意图。
图3(a)是本发明实施例二中拍摄冰箱瓶座内空间的示意简图(除用以说明的轮廓线及相邻两侧的物品外，省略图中其余内容)。
图3(b)是本发明实施例二中冰箱内物品信息识别方法的流程示意图。
图4(a)是本发明实施例三中拍摄冰箱瓶座内空间的示意简图(除用以说明的轮廓线及相邻两侧的物品外，省略图中其余内容)。
图4(b)是本发明实施例三中冰箱内物品信息识别方法的流程示意图。
图5(a)是本发明实施例四中拍摄冰箱瓶座内空间的示意简图(除用以说明的轮廓线及相邻两侧的物品外，省略图中其余内容)。
图5(b)是本发明实施例四中冰箱内物品信息识别方法的流程示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合本申请具体实施方式及相应的附图对本申请技术方案进行清楚、完整地描述。显然，所描述的实施方式仅是本申请一部分实施方式，而不是全部的实施方式。基于本申请中的实施方式，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施方式，都属于本申请保护的范围。
下面详细描述本发明的实施方式，实施方式的示例在附图中示出，其中自始至终相 同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施方式是示例性的，仅用于解释本发明，而不能理解为对本发明的限制。
为方便说明，本文使用表示空间相对位置的术语来进行描述，例如“上”、“下”、“后”、“前”等，用来描述附图中所示的一个单元或者特征相对于另一个单元或特征的关系。空间相对位置的术语可以包括设备在使用或工作中除了图中所示方位以外的不同方位。例如，如果将图中的装置翻转，则被描述为位于其他单元或特征“下方”或“上方”的单元将位于其他单元或特征“下方”或“上方”。因此，示例性术语“下方”可以囊括下方和上方这两种空间方位。
如图1所示，本发明提供一种冰箱内物品信息识别方法，包括步骤：
S1：获取冰箱瓶座内空间的鸟瞰图像。
在本实施方式中，摄像头设于所述瓶座顶部，从上方拍摄物品得到鸟瞰图像，为避免瓶座内物品间的相互遮挡而不利于物品信息进行检测识别，所以，本发明优选为将摄像头设于所述瓶座顶部中间位置。
当然，在本发明的另一些实施方式中，也可在瓶座内设置两个或更多个摄像头，用于获取两张或更多不同角度拍摄的冰箱瓶座内空间的鸟瞰图像。
S2：在所述鸟瞰图像中识别出相交轮廓线组成的轮廓框。
由于瓶座底部和瓶座壁面材质不同，因此通过摄像头拍摄得到的瓶座底部和瓶座壁面等区域在图像中通常具有较明显的像素灰度差异，能够被检测识别到明显的边缘轮廓线。
具体的，本发明一实施方式利用边缘检测算法，识别瓶座边缘轮廓线，标定出所述鸟瞰图像中部区域相交的轮廓线，所述轮廓线相交组成轮廓框1，如图2(a)-5(a)所示。这里，边缘检测算法可以采用诸如Roberts、Sobel、Prewitt、Canny、Log等目前常见的算法，具体算法内容为现有技术，在此不过多赘述。
S3：在所述鸟瞰图像中分割出多个检测框，每个所述检测框对应于一个物品。
在瓶座壁面不采用磨砂材质的情况下，瓶座内物品往往会在瓶座壁面投射出相应的影子，即其镜像物品，所以，本发明中拍摄获得的鸟瞰图像中的物品包括放置于所述瓶座内的真实物品以及对应于所述真实物品的镜像物品。
具体的，在鸟瞰图像中识别其内存在的物品，利用Yolo V3目标检测算法，标定出所述鸟瞰图像中每个物品所对应的检测框2。这里Yolo V3目标检测算法具体内容为现有技术，在此不过多赘述。
S4：选取一轮廓线，并识别所述轮廓线位于轮廓框的方位。
具体的，所述轮廓框1为四方形结构，其包括位于轮廓框上方位的轮廓线11、位于轮廓框下方位的轮廓线12、位于轮廓框左方位的轮廓线13、以及位于轮廓框右方位的 轮廓线14。
示例性的，在本发明实施方式中，选取一轮廓线可以是轮廓线11、轮廓线12、轮廓线13或轮廓线14。
S5：选取一检测线，所述检测线位于检测框的方位与所述轮廓线位于轮廓框的方位相反。
在本发明一实施方式中，所述检测框2为四方形结构，其包括位于检测框上方位的检测线21、位于检测框下方位的检测线22、位于检测框左方位的检测线23、以及位于检测框右方位的检测线24。
这里，选取的检测线可以是检测线21、检测线22、检测线23或检测线24。在本发明实施方式中，选取的检测线位于检测框2的方位与所述轮廓线位于轮廓框1的方位相反，示例性的，如果在步骤S4中选取轮廓线11，则步骤S5中选取检测线22；如果在步骤S4中选取轮廓线12，则步骤S5中选取检测线21；如果在步骤S4中选取轮廓线13，则步骤S5中选取检测线24；如果在步骤S4中选取轮廓线14，则步骤S5中选取检测线23。
更进一步的，在“选取一检测线”之前，还包括步骤：获取与所述轮廓线相邻的检测框。示例性的，如果在步骤S4中选取轮廓线11，则此时系统只获取与轮廓线11相邻两面上存在的物品检测框，对于其他面上的物品检测框不以此作判断标准。
根据选取的检测线与轮廓线相对方位与所述轮廓线位于轮廓框1的方位是否相同来判断鸟瞰图像中识别的物品为真实物品或镜像物品。
S61：若所述检测线与所述轮廓线相对方位与所述轮廓线位于轮廓框1的方位相同，则对应的物品为镜像物品。
S62：若所述检测线与所述轮廓线相对方位与所述轮廓线位于轮廓框1的方位相反，则对应的物品为真实物品。
示例性的，如图2(a)所示，对于鸟瞰图像中识别的物品a，其检测框2a包括位于上方位的21a、位于下方位的22a、位于左方位的23a以及位于右方位的24a。若在步骤S4中选取轮廓线11，则步骤S5中选取检测线22a来作为本发明实施方式中的判断标准，由于其检测线22a与所述轮廓线11的相对方位为检测线22a位于轮廓线11的上方位，且轮廓线11位于轮廓框1的上方位，所以对于物品a，其检测线22a与轮廓线11相对方位与轮廓线11位于轮廓框1的方位相同，即执行S61步骤，系统判断物品a为镜像物品。
对于鸟瞰图像中识别的物品b，其检测框2b包括位于上方位的21b、位于下方位的22b、位于左方位的23b以及位于右方位的24b。若在步骤S4中选取轮廓线11，则步骤S5中选取检测线22b来作为本发明实施方式中的判断标准，由于检测线22b与所述轮 廓线11的相对方位为检测线22b位于轮廓线11的下方位，且轮廓线11位于轮廓框1的上方位，所以对于物品b，其检测线22b与轮廓线11相对方位与轮廓线11位于轮廓框1的方位相反，即执行S62步骤，判断物品b为真实物品。
对于图2(a)中物品c和物品d，根据上述的判断方法即可判断物品c为镜像物品，物品d为真实物品。
由于选取的轮廓线和检测线不同，本发明给出四种具体的关于冰箱瓶座内物品的识别方法，以下以四种实施例具体说明。
实施例一
在实施例一中，选取轮廓线11，如图2(a)所示，所述识别方法(图2(b))包括步骤：
S4A1：选取一轮廓线11，所述轮廓线11位于所述轮廓框1的上边线。
S5A1：选取一检测线22，所述检测线22位于所述检测框2的下边线。
S61A1：若所述检测线22位于所述轮廓线11上方，则对应的物品为镜像物品。
S62A1：若所述检测线22位于所述轮廓线11下方，则对应的物品为真实物品。
同样的，在选取一检测线22之前还包括步骤：获取与所述轮廓线11相邻两侧的检测框。如图2(a)，对于图中物品a，由于检测线22a位于轮廓线11上方，即判断物品a为镜像物品；对于图中物品b，由于检测线22b位于轮廓线11下方，即判断物品b为真实物品；对于图中物品c，由于检测线22c位于轮廓线11上方，即判断物品c为镜像物品；对于图中物品d，由于物品d为高度较高的大物品件，如大瓶装的牛奶或饮料，在获取的鸟瞰图中，物品d所对应的检测框2d横跨所述轮廓线11相邻两侧，但由于检测线22d位于轮廓线11下方，同样可根据上述方法判断物品d为真实物品。以下另外三种实施中判断真实物品或是镜像物品的方法均可用于对瓶座内大物品的识别，不会出现因物品体积不同而识别效果有差异的情况。
在此实施方式中，鸟瞰图中获取的在轮廓线11相邻两侧的物品检测框均可采用图2(b)中的物品信息识别方法来判断所述物品为真实物品或是镜像物品。
实施例二
在实施例二中，选取轮廓线12，如图3(a)所示，所述识别方法(图3(b)包括步骤：
S4B1：选取一轮廓线12，所述轮廓线12位于所述轮廓框1的下边线。
S5B1：选取一检测线21，所述检测线21位于所述检测框2的上边线。
S61B1：若所述检测线21位于所述轮廓线12的上方，则对应的物品为真实物品。
S62B1：若所述检测线21位于所述轮廓线12的下方，则对应的物品为镜像物品。
同样的，在选取一检测线21之前还包括步骤：获取与所述轮廓线12相邻两侧的检 测框。如图3(a)，对于图中物品e，由于检测线21e位于轮廓线12上方，即判断物品e为真实物品；对于图中物品f，由于检测线21f位于轮廓线12的下方，即判断物品f为镜像物品。
在此实施方式中，鸟瞰图中获取的在轮廓线12相邻两侧的物品检测框均可采用图3(b)中的物品信息识别方法来判断所述物品为真实物品或是镜像物品。
实施例三
在实施例三中，选取轮廓线13，如图4(a)所示，所述识别方法(图4(b)包括步骤：
S4C1：选取一轮廓线13，所述轮廓线13位于所述轮廓框1的左边线。
S5C1：选取一检测线24，所述检测线24位于所述检测框2的右边线。
S61C1：若所述检测线24位于所述轮廓线13的左方，则对应的物品为镜像物品。
S62C1：若所述检测线24位于所述轮廓线13的右方，则对应的物品为真实物品。
同样的，在选取一检测线24之前还包括步骤：获取与所述轮廓线13相邻两侧的检测框。如图4(a)，对于图中物品g，由于检测线24g位于轮廓线13的左方，即判断物品g为镜像物品；对于图中物品h，由于检测线24h位于轮廓线13的右方，即判断物品h为真实物品。
在此实施方式中，鸟瞰图中获取的在轮廓线13相邻两侧的物品检测框均可采用图4(b)中的物品信息识别方法来判断所述物品为真实物品或是镜像物品。
实施例四
在实施例四中，选取轮廓线14，如图5(a)所示，所述识别方法(图5(b)包括步骤：
S4D1：选取一轮廓线14，所述轮廓线14位于所述轮廓框1的右边线。
S5D1：选取一检测线23，所述检测线23位于所述检测框2的左边线。
S61D1：若所述检测线23位于所述轮廓线14的左方，则对应的物品为真实物品。
S62D1：若所述检测线23位于所述轮廓线14的右方，则对应的物品为镜像物品。
同样的，在选取一检测线23之前还包括步骤：获取与所述轮廓线14相邻两侧的检测框。如图5(a)，对于图中物品i，由于检测线23i位于所述轮廓线14的左方，即判断物品i为真实物品；对于图中物品j，由于检测线23j位于所述轮廓线14的右方，即判断物品j为镜像物品。
在此实施方式中，鸟瞰图中获取的在轮廓线14相邻两侧的物品检测框均可采用图5(b)中的物品信息识别方法来判断所述物品为真实物品或是镜像物品。
更进一步的，所述方法在识别鸟瞰图中物品为真实物品或镜像物品之后还包括步骤：
S7：排除镜像物品，统计所述鸟瞰图像中真实物品的数量。
本发明还提供一种冰箱，所述冰箱包括瓶座、摄像头以及存储器和处理器。
进一步的，所述瓶座包括瓶座顶部、底部和瓶座壁面，所述摄像头设于所述瓶座顶部中间位置，竖直向下设置。
所述瓶座底部用于放置物品，所述瓶座壁面至少一面为玻璃面。
所述存储器存储可在所述处理器上运行的计算机程序，所述处理器执行所述程序时实现上述的冰箱内物品信息识别方法的步骤。
综上所述，本发明采用边缘检测算法和Yolo V3目标检测算法标定轮廓框和物品对应的检测框，获取轮廓线和检测线，检测线位于检测框的方位与轮廓线位于轮廓框的方位相反，通过比较检测线与轮廓线相对方位与轮廓线位于轮廓框的方位是否相同，来判断鸟瞰图像中检测到的物品是真实物品还是镜像物品，以此可排除镜像物品的干扰，提高识别算法的准确率。
应当理解，虽然本说明书按照实施方式加以描述，但并非每个实施方式仅包含一个独立的技术方案，说明书的这种叙述方式仅仅是为清楚起见，本领域技术人员应当将说明书作为一个整体，各实施方式中的技术方案也可以经适当组合，形成本领域技术人员可以理解的其他实施方式。
上文所列出的一系列的详细说明仅仅是针对本发明的可行性实施方式的具体说明，它们并非用以限制本发明的保护范围，凡未脱离本发明技艺精神所作的等效实施方式或变更均应包含在本发明的保护范围之内。

Claims (10)

  1. 一种冰箱内物品信息识别方法，其特征在于，包括步骤：
    获取冰箱瓶座内空间的鸟瞰图像；
    在所述鸟瞰图像中识别出相交轮廓线组成的轮廓框；
    在所述鸟瞰图像中分割出多个检测框，每个所述检测框对应于一个物品；
    选取一轮廓线，并识别所述轮廓线位于轮廓框的方位；
    选取一检测线，所述检测线位于检测框的方位与所述轮廓线位于轮廓框的方位相反；
    若所述检测线与所述轮廓线相对方位与所述轮廓线位于轮廓框的方位相同，则对应的物品为镜像物品；
    若所述检测线与所述轮廓线相对方位与所述轮廓线位于轮廓框的方位相反，则对应的物品为真实物品。
  2. 根据权利要求1所述的冰箱内物品信息识别方法，其特征在于，“获取冰箱瓶座内空间的鸟瞰图像”具体包括：
    控制设于所述瓶座顶部中间的摄像机拍摄得到所述瓶座内空间的鸟瞰图像。
  3. 根据权利要求2所述的冰箱内物品信息识别方法，其特征在于，“在所述鸟瞰图像中识别出相交轮廓线组成的轮廓框”具体包括：
    利用边缘检测算法，标定出所述鸟瞰图像中部区域相交的轮廓线；
    所述轮廓线相交组成轮廓框。
  4. 根据权利要求3所述的冰箱内物品信息识别方法，其特征在于，“在所述鸟瞰图像中分割出多个检测框，每个所述检测框对应于一个物品”具体包括：
    利用Yolo V3目标检测算法，标定出所述鸟瞰图像中每个物品所对应的检测框；
    所述鸟瞰图像中的物品包括放置于所述瓶座内的真实物品以及对应于所述真实物品的镜像物品。
  5. 根据权利要求4所述的冰箱内物品信息识别方法，其特征在于，在“选取一检测线，所述检测线位于检测框的方位与所述轮廓线位于轮廓框的方位相反”之前还包括步骤：
    获取与所述轮廓线相邻两侧的检测框。
  6. 根据权利要求5所述的冰箱内物品信息识别方法，其特征在于，所述方法具体包括：
    选取一轮廓线，所述轮廓线位于所述轮廓框的上边线；
    选取一检测线，所述检测线位于所述检测框的下边线；
    若所述检测线位于所述轮廓线上方，则对应的物品为镜像物品；
    若所述检测线位于所述轮廓线下方，则对应的物品为真实物品。
  7. 根据权利要求5所述的冰箱内物品信息识别方法，其特征在于，所述方法具体包括：
    选取一轮廓线，所述轮廓线位于所述轮廓框的下边线；
    选取一检测线，所述检测线位于所述检测框的上边线；
    若所述检测线位于所述轮廓线上方，则对应的物品为真实物品；
    若所述检测线位于所述轮廓线下方，则对应的物品为镜像物品。
  8. 根据权利要求5所述的冰箱内物品信息识别方法，其特征在于，所述方法具体包括：
    选取一轮廓线，所述轮廓线位于所述轮廓框的左边线；
    选取一检测线，所述检测线位于所述检测框的右边线；
    若所述检测线位于所述轮廓线左方，则对应的物品为镜像物品；
    若所述检测线位于所述轮廓线右方，则对应的物品为真实物品。
  9. 根据权利要求1所述的冰箱内物品信息识别方法，其特征在于，所述方法还包括步骤：
    排除镜像物品，统计所述鸟瞰图像中真实物品的数量。
  10. 一种冰箱，包括：瓶座，摄像头以及存储器和处理器，其特征在于，
    所述瓶座包括瓶座顶部、底部和瓶座壁面，所述摄像头设于所述瓶座顶部中间位置，竖直向下设置；
    所述瓶座底部用于放置物品；
    所述瓶座壁面至少一面为玻璃面；
    所述存储器存储可在所述处理器上运行的计算机程序，所述处理器执行所述程序时实现如权利要求1-9任意一项所述冰箱内物品信息识别方法的步骤。
PCT/CN2022/110226 2021-12-14 2022-08-04 冰箱内物品信息识别方法和冰箱 Ceased WO2023109151A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111529525.0 2021-12-14
CN202111529525.0A CN114863265B (zh) 2021-12-14 2021-12-14 冰箱内物品信息识别方法、冰箱和计算机存储介质

Publications (1)

Publication Number Publication Date
WO2023109151A1 true WO2023109151A1 (zh) 2023-06-22

Family

ID=82627596

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/110226 Ceased WO2023109151A1 (zh) 2021-12-14 2022-08-04 冰箱内物品信息识别方法和冰箱

Country Status (2)

Country Link
CN (1) CN114863265B (zh)
WO (1) WO2023109151A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116416515B (zh) * 2022-11-29 2026-03-20 青岛海尔电冰箱有限公司 储物设备的物品识别方法及装置
CN116863442B (zh) * 2023-06-19 2025-10-28 重庆长安汽车股份有限公司 车位的检测方法、装置、车辆及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110738125A (zh) * 2019-09-19 2020-01-31 平安科技（深圳）有限公司 利用Mask R-CNN选择检测框的方法、装置及存储介质
WO2021124654A1 (ja) * 2019-12-18 2021-06-24 コベルコ建機株式会社 作業支援サーバ、作業支援方法および作業支援システム
CN113158876A (zh) * 2021-04-16 2021-07-23 青岛海尔电冰箱有限公司 冰箱内物品信息识别方法、冰箱和计算机存储介质
US20210334944A1 (en) * 2020-04-28 2021-10-28 Oyla, Inc. Device and method of object detection

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101334900B (zh) * 2008-08-01 2011-07-27 北京大学 一种基于图像的绘制方法
DE102011001127A1 (de) * 2011-03-07 2012-09-13 Miho Holding-Gmbh Inspektionsvorrichtung für Leergut und Verfahren zur Inspektion von Leergut
WO2018146997A1 (ja) * 2017-02-07 2018-08-16 日本電気株式会社 立体物検出装置
CN112613392B (zh) * 2020-12-18 2024-07-23 北京国家新能源汽车技术创新中心有限公司 基于语义分割的车道线检测方法、装置、系统及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110738125A (zh) * 2019-09-19 2020-01-31 平安科技（深圳）有限公司 利用Mask R-CNN选择检测框的方法、装置及存储介质
WO2021124654A1 (ja) * 2019-12-18 2021-06-24 コベルコ建機株式会社 作業支援サーバ、作業支援方法および作業支援システム
US20210334944A1 (en) * 2020-04-28 2021-10-28 Oyla, Inc. Device and method of object detection
CN113158876A (zh) * 2021-04-16 2021-07-23 青岛海尔电冰箱有限公司 冰箱内物品信息识别方法、冰箱和计算机存储介质

Also Published As

Publication number Publication date
CN114863265A (zh) 2022-08-05
CN114863265B (zh) 2025-08-29

Similar Documents

Publication Publication Date Title
KR102319177B1 (ko) 이미지 내의 객체 자세를 결정하는 방법 및 장치, 장비, 및 저장 매체
TWI485650B (zh) 用於多相機校準之方法及配置
CN113076907B (zh) 冰箱内物品信息标识方法、冰箱和计算机存储介质
CN113095240B (zh) 冰箱内物品信息识别方法、冰箱和计算机存储介质
TWI694352B (zh) 互動行為檢測方法、裝置、系統及設備
WO2023185779A1 (zh) 冰箱内物品信息识别方法
WO2023109151A1 (zh) 冰箱内物品信息识别方法和冰箱
CN105704472A (zh) 一种识别儿童用户的电视控制方法和系统
CN116416556A (zh) 物品信息识别方法、制冷设备和计算机存储介质
WO2022217928A1 (zh) 冰箱内物品信息标识方法、冰箱和计算机存储介质
US9924865B2 (en) Apparatus and method for estimating gaze from un-calibrated eye measurement points
Pahwa et al. Locating 3D object proposals: A depth-based online approach
Svoboda et al. Matching in catadioptric images with appropriate windows, and outliers removal
Díaz-Vilariño et al. Door recognition in cluttered building interiors using imagery and LiDAR data
CN107886540B (zh) 制冷设备内物品识别定位方法和制冷设备
CN106845378B (zh) 一种自对向图像中识别人体目标的方法
CN117132564A (zh) 基于YOLOv3的蓝宝石衬底表面缺陷检测方法及系统
CN119687791B (zh) 基于多视角图像采集分析的智能测量方法及系统
JP6255819B2 (ja) 計測用コンピュータプログラム、計測装置及び計測方法
CN119648949A (zh) 三维场景重建方法及其装置
WO2023185280A1 (zh) 冰箱瓶座内物品信息识别方法和冰箱
CN113537199B (zh) 图像边界框筛选方法、系统、电子装置及介质
CN116091613A (zh) 一种摄像头清晰度检测方法
WO2019024498A1 (zh) 一种定位系统以及定位方法
CN117788742A (zh) 非常规件识别方法、装置、计算机设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22905913

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22905913

Country of ref document: EP

Kind code of ref document: A1