WO2022217928A1 - 冰箱内物品信息标识方法、冰箱和计算机存储介质 - Google Patents

冰箱内物品信息标识方法、冰箱和计算机存储介质 Download PDF

Info

Publication number
WO2022217928A1
WO2022217928A1 PCT/CN2021/132850 CN2021132850W WO2022217928A1 WO 2022217928 A1 WO2022217928 A1 WO 2022217928A1 CN 2021132850 W CN2021132850 W CN 2021132850W WO 2022217928 A1 WO2022217928 A1 WO 2022217928A1
Authority
WO
WIPO (PCT)
Prior art keywords
refrigerator
item
bottle
bottle holder
article
Prior art date
Application number
PCT/CN2021/132850
Other languages
English (en)
French (fr)
Inventor
高洪波
Original Assignee
青岛海尔电冰箱有限公司
海尔智家股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 青岛海尔电冰箱有限公司, 海尔智家股份有限公司 filed Critical 青岛海尔电冰箱有限公司
Publication of WO2022217928A1 publication Critical patent/WO2022217928A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
    • G06V20/36—Indoor scenes
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D29/00—Arrangement or mounting of control or safety devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components

Definitions

  • the invention relates to the field of refrigeration devices, in particular to a method for identifying item information in a refrigerator, a refrigerator and a computer storage medium.
  • the identification of internal ingredients has become a necessary function of smart refrigerators.
  • one or more cameras are installed in the internal storage space of the refrigerator to photograph the ingredients, and the captured photos are processed to generate an identification result and sent to the user terminal for viewing by the user.
  • the refrigerator bottle holder is usually used to store items with a short shelf life such as yogurt, and has higher requirements for the identification of ingredient information.
  • a method of taking a bird's-eye view of the items stored in the bottle holder has been proposed. Since the content of the top surface pattern of the item is relatively fixed, so It is less affected by the placement angle of the items and is easier to identify.
  • the purpose of the present invention is to provide a method for identifying information of items in a refrigerator, a refrigerator and a computer storage medium.
  • the invention provides a method for identifying information of items in a refrigerator, comprising the steps of:
  • a bird's-eye view image including a plurality of refrigerator bottle holders is obtained again, and the item information is identified according to the pattern on the top surface of the item.
  • obtaining a bird's-eye view image of the interior space of a plurality of refrigerator bottle holders specifically includes the steps:
  • the refrigerator door body moves to a preset angle with the refrigerator box body, and the camera located on the top of the refrigerator box body is controlled to capture a bird's-eye image of the space in the bottle holder.
  • At least the area between the adjacent edge contour lines between adjacent bottle bases is defined as the identification area
  • the object image is detected and identified in the identification area.
  • detecting and judging in the bird's-eye image whether the item in the bottle holder is blocked by the bottle holder above it specifically includes the steps:
  • edge contour line of the object intersects with the edge contour line of the bottle seat above it, it is determined that the object is blocked.
  • “adjusting the positional relationship between the bottle holder above the article and the bottle holder for storing the article until there is no obstruction between the article and the bottle holder above it” specifically includes:
  • detecting and identifying the image of the item in the identification area specifically includes:
  • Identify the top surface information of the item through the deep learning recognition algorithm to obtain the item category information, record the storage time and count the number of similar items.
  • the present invention also provides a refrigerator, comprising a plurality of bottle holders arranged on the door body of the refrigerator, the bottle holders are slidably arranged on the door body through a transmission device, and slide up and down along the frame of the door body;
  • the top of the refrigerator box is provided with a camera disposed toward the outside of the box, for photographing the inner space of the bottle holder;
  • the refrigerator further includes a memory and a processor, the memory stores a computer program that can be executed on the processor, and the processor implements the steps of the above method for identifying item information in the refrigerator when the processor executes the program.
  • the transmission device includes a sliding rail and a driving device
  • the bottle holder is arranged on the frame of the refrigerator door body through the sliding rail, and is controlled by the driving module to move along the sliding rail.
  • the present invention also provides a computer storage medium, in which a computer program is stored, and when the computer program runs, the device where the computer storage medium is located executes the steps of the above method for identifying item information in a refrigerator.
  • the beneficial effects of the present invention are: by judging the situation that the items in the captured image are blocked by the bottle holder, the height of the bottle holder is adjusted to completely expose the item in the image, thereby improving the accuracy of identifying the information of the items stored in the bottle holder, Reduce unrecognized cases due to obscured items.
  • FIG. 1 is a schematic flowchart of a method for identifying information of items in a refrigerator according to an embodiment of the present invention.
  • FIG. 2 is a schematic diagram of a bottle seat area photographed in an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a related process of defining an identification area in an embodiment of the present invention.
  • FIG. 4 is a schematic flow chart of step S2 in the method for identifying information of items in a refrigerator according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of the bottle base 1b and the bottle base 1c after the positions are adjusted in an embodiment of the present invention.
  • the term used to describe the relative position in space such as “upper”, “lower”, “rear”, “front”, etc., is used to describe one unit or feature shown in the drawings relative to another A unit or feature relationship.
  • the term spatially relative position may include different orientations of the device in use or operation other than the orientation shown in the figures. For example, if the device in the figures is turned over, elements described as “below” or “above” other elements or features would then be oriented “below” or “above” the other elements or features.
  • the exemplary term “below” can encompass both a spatial orientation of below and above.
  • the present invention provides a method for identifying information of items in a refrigerator, comprising the steps of:
  • S1 Acquire a bird's-eye image of the space inside the bottle holder including multiple refrigerators.
  • acquiring a bird's-eye view image specifically includes the steps:
  • the refrigerator door body moves to a preset angle with the refrigerator box body, and the camera located on the top of the refrigerator box body is controlled to capture a bird's-eye image of the space in the bottle holder.
  • the camera is provided on the top of the refrigerator box
  • the refrigerator door is provided with multiple rows of bottle seats, which are used to place items such as food and medicine that need to be stored at low temperature
  • the refrigerator door is provided with an angle sensor.
  • the camera is controlled to shoot to obtain a bird's-eye view image.
  • the preset angle is 30°.
  • a single camera can capture the complete situation in multiple bottle seat areas from a top-down angle, and can The upper end face pattern of the items stored in the bottle holder is photographed to facilitate the subsequent identification of the item category, and when shooting at a position of 30°, the obstruction between the objects is small and easy to identify, and the items occupy a large proportion in the image. Invalid information in images can be reduced.
  • FIG. 2 it is a schematic diagram of the bottle holder area obtained by shooting.
  • the contour line of the edge of the bottle base is identified. Further, in some embodiments, the edge of the bottle base is provided with a marking line to enhance the identification effect.
  • the edge detection algorithm can adopt the currently common algorithms such as Roberts, Sobel, Prewitt, Canny, Log, etc. The specific algorithm content will not be repeated here.
  • S12 Define at least an area between adjacent edge contour lines between adjacent bottle holders as an identification area.
  • the contour lines are numbered according to the arrangement order, and the area between the adjacent edge contour lines between two adjacent bottle seats is selected as the identification area according to the arrangement order of the bottle bases.
  • the door body of the refrigerator is provided with a first bottle base 1a, a second bottle base 1b and a third bottle base 1c in sequence, and the identified edge contour lines are numbered as 11, 12, 13, 14, 15, 16, wherein, the edge contour line 11 and the edge contour line 12 are respectively the contour lines of the first bottle base 1a, and the edge contour line 13 and the edge contour line 14 are respectively the contour lines of the second bottle base 1b,
  • the edge contour line 15 and the edge contour line 16 are respectively the contour lines of the third bottle holder 1c.
  • the edge contour line 12 and the edge contour line 13, the edge contour line 14 and the edge contour line 15, and the edge contour line 16 are respectively The area to the top of the image is defined as the recognition area.
  • the position of the item to be identified in the image can be accurately defined, the invalid information in the identification area is reduced, and the speed and accuracy of the subsequent detection and identification of the item information can be improved.
  • the camera can also be installed on the refrigerator door, facing the refrigerator box. After detecting that the refrigerator door and the refrigerator box are at a preset angle, the camera is activated to shoot the refrigerator compartment and the freezer. Storage of indoor items.
  • S2 Detect in the bird's-eye image to determine whether the item in the bottle holder is blocked by the bottle holder above it, and when there is an blocked item, adjust the positional relationship between the bottle holder above the item and the bottle holder where the item is stored, until there is no obstruction between the item and the bottle holder above it.
  • step S2 includes steps:
  • the edge contours between the two must have a partial intersection. By judging whether there is an intersecting part between the two edge contours, it can be accurately judged whether the object is blocked.
  • the top surface patterns of the items 2 and 3 stored in the refrigerator are respectively blocked by the bottle holder 1b and the bottle holder 1c above them.
  • the edge contour line and the edge contour line 13 of the item 2 are blocked.
  • the accurate information detection and identification of the item 2 and the item 3 cannot be performed.
  • the positions of the bottle seats are adjusted in sequence from bottom to top or in sequence from top to bottom.
  • the relationship between the blocked object and the bottle holder can be monitored and calculated in real time through the camera, or the fixed distance of each movement of the bottle holder can be set, and each time a movement is performed, the camera will be re-photographed to determine whether there are still blocked objects, until no more There is an obstructed item.
  • the position of the bottle seat when adjusting the position of the bottle seat, it further includes the step of: when it is judged that there is resistance to the movement of the bottle seat, stopping the movement of the bottle seat.
  • an adjustment distance threshold is set for each bottle base, so as to avoid collision between the items in the bottle base being adjusted and the bottle base above it, which may cause damage to the structure of the refrigerator and the stored items.
  • FIG. 5 it is a schematic diagram of the bottle holder 1b and the bottle holder 1c after adjusting their positions. At this time, the article 2 and the bottle holder 1b, and the article 3 and the bottle holder 1c are just in separate positions respectively on the image. separated state.
  • step S23 after step S23 is performed, the following steps are further included:
  • the above two steps are applicable when the distance between the bottle holder and the items below it is too large.
  • adjust the downward movement of the bottle seat thereby reducing the size of the identification area, reducing the invalid information in the identification area, and improving the accuracy of subsequent identification and detection. Accuracy, and can effectively use the space in the refrigerator bottle holder.
  • S3 Obtain a bird's-eye view image of the space in the bottle holder of the refrigerator again, and identify the item information according to the pattern on the top surface of the item.
  • Identify the top surface information of the item through the deep learning recognition algorithm, detect and identify the category information of the item, record the storage time and count the number of similar items.
  • the present invention also provides a refrigerator, which includes a plurality of bottle holders arranged on the door body of the refrigerator.
  • the top of the refrigerator box is provided with a camera facing the outside of the box, which is used to photograph the inner space of the bottle holder;
  • the refrigerator further includes a memory and a processor.
  • the memory stores a computer program that can be executed on the processor.
  • the processor executes the program, the steps of the above-mentioned method for recognizing item information in the refrigerator are implemented.
  • the transmission device includes a sliding rail and a driving device
  • the bottle holder is arranged on the frame of the refrigerator door body through the sliding rail, and is controlled by the driving module to move along the sliding rail.
  • the refrigerator is also provided with an angle sensing device for detecting the angle between the refrigerator door and the refrigerator box.
  • the present invention also provides a computer storage medium, in which a computer program is stored, and when the computer program runs, the device where the computer storage medium is located executes the steps according to the above-mentioned method for identifying item information in a refrigerator.
  • the present invention adjusts the height of the bottle holder to fully expose the article in the image by judging the situation that the item in the captured image is blocked by the bottle holder, thereby improving the accuracy of identifying the information of the item stored in the bottle holder, Reduce the situation that the item cannot be recognized due to being occluded, and improve the user experience.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Mechanical Engineering (AREA)
  • Thermal Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Cold Air Circulating Systems And Constructional Details In Refrigerators (AREA)

Abstract

本发明提供一种冰箱内物品信息识别方法、冰箱和计算机存储介质，冰箱内物品信息识别方法包括步骤：获取包括多个冰箱瓶座内空间的鸟瞰图像；在鸟瞰图像内检测判断瓶座内物品是否被其上方的瓶座遮挡；根据物品顶面图案识别物品信息。通过对所拍摄图像内物品被瓶座遮挡的情况进行判断，来调整瓶座高度以在图像中完全暴露物品，从而提高对瓶座内存放物品信息识别的准确度，减少因物品被遮挡而无法识别的情况。

Description

冰箱内物品信息标识方法、冰箱和计算机存储介质
本申请要求了申请日为2021年04月16日，申请号为202110413457.5，发明名称为“冰箱内物品信息标识方法、冰箱和计算机存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及制冷装置领域，具体地涉及一种冰箱内物品信息标识方法、冰箱和计算机存储介质。
背景技术
随着家用电器智能化的发展，对内部食材进行识别已经是智能冰箱所必需的功能。通常为了实现对冰箱内部食材的识别，冰箱的内部储存空间内都安装有一个或多个摄像头用于拍摄食材，并对拍摄的照片进行处理后生成识别结果发送到使用终端，以供用户查看。冰箱瓶座内通常用于存放酸奶等保质期较短的物品，对于食材信息识别的要求更高，目前已经提出拍摄瓶座内存放物品的鸟瞰图的方法，由于物品顶面图案内容相对固定，所受物品摆放角度影响较小，更加易于识别。
然而，拍摄瓶座内存放物品时，当存在部分高度较高的物品时，其上方的瓶座会遮挡住该物品的顶面而造成难以识别的情况。
发明内容
本发明的目的在于提供一种冰箱内物品信息识别方法、冰箱和计算机存储介质。
本发明提供一种冰箱内物品信息识别方法，包括步骤：
获取包括多个冰箱瓶座内空间的鸟瞰图像，
在所述鸟瞰图像内检测判断瓶座内物品是否被其上方的瓶座遮挡，当存在被遮挡物品时，调整所述物品上方的瓶座与存放所述物品的瓶座之间的位置关系，直至所述物品与其上方瓶座之间无遮挡；
再次获取包括多个冰箱瓶座内空间的鸟瞰图像，根据物品顶面图案识别物品信息。
作为本发明的进一步改进，“获取包括多个冰箱瓶座内空间的鸟瞰图像”具体包括步骤：
检测到冰箱门体运动至与冰箱箱体之间夹角呈预设角度，控制设于冰箱箱体顶部的摄像机拍摄得到瓶座内空间的鸟瞰图像。
作为本发明的进一步改进，在“获取包括多个冰箱瓶座内空间的鸟瞰图像”之后还包括步骤：
检测识别多个瓶座的边缘轮廓线；
至少将相邻瓶座之间的相邻边缘轮廓线之间的区域定义为识别区；
在所述识别区内进行物品图像进行检测识别。
作为本发明的进一步改进，“在所述鸟瞰图像内检测判断瓶座内物品是否被其上方的瓶座遮挡”具体包括步骤：
检测识别物品边缘轮廓线；
若所述物品边缘轮廓线与其上方的瓶座边缘轮廓线相交，则判断所述物品被遮挡。
作为本发明的进一步改进，“调整所述物品上方的瓶座与存放所述物品的瓶座之间的位置关系，直至所述物品与其上方瓶座之间无遮挡”具体包括：
控制所述物品上方的瓶座沿传动装置向上运动，或控制存放所述物品的瓶座沿传动装置向下运动，直至所述被遮挡物品的边缘轮廓线和其上方所述瓶座边缘轮廓线之间不存在重叠或相交部分。
作为本发明的进一步改进，还包括步骤：
当判断所述瓶座运动出现阻力时，停止所述瓶座运动。
作为本发明的进一步改进，还包括步骤：
于所述检测区内检测物品边缘轮廓线与其上方瓶座底部边缘轮廓线之间的最近距离；
当所述最近距离大于预设间隔距离时，调整所述物品上方的瓶座与存放所述物品的瓶座之间的位置关系，直至所述被遮挡物品的边缘轮廓线和其上方所述瓶座边缘轮廓线之间达到预设距离。
作为本发明的进一步改进，“在所述识别区内进行物品图像进行检测识别”具体包括：
通过深度学习识别算法识别物品顶面信息获得物品类别信息，并记录存放时间和统计同类物品数量。
本发明还提供一种冰箱，包括多个设于冰箱门体上的瓶座，所述瓶座通过传动装置滑动设于所述门体上，沿所述门体边框上下滑动运动；
所述冰箱箱体顶部设有朝向箱体外部设置的摄像机，用于拍摄所述瓶座内空间；
所述冰箱还包括存储器和处理器，所述存储器存储有可在所述处理器上运行的计算机程序，所述处理器执行所述程序时实现上述冰箱内物品信息识别方法的步骤。
作为本发明的进一步改进，所述传动装置包括滑轨和驱动装置，所述瓶座通过所述滑轨设于冰箱门体边框上，并受驱动模块控制而沿所述滑轨运动。
本发明还提供一种计算机存储介质，其中存储有计算机程序，并且所述计算机程序运行时导致所述计算机存储介质的所在设备执行上述冰箱内物品信息识别方法的步骤。
本发明的有益效果是：通过对所拍摄图像内物品被瓶座遮挡的情况进行判断，来调整瓶座高度以在图像中完全暴露物品，从而提高对瓶座内存放物品信息识别的准确度，减少因物品被遮挡而无法识别的情况。
附图说明
图1是本发明一实施方式中的冰箱内物品信息识别方法流程示意图。
图2是本发明一实施方式中拍摄得到的瓶座区域示意简图。
图3是本发明一实施方式中的定义识别区相关流程示意图。
图4是本发明一实施方式中的冰箱内物品信息识别方法中步骤S2的具体流程示意图。
图5是本发明一实施方式中瓶座1b和瓶座1c调整位置后的示意简图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合本申请具体实施方式及相应的附图对本申请技术方案进行清楚、完整地描述。显然，所描述的实施方式仅是本申请一部分实施方式，而不是全部的实施方式。基于本申请中的实施方式，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施方式，都属于本申请保护的范围。
下面详细描述本发明的实施方式，实施方式的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施方式是示例性的，仅用于解释本发明，而不能理解为对本发明的限制。
为方便说明，本文使用表示空间相对位置的术语来进行描述，例如“上”、“下”、“后”、“前”等，用来描述附图中所示的一个单元或者特征相对于另一个单元或特征的关系。空间相对位置的术语可以包括设备在使用或工作中除了图中所示方位以外的不同方位。例如，如果将图中的装置翻转，则被描述为位于其他单元或特征“下方”或“上方”的单元将位于其他单元或特征“下方”或“上方”。因此，示例性术语“下方”可以囊括下方和上方这两种空间方位。
如图1所示，本发明提供一种冰箱内物品信息识别方法，包括步骤：
S1：获取包括多个冰箱瓶座内空间的鸟瞰图像。
具体的，在本实施方式中，获取鸟瞰图像具体包括步骤：
检测到冰箱门体运动至与冰箱箱体之间夹角呈预设角度，控制设于冰箱箱体顶部的摄像机拍摄得到瓶座内空间的鸟瞰图像。
示例性的，摄像机设于冰箱箱体顶部，冰箱门体上设有多排瓶座，用于放置诸如食材、药品等需要在低温下储存的物品，冰箱门体处设有角度传感器，在检测到冰箱门体 运动至与冰箱箱体之间夹角呈预设角度时，控制摄像机拍摄得到鸟瞰图像。优选的，预设角度为30°，此时位于冰箱箱体顶部的摄像机拍摄位于门体上的瓶座区域时，以俯视角度能够单个摄像机拍摄到完整的多个瓶座区域内情况，并且可拍摄到瓶座内所存放物品的上端面图案，以方便后续对物品类别识别，而且在30°的位置进行拍摄时，物体之间的阻挡较小，易于识别，物品在图像中占比大，可以减少图像中的无效信息。
如图2所示，为拍摄得到的瓶座区域示意简图。
进一步的，如图3所示，在本发明的一些实施方式中，获得鸟瞰图像之后，还包括步骤：
S11：检测识别多个瓶座的边缘轮廓线。
通过边缘检测算法，识别瓶座边缘轮廓线，进一步的，在一些实施方式中，瓶座边缘设有用以加强识别效果的标示线。边缘检测算法可以采用诸如Roberts、Sobel、Prewitt、Canny、Log等目前常见的算法，具体算法内容再此不再赘述。
S12：至少将相邻瓶座之间的相邻边缘轮廓线之间的区域定义为识别区。
具体的，对轮廓线按照排布顺序进行编号，依照瓶座排布顺序选择相邻两瓶座之间相邻的边缘轮廓线之间的区域为识别区，比如，如图2所示，从下往上冰箱门体上依次设有第一瓶座1a、第二瓶座1b和第三瓶座1c，按从下往上的顺序额依次将识别到的边缘轮廓线编号为11、12、13、14、15、16，其中，边缘轮廓线11和边缘轮廓线12分别为第一瓶座1a的轮廓线，边缘轮廓线13和边缘轮廓线14分别为第二瓶座1b的轮廓线，边缘轮廓线15和边缘轮廓线16分别为第三瓶座1c的轮廓线，此时，分别将边缘轮廓线12和边缘轮廓线13、边缘轮廓线14和边缘轮廓线15，以及边缘轮廓线16至图像顶端区域之间的区域定义为识别区。
S13：在识别区内进行物品图像进行检测识别。
通过上述方法划分识别区，可以准确定义出所需识别物品在图像中的位置，减少识别区内的无效信息，提高后续对物品信息检测识别的速度和准确率。
于本发明的另一些实施方式中，也可将摄像机设于冰箱门体上，朝向冰箱箱体内设置，在检测到冰箱门体与冰箱箱体呈预设角度后，启动摄像机拍摄冷藏室和冷冻室内物品存放情况。
S2：在鸟瞰图像内检测判断瓶座内物品是否被其上方的瓶座遮挡，当存在被遮挡物品时，调整所述物品上方的瓶座与存放所述物品的瓶座之间的位置关系，直至所述物品与其上方瓶座之间无遮挡。
具体的，如图4所示，步骤S2包括步骤：
S21：检测识别物品边缘轮廓线。
S22：若物品边缘轮廓线与其上方的瓶座边缘轮廓线相交，则判断物品被遮挡。
由于物品在图像中若被其上方的瓶座遮挡，则其两者之间的边缘轮廓必定存在部分相交，通过判断两者的边缘轮廓线是否存在相交的部分即可准确判断物品是否被遮挡。
示例性的，如图2所示，冰箱内存放的物品2和物品3顶面图案分别被其上方的瓶座1b和瓶座1c所遮挡，此时物品2的边缘轮廓线和边缘轮廓线13存在部分重叠，物品3的边缘轮廓线和边缘轮廓线15存在部分重叠，此时无法对物品2和物品3进行准确的信息检测识别。
S23：控制所述物品上方的瓶座沿传动装置向上运动，或控制存放所述物品的瓶座沿传动装置向下运动，直至被遮挡物品的边缘轮廓线和其上方瓶座边缘轮廓线之间不存在重叠或相交部分。
进一步的，当设有多个瓶座时，依照从下往上的顺序或依照从上往下的顺序依次对瓶座位置进行调整。
具体的，可以通过摄像机实时监控及计算被遮挡物品与瓶座之间的关系，或者设置瓶座每次运动的固定距离，每进行一次运动，便重新拍照判断是否还存在被遮挡物品，直至不存在被遮挡物品。
更进一步的，在调整瓶座位置时，还包括步骤：当判断所述瓶座运动出现阻力时，停止所述瓶座运动。或者，针对每个瓶座设置一调整距离阈值，以避免调整中的瓶座内物品与其上方的瓶座之间发生碰撞而对冰箱结构及存放物品造成损坏。
示例性的，如图5所示，为瓶座1b和瓶座1c调整位置后的示意简图，此时，物品2和瓶座1b、以及物品3和瓶座1c在图像上刚好分别处于分隔开的状态。
更进一步的，在本发明的一些实施方式中，在进行步骤S23之后还包括步骤：
S24：于检测区内检测物品边缘轮廓线与其上方瓶座底部边缘轮廓线之间的最近距离。
S25：当最近距离大于预设间隔距离时，调整物品上方的瓶座与存放物品的瓶座之间的位置关系，直至被遮挡物品的边缘轮廓线和其上方瓶座边缘轮廓线之间达到预设距离。
上述两个步骤，适用于瓶座与其下方之间的物品间隔距离过大的情况。当物品边缘轮廓线与其上方瓶座底部边缘轮廓线之间的距离超过预设距离时，调整瓶座向下运动，从而减小识别区的大小，减少识别区内无效信息，提高后续识别检测的准确率，并且可以有效利用冰箱瓶座内空间。
通过调整调整瓶座高度以减少所拍摄的图像中物品被瓶座所遮挡的情况，从而提高后续对瓶座内存放物品信息识别的准确度，减少了物品因被遮挡而无法识别的情况，提高了用户体验。
S3：再次获取包括多个冰箱瓶座内空间的鸟瞰图像，根据物品顶面图案识别物品信 息。
通过深度学习识别算法识别物品顶面信息检测识别物品类别信息，并记录存放时间和统计同类物品数量。
根据顶面图像信息检测识别物品种类信息，相比于识别物品侧面图案，顶部图案设别更加简便且识别准确率更高。
本发明还提供一种冰箱，包括多个设于冰箱门体上的瓶座，瓶座通过传动装置滑动设于门体上，沿门体边框上下滑动运动。
冰箱箱体顶部设有朝向箱体外部设置的摄像机，用于拍摄瓶座内空间；
冰箱还包括存储器和处理器，存储器存储有可在处理器上运行的计算机程序，处理器执行程序时实现上述的冰箱内物品信息识别方法的步骤。
进一步的，传动装置包括滑轨和驱动装置，瓶座通过滑轨设于冰箱门体边框上，并受驱动模块控制而沿滑轨运动。
进一步的，冰箱还设有角度传感装置，用以检测冰箱门第与冰箱箱体之间的夹角角度。
本发明还提供一种计算机存储介质，其中存储有计算机程序，并且计算机程序运行时导致计算机存储介质的所在设备执行根据上述所述的冰箱内物品信息识别方法的步骤。
综上所述，本发明通过对所拍摄图像内物品被瓶座遮挡的情况进行判断，来调整瓶座高度以在图像中完全暴露物品，从而提高对瓶座内存放物品信息识别的准确度，减少因物品被遮挡而无法识别的情况，提高了用户体验。
应当理解，虽然本说明书按照实施方式加以描述，但并非每个实施方式仅包含一个独立的技术方案，说明书的这种叙述方式仅仅是为清楚起见，本领域技术人员应当将说明书作为一个整体，各实施方式中的技术方案也可以经适当组合，形成本领域技术人员可以理解的其他实施方式。
上文所列出的一系列的详细说明仅仅是针对本发明的可行性实施方式的具体说明，并非用以限制本发明的保护范围，凡未脱离本发明技艺精神所作的等效实施方式或变更均应包含在本发明的保护范围之内。

Claims (11)

  1. 一种冰箱内物品信息识别方法，其特征在于，包括步骤：
    获取包括多个冰箱瓶座内空间的鸟瞰图像；
    在所述鸟瞰图像内检测判断瓶座内物品是否被其上方的瓶座遮挡，当存在被遮挡物品时，调整所述物品上方的瓶座与存放所述物品的瓶座之间的位置关系，直至所述物品与其上方瓶座之间无遮挡；
    再次获取包括多个冰箱瓶座内空间的鸟瞰图像，根据物品顶面图案识别物品信息。
  2. 根据权利要求1所述的冰箱内物品信息识别方法，其特征在于，“获取包括多个冰箱瓶座内空间的鸟瞰图像”具体包括步骤：
    检测到冰箱门体运动至与冰箱箱体之间夹角呈预设角度，控制设于冰箱箱体顶部的摄像机拍摄得到瓶座内空间的鸟瞰图像。
  3. 根据权利要求1所述的冰箱内物品信息识别方法，其特征在于，在“获取包括多个冰箱瓶座内空间的鸟瞰图像”之后还包括步骤：
    检测识别多个瓶座的边缘轮廓线；
    至少将相邻瓶座之间的相邻边缘轮廓线之间的区域定义为识别区；
    在所述识别区内进行物品图像进行检测识别。
  4. 根据权利要求3所述的冰箱内物品信息识别方法，其特征在于，“在所述鸟瞰图像内检测判断瓶座内物品是否被其上方的瓶座遮挡”具体包括步骤：
    检测识别物品边缘轮廓线；
    若所述物品边缘轮廓线与其上方的瓶座边缘轮廓线相交，则判断所述物品被遮挡。
  5. 根据权利要求4所述的冰箱内物品信息识别方法，其特征在于，“调整所述物品上方的瓶座与存放所述物品的瓶座之间的位置关系，直至所述物品与其上方瓶座之间无遮挡”具体包括：
    控制所述物品上方的瓶座沿传动装置向上运动，或控制存放所述物品的瓶座沿传动装置向下运动，直至所述被遮挡物品的边缘轮廓线和其上方所述瓶座边缘轮廓线之间不存在重叠或相交部分。
  6. 根据权利要求4所述的冰箱内物品信息识别方法，其特征在于，还包括步骤：
    当判断所述瓶座运动出现阻力时，停止所述瓶座运动。
  7. 根据权利要求5所述的冰箱内物品信息识别方法，其特征在于，还包括步骤：
    于所述检测区内检测物品边缘轮廓线与其上方瓶座底部边缘轮廓线之间的最近距离；
    当所述最近距离大于预设间隔距离时，调整所述物品上方的瓶座与存放所述物品的 瓶座之间的位置关系，直至所述被遮挡物品的边缘轮廓线和其上方所述瓶座边缘轮廓线之间达到预设距离。
  8. 根据权利要求3所述的冰箱内物品信息识别方法，其特征在于，“在所述识别区内进行物品图像进行检测识别”具体包括：
    通过深度学习识别算法识别物品顶面信息获得物品类别信息，并记录存放时间和统计同类物品数量。
  9. 一种冰箱，包括多个设于冰箱门体上的瓶座，其特征在于，
    所述瓶座通过传动装置滑动设于所述门体上，沿所述门体边框上下滑动运动；
    所述冰箱箱体顶部设有朝向箱体外部设置的摄像机，用于拍摄所述瓶座内空间；
    所述冰箱还包括存储器和处理器，所述存储器存储有可在所述处理器上运行的计算机程序，所述处理器执行所述程序时实现权利要求1-8中任意一项所述冰箱内物品信息识别方法的步骤。
  10. 根据权利要求9所述的冰箱，其特征在于，所述传动装置包括滑轨和驱动装置，所述瓶座通过所述滑轨设于冰箱门体边框上，并受驱动模块控制而沿所述滑轨运动。
  11. 一种计算机存储介质，其中存储有计算机程序，并且所述计算机程序运行时导致所述计算机存储介质的所在设备执行根据权利要求1-8中任意一项所述冰箱内物品信息识别方法的步骤。
PCT/CN2021/132850 2021-04-16 2021-11-24 冰箱内物品信息标识方法、冰箱和计算机存储介质 WO2022217928A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110413457.5A CN113158876A (zh) 2021-04-16 2021-04-16 冰箱内物品信息识别方法、冰箱和计算机存储介质
CN202110413457.5 2021-04-16

Publications (1)

Publication Number Publication Date
WO2022217928A1 true WO2022217928A1 (zh) 2022-10-20

Family

ID=76868520

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/132850 WO2022217928A1 (zh) 2021-04-16 2021-11-24 冰箱内物品信息标识方法、冰箱和计算机存储介质

Country Status (2)

Country Link
CN (1) CN113158876A (zh)
WO (1) WO2022217928A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113158876A (zh) * 2021-04-16 2021-07-23 青岛海尔电冰箱有限公司 冰箱内物品信息识别方法、冰箱和计算机存储介质
CN114863265A (zh) * 2021-12-14 2022-08-05 青岛海尔电冰箱有限公司 冰箱内物品信息识别方法、冰箱和计算机存储介质
CN114862867A (zh) * 2021-12-27 2022-08-05 青岛海尔电冰箱有限公司 冰箱内物品信息识别方法、冰箱和计算机存储介质
CN114882349B (zh) * 2022-03-29 2024-05-24 青岛海尔制冷电器有限公司 冰箱内物品目标同一性判断方法、冰箱和计算机存储介质
CN114708503A (zh) * 2022-03-31 2022-07-05 青岛海尔电冰箱有限公司 冰箱瓶座内物品信息识别方法、冰箱和计算机存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109696020A (zh) * 2018-03-28 2019-04-30 青岛海尔股份有限公司 冰箱
US20190392383A1 (en) * 2019-08-09 2019-12-26 Lg Electronics Inc. Refrigerator for providing information on item using artificial intelligence and method of operating the same
CN110765815A (zh) * 2018-07-26 2020-02-07 北京京东尚科信息技术有限公司 展示架遮挡检测方法和装置
CN211650920U (zh) * 2020-03-10 2020-10-09 合肥市天马塑胶有限公司 一种位置可调式的冰箱瓶座
CN112432414A (zh) * 2020-12-11 2021-03-02 恩亿科(北京)数据科技有限公司 一种基于图像识别的冰箱鲜度管理方法与系统
CN113158876A (zh) * 2021-04-16 2021-07-23 青岛海尔电冰箱有限公司 冰箱内物品信息识别方法、冰箱和计算机存储介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN205980518U (zh) * 2016-08-12 2017-02-22 合肥华凌股份有限公司 一种可移动便利性置物盒、冰箱门体及冰箱
JP7211700B2 (ja) * 2017-10-06 2023-01-24 東芝ライフスタイル株式会社 冷蔵庫、表示システム
CN109330284B (zh) * 2018-09-21 2020-08-11 京东方科技集团股份有限公司 一种货架系统
CN109951636A (zh) * 2019-03-18 2019-06-28 Oppo广东移动通信有限公司 拍照处理方法、装置、移动终端以及存储介质
CN110455027A (zh) * 2019-07-16 2019-11-15 海信集团有限公司 一种图像采集装置及其冰箱、控制方法
CN111444880B (zh) * 2020-04-10 2023-10-31 海信集团有限公司 一种食材识别方法及冰箱
CN112487962A (zh) * 2020-11-27 2021-03-12 青岛海尔科技有限公司 一种物品信息确定方法、装置、存储介质及电子装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109696020A (zh) * 2018-03-28 2019-04-30 青岛海尔股份有限公司 冰箱
CN110765815A (zh) * 2018-07-26 2020-02-07 北京京东尚科信息技术有限公司 展示架遮挡检测方法和装置
US20190392383A1 (en) * 2019-08-09 2019-12-26 Lg Electronics Inc. Refrigerator for providing information on item using artificial intelligence and method of operating the same
CN211650920U (zh) * 2020-03-10 2020-10-09 合肥市天马塑胶有限公司 一种位置可调式的冰箱瓶座
CN112432414A (zh) * 2020-12-11 2021-03-02 恩亿科(北京)数据科技有限公司 一种基于图像识别的冰箱鲜度管理方法与系统
CN113158876A (zh) * 2021-04-16 2021-07-23 青岛海尔电冰箱有限公司 冰箱内物品信息识别方法、冰箱和计算机存储介质

Also Published As

Publication number Publication date
CN113158876A (zh) 2021-07-23

Similar Documents

Publication Publication Date Title
WO2022217928A1 (zh) 冰箱内物品信息标识方法、冰箱和计算机存储介质
WO2022217927A1 (zh) 冰箱内物品信息标识方法、冰箱和计算机存储介质
US20150002660A1 (en) Electric product
KR102391425B1 (ko) 식기 세척기
WO2022217935A1 (zh) 冰箱内物品信息识别方法和冰箱
US8929592B2 (en) Camera-based 3D climate control
WO2019218303A1 (zh) 智能售货柜、物品识别方法、装置、服务器和存储介质
WO2023193752A1 (zh) 冰箱内物品信息识别方法和冰箱
JP7482410B2 (ja) 入出庫管理装置、入出庫管理システム、及び、入出庫管理方法
WO2023185779A1 (zh) 冰箱内物品信息识别方法
CN108363977B (zh) 一种人体卧姿状态识别方法及装置
WO2023185280A1 (zh) 冰箱瓶座内物品信息识别方法和冰箱
CN114818796A (zh) 冰箱内物品信息识别方法、冰箱和计算机存储介质
JP7475000B2 (ja) 入出庫管理装置、入出庫管理システム、及び、入出庫管理方法
CN113824864A (zh) 食材存储设备及图像处理方法
CN114863265A (zh) 冰箱内物品信息识别方法、冰箱和计算机存储介质
CN112629105A (zh) 虚拟控制面板操作识别方法及冰箱
CN112307807B (zh) 图像识别方法、图像识别装置及冰箱
CN218495468U (zh) 制冷设备
CN116416556A (zh) 物品信息识别方法、制冷设备和计算机存储介质
US20230316755A1 (en) Refrigerator appliance and methods for tracking stored items in a freezer chamber
CN117367022A (zh) 完整运动轨迹的生成方法、制冷设备和计算机存储介质
CN117560568A (zh) 设备内物品图像采集方法、设备和计算机存储介质
WO2023124088A1 (zh) 冰箱内物品信息识别方法和冰箱
EP4246441A1 (en) A refrigerator with a camera for automatic image cropping

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21936794

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21936794

Country of ref document: EP

Kind code of ref document: A1