WO2023125491A1 - 特定轮廓食材管理方法、存储介质以及冰箱 - Google Patents

特定轮廓食材管理方法、存储介质以及冰箱 Download PDF

Info

Publication number
WO2023125491A1
WO2023125491A1 PCT/CN2022/142205 CN2022142205W WO2023125491A1 WO 2023125491 A1 WO2023125491 A1 WO 2023125491A1 CN 2022142205 W CN2022142205 W CN 2022142205W WO 2023125491 A1 WO2023125491 A1 WO 2023125491A1
Authority
WO
WIPO (PCT)
Prior art keywords
ingredients
contour line
food
contour
closed
Prior art date
Application number
PCT/CN2022/142205
Other languages
English (en)
French (fr)
Inventor
高洪波
孔令磊
Original Assignee
青岛海尔电冰箱有限公司
海尔智家股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 青岛海尔电冰箱有限公司, 海尔智家股份有限公司 filed Critical 青岛海尔电冰箱有限公司
Publication of WO2023125491A1 publication Critical patent/WO2023125491A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Definitions

  • the invention relates to the field of smart home appliances, in particular to a method for managing ingredients with specific contours in a refrigerator, a storage medium and a refrigerator.
  • refrigerators In family life, refrigerators have become an indispensable household appliance, and users often store a large amount of ingredients in the refrigerator.
  • some refrigerators are equipped with an image acquisition module, which can obtain the quantity of ingredients by collecting images of ingredients stored.
  • ingredients with specific outlines such as eggs, apples, and oranges are often stacked and stored in the refrigerator, and the ingredients may be blocked. It is difficult to accurately determine the quantity of ingredients through the collected related images.
  • the present invention proposes a method for managing ingredients with specific outlines in a refrigerator, which can determine whether there are stacks of ingredients with specific outlines stored in a predetermined area.
  • an embodiment of the present invention provides a method for managing ingredients with specific contours in a refrigerator, including:
  • controlling the image acquisition module to acquire images of the ingredients stored in the predetermined area
  • the number of occluded areas greater than the preset area is greater than the preset value, then mark the food material corresponding to the closed-loop contour line as the upper layer food material, otherwise, the closed-loop contour line
  • the ingredients corresponding to the line are marked as lower ingredients;
  • the method for managing ingredients with specific contours in the refrigerator further includes:
  • ingredients stored in the predetermined area include ingredients on the upper layer, a prompt message indicating that the ingredients are sufficient is output.
  • the method for managing ingredients with specific contours in the refrigerator further includes:
  • the ingredients corresponding to the shaded ingredient contour lines connected to the closed-loop contour lines whose shading area is larger than a preset area are marked as lower layer ingredients.
  • the method for managing ingredients with specific contours in the refrigerator further includes:
  • ingredients corresponding to the closed-loop contour lines are all lower-layer ingredients, the ingredients corresponding to each ingredient edge contour line are marked as lower-layer ingredients.
  • the method for managing ingredients with specific contours in the refrigerator further includes:
  • the non-severe occlusion contour corresponding to The ingredient is marked as an upper-layer ingredient; otherwise, the ingredient corresponding to the non-severely occluded contour line is marked as a lower-layer ingredient.
  • controlling the image acquisition module to acquire images of the ingredients stored in a predetermined area includes:
  • the image acquisition module is controlled to acquire a bird's-eye view of the ingredients stored in the predetermined area.
  • Performing edge detection on the image to obtain the edge contour of the food includes:
  • Edge detection is performed in the detection frame to identify the edge contour line of the food material.
  • the method for managing ingredients with specific contours in the refrigerator further includes:
  • the specific profile food is eggs
  • Performing edge detection in the detection frame to identify the edge outline of the food includes:
  • Edge detection is performed in the detection frame to identify the edge contour line conforming to the circle-like feature.
  • one embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it realizes the specific contour food material in the refrigerator in any of the above-mentioned embodiments. Steps in a management method.
  • an embodiment of the present invention provides a refrigerator, the refrigerator is provided with a storage compartment, the storage compartment has a predetermined area for storing the food with a specific outline, and An image acquisition module for capturing the predetermined area, the refrigerator also includes a memory and a processor, the memory stores a computer program that can run on the processor, and when the processor executes the computer program, the above-mentioned Steps in the method for managing ingredients with specific contours in the refrigerator in any one of the embodiments.
  • the method for managing ingredients with specific outlines in the refrigerator acquires the edge outlines of ingredients through images, and analyzes the occlusion of the edge outlines, so as to obtain the storage status of ingredients with specific outlines in a predetermined area, and quickly prepares to judge whether they are stacked, thereby Judging whether the quantity is sufficient is convenient for users to manage ingredients.
  • Fig. 1 is a flow chart of a method for managing ingredients with specific contours in a refrigerator according to an embodiment of the present invention
  • Fig. 2 is a schematic diagram of a bird's-eye view of an embodiment of the present invention.
  • Fig. 3 is a schematic diagram of each module in the refrigerator according to an embodiment of the present invention.
  • An embodiment of the present invention provides a refrigerator 100.
  • the refrigerator 100 includes a box body and a door body for opening and closing the box body.
  • a storage space for storing ingredients is formed in the box, and a number of bottle holders 200 for storing ingredients can be installed on the door.
  • An image acquisition module such as a camera can also be installed on the door body, and the image acquisition module can be installed above one of the bottle holders 200 to collect images of food materials stored in the bottle holder 200.
  • image acquisition modules can also be installed above each bottle holder 200. module.
  • the image acquisition module may include two cameras respectively installed on the left and right sides of the bottle holder 200 to obtain a bird's-eye view of the ingredients stored in the bottle holder 200. Cameras in three positions. The camera can be installed on the bottom of the upper bottle holder 200 and shoot vertically downwards. When multiple cameras are installed, the image information collected by multiple cameras can be transmitted to the image processing module, and the multiple images can be pre-corrected and processed by a preset algorithm to obtain the best bird's-eye view.
  • the bottle base 200 provided with the image acquisition module can form a predetermined area for storing ingredients with a specific profile, and only one type of food with a specific profile can be stored in each predetermined area.
  • the specific profile ingredients may be poultry and egg ingredients such as eggs, and may also be ingredients such as oranges and apples.
  • the food with a specific outline to be stored in this specific area can be the food that is preset when the refrigerator 100 leaves the factory. In this way, a pre-trained machine learning model and processing model can be set in the refrigerator 100 to facilitate accurate identification according to the outline characteristics of the food.
  • the edge outline of the ingredients can be set in the refrigerator 100 to facilitate accurate identification according to the outline characteristics of the food.
  • the refrigerator 100 provided by an embodiment of the present invention can also be provided with a display screen on the door, through which the food information in the refrigerator 100 can be displayed, such as the image information of the relevant food materials captured, and the image information can also be processed After the pre-processing analysis obtains the storage information of the ingredients, it will be displayed on the display.
  • Interaction modules such as a voice module can also be set on the refrigerator 100, or a WiFi module can be installed, and the refrigerator 100 can be connected to a cloud server or a mobile terminal such as a mobile phone through the WiFi module, so that relevant information can be sent to the cloud for processing, or sent to the mobile phone
  • the mobile terminal is convenient for users to view and manage.
  • an embodiment of the present invention provides a method for managing ingredients with a specific profile in a refrigerator 100 , which can be used in the above-mentioned refrigerator 100 .
  • the specific profile ingredients management method in the refrigerator 100 may include:
  • the number of occlusion contour lines connected to the closed-loop contour line has an occlusion area greater than the preset area is greater than the preset value, then mark the food material corresponding to the closed-loop contour line as the upper-level food material, otherwise, mark the food material corresponding to the closed-loop contour line as lower ingredients;
  • conventional edge detection algorithms such as the Canny edge detection algorithm can be used to identify the edge contours of the ingredients in the bird's-eye view, and processed by a pre-trained machine learning model to obtain the edge contours corresponding to the ingredients.
  • the specific outline of the food is circular or oval or similar to a circle.
  • the closed-loop contour of the unoccluded food and the occluded contour of the occluded food connected to the closed-loop contour can be further identified according to the characteristics of the predetermined specific contour.
  • the shape of the closed-loop contour line of the non-occluded food material may be a complete circle, and the occluded contour line of the covered food material may have a partially discontinuous and complete circle-like feature.
  • the ingredients corresponding to the shaded ingredient contour lines connected to the closed-loop contour line may be partially occluded by the ingredients corresponding to the closed-loop contour line.
  • the occluded area of the occluded food can be the area occluded by the closed-loop contour line connected to it, and the contour of the occluded food can be restored through the pre-trained machine learning model, so that the contour of the occluded food can be calculated
  • the preset area can be a certain proportion of the complete area, for example, it can be a quarter of the complete area, and the preset area can also be a fixed value.
  • the occlusion area When it is detected that the occlusion area is larger than the preset area, it can be determined that the food corresponding to the occluded food is seriously occluded.
  • the number of occluded ingredients connected by a closed-loop contour line is more than one, for example, there are two or more, that is, the ingredients corresponding to the closed-loop contour line seriously block at least two ingredients, according to the ingredients
  • the stacking situation during storage can determine that the ingredients in the predetermined area are stacked and stored.
  • the ingredients corresponding to the closed-loop contour line are stacked on top of other ingredients. . Otherwise, the ingredient corresponding to the closed-loop contour line is the lower layer ingredient.
  • the storage status of the ingredients in the predetermined area can be obtained, and the prompt information of the storage status of the ingredients in the predetermined area can be output through interactive modules such as a display screen and a voice module, so as to facilitate management by the user.
  • the food storage status prompt information can be stacking or non-stacking of the food, one or two layers of food storage, etc., or specific quantity information, so that the user can know the storage status of the relevant food.
  • the method for managing ingredients with specific contours in the refrigerator 100 further includes:
  • ingredients stored in the predetermined area include ingredients on the upper layer, a prompt message indicating that the ingredients are sufficient is output.
  • the ingredients stored in the predetermined area include the ingredients on the upper layer, it can be determined that the ingredients in the predetermined area are stacked and stored. Therefore, it can be determined that there are more ingredients stored in the area, so the ingredients can be directly output. Sufficient prompt information is convenient for users to intuitively understand the quantity information of related ingredients.
  • the method for managing ingredients with specific contours in the refrigerator 100 further includes:
  • the ingredients corresponding to the occluded ingredient contour lines connected to the closed-loop contour line and whose occlusion area is larger than the preset area are marked as lower layer ingredients.
  • the food material corresponding to the closed-loop contour line when the food material corresponding to the closed-loop contour line is an upper-layer food material, the food material corresponding to the blocked food material contour line with an occluded area larger than the preset area is severely blocked by the food material corresponding to the closed-loop contour line.
  • the closed-loop contour line The ingredients corresponding to the line are stacked on top of the ingredients that are heavily shaded by it, and the ingredients that are heavily shaded by it can be the lower layer ingredients.
  • the food material corresponding to the closed-loop contour line is a lower-layer food material
  • the food material seriously blocked by it is at least partly located below the food material corresponding to the closed-loop contour line, therefore, the food material heavily covered by it is also a lower-layer food material.
  • the stacking state of the ingredients stored in the predetermined area can be further confirmed according to the characteristics of the edge contour line, so as to facilitate more accurate acquisition and presentation of the food storage status information in the predetermined area.
  • FIG. 2 it is a bird's-eye view of a specific example of an embodiment of the present invention.
  • a number of eggs are stored on the bottle holder 200 of the door body of the refrigerator 100, and the edge contour map of the internal eggs can be obtained by analyzing the bird's-eye view, wherein, egg A and egg B are unshielded eggs with closed-loop contour lines.
  • the occluded material contour lines connected to the closed-loop contour line of egg A are the contour lines of eggs A1, A2, A3, A4, A5, and A6.
  • the pre-trained machine learning model it is possible to calculate the area where the contour lines of eggs A1, A2, A3, A4, A5, and A6 are blocked by the contour line of egg A, and the overall area after restoration.
  • A1, A5, and A6 The occluded area is greater than one-fourth of its corresponding overall area, which is serious occlusion, while the areas of A2 and A3 are less than one-quarter of the overall area occluded by A, which is non-serious occlusion, because the eggs that are seriously occluded
  • the number is greater than 1. Therefore, egg A can be marked as the upper layer egg, and eggs A1, A5, and A6 are marked as the lower layer ingredients.
  • the contour line of the covered material connected to it is the contour line of egg B1
  • the area of egg B1 covered by the blocked egg B is less than a quarter, so it can be determined that egg B is the lower egg.
  • the eggs are stacked, which can indicate that the number of eggs is sufficient.
  • the method for managing ingredients with specific contours in the refrigerator 100 further includes:
  • ingredients corresponding to the closed-loop contour lines are all lower-layer ingredients, the ingredients corresponding to each ingredient edge contour line are marked as lower-layer ingredients.
  • the method for managing ingredients with a specific profile in the refrigerator 100 further includes:
  • the ingredients corresponding to the non-serious occlusion contour lines are marked as upper-level ingredients, and the occlusion area is greater than the preset area.
  • the ingredients corresponding to the shaded ingredient outlines are marked as lower layer ingredients. Otherwise, the ingredient corresponding to the non-severely occluded contour line may be marked as the lower layer ingredient.
  • the contour lines of the occluded ingredients connected to its contour can be obtained, that is, the contour lines of eggs B11, B12, and B13.
  • ingredients stored in the predetermined area include upper-layer ingredients, identify the number of corresponding upper-layer ingredients, lower-layer ingredients in the edge contour line, and the number of occlusion contour lines whose occlusion area is less than or equal to the preset area, and output quantity prompt information.
  • the calculation can be performed on the basis of closed-loop contours.
  • the number of closed-loop contours and the number of seriously occluded ingredients connected to the closed-loop contours are counted. Occlusion is deduplicated when counting.
  • the number of ingredients marked as upper-level ingredients, the number of ingredients marked as lower-level ingredients, and the number of unmarked ingredients can be counted, and deduplication processing is performed for ingredients that have been marked multiple times.
  • the ingredients corresponding to the closed-loop contour lines are all lower-level ingredients, it can be determined that there is no stacking of ingredients in the predetermined area, and the number of edge contour lines corresponding to the ingredients can be directly determined as the number of ingredients.
  • performing edge detection on images to obtain edge outlines of ingredients includes:
  • Edge detection is performed in the detection frame to identify the edge outline of the food.
  • the detection frame can be determined according to the outline of the food, for example, when the food is an egg, the detection frame can be a rectangular frame, and each detection frame can correspond to one food.
  • the accuracy of recognition can be improved by first processing the image information into detection frames, and then performing edge detection and recognition in each detection frame.
  • performing edge detection in the detection frame to identify the edge contours of the ingredients includes:
  • Edge detection is performed in the detection frame to identify the edge contour line that conforms to the circular feature.
  • identifying the corresponding edge contour lines in the detection frame can avoid recognition errors and improve recognition accuracy.
  • the method for managing ingredients with specific contours in the refrigerator 100 detects and recognizes the edge contour lines of the ingredients, and judges the stacking state according to the area and quantity that the contour lines block each other, so that the ingredients in the predetermined area can be accurately judged Store state for easy user management.
  • an embodiment of the present invention also provides a refrigerator 100 , including a memory 102 and a processor 101 , and the memory 102 and the processor 101 are communicatively connected through a communication bus 104 .
  • a computer program that can run on the processor 101 is stored in the memory 102.
  • the processor 101 executes the computer program, the steps in the refrigerator control method in the above-mentioned embodiments are realized.
  • the refrigerator also includes a communication interface 103 connected to a communication bus 104 for communicating with other devices of the refrigerator 100 .
  • An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored.
  • a computer program is stored on which a computer program is stored.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Geometry (AREA)
  • Cold Air Circulating Systems And Constructional Details In Refrigerators (AREA)
  • Image Analysis (AREA)

Abstract

冰箱内特定轮廓食材管理方法包括：控制图像采集模块采集预定区域内存放的所述食材的图像；对所述图像进行边缘检测获取所述食材的边缘轮廓线；识别所述边缘轮廓线中未遮挡食材的闭环轮廓线，以及与所述闭环轮廓线相连的被遮挡食材轮廓线；通过预训练的机器学习模型分析所述被遮挡食材轮廓线对应的遮挡面积；若与闭环轮廓线相连的被遮挡食材轮廓线中，遮挡面积大于预设面积的个数大于预设值，则将所述闭环轮廓线对应的食材标记为上层食材。

Description

特定轮廓食材管理方法、存储介质以及冰箱 技术领域
本发明涉及智能家电领域，尤其是一种冰箱内特定轮廓食材管理方法、存储介质以及冰箱。
背景技术
在家庭生活中，冰箱已经成为一种不可或缺的家用电器，用户往往会在冰箱内存储大量的食材。为了方便用户进行食材管理，部分冰箱上设置有图像采集模块，通过采集食材存储图像获取食材的数量。但是对于鸡蛋、苹果、橘子等特定轮廓的食材，在冰箱内存储时往往会进行堆叠存放，食材可能被遮挡，通过采集到的相关图像很难准确判断食材的数量。
发明内容
为了解决上述问题，本发明提出了一种冰箱内特定轮廓食材管理方法，能够判断预定区域内存储的特定轮廓食材是否存在堆叠。
为实现上述发明目的之一，本发明一实施方式提供了一种冰箱内特定轮廓食材管理方法，包括：
控制图像采集模块采集预定区域内存放的所述食材的图像；
对所述图像进行边缘检测获取所述食材的边缘轮廓线；
识别所述边缘轮廓线中未遮挡食材的闭环轮廓线，以及与所述闭环轮廓线相连的被遮挡食材轮廓线；
通过预训练的机器学习模型分析所述被遮挡食材轮廓线对应的遮挡面积；
若与闭环轮廓线相连的被遮挡食材轮廓线中，遮挡面积大于预设面积的个数大于预设值，则将所述闭环轮廓线对应的食材标记为上层食材，否则，将所述闭环轮廓线对应的食材标记为下层食材；
输出所述预定区域内食材存储状态提示信息。
作为本发明一实施方式的进一步改进，所述冰箱内特定轮廓食材管理方法还包括：
若所述预定区域存放的食材包括上层食材，则输出食材充足的提示信息。
作为本发明一实施方式的进一步改进，所述冰箱内特定轮廓食材管理方法还包括：
将与所述闭环轮廓线相连的遮挡面积大于预设面积的被遮挡食材轮廓线对应的食材标记为下层食材。
作为本发明一实施方式的进一步改进，所述冰箱内特定轮廓食材管理方法还包括：
若闭环轮廓线对应的食材均为下层食材，则将每个食材边缘轮廓线对应的食材标记为下层食材。
作为本发明一实施方式的进一步改进，所述冰箱内特定轮廓食材管理方法还包括：
识别与所述闭环轮廓线相连的非严重遮挡轮廓线，以及与所述非严重遮挡轮廓线相连的被遮挡食材轮廓线；
通过预训练的机器学习模型分析与所述非严重遮挡轮廓线遮挡的所述被遮挡食材轮廓线的面积；
若与所述非严重遮挡廓线相连的被遮挡轮廓线中，被所述非严重遮挡轮廓线遮挡的面积大于预设面积的数量大于预设值，则将所述非严重遮挡轮廓线对应的食材标记为上层食材，否则，将所述非严重遮挡轮廓线对应的食材标记为下层食材。
作为本发明一实施方式的进一步改进，控制图像采集模块采集预定区域内存放的所述食材的图像”包括：
控制图像采集模块采集预定区域内存放的所述食材的鸟瞰图。
“对所述图像进行边缘检测获取所述食材的边缘轮廓线”包括：
采用预先训练的机器学习模型在所述鸟瞰图中分割出与每个所述食材对应的检测框；
在所述检测框内进行边缘检测识别所述食材的边缘轮廓线。
作为本发明一实施方式的进一步改进，所述冰箱内特定轮廓食材管理方法，还包括：
分别获取被标记为上层食材和被标记为下层食材的边缘轮廓线数量；
输出上层食材和下层食材的数量信息。
作为本发明一实施方式的进一步改进，所述特定轮廓食材为蛋类；
“在所述检测框内进行边缘检测识别所述食材的边缘轮廓线”包括：
在所述检测框内进行边缘检测识别符合类圆形特征的边缘轮廓线。
为实现上述发明目的之一，本发明一实施方式提供一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述任一实施方式的冰箱内特定轮廓食材管理方法中的步骤。
为实现上述发明目的之一，本发明一实施方式提供一种冰箱，所述冰箱内设置有储物间室，所述储物间室内具有用于存放所述特定轮廓食材的预定区域，以及用采集所述预定区域的图像采集模块，所述冰箱还包括存储器和处理器，所述存储器存储有可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时，实现上述任一实施方式的冰箱内特定轮廓食材管理方法中的步骤。
本发明提供的冰箱内特定轮廓食材管理方法，通过图像获取食材的边缘轮廓线，分析边缘轮廓线被遮挡情况，从而获取预定区域内特定轮廓食材的存储状态，快速准备的判断其是否堆叠，从而判断数量是否充足，便于用户进行食材管理。
附图说明
图1为本发明一实施方式的冰箱内特定轮廓食材管理方法流程图；
图2为本发明一实施方式的俯瞰图示意图；
图3为本发明一实施方式的冰箱内各模块示意图。
具体实施方式
为了使本技术领域的人员更好地理解本发明中的技术方案，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本发明保护的范围。
本发明一实施方式提供一种冰箱100，冰箱100包括箱体以及用于开闭箱体的门体。箱体内形成有用于存储食材的存储空间，门体上可安装有若干的用于存放食材的瓶座200。门体上还可安装有摄像头等图像采集模块，图像采集模块可安装在其中一个瓶座200上方以采集瓶座200内的食材存储图像，当然也可以在每个瓶座200上方均安装图像采集模块。
在本实施方式中，图像采集模块可以包括两个分别安装于瓶座200左右两侧的摄像头，可以获取瓶座200内存储食材的鸟瞰图，图像采集模块也可以包括分别安 装于瓶座200左右中三个位置的摄像头。摄像头可安装于上层瓶座200的底部，垂直向下拍摄。当安装有多个摄像头，可以将多个摄像头采集到的图像信息均传输至图像处理模块，通过预先设定的算法对多个图像进行预先修正处理，获得最佳的鸟瞰图。
设置有图像采集模块的瓶座200可以形成用于存储特定轮廓食材的预定区域，每个预定区域内可仅存储一种特定轮廓食材。特定轮廓食材可以为鸡蛋等禽蛋类食材，也可以为橘子、苹果等食材。该特定区域待存储的特定轮廓食材可以可为冰箱100出厂时预先设定的食材，如此，可以在冰箱100内设置预训练的机器学习模型和处理模型等，以便于根据食材的轮廓特点准确识别食材的边缘轮廓线。
本发明一实施方式提供的冰箱100，还可以在门体上设置有显示屏，通过显示屏可以显示冰箱100内的食材信息，如显示拍摄到的相关食材的图像信息，也可以对图像信息进行预先处理分析获取食材存储信息后通过显示屏显示。冰箱100上还可以设置语音模块等交互模块，或者可以设置WiFi模块，通过WiFi模块可以将冰箱100与云端服务器或者手机等移动终端通信连接，从而将相关信息发送至云端进行处理，或者发送至手机等移动端便于用户查看管理。
参见图1，本发明一实施方式提供了一种冰箱100内特定轮廓食材的管理方法可用于上述冰箱100。冰箱100内特定轮廓食材管理方法可包括：
控制图像采集模块采集预定区域内存放的食材的图像；
对图像进行边缘检测获取食材的边缘轮廓线；
识别边缘轮廓线中未遮挡食材的闭环轮廓线，以及与闭环轮廓线相连的被遮挡食材轮廓线；
通过预训练的机器学习模型分析遮挡轮廓线对应的遮挡面积；
若与闭环轮廓线相连的遮挡轮廓线中，遮挡面积大于预设面积的个数大于预设值，则将闭环轮廓线对应的食材标记为上层食材，否则，将闭环轮廓线对应的食材标记为下层食材；
输出预定区域内食材存储状态提示信息。
在本实施方式中，可以通过Canny边缘检测算法等常规边缘检测算法识别鸟瞰图中食材的边缘轮廓线，通过预先训练的机器学习模型进行处理，获取与食材对应的边缘轮廓线。
以特定轮廓食材为蛋类作为示例，食材的特定轮廓呈圆形或者椭圆形等类圆形。 在获取食材的边缘轮廓线后，可以进一步的根据预定的特定轮廓的特征识别其中未遮挡食材的闭环轮廓线，以及与该闭环轮廓线相连的被遮挡食材的遮挡轮廓线。其中，未遮挡食材的闭环轮廓线的形状可为完整的类圆形，被遮挡食材的遮挡轮廓线可具有部分不连续且完整的类圆形特征。与闭环轮廓线相连的被遮挡食材轮廓线对应的食材，可为被该闭环轮廓线对应的食材部分遮挡。
在本实施方式中，被遮挡食材的遮挡面积可以为被与其相连的闭环轮廓线遮挡的面积，通过预先训练的机器学习模型可以将被遮挡食材的轮廓线还原，从而可计算出被遮挡食材轮廓线还原后的完整面积、以及被与相连的闭环轮廓线遮挡的遮挡面积。预设面积可以为完整面积的一定比例，如可以为完整面积的四分之一，预设面积也可以为一个固定值。
当检测到遮挡面积大于预设面积时，可以判定该被遮挡食材对应的食材被严重遮挡。当一个闭环轮廓线相连的被遮挡食材轮廓线中被严重遮挡的数量大于1个，如具有两个或者两个以上，也就是该闭环轮廓线对应的食材严重遮挡了至少两个食材，根据食材存放时的堆叠情况可以判定预定区域内的食材存在堆叠存放的情况，该闭环轮廓线对应的食材堆叠于其他食材上方，该闭环轮廓线对应的食材为上层食材，其对下层食材造成了严重遮挡。否则，该闭环轮廓线对应的食材为下层食材。
如此，可以获取预定区域内食材存储状态，并可通过显示屏、语音模块等交互模块输出预定区域内食材存储状态提示信息，以便于用户进行管理。其中，食材存储状态提示信息可以为食材存在堆叠或者不存在堆叠情况、食材存储有一层或者两层等，也可为具体的数量信息等，从而用户可以了解相关食材的存储情况。
进一步的，在本发明一实施方式中，冰箱100内特定轮廓食材管理方法还包括：
若预定区域内存放的食材包括上层食材，则输出食材充足的提示信息。
在本实施方式中，若预定区域内存放的食材包括上层食材，可以判定预定区域内的食材存在堆叠存放的情况，因此，可以判定该区域内存储有较多的食材，因而，可以直接输出食材充足的提示信息，方便用户直观了解相关食材的数量信息。
进一步的，在本发明一实施方式中，冰箱100内特定轮廓食材管理方法还包括：
将与闭环轮廓线相连的遮挡面积大于预设面积的被遮挡食材轮廓线对应的食材标记为下层食材。
在本实施方式中，当闭环轮廓线对应的食材为上层食材时，遮挡面积大于预设面积的被遮挡食材轮廓线对应的食材被该闭环轮廓线对应的食材严重遮挡，可以理 解，该闭环轮廓线对应的食材堆叠于被其严重遮挡的食材上方，被其严重遮挡的食材可为下层食材。
当闭环轮廓线对应的食材为下层食材时，可以理解，被其严重的遮挡的食材至少部分位于该闭环轮廓线对应的食材下方，因此，被其严重遮挡的食材也为下层食材。
如此，可以根据边缘轮廓线的特征进一步确认预定区域内存储的食材的堆叠状态，便于更准确的获取并提示预定区域内食材存储状态信息。
参见图2，为本发明一实施方式的具体示例中的俯瞰图。冰箱100门体的瓶座200上存储有若干鸡蛋，通过对俯瞰图进行分析可以得出内部鸡蛋的边缘轮廓图，其中，鸡蛋A和鸡蛋B为未遮挡鸡蛋，具有闭环轮廓线。与鸡蛋A的闭环轮廓线相连的被遮挡食材轮廓线为鸡蛋A1、A2、A3、A4、A5、A6的轮廓线。通过预先训练的机器学习模型可以计算出鸡蛋A1、A2、A3、A4、A5、A6的轮廓线被鸡蛋A的轮廓线遮挡的面积，以及其还原后的整体面积，若其中A1、A5、A6被遮挡面积均大于其对应的整体面积的四分之一，为严重遮挡，而A2、A3被A的遮挡的面积小于整体面积的四分之一，为非严重遮挡，因被严重遮挡的鸡蛋个数大于1，因此，可以将鸡蛋A标记为上层鸡蛋，鸡蛋A1、A5、A6被标记为下层食材。
同样，对于鸡蛋B，与其相连的被遮挡食材轮廓线为鸡蛋B1的轮郭线，鸡蛋B1被遮挡鸡蛋B遮挡的面积小于四分之一，可以判定鸡蛋B为下层鸡蛋。此时，因存在上层鸡蛋，因此，可以判定鸡蛋存在堆叠现象，可提示鸡蛋数量充足。
进一步的，在本发明一实施方式中，冰箱100内特定轮廓食材管理方法还包括：
若闭环轮廓线对应的食材均为下层食材，则将每个食材边缘轮廓线对应的食材标记为下层食材。
在本实施方式中，若所有闭环轮廓线对应的食材均为下层食材，可以认为所有完全裸露于外侧的未遮挡食材均为下层食材，此时，预定区域内存储的食材不存在堆叠状态，可以将所有食材边缘轮廓线对应的食材标记为下层食材，此时，也可发出食材数量不足的提示信息，或者发出食材仅剩有一层的提示信息。
进一步的，在本发明一实施方式中，冰箱100内特定轮廓食材管理方法，还包括：
获取与所述闭环轮廓线相连的非严重遮挡轮廓线，非严重遮挡轮廓线的遮挡面积小于或等于预设面积；
识别与该非严重遮挡轮廓线相连的被遮挡食材轮廓线；
通过预训练的机器学习模型分析被该非严重遮挡轮廓线遮挡的被遮挡轮廓线的遮挡面积；
若与非严重遮挡轮廓线相连的被遮挡轮廓线中，遮挡面积大于预设面积的数量大于预设值，则将非严重遮挡轮廓线对应的食材标记为上层食材，遮挡面积大于预设面积的被遮挡食材轮廓线对应的食材标记为下层食材。否则，可以将该非严重遮挡轮廓线对应的食材标记为下层食材。
具体示例可继续参见图3，对于非严重遮挡的鸡蛋B1，可以获取与其轮廓相连的被其遮挡的被遮挡食材轮廓线，也就是鸡蛋B11、B12、B13的轮廓线，通过重复上述算法，可以计算出鸡蛋B11、B12、B13的轮廓线被鸡蛋B1轮廓线遮挡的遮挡面积，因其遮挡面积均大于四分之一，数量超过一个，因此，可以判定鸡蛋B1为上层鸡蛋，而鸡蛋B11、B12、B13为下层鸡蛋。
如此，可以进一步确认食材的分布位置，并获取上层食材和下层食材的数量，给予用户更精准的数量提示。
若预定区域内存放的食材包括上层食材，识别边缘轮廓线中对应的上层食材的数量、下层食材的数量以及遮挡面积小于或等于预设面积的遮挡轮廓线数量，输出数量提示信息。
在本实施方式中，可以以闭环轮廓线为基础进行计算，先统计闭环轮廓线的数量以及与闭环轮廓线相连的严重遮挡的食材数量，对于一个被遮挡食材轮廓线若同时被多个轮廓线遮挡则在计数时进行去重处理。或者，可以统计被标记为上层食材的食材数量和被标记为下层食材的食材数量，以及未被标记的食材数量，对于其中被多次标记的食材，进行去重处理。
进一步的，在本发明一实施方式中，若闭环轮廓线对应的食材均为下层食材，可判定预定区域的食材不存在堆叠情况，食材对应的边缘轮廓线的个数可直接判定为食材数量。
进一步的，在本发明一实施方式中，冰箱100内特定轮廓食材管理方法中，“对图像进行边缘检测获取食材的边缘轮廓线”包括：
采用预先训练的机器学习模型在鸟瞰图中分割出每个食材对应的检测框；
在检测框内进行边缘检测识别食材的边缘轮廓线。
在本实施方式中，检测框可根据食材的轮廓确定，如当食材为鸡蛋时，检测框 可以为矩形框，每个检测框红可对应一个食材。通过先将图像信息处理分割出检测框，再在每个检测框内进行边缘检测识别，能够提升识别的准确度。
当食材为蛋类时，“在检测框内进行边缘检测识别所述食材的边缘轮廓线”包括：
在检测框内进行边缘检测识别符合类圆形特征的边缘轮廓线。
如此，根据食材的特定轮廓特征，在检测框内识别与之相符合的边缘轮廓线，可以避免识别错误，提升识别准确度。
综上所述，本发明提供的冰箱100内特定轮廓食材管理方法，通过检测识别食材的边缘轮廓线，并根据轮廓线相互遮挡的面积和数量判断堆叠状态，从而能够准确的判断预定区域的食材存储状态，便于用户进行管理。
参见图3，本发明一实施方式还提供一种冰箱100，包括存储器102和处理器101，存储器102和处理器101通过通信总线104通信连接。存储器102上存储有可在处理器101上运行的计算机程序，所述处理器101执行所述计算机程序时，实现上述实施方式中的冰箱控制方法中的步骤。冰箱还包括与通信总线104连接的通信接口103，用于与冰箱100的其他设备通信。
本发明一实施方式还提供一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时，实现上述实施方式中的冰箱控制方法中的步骤。
应当理解，虽然本说明书按照实施例加以描述，但并非每个实施例仅包含一个独立的技术方案，说明书的这种叙述方式仅仅是为清楚起见，本领域技术人员应当将说明书作为一个整体，各实施例中的技术方案也可以经适当组合，形成本领域技术人员可以理解的其他实施例。
上文所列出的一系列的详细说明仅仅是针对本发明的可行性实施例的具体说明，并非用以限制本发明的保护范围，凡未脱离本发明技艺精神所作的等效实施例或变更均应包含在本发明的保护范围之内。

Claims (10)

  1. 一种冰箱内特定轮廓食材管理方法，其特征在于，包括：
    控制图像采集模块采集预定区域内存放的所述食材的图像；
    对所述图像进行边缘检测获取所述食材的边缘轮廓线；
    识别所述边缘轮廓线中未遮挡食材的闭环轮廓线，以及与所述闭环轮廓线相连的被遮挡食材轮廓线；
    通过预训练的机器学习模型分析所述被遮挡食材轮廓线对应的遮挡面积；
    若与闭环轮廓线相连的被遮挡食材轮廓线中，遮挡面积大于预设面积的个数大于预设值，则将所述闭环轮廓线对应的食材标记为上层食材，否则，将所述闭环轮廓线对应的食材标记为下层食材；
    输出所述预定区域内食材存储状态提示信息。
  2. 如权利要求1所述的冰箱内特定轮廓食材管理方法，其特征在于，还包括：
    若所述预定区域存放的食材包括上层食材，则输出食材充足的提示信息。
  3. 如权利要求1所述的冰箱内特定轮廓食材管理方法，其特征在于，还包括：
    将与所述闭环轮廓线相连的遮挡面积大于预设面积的被遮挡食材轮廓线对应的食材标记为下层食材。
  4. 如权利要求1所述的冰箱内特定轮廓食材管理方法，其特征在于，还包括：
    若闭环轮廓线对应的食材均为下层食材，则将每个食材边缘轮廓线对应的食材均标记为下层食材。
  5. 如权利要求1所述的冰箱内特定轮廓食材管理方法，其特征在于，还包括：
    获取与所述闭环轮廓线相连的非严重遮挡轮廓线，所述非严重遮挡轮廓线被所述闭环轮廓线遮挡的遮挡面积小于或等于所述预设面积；
    识别与所述非严重遮挡轮廓线相连的被遮挡食材轮廓线；
    通过预训练的机器学习模型分析被所述非严重遮挡轮廓线遮挡的被遮挡食材轮廓线的遮挡面积；
    若与所述非严重遮挡廓线相连的被遮挡轮廓线中，遮挡面积大于预设面积的数量大于预设值，则将所述非严重遮挡轮廓线对应的食材标记为上层食材，否则，将 所述非严重遮挡轮廓线对应的食材标记为下层食材。
  6. 如权利要求1所述的冰箱内特定轮廓食材管理方法，其特征在于，“控制图像采集模块采集预定区域内存放的所述食材的图像”包括：
    控制图像采集模块采集预定区域内存放的所述食材的鸟瞰图；
    “对所述图像进行边缘检测获取所述食材的边缘轮廓线”包括：
    采用预先训练的机器学习模型在所述鸟瞰图中分割出与每个所述食材对应的检测框；
    在所述检测框内进行边缘检测识别所述食材的边缘轮廓线。
  7. 如权利要求6所述的冰箱内特定轮廓食材管理方法，其特征在于，还包括：
    分别获取被标记为上层食材和被标记为下层食材的边缘轮廓线数量；
    输出上层食材和下层食材的数量信息。
  8. 如权利要求1所述的冰箱内特定轮廓食材管理方法，其特征在于，所述特定轮廓食材为蛋类；
    “在所述检测框内进行边缘检测识别所述食材的边缘轮廓线”包括：
    在所述检测框内进行边缘检测识别符合类圆形特征的边缘轮廓线。
  9. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1所述的冰箱内特定轮廓食材管理方法中的步骤。
  10. 一种冰箱，所述冰箱内设置有储物间室，所述储物间室内具有用于存放所述特定轮廓食材的预定区域，以及用采集所述预定区域的图像采集模块，其特征在于，还包括存储器和处理器，所述存储器存储有可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时，实现权利要求1所述的冰箱内特定轮廓食材管理方法中的步骤。
PCT/CN2022/142205 2021-12-30 2022-12-27 特定轮廓食材管理方法、存储介质以及冰箱 WO2023125491A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111649594.5 2021-12-30
CN202111649594.5A CN114882057A (zh) 2021-12-30 2021-12-30 特定轮廓食材管理方法、存储介质以及冰箱

Publications (1)

Publication Number Publication Date
WO2023125491A1 true WO2023125491A1 (zh) 2023-07-06

Family

ID=82667594

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/142205 WO2023125491A1 (zh) 2021-12-30 2022-12-27 特定轮廓食材管理方法、存储介质以及冰箱

Country Status (2)

Country Link
CN (1) CN114882057A (zh)
WO (1) WO2023125491A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114882057A (zh) * 2021-12-30 2022-08-09 青岛海尔电冰箱有限公司 特定轮廓食材管理方法、存储介质以及冰箱

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018040105A1 (zh) * 2016-09-05 2018-03-08 合肥华凌股份有限公司 食材识别系统和方法、食材模型训练方法、冰箱和服务器
CN108240732A (zh) * 2016-12-23 2018-07-03 青岛海尔智能技术研发有限公司 一种冰箱及具有特定轮廓的食材统计方法和系统
CN112699888A (zh) * 2020-12-31 2021-04-23 上海肇观电子科技有限公司 图像识别方法、目标对象的提取方法、装置、介质及设备
CN113076907A (zh) * 2021-04-16 2021-07-06 青岛海尔电冰箱有限公司 冰箱内物品信息标识方法、冰箱和计算机存储介质
CN114882057A (zh) * 2021-12-30 2022-08-09 青岛海尔电冰箱有限公司 特定轮廓食材管理方法、存储介质以及冰箱

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018040105A1 (zh) * 2016-09-05 2018-03-08 合肥华凌股份有限公司 食材识别系统和方法、食材模型训练方法、冰箱和服务器
CN108240732A (zh) * 2016-12-23 2018-07-03 青岛海尔智能技术研发有限公司 一种冰箱及具有特定轮廓的食材统计方法和系统
CN112699888A (zh) * 2020-12-31 2021-04-23 上海肇观电子科技有限公司 图像识别方法、目标对象的提取方法、装置、介质及设备
CN113076907A (zh) * 2021-04-16 2021-07-06 青岛海尔电冰箱有限公司 冰箱内物品信息标识方法、冰箱和计算机存储介质
CN114882057A (zh) * 2021-12-30 2022-08-09 青岛海尔电冰箱有限公司 特定轮廓食材管理方法、存储介质以及冰箱

Also Published As

Publication number Publication date
CN114882057A (zh) 2022-08-09

Similar Documents

Publication Publication Date Title
WO2023125491A1 (zh) 特定轮廓食材管理方法、存储介质以及冰箱
CN105989347B (zh) 客观题智能阅卷方法及系统
CN108764100B (zh) 一种目标行为检测方法及服务器
CN107569248B (zh) 一种乳腺机设备的曝光方法及乳腺机设备
CN109492577A (zh) 一种手势识别方法、装置及电子设备
CN108711161A (zh) 一种图像分割方法、图像分割装置及电子设备
CN103383774A (zh) 图像处理方法及其设备
CN110348519A (zh) 金融产品欺诈团伙的识别方法和装置
CN108240732B (zh) 一种冰箱及具有特定轮廓的食材统计方法和系统
CN108932512A (zh) 一种冰箱食材录入方法
CN109685377A (zh) 推荐方法及装置、电子设备及存储介质
WO2023273895A1 (zh) 一种基于聚类的目标检测模型的评估方法
CN108596187A (zh) 商品纯净度检测方法及展示柜
CN110136153A (zh) 一种图像处理方法、设备及存储介质
CN109389019A (zh) 人脸图像选择方法、装置及计算机设备
CN111080493A (zh) 一种菜品信息识别方法、装置及菜品自助结算系统
WO2023193752A1 (zh) 冰箱内物品信息识别方法和冰箱
CN107223046A (zh) 智能导盲方法和装置
CN110149531A (zh) 一种识别视频数据中视频场景的方法和装置
CN114511820A (zh) 货架商品检测方法、装置、计算机设备及存储介质
CN107918798A (zh) 食物处理方法及装置
WO2023185779A1 (zh) 冰箱内物品信息识别方法
CN113569594A (zh) 一种人脸关键点的标注方法及装置
CN109063588B (zh) 一种基于人脸框提取图像中人脸部肤色的方法
CN107886540A (zh) 制冷设备内物品识别定位方法和制冷设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22914745

Country of ref document: EP

Kind code of ref document: A1