CN110826574A - Food material maturity determination method and device, kitchen electrical equipment and server - Google Patents

Food material maturity determination method and device, kitchen electrical equipment and server Download PDF

Info

Publication number
CN110826574A
CN110826574A CN201910918784.9A CN201910918784A CN110826574A CN 110826574 A CN110826574 A CN 110826574A CN 201910918784 A CN201910918784 A CN 201910918784A CN 110826574 A CN110826574 A CN 110826574A
Authority
CN
China
Prior art keywords
maturity
image
frame image
food material
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910918784.9A
Other languages
Chinese (zh)
Other versions
CN110826574B (en
Inventor
张坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao Guochuang Intelligent Home Appliance Research Institute Co ltd
Qingdao Haier Smart Technology R&D Co Ltd
Haier Smart Home Co Ltd
Original Assignee
Qingdao Guochuang Intelligent Home Appliance Research Institute Co ltd
Qingdao Haier Smart Technology R&D Co Ltd
Haier Smart Home Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qingdao Guochuang Intelligent Home Appliance Research Institute Co ltd, Qingdao Haier Smart Technology R&D Co Ltd, Haier Smart Home Co Ltd filed Critical Qingdao Guochuang Intelligent Home Appliance Research Institute Co ltd
Priority to CN201910918784.9A priority Critical patent/CN110826574B/en
Publication of CN110826574A publication Critical patent/CN110826574A/en
Application granted granted Critical
Publication of CN110826574B publication Critical patent/CN110826574B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/44Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/25Fusion techniques
    • G06F18/253Fusion techniques of extracted features

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application relates to the technical field of intelligent equipment, and discloses a method and a device for determining food material maturity, kitchen electrical equipment and a server. The method comprises the following steps: acquiring a current frame image and a starting frame image of a cooked food material in electric equipment, wherein the starting frame image is an image of the food material when cooking is started; performing fusion processing on the current frame image and the initial frame image to obtain a current fusion processing image; and inputting the current fusion processing image into a configured maturity machine learning training algorithm model, and determining the maturity corresponding to the food material. Therefore, the maturity of the food materials is determined through feature extraction and model prediction of the fusion processing images after the double-image fusion, and the reliability of determining the maturity of the food materials is improved.

Description

食材成熟度确定的方法、装置、厨电设备及服务器Method, device, kitchen electrical equipment and server for determining maturity of food materials

技术领域technical field

本申请涉及智能设备技术领域,例如涉及食材成熟度确定的方法、装置、厨电设备及服务器。The present application relates to the technical field of smart devices, for example, to a method, a device, a kitchen appliance, and a server for determining the maturity of an ingredient.

背景技术Background technique

随着科学技术的进步和人工智能的发展,智能算法也越来越多的应用到智能家电,例如:冰箱,空调、烤箱等,其中,烤箱、微波炉、空气炸锅等等厨电设备可以根据食材的类型、重量等信息,智能确定烹饪时间,烹饪火力等等。With the advancement of science and technology and the development of artificial intelligence, intelligent algorithms are increasingly applied to smart home appliances, such as refrigerators, air conditioners, ovens, etc. Among them, ovens, microwave ovens, air fryers and other kitchen appliances can be Information such as type and weight of ingredients, intelligently determine cooking time, cooking power, etc.

目前,对于一些烤制厨电设备还可进行食材成熟度的判断,可通过温度传感器探测食材内部的温度后,根据温度判断食材的成熟度。但是,烤制过程中,食材的温度不稳定,使得成熟度判断还不够可靠,并且,通过接触式传感器探测温度,可能会破坏食材的外观,也可能会污染食材。At present, some grilling kitchen appliances can also judge the maturity of the ingredients. After detecting the temperature inside the ingredients through a temperature sensor, the maturity of the ingredients can be judged according to the temperature. However, during the roasting process, the temperature of the ingredients is unstable, making the maturity judgment unreliable, and detecting the temperature through the contact sensor may damage the appearance of the ingredients and may also contaminate the ingredients.

发明内容SUMMARY OF THE INVENTION

为了对披露的实施例的一些方面有基本的理解,下面给出了简单的概括。所述概括不是泛泛评述,也不是要确定关键/重要组成元素或描绘这些实施例的保护范围,而是作为后面的详细说明的序言。In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended to be an extensive review, nor to identify key/critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

本公开实施例提供了一种食材成熟度确定的方法、装置、厨电设备、以及服务器,以解决确定食材成熟度可靠性不高的技术问题。Embodiments of the present disclosure provide a method, a device, a kitchen appliance, and a server for determining the maturity of an ingredient, so as to solve the technical problem of low reliability in determining the maturity of an ingredient.

在一些实施例中,所述方法包括:In some embodiments, the method includes:

获取电设备内被烹饪食材的当前帧图像、以及起始帧图像,其中,所述起始帧图像是烹饪启动时,所述食材的图像;acquiring a current frame image and an initial frame image of the ingredient to be cooked in the electrical device, wherein the initial frame image is an image of the ingredient when cooking is started;

将所述当前帧图像、以及所述起始帧图像进行融合处理,得到当前融合处理图像;Perform fusion processing on the current frame image and the initial frame image to obtain a current fusion processed image;

将所述当前融合处理图像,输入已配置的成熟度机器学习训练算法模型中,确定所述食材对应的成熟度。The current fusion processing image is input into the configured maturity machine learning training algorithm model to determine the corresponding maturity of the ingredients.

在一些实施例中,所述装置包括:In some embodiments, the apparatus includes:

信息获取模块,被配置为获取厨电设备内被烹饪食材的当前帧图像、以及起始帧图像,其中,所述起始帧图像是烹饪启动时,所述食材的图像;an information acquisition module, configured to acquire a current frame image and a start frame image of the ingredients to be cooked in the kitchen appliance, wherein the start frame image is an image of the ingredients when cooking is started;

特征提取模块,被配置为将所述当前帧图像、以及所述起始帧图像进行融合处理,得到当前融合处理图像;a feature extraction module, configured to perform fusion processing on the current frame image and the initial frame image to obtain a current fusion processed image;

预测确定模块,被配置为将所述当前融合处理图像,输入已配置的成熟度机器学习训练算法模型中,确定所述食材对应的成熟度。The prediction and determination module is configured to input the current fusion processed image into the configured maturity machine learning training algorithm model to determine the corresponding maturity of the food material.

在一些实施例中,所述厨电设备包括:上述食材成熟度确定的装置。In some embodiments, the kitchen electrical equipment includes: the above-mentioned device for determining the maturity of the food material.

在一些实施例中,所述服务器包括:上述食材成熟度确定的装置。In some embodiments, the server includes: the above-mentioned device for determining the maturity of the food material.

本公开实施例提供的食材成熟度确定的方法、装置、厨电设备及服务器,可以实现以下技术效果:The method, device, kitchen appliance, and server for determining the maturity of ingredients provided by the embodiments of the present disclosure can achieve the following technical effects:

获取食材烹饪过程中的当前帧图像和起始帧图像,对双图进行融合处理得到对应的当前融合处理图像,并输入到已配置的成熟度机器学习训练算法模型中,确定食材的成熟度。这样,通过对双图融合后的融合处理图像的特征提取以及模型预测,根据融合处理图像中双图之间的差异程度,确定食材成熟度,提高了确定食材成熟度的可靠性,并且,不需要接触食材,即可进行可靠性的确定,减少了破坏食材外观以及污染食材的几率。Obtain the current frame image and the starting frame image during the cooking process of the ingredients, fuse the two images to obtain the corresponding current fusion image, and input them into the configured maturity machine learning training algorithm model to determine the maturity of the ingredients. In this way, through the feature extraction and model prediction of the fused image after the fusion of the two images, the maturity of the ingredients is determined according to the degree of difference between the two images in the fusion image, which improves the reliability of determining the maturity of the ingredients. Reliability can be determined by contacting the ingredients, reducing the chance of damaging the appearance of the ingredients and contaminating the ingredients.

以上的总体描述和下文中的描述仅是示例性和解释性的,不用于限制本申请。The foregoing general description and the following description are exemplary and explanatory only and are not intended to limit the application.

附图说明Description of drawings

一个或多个实施例通过与之对应的附图进行示例性说明,这些示例性说明和附图并不构成对实施例的限定,附图中具有相同参考数字标号的元件示为类似的元件,附图不构成比例限制,并且其中:One or more embodiments are exemplified by the accompanying drawings, which are not intended to limit the embodiments, and elements with the same reference numerals in the drawings are shown as similar elements, The drawings do not constitute a limitation of scale, and in which:

图1是本公开实施例提供的一种食材成熟度确定方法的流程示意图;1 is a schematic flowchart of a method for determining the maturity of an ingredient provided by an embodiment of the present disclosure;

图1-1是本公开实施例提供的一种融合处理图像的示意图;1-1 is a schematic diagram of a fusion processing image provided by an embodiment of the present disclosure;

图2是本公开实施例提供的一种食材成熟度确定方法的流程示意图;2 is a schematic flowchart of a method for determining the maturity of an ingredient provided by an embodiment of the present disclosure;

图3是本公开实施例提供的一种食材成熟度确定系统的结构示意图;3 is a schematic structural diagram of a food material maturity determination system provided by an embodiment of the present disclosure;

图4是本公开实施例提供的一种食材成熟度确定方法的流程示意图;4 is a schematic flowchart of a method for determining the maturity of an ingredient provided by an embodiment of the present disclosure;

图5是本公开实施例提供的一种食材成熟度确定装置的结构示意图;FIG. 5 is a schematic structural diagram of a device for determining the maturity of an ingredient provided by an embodiment of the present disclosure;

图6是本公开实施例提供的一种食材成熟度确定装置的结构示意图;FIG. 6 is a schematic structural diagram of a device for determining the maturity of an ingredient provided by an embodiment of the present disclosure;

图7是本公开实施例提供的一种食材成熟度确定装置的结构示意图;FIG. 7 is a schematic structural diagram of a device for determining the maturity of an ingredient provided by an embodiment of the present disclosure;

图8是本公开实施例提供的一种食材成熟度确定装置的结构示意图。FIG. 8 is a schematic structural diagram of an apparatus for determining the maturity of an ingredient provided by an embodiment of the present disclosure.

具体实施方式Detailed ways

为了能够更加详尽地了解本公开实施例的特点与技术内容,下面结合附图对本公开实施例的实现进行详细阐述,所附附图仅供参考说明之用,并非用来限定本公开实施例。在以下的技术描述中,为方便解释起见,通过多个细节以提供对所披露实施例的充分理解。然而,在没有这些细节的情况下,一个或多个实施例仍然可以实施。在其它情况下,为简化附图,熟知的结构和装置可以简化展示。In order to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings, which are for reference only and are not intended to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, numerous details are provided to provide a thorough understanding of the disclosed embodiments. However, one or more embodiments may be practiced without these details. In other instances, well-known structures and devices may be shown simplified in order to simplify the drawings.

本公开实施例中,获取食材烹饪过程中的当前帧图像和起始帧图像,对双图进行融合处理得到对应的当前融合处理图像,并输入到已配置的成熟度机器学习训练算法模型中,确定食材的成熟度。这样,通过对双图融合后的融合处理图像的特征提取以及模型预测,根据融合处理图像中双图之间的差异程度,确定食材成熟度,提高了确定食材成熟度的可靠性,并且,在配置成熟度机器学习训练算法模型时,通过双图融合处理,可有效避免由于不同食材的形状、颜色和纹理等差异引起的成熟度回归网络不收敛的问题。另外,不需要接触食材,即可进行可靠性的确定,减少了破坏食材外观以及污染食材的几率。In the embodiment of the present disclosure, the current frame image and the starting frame image during the cooking process of the ingredient are acquired, the two images are fused to obtain the corresponding current fusion processed image, and the image is input into the configured maturity machine learning training algorithm model, Determine the maturity of the ingredients. In this way, through the feature extraction and model prediction of the fused image after the fusion of the two images, the maturity of the ingredients is determined according to the degree of difference between the two images in the fusion image, which improves the reliability of determining the maturity of the ingredients. When configuring the maturity machine learning training algorithm model, the dual-image fusion process can effectively avoid the problem of non-convergence of the maturity regression network caused by differences in the shape, color, and texture of different ingredients. In addition, the reliability can be determined without touching the ingredients, reducing the probability of damaging the appearance of the ingredients and contaminating the ingredients.

图1是本公开实施例提供的一种食材成熟度确定方法的流程示意图。如图1所示,食材成熟度确定的过程包括:FIG. 1 is a schematic flowchart of a method for determining the maturity of an ingredient provided by an embodiment of the present disclosure. As shown in Figure 1, the process of determining the maturity of ingredients includes:

步骤101:获取厨电设备内被烹饪食材的当前帧图像、以及起始帧图像。Step 101: Acquire a current frame image and an initial frame image of the ingredients to be cooked in the kitchen appliance.

本公开实施例中,厨电设备可包括:烤箱、微波炉、或空气炸锅等等这些可封闭,能烤制食材的电器。一般,可在厨电设备中配置图像采集设备,例如:顶部配置摄像头,通过图像采集设备可采集厨电设备中的图像或视频信息。In the embodiment of the present disclosure, the kitchen electrical equipment may include: an oven, a microwave oven, or an air fryer, and other electrical appliances that can be closed and can bake food. Generally, an image capture device can be configured in the kitchen appliance, for example, a camera is configured on the top, and the image or video information in the kitchen appliance can be captured through the image capture device.

图像采集设备可定时启动,获取厨电设备中的图像,或录取设定时间的视频信息。或者,图像采集设备可在触发信号的控制下开启或关闭,从而获取到对应的图像或视频信息。而触发信号可是厨电设备发送的,或者,服务器通过厨电设备发送的。例如,在确定厨电设备启动烹饪情况下,厨电设备或服务器发送启动指令,启动配置的图像采集设备,进行视频信息采集;而在确定厨电设备中门被开启,或者,到达设定时间的情况下,厨电设备或服务器发送关闭指令,关闭图像采集设备,停止视频信息采集。The image acquisition device can be started regularly to acquire images in the kitchen appliances, or record video information at a set time. Alternatively, the image acquisition device can be turned on or off under the control of the trigger signal, so as to acquire corresponding image or video information. The trigger signal may be sent by the kitchen appliance, or the server may be sent by the kitchen appliance. For example, when it is determined that the kitchen appliance starts cooking, the kitchen appliance or the server sends a start instruction to start the configured image capture device to collect video information; while it is determined that the kitchen appliance is opened, or the set time is reached Under the circumstance, the kitchen appliance or the server sends a shutdown command to turn off the image capture device and stop the video information capture.

因此,在一些实施例中,在该方法应用于厨电设备的情况下,获取厨电设备内被烹饪食材的当前帧图像、以及起始帧图像可包括:在烹饪启动时,控制摄像头启动烹饪视频的录制,获取食材对应的起始帧图像并保存,以及通过摄像头获取食材的当前帧图像。Therefore, in some embodiments, when the method is applied to a kitchen appliance, acquiring the current frame image and the start frame image of the ingredients to be cooked in the kitchen appliance may include: when cooking is started, controlling the camera to start cooking For video recording, obtain and save the starting frame image corresponding to the ingredient, and obtain the current frame image of the ingredient through the camera.

在一些实施例中,与厨电设备通讯的服务器来获取厨电设备内食材的当前帧图像、以及起始帧图像,即在该方法应用于服务器的情况下,接收厨电设备发送的食材的当前帧图像,以及保存的食材的起始帧图像。In some embodiments, the server that communicates with the kitchen appliance acquires the current frame image and the starting frame image of the ingredients in the kitchen appliance, that is, when the method is applied to the server, receives the ingredients sent by the kitchen appliance. The current frame image, and the starting frame image of the saved ingredients.

步骤102:将当前帧图像、以及起始帧图像进行融合处理,得到当前融合处理图像。Step 102: Perform fusion processing on the current frame image and the initial frame image to obtain the current fusion processed image.

本实施例中,需要对当前帧图像、以及起始帧图像进行双图融合处理,将融合为一帧图像,在一些实施例中,可包括:可通过双图交错排列像素行的方式,将当前帧图像、以及起始帧图像进行融合,得到当前融合处理图像。或者,通过双图上下拼接的方式,将当前帧图像、以及起始帧图像进行融合,得到当前融合处理图像。In this embodiment, the current frame image and the starting frame image need to be double-image fusion processing, which will be merged into one frame image. The current frame image and the starting frame image are fused to obtain the current fusion processed image. Alternatively, the current frame image and the starting frame image are fused by splicing the upper and lower images of the two images to obtain the current fusion processed image.

图1-1是本公开实施例提供的一种融合处理图像的示意图。本实施例中,厨电设备可为烤箱,食材可为蛋糕。获取了烤箱烘焙蛋糕过程中蛋糕的当前帧图像,然后,将当前帧图像,与蛋糕放入烤箱启动时对应的起始帧图像,通过双图上下拼接的方式,得到了如图1-1所示的当前融合处理图像。当然,本公开不限于此,其他可将两帧图像融合为一帧图像的方式也可应用于此。FIG. 1-1 is a schematic diagram of a fusion-processed image provided by an embodiment of the present disclosure. In this embodiment, the kitchen appliance may be an oven, and the ingredients may be cakes. The current frame image of the cake in the process of baking the cake in the oven is obtained, and then, the current frame image and the starting frame image corresponding to the cake when the cake is put into the oven are obtained by splicing the upper and lower images of the two images, as shown in Figure 1-1. the current fused image shown. Of course, the present disclosure is not limited to this, and other ways of fusing two frames of images into one frame of image can also be applied to this.

步骤103:将当前融合处理图像,输入已配置的成熟度机器学习训练算法模型中,确定食材对应的成熟度。Step 103: Input the current fusion processing image into the configured maturity machine learning training algorithm model to determine the maturity corresponding to the ingredients.

卷积神经网络(Convolutional Neural Networks,CNN)可通过卷积和池化操作自动学习图像在各个层次上的特征,这符合人们理解图像的常识。人在认知图像时是分层抽象的,首先理解的是颜色和亮度,然后是边缘、角点、直线等局部细节特征,接下来是纹理、几何形状等更复杂的信息和结构,最后形成整个物体的概念。每个卷积层包含多个卷积核,用这些卷积核从左向右、从上往下依次扫描整个图像,得到称为特征图(feature map)的输出数据,即得到图像特征信息。卷积神经网络中,前面的卷积层捕捉图像局部、细节信息,有小的感受野,即输出图像的每个像素只利用输入图像很小的一个范围;而后面的卷积层感受野逐层加大,用于捕获图像更复杂,更抽象的信息。经过多个卷积层的运算,最后得到图像在各个不同尺度的抽象表示。Convolutional Neural Networks (CNN) can automatically learn the features of images at various levels through convolution and pooling operations, which is in line with people's common sense in understanding images. When people perceive images, they are layered and abstract. First, they understand color and brightness, then local details such as edges, corners, and lines, and then more complex information and structures such as textures and geometric shapes. The concept of the whole object. Each convolutional layer contains multiple convolution kernels, and these convolution kernels are used to scan the entire image sequentially from left to right and from top to bottom to obtain output data called feature map, that is, to obtain image feature information. In the convolutional neural network, the previous convolutional layer captures the local and detailed information of the image, and has a small receptive field, that is, each pixel of the output image only uses a small range of the input image; The layers are enlarged to capture the more complex and abstract information of the image. After the operation of multiple convolutional layers, the abstract representation of the image at different scales is finally obtained.

在卷积神经网络中,每一个卷积核所对应的卷积层实际上就是一个系统,一个用于判断图像中某一个特征的系统,当所有的单个卷积层,即单个特征判断系统,都能有效地完成特征判断的任务时,由这些数量众多卷积层所组成的复杂系统CNN就能够完成人类所需要的给予卷积神经网络的复杂任务。In a convolutional neural network, the convolutional layer corresponding to each convolutional kernel is actually a system, a system for judging a certain feature in the image. When all the single convolutional layers, that is, a single feature judgment system, When the task of feature judgment can be effectively completed, the complex system CNN composed of these many convolutional layers can complete the complex tasks given to the convolutional neural network that humans need.

因此,本公开实施例中,可基于卷积神经网络CNN,并通过大量的样本数据,配置了成熟度机器学习训练算法模型。Therefore, in the embodiment of the present disclosure, a mature machine learning training algorithm model can be configured based on the convolutional neural network CNN and through a large amount of sample data.

在一些实施例中,成熟度机器学习训练算法模型的配置过程包括:基于卷积神经网络CNN,提取多个样本图像的特征图像信息,其中,样本图像是已标定成熟度的食材图像与对应的食材起始图像融合处理后生成的;通过回归网络,对每个特征图像信息进行监督训练,生成成熟度机器学习训练算法模型。In some embodiments, the configuration process of the maturity machine learning training algorithm model includes: extracting feature image information of a plurality of sample images based on a convolutional neural network CNN, wherein the sample images are images of food ingredients whose maturity has been calibrated and corresponding It is generated after the fusion processing of the starting image of the food; through the regression network, each feature image information is supervised and trained to generate a maturity machine learning training algorithm model.

食材图像已被标定了成熟度,并且每个食材图像都与食材起始图像对应,这样,同样需要进行双图的融合处理,即可通过双图交错排列像素行的方式,或,双图上下拼接的方式,将已标定成熟度的食材图像与对应的食材起始图像进行融合,得到融合处理图像即样本图像。然后,基于卷积神经网络CNN,可进行图像的特征图像信息提取,并将特征图像信息,通过回归网络,进行监督训练,生成所成熟度机器学习训练算法模型。The image of the food has been calibrated for maturity, and each image of the food corresponds to the starting image of the food. In this way, it is also necessary to perform the fusion processing of the two images, which can be arranged by staggering the pixel rows of the two images, or, the top and bottom of the two images. In the splicing method, the image of the food material whose maturity has been calibrated is fused with the corresponding initial image of the food material to obtain the fusion processed image, that is, the sample image. Then, based on the convolutional neural network CNN, the feature image information of the image can be extracted, and the feature image information can be supervised and trained through the regression network to generate the machine learning training algorithm model of the maturity.

将双图进行融合处理后,基于卷积神经网络CNN,进行融合处理图像的特征图像信息提取时,卷积操作很好地考虑到双图中,像素与像素之间的位置关系,这样判断出的食物成熟度结果,能融合食材图像和起始图像之间的信息,衡量两图之间的差异程度,这样,提取的特征信息利用的是图像之间的差异程度,如果食材图像与起始图像差异越大,那么其成熟度值越高,反之则越低。After the double image is fused, based on the convolutional neural network CNN, when extracting the feature image information of the fusion image, the convolution operation takes into account the positional relationship between the pixels and the pixels in the double image. The result of the food maturity can be integrated with the information between the food image and the starting image to measure the degree of difference between the two images. In this way, the extracted feature information uses the degree of difference between the images. If the food image is different from the starting image The greater the image difference, the higher its maturity value, and vice versa.

并且,以双图融合处理后的融合处理图像的特征图像信息,通过回归网络,进行监督训练时,可有效规避“由于不同食材的形状、颜色和纹理等差异”引起的成熟度回归网络不收敛。其中,不同成熟度的相同食物之间的差异被放大了;同时,相同成熟度的不同食物间的差异被缩小,这样,判断食物成熟度的特征更加明显,这样使网络更容易收敛。In addition, the feature image information of the fusion-processed image after the double-image fusion processing can effectively avoid the maturity regression network caused by "differences in the shape, color and texture of different ingredients" when performing supervised training through the regression network. . Among them, the difference between the same food of different maturity is enlarged; at the same time, the difference between different food of the same maturity is reduced, so that the characteristics of judging the maturity of food are more obvious, which makes the network easier to converge.

在一些实施例中,为使得进行监督训练的回归网络的收敛性更好,可采用公式(1)对应的损失函数:In some embodiments, in order to make the convergence of the regression network for supervised training better, the loss function corresponding to formula (1) can be used:

Figure BDA0002216906000000061
Figure BDA0002216906000000061

其中,ε为收敛系数,Δx为预测值与标定值之间的差值。Among them, ε is the convergence coefficient, and Δx is the difference between the predicted value and the calibration value.

本实施例中,采用回归网络预测食材的成熟度,因此,预测值可为预测成熟度,标定值为标定成熟度。目前,在回归网络的采用损失函数中,并没有收敛系数ε。这样,当Δx的绝对值大于1时可以收敛得更快,一次线性求导可以使网络快速收敛;而Δx的绝对值小于1时,原smoothL1对离群点、异常值不敏感,梯度变化相对更小,训练时不容易跑飞。但与之带来的是收敛过慢,特别是大网络,大样本的训练。因此,本实施例中,可为smoothL1Plus(Δx)引入ε,ε是一个动态的系数,Δx的绝对值小于1阶段,可调节ε,使得训练收敛时既有平均绝对值误差(MAE,也称L1损失)的梯度不致过小的效果,又有均方误差(MSE,也称L2损失)的防止网络发散的效果。In this embodiment, the regression network is used to predict the maturity of the ingredients. Therefore, the predicted value may be the predicted maturity, and the calibration value may be the calibration maturity. At present, there is no convergence coefficient ε in the loss function adopted by the regression network. In this way, when the absolute value of Δx is greater than 1, it can converge faster, and a linear derivation can make the network converge quickly; and when the absolute value of Δx is less than 1, the original smooth L1 is insensitive to outliers and outliers, and the gradient changes. Relatively smaller, it is not easy to run and fly during training. But the result is that the convergence is too slow, especially the training of large networks and large samples. Therefore, in this embodiment, ε can be introduced for smooth L1 Plus (Δx), ε is a dynamic coefficient, the absolute value of Δx is less than 1 stage, and ε can be adjusted, so that there is a mean absolute value error (MAE, Also known as L1 loss), the gradient is not too small, and there is also the effect of mean square error (MSE, also known as L2 loss) to prevent network divergence.

在配置成熟度机器学习训练算法模型的过程,样本图像融合了已标定成熟度的食材图像,即每个食材图像都有对应的标定成熟度,而成熟度标定是一个难题,一般很有经验的烹饪师才能在各形形色色食物中准确判断某食物的熟度,而这又是深度学习的有监督网络训练的前提。在一些实施例中,成熟度的标定过程包括:在食材烹饪启动的情况下,控制摄像头启动烹饪视频的录制,且在食材成熟度到达设定成熟度的情况下,控制摄像头停止烹饪视频的录制;确定录制的烹饪视频的总帧数;根据第一图像对应帧序号,总帧数,以及设定成熟度,确定第一图像对应的成熟度。In the process of configuring the maturity machine learning training algorithm model, the sample images are combined with the images of the ingredients that have been calibrated for maturity, that is, each food image has a corresponding calibration maturity, and maturity calibration is a difficult problem, generally very experienced. Only cooks can accurately judge the doneness of a certain food among all kinds of food, which is the premise of supervised network training of deep learning. In some embodiments, the maturity calibration process includes: when the cooking of the ingredients is started, controlling the camera to start the recording of the cooking video, and when the maturity of the ingredients reaches the set maturity, controlling the camera to stop the recording of the cooking video ; determine the total number of frames of the recorded cooking video; determine the maturity corresponding to the first image according to the frame serial number corresponding to the first image, the total number of frames, and the set maturity.

其中,食材成熟度到达设定成熟度的确定过程可以通过烹饪师观察食材来确定,当然,也不限于此,通过图像比对等等来确定。The process of determining that the maturity of the ingredients reaches the set maturity may be determined by the cook observing the ingredients, of course, it is not limited to this, but is determined through image comparison and the like.

例如:在需标定的食材放入烤箱中,开始录制视频,人为观察食物烘焙烤制的成熟临界状态,当确定到达时,停止录制视频。从而,可获取视频的总帧数nall,而第n帧的熟度的计算公式即可为doneness=n/nall。For example, when the ingredients to be calibrated are placed in the oven, video recording is started, and the critical state of maturity of food baking is observed manually. When it is determined to arrive, the video recording is stopped. Therefore, the total number of frames of the video can be obtained, and the calculation formula of the doneness of the nth frame can be doneness=n/nall.

配置了成熟度机器学习训练算法模型后,即可输入步骤102中融合处理后的当前融合处理图像,这样,基于卷积神经网络CNN,提取当前融合处理图像的当前图像特征信息,通过回归网络,对当前图像特征信息进行监督训练,预测出食材的成熟度,即通过模型进行预测,得到了预测结果,即确定食材对应的成熟度。After the maturity machine learning training algorithm model is configured, the current fusion processing image after fusion processing in step 102 can be input. In this way, based on the convolutional neural network CNN, the current image feature information of the current fusion processing image is extracted, and through the regression network, The current image feature information is supervised and trained to predict the maturity of the ingredients, that is, the model is used to predict, and the prediction result is obtained, that is, the corresponding maturity of the ingredients is determined.

可见,本公开实施例中,获取食材烹饪过程中的当前帧图像和起始帧图像,对双图进行融合处理,并将处理后的融合处理图像输入到已配置的成熟度机器学习训练算法模型中,确定食材的成熟度。这样,通过融合双图的特征的融合处理图像进行模型预测,根据融合处理图像中双图之间差异程度,确定食材成熟度,提高了确定食材成熟度的可靠性,并且,在配置成熟度机器学习训练算法模型时,通过双图融合处理,可有效避免由于不同食材的形状、颜色和纹理等差异引起的成熟度回归网络不收敛的问题。另外,不需要接触食材,即可进行可靠性的确定,减少了破坏食材外观以及污染食材的几率。It can be seen that in the embodiment of the present disclosure, the current frame image and the starting frame image during the cooking process of the ingredients are obtained, the double images are fused, and the processed fusion image is input into the configured maturity machine learning training algorithm model , to determine the maturity of the ingredients. In this way, the model prediction is performed by merging the features of the two images, and the maturity of the ingredients is determined according to the degree of difference between the two images in the fusion image, which improves the reliability of determining the maturity of the ingredients. When learning and training the algorithm model, the dual-image fusion processing can effectively avoid the problem of non-convergence of the maturity regression network caused by the differences in the shape, color and texture of different ingredients. In addition, the reliability can be determined without touching the ingredients, reducing the probability of damaging the appearance of the ingredients and contaminating the ingredients.

本公开实施例中,厨电设备可以本地确定食材成熟度,也可将当前帧图像、以及起始帧图像发送给服务器,从而服务器确定食材成熟度,因此,在一些实施例中,在该方法应用于厨电设备的情况下,确定食材对应的成熟度之后,还可进行成熟度的信息提醒。可包括以下至少一种方式:在厨电设备的显示屏幕上显示成熟度信息,通过语音播报装置播放成熟度信息;将成熟度信息发送给终端进行显示以及提醒。这样,厨电设备本地进行食材成熟度的确定,提高了成熟度确定的速度,也减少对网络资源的占用。In the embodiments of the present disclosure, the kitchen appliance may locally determine the maturity of the ingredients, and may also send the current frame image and the starting frame image to the server, so that the server determines the maturity of the ingredients. Therefore, in some embodiments, in this method In the case of kitchen appliances, after determining the corresponding maturity of the ingredients, the maturity information can also be reminded. It may include at least one of the following ways: displaying maturity information on the display screen of the kitchen appliance, playing the maturity information through a voice broadcasting device; sending the maturity information to a terminal for display and reminder. In this way, the kitchen electrical equipment locally determines the maturity of the ingredients, which improves the speed of determining the maturity and reduces the occupation of network resources.

在一些实施例中,在该方法应用于服务器的情况下,确定食材对应的成熟度之后,将成熟度发送给厨电设备进行成熟度的信息提醒。厨电设备的具体提醒方式可如上述。本实施例中,服务器进行食材成熟度的确定,减少了对厨电设备的资源的占用,节省了内存,提高了厨电设备多控制功能。In some embodiments, when the method is applied to the server, after determining the maturity level corresponding to the ingredients, the maturity level is sent to the kitchen electrical device for information reminder of the maturity level. The specific reminder method of the kitchen appliance can be as above. In this embodiment, the server determines the maturity of the ingredients, which reduces the occupation of resources of the kitchen electrical equipment, saves memory, and improves the multi-control function of the kitchen electrical equipment.

下面将操作流程集合到具体实施例中,举例说明本发明实施例提供的食材成熟度确定过程。The operation procedures are collected into specific embodiments below, and the process of determining the maturity of the food material provided by the embodiments of the present invention is exemplified.

本公开一实施例中,厨电设备可为烤箱,烤箱顶部配置有摄像头。并且,厨电设备通过机器学习,已配置成熟度机器学习训练算法模型。In an embodiment of the present disclosure, the kitchen appliance may be an oven, and a camera is disposed on the top of the oven. In addition, kitchen appliances have been equipped with a mature machine learning training algorithm model through machine learning.

图2是本公开实施例提供的一种食材成熟度确定方法的流程示意图。如图2所示,食材成熟度确定的过程包括:FIG. 2 is a schematic flowchart of a method for determining the maturity of an ingredient provided by an embodiment of the present disclosure. As shown in Figure 2, the process of determining the maturity of ingredients includes:

步骤201:获取烤箱内被烹饪食材的当前帧图像。Step 201: Acquire a current frame image of the ingredients to be cooked in the oven.

步骤202:判断当前帧图像是否为起始帧图像?若是,执行步骤203,否则,执行步骤204。Step 202: Determine whether the current frame image is the starting frame image? If yes, go to step 203; otherwise, go to step 204.

步骤203:将当前帧图像保存为起始帧图像,返回步骤201。Step 203 : save the current frame image as the initial frame image, and return to step 201 .

步骤204:通过双图交错排列像素行的方式,将当前帧图像、以及起始帧图像进行融合,得到当前融合处理图像。Step 204 : fuse the current frame image and the starting frame image by arranging the pixel rows alternately in two images to obtain the current fused processed image.

步骤205:将当前融合处理图像,输入已配置的成熟度机器学习训练算法模型中,确定食材对应的成熟度。Step 205: Input the current fusion processing image into the configured maturity machine learning training algorithm model to determine the maturity corresponding to the ingredients.

步骤206:在烤箱的显示屏上显示成熟度信息。Step 206: Display maturity information on the display screen of the oven.

可见,本实施例中,烤箱可获取食材烤制过程中的当前帧图像和起始帧图像,对双图进行融合处理得到对应的当前融合处理图像,并输入到已配置的成熟度机器学习训练算法模型中,确定食材的成熟度。这样,通过对双图融合后的融合处理图像的特征提取以及模型预测,根据融合处理图像中双图之间的差异程度,确定食材成熟度,提高了确定食材成熟度的可靠性,并且,不需要接触食材,即可进行可靠性的确定,减少了破坏食材外观以及污染食材的几率。并且,烤箱本地进行食材成熟度的确定,提高了成熟度确定的速度,也减少对网络资源的占用。It can be seen that in this embodiment, the oven can obtain the current frame image and the starting frame image during the baking process of the ingredients, perform fusion processing on the two images to obtain the corresponding current fusion processing image, and input them into the configured maturity machine learning training. In the algorithm model, the maturity of the ingredients is determined. In this way, through the feature extraction and model prediction of the fused image after the fusion of the two images, the maturity of the ingredients is determined according to the degree of difference between the two images in the fusion image, which improves the reliability of determining the maturity of the ingredients. Reliability can be determined by contacting the ingredients, reducing the chance of damaging the appearance of the ingredients and contaminating the ingredients. In addition, the oven determines the maturity of the ingredients locally, which improves the speed of determining the maturity and reduces the occupation of network resources.

本公开一实施例中,可由服务器进行食材成熟度的确定。In an embodiment of the present disclosure, the determination of the maturity of the food material may be performed by the server.

图3是本公开实施例提供的一种食材成熟度确定系统的结构示意图。如图3所示,该系统包括:厨电设备100,配置在厨电设备上的图像采集设备200,以及与厨电设备100进行通信的服务器300。FIG. 3 is a schematic structural diagram of a system for determining the maturity of an ingredient provided by an embodiment of the present disclosure. As shown in FIG. 3 , the system includes: a kitchen appliance 100 , an image acquisition device 200 configured on the kitchen appliance, and a server 300 that communicates with the kitchen appliance 100 .

其中,厨电设备100可通过图像采集设备200获取厨电设备内被烹饪食材的当前帧图像、以及起始帧图像,并可发送给服务器300。而服务器300通过机器学,配置了成熟度机器学习训练算法模型,从而,服务器300可基于成熟度机器学习训练算法模型,对当前帧图像、以及起始帧图像融合后的当前融合处理图像进行特征提取以及预测,得到食材的成熟度,并可将成熟度发送给厨电设备100进行提示处理。Wherein, the kitchen appliance 100 can acquire the current frame image and the starting frame image of the ingredients to be cooked in the kitchen appliance through the image acquisition device 200 , and can send them to the server 300 . The server 300 is configured with a maturity machine learning training algorithm model through machine learning, so that the server 300 can characterize the current frame image and the current fusion processed image after fusion of the initial frame image based on the maturity machine learning training algorithm model. After extraction and prediction, the maturity of the ingredients can be obtained, and the maturity can be sent to the kitchen appliance 100 for prompt processing.

图4是本公开实施例提供的一种食材成熟度确定方法的流程示意图。食材成熟度确定系统可如图3,如图4所示,食材成熟度确定的过程包括:FIG. 4 is a schematic flowchart of a method for determining the maturity of an ingredient provided by an embodiment of the present disclosure. The food material maturity determination system can be shown in Figure 3. As shown in Figure 4, the process of food maturity determination includes:

步骤401:厨电设备通过图像采集设备获取厨电设备内被烹饪食材的当前帧图像。Step 401 : the kitchen appliance acquires a current frame image of the food to be cooked in the kitchen appliance through the image acquisition device.

步骤402:厨电设备判断当前帧图像是否为起始帧图像?若是,执行步骤403,否则,执行步骤404。Step 402: The kitchen appliance determines whether the current frame image is the starting frame image? If yes, go to step 403; otherwise, go to step 404.

步骤403:厨电设备将当前帧图像保存为起始帧图像,返回步骤401。Step 403 : the kitchen appliance saves the current frame image as the initial frame image, and returns to step 401 .

步骤404:厨电设备将当前帧图像、以及起始帧图像发送给服务器。Step 404: The kitchen appliance sends the current frame image and the starting frame image to the server.

步骤405:服务器通过双图上下拼接的方式,将当前帧图像、以及起始帧图像进行融合,得到当前融合处理图像。Step 405 : The server fuses the current frame image and the starting frame image by splicing two images up and down to obtain the current fusion processed image.

步骤406:服务器将当前融合处理图像,输入已配置的成熟度机器学习训练算法模型中,确定食材对应的成熟度。Step 406: The server inputs the current fusion processing image into the configured maturity machine learning training algorithm model to determine the maturity corresponding to the ingredients.

步骤407:服务器将成熟度发送给厨电设备。Step 407: The server sends the maturity level to the kitchen appliance.

步骤408:厨电设备在显示屏上显示成熟度信息,以及在成熟度大于设定值的情况下,进行语音播报。Step 408: The kitchen appliance displays the maturity information on the display screen, and performs a voice broadcast when the maturity is greater than the set value.

可见,本实施例中,服务器可通过厨电设备获取食材烹饪过程中的当前帧图像和起始帧图像,对双图进行融合处理得到对应的当前融合处理图像,并输入到已配置的成熟度机器学习训练算法模型中,确定食材的成熟度。这样,通过对双图融合后的融合处理图像的特征提取以及模型预测,根据融合处理图像中双图之间的差异程度,确定食材成熟度,提高了确定食材成熟度的可靠性,并且,不需要接触食材,即可进行可靠性的确定,减少了破坏食材外观以及污染食材的几率。另外,服务器进行食材成熟度的确定,减少了对厨电设备的资源的占用,提高了厨电设备多控制功能。It can be seen that, in this embodiment, the server can obtain the current frame image and the starting frame image of the ingredients during the cooking process through the kitchen appliance, perform fusion processing on the two images to obtain the corresponding current fusion processing image, and input them into the configured maturity level. In the machine learning training algorithm model, the maturity of the ingredients is determined. In this way, through the feature extraction and model prediction of the fused image after the fusion of the two images, the maturity of the ingredients is determined according to the degree of difference between the two images in the fusion image, which improves the reliability of determining the maturity of the ingredients. Reliability can be determined by contacting the ingredients, reducing the chance of damaging the appearance of the ingredients and contaminating the ingredients. In addition, the server determines the maturity of the ingredients, which reduces the resource occupation of the kitchen electrical equipment and improves the multi-control function of the kitchen electrical equipment.

根据上述食材成熟度确定的过程,可构建一种食材成熟度确定的装置。According to the above process of determining the maturity of the food, a device for determining the maturity of the food can be constructed.

图5是本公开实施例提供的一种食材成熟度确定装置的结构示意图。如图5所示,食材成熟度确定装置包括:信息获取模块510、图像融合模块520和预测确定模块530。FIG. 5 is a schematic structural diagram of an apparatus for determining the maturity of an ingredient provided by an embodiment of the present disclosure. As shown in FIG. 5 , the apparatus for determining the maturity of the food material includes: an information acquisition module 510 , an image fusion module 520 and a prediction determination module 530 .

信息获取模块510,被配置为获取厨电设备内被烹饪食材的当前帧图像、以及起始帧图像,其中,起始帧图像是烹饪启动时,食材的图像。The information acquisition module 510 is configured to acquire the current frame image and the starting frame image of the ingredients to be cooked in the kitchen appliance, wherein the starting frame image is the image of the ingredients when cooking is started.

图像融合模块520,被配置为将当前帧图像、以及起始帧图像进行融合处理,得到当前融合处理图像。The image fusion module 520 is configured to perform fusion processing on the current frame image and the initial frame image to obtain the current fusion processed image.

预测确定模块530,被配置为将当前融合处理图像,输入已配置的成熟度机器学习训练算法模型中,确定食材对应的成熟度。The prediction and determination module 530 is configured to input the current fusion processing image into the configured maturity machine learning training algorithm model to determine the corresponding maturity of the ingredients.

在一些实施例中,还包括:模型配置模块,被配置为基于卷积神经网络CNN,提取多个样本图像的特征图像信息,其中,样本图像是已标定成熟度的食材图像与对应的食材起始图像融合处理后得到的;以及,通过回归网络,对每个特征图像信息进行监督训练,生成成熟度机器学习训练算法模型;回归网络中的损失函数包括:In some embodiments, the method further includes: a model configuration module configured to extract feature image information of a plurality of sample images based on a convolutional neural network CNN, wherein the sample images are images of ingredients whose maturity has been calibrated and corresponding ingredients. obtained after the initial image fusion processing; and, through the regression network, supervised training is performed on each feature image information to generate a mature machine learning training algorithm model; the loss function in the regression network includes:

Figure BDA0002216906000000101
Figure BDA0002216906000000101

其中,ε为收敛系数,Δx为预测值与标定值之间的差值。Among them, ε is the convergence coefficient, and Δx is the difference between the predicted value and the calibration value.

在一些实施例中,还包括:标定模块,被配置为在食材烹饪启动的情况下,控制摄像头启动烹饪视频的录制,且在食材成熟度到达设定成熟度的情况下,控制摄像头停止烹饪视频的录制;确定录制的烹饪视频的总帧数;根据第一图像对应帧序号,总帧数,以及设定成熟度,确定第一图像对应的成熟度。In some embodiments, it further includes: a calibration module, configured to control the camera to start the recording of the cooking video when the cooking of the ingredients is started, and to control the camera to stop the cooking video when the maturity of the ingredients reaches the set maturity. determine the total number of frames of the recorded cooking video; determine the maturity corresponding to the first image according to the frame serial number corresponding to the first image, the total number of frames, and the set maturity.

下面举例说明本公开实施例提供的食材成熟度确定装置。The following is an example to illustrate the device for determining the maturity of the food material provided by the embodiments of the present disclosure.

图6是本公开实施例提供的一种食材成熟度确定装置的结构示意图。如图6所示,食材成熟度确定装置可应用于厨电设备中,包括:信息获取模块510、图像融合模块520、预测确定模块530,还包括模型配置模块540,标定模块550和处理模块560。FIG. 6 is a schematic structural diagram of an apparatus for determining the maturity of an ingredient provided by an embodiment of the present disclosure. As shown in FIG. 6 , the device for determining the maturity of food materials can be applied to kitchen electrical equipment, including: an information acquisition module 510 , an image fusion module 520 , a prediction and determination module 530 , and a model configuration module 540 , a calibration module 550 and a processing module 560 .

其中,标定模块550可对样本食材的成熟度进行标定,即在食材烹饪启动的情况下,控制摄像头启动烹饪视频的录制,且在食材成熟度到达设定成熟度的情况下,控制摄像头停止烹饪视频的录制;确定录制的烹饪视频的总帧数;根据第一图像对应帧序号,总帧数,以及设定成熟度,确定第一图像对应的成熟度。The calibration module 550 can calibrate the maturity of the sample ingredients, that is, when the cooking of the ingredients is started, the camera is controlled to start recording the cooking video, and when the maturity of the ingredients reaches the set maturity, the camera is controlled to stop cooking Video recording; determine the total number of frames of the recorded cooking video; determine the maturity corresponding to the first image according to the frame number corresponding to the first image, the total number of frames, and the set maturity.

这样,模型配置模块540可基于卷积神经网络CNN,提取多个样本图像的特征图像信息,其中,样本图像是已标定成熟度的食材图像与对应的食材起始图像融合处理后得到的;以及,通过回归网络,对每个特征图像信息进行监督训练,生成成熟度机器学习训练算法模型;回归网络中的损失函数包括:In this way, the model configuration module 540 can extract feature image information of a plurality of sample images based on the convolutional neural network CNN, wherein the sample images are obtained after fusion processing of the image of the food material whose maturity has been calibrated and the corresponding initial image of the food material; and , through the regression network, supervised training is performed on each feature image information to generate a mature machine learning training algorithm model; the loss function in the regression network includes:

Figure BDA0002216906000000111
Figure BDA0002216906000000111

其中,ε为收敛系数,Δx为预测值与标定值之间的差值。Among them, ε is the convergence coefficient, and Δx is the difference between the predicted value and the calibration value.

在烹饪启动时,摄像头启动烹饪视频的录制,信息获取模块510可获取食材对应的起始帧图像并保存,以及通过摄像头,信息获取模块510可获取食材的当前帧图像。When cooking starts, the camera starts recording the cooking video, the information acquisition module 510 can acquire and save the starting frame image corresponding to the ingredient, and through the camera, the information acquisition module 510 can acquire the current frame image of the ingredient.

这样,通过图像融合模块520可将当前帧图像、以及起始帧图像进行融合处理,得到当前融合处理图像。In this way, the image fusion module 520 can perform fusion processing on the current frame image and the initial frame image to obtain the current fusion processed image.

而预测确定模块530可将当前融合处理图像,输入模型配置模块540已配置的成熟度机器学习训练算法模型中,确定食材对应的成熟度。而处理模块560可在显示界面上显示成熟度信息。The prediction and determination module 530 can input the current fusion processing image into the maturity machine learning training algorithm model configured by the model configuration module 540 to determine the corresponding maturity of the ingredients. The processing module 560 can display maturity information on the display interface.

可见,本实施例中,应用于厨电设备中食材成熟度确定装置可获取食材烹饪过程中的当前帧图像和起始帧图像,对双图进行融合处理得到对应的当前融合处理图像,并输入到已配置的成熟度机器学习训练算法模型中,确定食材的成熟度。这样,通过对双图融合后的融合处理图像的特征提取以及模型预测,根据融合处理图像中双图之间的差异程度,确定食材成熟度,提高了确定食材成熟度的可靠性,并且,不需要接触食材,即可进行可靠性的确定,减少了破坏食材外观以及污染食材的几率。并且,厨电设备本地进行食材成熟度的确定,提高了成熟度确定的速度,也减少对网络资源的占用。It can be seen that in this embodiment, the device for determining the maturity of the ingredients in the kitchen electrical equipment can obtain the current frame image and the starting frame image during the cooking process of the ingredients, perform fusion processing on the two images to obtain the corresponding current fusion processing image, and input Go to the configured maturity machine learning training algorithm model to determine the maturity of the ingredients. In this way, through the feature extraction and model prediction of the fused image after the fusion of the two images, the maturity of the ingredients is determined according to the degree of difference between the two images in the fusion image, which improves the reliability of determining the maturity of the ingredients. Reliability can be determined by contacting the ingredients, reducing the chance of damaging the appearance of the ingredients and contaminating the ingredients. In addition, the kitchen electrical equipment determines the maturity of the ingredients locally, which improves the speed of determining the maturity and reduces the occupation of network resources.

图7是本公开实施例提供的一种食材成熟度确定装置的结构示意图。如图7所示,食材成熟度确定装置可应用于服务器中,包括:信息获取模块510、图像融合模块520、预测确定模块530,还包括模型配置模块540,标定模块550和发送模块570。FIG. 7 is a schematic structural diagram of an apparatus for determining the maturity of an ingredient provided by an embodiment of the present disclosure. As shown in FIG. 7 , the food material maturity determination device can be applied to the server, including: an information acquisition module 510 , an image fusion module 520 , a prediction determination module 530 , a model configuration module 540 , a calibration module 550 and a transmission module 570 .

其中,标定模块550可对样本食材的成熟度进行标定,即在食材烹饪启动的情况下,控制摄像头启动烹饪视频的录制,且在食材成熟度到达设定成熟度的情况下,控制摄像头停止烹饪视频的录制;确定录制的烹饪视频的总帧数;根据第一图像对应帧序号,总帧数,以及设定成熟度,确定第一图像对应的成熟度。The calibration module 550 can calibrate the maturity of the sample ingredients, that is, when the cooking of the ingredients is started, the camera is controlled to start recording the cooking video, and when the maturity of the ingredients reaches the set maturity, the camera is controlled to stop cooking Video recording; determine the total number of frames of the recorded cooking video; determine the maturity corresponding to the first image according to the frame number corresponding to the first image, the total number of frames, and the set maturity.

这样,模型配置模块540可基于卷积神经网络CNN,提取多个样本图像的特征图像信息,其中,样本图像是已标定成熟度的食材图像与对应的食材起始图像融合处理后得到的;以及,通过回归网络,对每个特征图像信息进行监督训练,生成成熟度机器学习训练算法模型;回归网络中的损失函数包括:In this way, the model configuration module 540 can extract feature image information of a plurality of sample images based on the convolutional neural network CNN, wherein the sample images are obtained after fusion processing of the image of the food material whose maturity has been calibrated and the corresponding initial image of the food material; and , through the regression network, supervised training is performed on each feature image information to generate a mature machine learning training algorithm model; the loss function in the regression network includes:

Figure BDA0002216906000000121
Figure BDA0002216906000000121

其中,ε为收敛系数,Δx为预测值与标定值之间的差值。Among them, ε is the convergence coefficient, and Δx is the difference between the predicted value and the calibration value.

在烹饪启动时,摄像头启动烹饪视频的录制,厨电设备可获取食材对应的起始帧图像以及当前帧图像,这样,信息获取模块510可接收厨电设备发送的食材的当前帧图像,以及保存的食材的起始帧图像。When cooking is started, the camera starts recording the cooking video, and the kitchen appliance can acquire the starting frame image and the current frame image corresponding to the ingredient. In this way, the information acquisition module 510 can receive the current frame image of the ingredient sent by the kitchen appliance, and save it. The starting frame image of the ingredients.

同样,通过图像融合模块520可将当前帧图像、以及起始帧图像进行融合处理,得到当前融合处理图像。Similarly, the current frame image and the starting frame image can be fused through the image fusion module 520 to obtain the current fusion processed image.

而预测确定模块530可将当前融合处理图像,输入模型配置模块540已配置的成熟度机器学习训练算法模型中,确定食材对应的成熟度。而发送模块570可将成熟度发送给厨电设备进行成熟度的信息提醒。The prediction and determination module 530 can input the current fusion processing image into the maturity machine learning training algorithm model configured by the model configuration module 540 to determine the corresponding maturity of the ingredients. The sending module 570 can send the maturity to the kitchen electrical equipment to remind the maturity of the information.

可见,本实施例中,应用于服务器中食材成熟确定装置可通过厨电设备获取食材烹饪过程中的当前帧图像和起始帧图像,对双图进行融合处理得到对应的当前融合处理图像,并输入到已配置的成熟度机器学习训练算法模型中,确定食材的成熟度。这样,通过对双图融合后的融合处理图像的特征提取以及模型预测,根据融合处理图像中双图之间的差异程度,确定食材成熟度,提高了确定食材成熟度的可靠性,并且,不需要接触食材,即可进行可靠性的确定,减少了破坏食材外观以及污染食材的几率。另外,服务器进行食材成熟度的确定,减少了对厨电设备的资源的占用,提高了厨电设备多控制功能。It can be seen that in this embodiment, the device for determining the maturity of the ingredients in the server can obtain the current frame image and the starting frame image during the cooking process of the ingredients through the kitchen electrical equipment, and perform fusion processing on the two images to obtain the corresponding current fusion processing image, and Input into the configured maturity machine learning training algorithm model to determine the maturity of the ingredients. In this way, through the feature extraction and model prediction of the fused image after the fusion of the two images, the maturity of the ingredients is determined according to the degree of difference between the two images in the fusion image, which improves the reliability of determining the maturity of the ingredients. Reliability can be determined by contacting the ingredients, reducing the chance of damaging the appearance of the ingredients and contaminating the ingredients. In addition, the server determines the maturity of the ingredients, which reduces the resource occupation of the kitchen electrical equipment and improves the multi-control function of the kitchen electrical equipment.

本公开实施例提供了一种食材成熟度确定的装置,其结构如图8所示,包括:An embodiment of the present disclosure provides a device for determining the maturity of an ingredient, the structure of which is shown in FIG. 8 and includes:

处理器(processor)1000和存储器(memory)1001,还可以包括通信接口(Communication Interface)1002和总线1003。其中,处理器1000、通信接口1002、存储器1001可以通过总线1003完成相互间的通信。通信接口102可以用于信息传输。处理器1000可以调用存储器1001中的逻辑指令,以执行上述实施例的食材成熟度确定的方法。A processor (processor) 1000 and a memory (memory) 1001 may also include a communication interface (Communication Interface) 1002 and a bus 1003 . The processor 1000 , the communication interface 1002 , and the memory 1001 can communicate with each other through the bus 1003 . The communication interface 102 may be used for information transfer. The processor 1000 may invoke the logic instructions in the memory 1001 to execute the method for determining the maturity of the food material in the above-mentioned embodiment.

此外,上述的存储器1001中的逻辑指令可以通过软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。In addition, the above-mentioned logic instructions in the memory 1001 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

存储器1001作为一种计算机可读存储介质,可用于存储软件程序、计算机可执行程序,如本公开实施例中的方法对应的程序指令/模块。处理器1000通过运行存储在存储器1001中的程序指令/模块,从而执行功能应用以及数据处理,即实现上述方法实施例中的食材成熟度确定的方法。As a computer-readable storage medium, the memory 1001 can be used to store software programs and computer-executable programs, such as program instructions/modules corresponding to the methods in the embodiments of the present disclosure. The processor 1000 executes the function application and data processing by running the program instructions/modules stored in the memory 1001, that is, the method for determining the maturity of the food material in the above method embodiments.

存储器1001可包括存储程序区和存储数据区,其中,存储程序区可存储操作系统、至少一个功能所需的应用程序;存储数据区可存储根据终端设备的使用所创建的数据等。此外,存储器101可以包括高速随机存取存储器,还可以包括非易失性存储器。The memory 1001 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal device, and the like. In addition, the memory 101 may include high-speed random access memory, and may also include non-volatile memory.

本公开实施例提供了一种厨电设备,包含上述的食材成熟度确定装置。An embodiment of the present disclosure provides a kitchen appliance, including the above-mentioned device for determining the maturity of an ingredient.

本公开实施例提供了一种服务器,包含上述的食材成熟度确定装置。An embodiment of the present disclosure provides a server, including the above-mentioned device for determining the maturity of an ingredient.

本公开实施例提供了一种计算机可读存储介质,存储有计算机可执行指令,所述计算机可执行指令设置为执行上述食材成熟度确定方法。An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, where the computer-executable instructions are configured to execute the foregoing method for determining the maturity of an ingredient.

本公开实施例提供了一种计算机程序产品,所述计算机程序产品包括存储在计算机可读存储介质上的计算机程序,所述计算机程序包括程序指令,当所述程序指令被计算机执行时,使所述计算机执行上述食材成熟度确定方法。Embodiments of the present disclosure provide a computer program product, where the computer program product includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions that, when executed by a computer, cause all The computer executes the above-mentioned method for determining the maturity of the food material.

上述的计算机可读存储介质可以是暂态计算机可读存储介质,也可以是非暂态计算机可读存储介质。The above-mentioned computer-readable storage medium may be a transient computer-readable storage medium, and may also be a non-transitory computer-readable storage medium.

本公开实施例的技术方案可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括一个或多个指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本公开实施例所述方法的全部或部分步骤。而前述的存储介质可以是非暂态存储介质,包括:U盘、移动硬盘、只读存储器(ROM,Read-Only Memory)、随机存取存储器(RAM,Random Access Memory)、磁碟或者光盘等多种可以存储程序代码的介质,也可以是暂态存储介质。The technical solutions of the embodiments of the present disclosure may be embodied in the form of software products, and the computer software products are stored in a storage medium and include one or more instructions to enable a computer device (which may be a personal computer, a server, or a network equipment, etc.) to execute all or part of the steps of the methods described in the embodiments of the present disclosure. The aforementioned storage medium may be a non-transitory storage medium, including: U disk, removable hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk, etc. A medium that can store program codes, and can also be a transient storage medium.

以上描述和附图充分地示出了本公开的实施例,以使本领域的技术人员能够实践它们。其他实施例可以包括结构的、逻辑的、电气的、过程的以及其他的改变。实施例仅代表可能的变化。除非明确要求,否则单独的部件和功能是可选的,并且操作的顺序可以变化。一些实施例的部分和特征可以被包括在或替换其他实施例的部分和特征。本公开实施例的范围包括权利要求书的整个范围,以及权利要求书的所有可获得的等同物。当用于本申请中时,虽然术语“第一”、“第二”等可能会在本申请中使用以描述各元件,但这些元件不应受到这些术语的限制。这些术语仅用于将一个元件与另一个元件区别开。比如,在不改变描述的含义的情况下,第一元件可以叫做第二元件,并且同样第,第二元件可以叫做第一元件,只要所有出现的“第一元件”一致重命名并且所有出现的“第二元件”一致重命名即可。第一元件和第二元件都是元件,但可以不是相同的元件。而且,本申请中使用的用词仅用于描述实施例并且不用于限制权利要求。如在实施例以及权利要求的描述中使用的,除非上下文清楚地表明,否则单数形式的“一个”(a)、“一个”(an)和“所述”(the)旨在同样包括复数形式。类似地,如在本申请中所使用的术语“和/或”是指包含一个或一个以上相关联的列出的任何以及所有可能的组合。另外,当用于本申请中时,术语“包括”(comprise)及其变型“包括”(comprises)和/或包括(comprising)等指陈述的特征、整体、步骤、操作、元素,和/或组件的存在,但不排除一个或一个以上其它特征、整体、步骤、操作、元素、组件和/或这些的分组的存在或添加。在没有更多限制的情况下,由语句“包括一个…”限定的要素,并不排除在包括所述要素的过程、方法或者设备中还存在另外的相同要素。本文中,每个实施例重点说明的可以是与其他实施例的不同之处,各个实施例之间相同相似部分可以互相参见。对于实施例公开的方法、产品等而言,如果其与实施例公开的方法部分相对应,那么相关之处可以参见方法部分的描述。The foregoing description and drawings sufficiently illustrate the embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, process, and other changes. The examples are only representative of possible variations. Unless expressly required, individual components and functions are optional and the order of operations may vary. Portions and features of some embodiments may be included in or substituted for those of other embodiments. The scope of the disclosed embodiments includes the full scope of the claims, along with all available equivalents of the claims. When used in this application, although the terms "first," "second," etc. may be used in this application to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without changing the meaning of the description, a first element could be termed a second element, and similarly, a second element could be termed a first element, so long as all occurrences of "the first element" were consistently renamed and all occurrences of "the first element" were named consistently The "second element" can be renamed consistently. The first element and the second element are both elements, but may not be the same element. Also, the terms used in this application are used to describe the embodiments only and not to limit the claims. As used in the description of the embodiments and the claims, the singular forms "a" (a), "an" (an) and "the" (the) are intended to include the plural forms as well, unless the context clearly dictates otherwise. . Similarly, the term "and/or" as used in this application is meant to include any and all possible combinations of one or more of the associated listings. Additionally, as used in this application, the term "comprise" and its variations "comprises" and/or including and/or the like refer to stated features, integers, steps, operations, elements, and/or The presence of a component does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and/or groupings of these. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, or device that includes the element. Herein, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method sections disclosed in the embodiments, reference may be made to the descriptions of the method sections for relevant parts.

本领域技术人员可以意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行,可以取决于技术方案的特定应用和设计约束条件。所述技术人员可以对每个特定的应用来使用不同方法以实现所描述的功能,但是这种实现不应认为超出本公开实施例的范围。所述技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的系统、装置和单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。Those skilled in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. Skilled artisans may use different methods for implementing the described functionality for each particular application, but such implementations should not be considered beyond the scope of the disclosed embodiments. The skilled person can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described systems, devices and units can refer to the corresponding processes in the foregoing method embodiments, and details are not repeated here.

本文所披露的实施例中,所揭露的方法、产品(包括但不限于装置、设备等),可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述单元的划分,可以仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另外,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例。另外,在本公开实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。In the embodiments disclosed herein, the disclosed methods and products (including but not limited to apparatuses, devices, etc.) may be implemented in other ways. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units may only be a logical function division. In actual implementation, there may be other division methods, for example, multiple units or components may be combined Either it can be integrated into another system, or some features can be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. This embodiment may be implemented by selecting some or all of the units according to actual needs. In addition, each functional unit in the embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

附图中的流程图和框图显示了根据本公开实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上,流程图或框图中的每个方框可以代表一个模块、程序段或代码的一部分,所述模块、程序段或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中,方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如,两个连续的方框实际上可以基本并行地执行,它们有时也可以按相反的顺序执行,这可以依所涉及的功能而定。在附图中的流程图和框图所对应的描述中,不同的方框所对应的操作或步骤也可以以不同于描述中所披露的顺序发生,有时不同的操作或步骤之间不存在特定的顺序。例如,两个连续的操作或步骤实际上可以基本并行地执行,它们有时也可以按相反的顺序执行,这可以依所涉及的功能而定。框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合,可以用执行规定的功能或动作的专用的基于硬件的系统来实现,或者可以用专用硬件与计算机指令的组合来实现。The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more functions for implementing the specified logical function(s) executable instructions. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, operations or steps corresponding to different blocks may also occur in different sequences than those disclosed in the description, and sometimes there is no specific relationship between different operations or steps. order. For example, two consecutive operations or steps may, in fact, be performed substantially concurrently, or they may sometimes be performed in the reverse order, depending upon the functionality involved. Each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in special purpose hardware-based systems that perform the specified functions or actions, or special purpose hardware implemented in combination with computer instructions.

Claims (10)

1. A method for determining food material maturity is characterized by comprising the following steps:
acquiring a current frame image and a starting frame image of a cooked food material in kitchen electrical equipment, wherein the starting frame image is an image of the food material when cooking is started;
performing fusion processing on the current frame image and the initial frame image to obtain a current fusion processing image;
and inputting the current fusion processing image into a configured maturity machine learning training algorithm model, and determining the maturity corresponding to the food material.
2. The method of claim 1, wherein before the obtaining the current frame image and the starting frame image of the food material cooked in the kitchen electrical appliance, further comprising:
extracting characteristic image information of a plurality of sample images based on a Convolutional Neural Network (CNN), wherein the sample images are obtained by fusing food material images with the ripeness calibrated and corresponding food material initial images;
and performing supervision training on each feature image information through a regression network to generate the maturity machine learning training algorithm model.
3. The method of claim 2, wherein the loss function in the regression network comprises:
Figure FDA0002216905990000011
wherein epsilon is a convergence coefficient, and deltax is a difference value between a predicted value and a calibrated value.
4. The method of claim 2, wherein the maturity calibration process comprises:
under the condition that the cooking of food materials is started, controlling a camera to start recording of cooking videos, and under the condition that the maturity of the food materials reaches the set maturity, controlling the camera to stop recording of the cooking videos;
determining the total frame number of the recorded cooking videos;
and determining the maturity corresponding to the first image according to the frame number corresponding to the first image, the total frame number and the set maturity.
5. The method of claim 1, wherein the obtaining the current frame image and the starting frame image of the cooked food material in the kitchen appliance comprises:
when the method is applied to kitchen electrical equipment, when cooking is started, a camera is controlled to start recording of a cooking video, a starting frame image corresponding to the food material is obtained and stored, and a current frame image of the food material is obtained through the camera;
in the case that the method is applied to a server, receiving a current frame image of the food material sent by the kitchen electrical equipment and a stored starting frame image of the food material.
6. The method of claim 1 or 5, wherein after determining the maturity corresponding to the food material, further comprising:
when the method is applied to kitchen electrical equipment, information reminding of maturity is carried out;
and under the condition that the method is applied to a server, sending the maturity to the kitchen electric equipment for information reminding of the maturity.
7. An apparatus for determining the ripeness of a food material, comprising:
the cooking system comprises an information acquisition module, a processing module and a display module, wherein the information acquisition module is configured to acquire a current frame image and a starting frame image of a cooked food material in the kitchen electrical equipment, and the starting frame image is an image of the food material when cooking is started;
the image fusion module is configured to perform fusion processing on the current frame image and the initial frame image to obtain a current fusion processing image;
and the prediction determining module is configured to input the current fusion processing image into a configured maturity machine learning training algorithm model, and determine the maturity corresponding to the food material.
8. The apparatus of claim 7, further comprising:
the model configuration module is configured to extract feature image information of a plurality of sample images based on a convolutional neural network CNN, wherein the sample images are obtained by performing fusion processing on the food material images with the calibrated maturity and corresponding food material initial images; performing supervision training on each feature image information through a regression network to generate the maturity machine learning training algorithm model; the loss function in the regression network includes:
wherein epsilon is a convergence coefficient, and deltax is a difference value between a predicted value and a calibration value;
the calibration module is configured to control the camera to start recording of the cooking video when the cooking of the food materials is started, and control the camera to stop recording of the cooking video when the maturity of the food materials reaches the set maturity; determining the total frame number of the recorded cooking videos; and determining the maturity corresponding to the first image according to the frame number corresponding to the first image, the total frame number and the set maturity.
9. A kitchen appliance, characterized in that it comprises a device according to any one of claims 7 to 8.
10. A server, characterized in that it comprises a device according to any one of claims 7 to 8.
CN201910918784.9A 2019-09-26 2019-09-26 Method, device, kitchen appliance and server for determining food maturity Active CN110826574B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910918784.9A CN110826574B (en) 2019-09-26 2019-09-26 Method, device, kitchen appliance and server for determining food maturity

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910918784.9A CN110826574B (en) 2019-09-26 2019-09-26 Method, device, kitchen appliance and server for determining food maturity

Publications (2)

Publication Number Publication Date
CN110826574A true CN110826574A (en) 2020-02-21
CN110826574B CN110826574B (en) 2025-06-24

Family

ID=69548599

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910918784.9A Active CN110826574B (en) 2019-09-26 2019-09-26 Method, device, kitchen appliance and server for determining food maturity

Country Status (1)

Country Link
CN (1) CN110826574B (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112741508A (en) * 2021-01-27 2021-05-04 九阳股份有限公司 Control method of cooking equipment and cooking equipment
CN112926377A (en) * 2021-01-02 2021-06-08 黑龙江北草堂中药材有限责任公司 CNN-based traditional Chinese medicine material stir-frying degree determination method
WO2021110066A1 (en) * 2019-12-06 2021-06-10 广东美的白色家电技术创新中心有限公司 Food maturity level identification method and device, and computer storage medium
CN113283447A (en) * 2021-05-28 2021-08-20 青岛海尔科技有限公司 Food baking method and device, storage medium and electronic device
CN113436159A (en) * 2021-06-21 2021-09-24 青岛海尔科技有限公司 Method and device for detecting maturity of food material, storage medium and electronic device
CN113793314A (en) * 2021-09-13 2021-12-14 河南丹圣源农业开发有限公司 Pomegranate maturity identification equipment and use method
CN116594367A (en) * 2023-07-19 2023-08-15 烟台金潮果蔬食品有限公司 Cooking degree control system of sweet potato juice spiral precooking machine
CN117237939A (en) * 2023-11-16 2023-12-15 沈阳东方和利厨业有限公司 Image data-based detection method and device for food maturity of young cooker

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190110638A1 (en) * 2017-10-16 2019-04-18 Midea Group Co., Ltd Machine learning control of cooking appliances
CN109998360A (en) * 2019-04-11 2019-07-12 上海长膳智能科技有限公司 A kind of method and apparatus for automatic cooking food
CN110123149A (en) * 2019-05-10 2019-08-16 九阳股份有限公司 A kind of cooking control method and cooking equipment of cooking equipment
CN110275456A (en) * 2018-03-14 2019-09-24 佛山市顺德区美的电热电器制造有限公司 Cooking control method, system and computer readable storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190110638A1 (en) * 2017-10-16 2019-04-18 Midea Group Co., Ltd Machine learning control of cooking appliances
CN110275456A (en) * 2018-03-14 2019-09-24 佛山市顺德区美的电热电器制造有限公司 Cooking control method, system and computer readable storage medium
CN109998360A (en) * 2019-04-11 2019-07-12 上海长膳智能科技有限公司 A kind of method and apparatus for automatic cooking food
CN110123149A (en) * 2019-05-10 2019-08-16 九阳股份有限公司 A kind of cooking control method and cooking equipment of cooking equipment

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BÜHLMANN P, ET AL.: "Boosting with the L 2 loss: regression and classification", 《JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION》, 31 December 2003 (2003-12-31), pages 324 - 339 *
SCHMIDT M, ET AL.: "Fast optimization methods for l1 regularization: A comparative study and two new approaches", 《MACHINE LEARNING》, 31 December 2007 (2007-12-31), pages 286 - 297 *

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220262143A1 (en) * 2019-12-06 2022-08-18 Guangdong Midea White Home Appliance Technology Innovation Center Co., Ltd. Method of Identifying Level of Doneness of Food, Device, and Computer Storage Medium
WO2021110066A1 (en) * 2019-12-06 2021-06-10 广东美的白色家电技术创新中心有限公司 Food maturity level identification method and device, and computer storage medium
US12094228B2 (en) * 2019-12-06 2024-09-17 Guangdong Midea White Home Appliance Technology Innovation Center Co., Ltd Method of identifying level of doneness of food, device, and computer storage medium
CN112926377A (en) * 2021-01-02 2021-06-08 黑龙江北草堂中药材有限责任公司 CNN-based traditional Chinese medicine material stir-frying degree determination method
CN112741508A (en) * 2021-01-27 2021-05-04 九阳股份有限公司 Control method of cooking equipment and cooking equipment
CN113283447A (en) * 2021-05-28 2021-08-20 青岛海尔科技有限公司 Food baking method and device, storage medium and electronic device
CN113283447B (en) * 2021-05-28 2023-12-19 青岛海尔科技有限公司 Food baking methods and devices, storage media and electronic devices
CN113436159A (en) * 2021-06-21 2021-09-24 青岛海尔科技有限公司 Method and device for detecting maturity of food material, storage medium and electronic device
CN113793314A (en) * 2021-09-13 2021-12-14 河南丹圣源农业开发有限公司 Pomegranate maturity identification equipment and use method
CN116594367A (en) * 2023-07-19 2023-08-15 烟台金潮果蔬食品有限公司 Cooking degree control system of sweet potato juice spiral precooking machine
CN116594367B (en) * 2023-07-19 2023-09-19 烟台金潮果蔬食品有限公司 Cooking degree control system of sweet potato juice spiral precooking machine
CN117237939A (en) * 2023-11-16 2023-12-15 沈阳东方和利厨业有限公司 Image data-based detection method and device for food maturity of young cooker
CN117237939B (en) * 2023-11-16 2024-01-30 沈阳东方和利厨业有限公司 Image data-based detection method and device for food maturity of young cooker

Also Published As

Publication number Publication date
CN110826574B (en) 2025-06-24

Similar Documents

Publication Publication Date Title
CN110826574A (en) Food material maturity determination method and device, kitchen electrical equipment and server
US12094228B2 (en) Method of identifying level of doneness of food, device, and computer storage medium
KR102329592B1 (en) Food preparation methods and systems based on ingredient recognition
CN113194792B (en) System and method for training cooking utensil, positioning food and determining cooking progress
CN106897661A (en) A kind of Weigh sensor method of food materials image, system and household electrical appliance
CN110488672B (en) Control method and device of cooking equipment, cooking equipment and storage medium
CN107752794B (en) Baking method and device
CN111752170B (en) Intelligent cooking method and device
JP2022552383A (en) Adjusting Equipment Settings with Multi-Pass Training of Target Detection Models
CN112017754A (en) Recipe recommendation method, device, range hood and storage medium
CN113239780A (en) Food material determining method and device, electronic equipment, refrigerator and storage medium
CN111435229A (en) Method and device for controlling cooking mode and cooking appliance
CN118235970A (en) Cooking appliance, system and control method thereof
WO2021082285A1 (en) Method and device for measuring volume of ingredient, and kitchen appliance apparatus
CN118366152A (en) Prefabricated vegetable processing control method and system
US20230089725A1 (en) Cooking apparatus and controlling method thereof
CN113283447B (en) Food baking methods and devices, storage media and electronic devices
CN117322780A (en) Intelligent cooking method, device, equipment and medium for food
US20230389578A1 (en) Oven appliances and methods of automatic reverse sear cooking
WO2021082284A1 (en) Baking mold specification detection method and apparatus, and kitchen appliance
CN112413989B (en) Object monitoring method, refrigerator, storage medium and electronic device
CN116129334A (en) Cleaning judging method and device, storage medium and cooking equipment
CN114208993A (en) Unfreezing method and device and unfreezing equipment
CN111752178A (en) A kind of intelligent cooking method and device
CN115183290A (en) Method, apparatus and microwave oven for controlling microwave oven

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant