WO2021082285A1 - 食材体积检测的方法及装置、厨电设备 - Google Patents
食材体积检测的方法及装置、厨电设备 Download PDFInfo
- Publication number
- WO2021082285A1 WO2021082285A1 PCT/CN2020/071725 CN2020071725W WO2021082285A1 WO 2021082285 A1 WO2021082285 A1 WO 2021082285A1 CN 2020071725 W CN2020071725 W CN 2020071725W WO 2021082285 A1 WO2021082285 A1 WO 2021082285A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- food material
- food
- information
- volume
- area
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 239000004615 ingredient Substances 0.000 title claims abstract description 32
- 238000012549 training Methods 0.000 claims abstract description 18
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 14
- 239000000463 material Substances 0.000 claims description 106
- 239000011159 matrix material Substances 0.000 claims description 54
- 238000013507 mapping Methods 0.000 claims description 24
- 230000008569 process Effects 0.000 claims description 22
- 238000012545 processing Methods 0.000 claims description 9
- 230000006835 compression Effects 0.000 claims description 7
- 238000007906 compression Methods 0.000 claims description 7
- 230000001537 neural effect Effects 0.000 claims description 7
- 230000011218 segmentation Effects 0.000 claims description 7
- 238000001514 detection method Methods 0.000 description 25
- 230000006870 function Effects 0.000 description 20
- 238000010586 diagram Methods 0.000 description 18
- 238000004891 communication Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 6
- 230000000717 retained effect Effects 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 5
- 238000005259 measurement Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 235000012041 food component Nutrition 0.000 description 1
- 239000005417 food ingredient Substances 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/68—Food, e.g. fruit or vegetables
Definitions
- This application relates to the technical field of smart devices, such as methods and devices for detecting the volume of foodstuffs, and kitchen appliances.
- kitchen electrical equipment such as ovens, microwave ovens, air fryer, etc.
- the size and weight of the food can be determined by identifying the type of food and the weight detection device, thereby determining the kitchen
- the set temperature and the set time of the electric equipment to roast the food to achieve the purpose of roasting the food.
- weight detection devices such as gravity sensors
- adding weight detection devices, such as gravity sensors, to kitchen electrical equipment will increase the manufacturing cost of the kitchen electrical equipment.
- performance and service life of the weight detection device will affect the size and weight of the ingredients, resulting in the size and weight of the ingredients. The weight is not accurate enough.
- the embodiments of the present disclosure provide a method, a device, and a kitchen appliance for detecting the volume of food materials, so as to solve the technical problem of insufficient accuracy in determining the volume of food materials by the kitchen appliance.
- the method includes:
- the volume of the food material corresponding to the area information is determined.
- the device includes:
- the image acquisition module is configured to acquire an image of the working area of the kitchen appliance where the ingredients are placed;
- the model training module is configured to train the image through a convolutional neural network model to determine the image information of the location of the food in the kitchen appliance, and the segmented image information of the set surface of the food ;
- An area determination module configured to determine area information of the set surface of the food material according to the position image information and the segmented image information
- the volume determination module is configured to determine the volume of the food material corresponding to the area information according to the corresponding relationship between the area of the food material stored and the volume of the food material.
- the device for detecting food volume includes a processor and a memory storing program instructions, and the processor is configured to execute the foregoing food volume detection method when the program instructions are executed.
- the kitchen appliance includes: the above-mentioned food volume detection device.
- the food volume detection method, device and kitchen electrical equipment provided by the embodiments of the present disclosure can achieve the following technical effects:
- the convolutional neural network model For the image of the working area of the kitchen appliance, deep learning is performed through the convolutional neural network model to obtain the volume information of the food placed in the kitchen appliance. In this way, there is no need to add any hardware equipment. While identifying the category of the food, That is, the volume of the food material can be determined and then the quality of the food material can be determined, which reduces the manufacturing resources of the kitchen electrical equipment, and does not rely on the performance of the weight measurement hardware device, which improves the accuracy of determining the volume of the food material.
- Fig. 1 is a schematic flow chart of a method for detecting volume of foodstuffs provided by an embodiment of the present disclosure
- Figure 2 is a schematic diagram of a MobileNet model training provided by an embodiment of the present disclosure
- FIG. 3 is a schematic diagram of a grid mapping matrix provided by an embodiment of the present disclosure.
- FIG. 4 is a schematic diagram of a binary mask matrix provided by an embodiment of the present disclosure.
- FIG. 5 is a schematic diagram of a mask image of ingredients provided by an embodiment of the present disclosure.
- FIG. 6 is a schematic diagram of a fitting function mapping provided by an embodiment of the present disclosure.
- FIG. 7 is a schematic flowchart of a method for detecting volume of foodstuffs according to an embodiment of the present disclosure
- FIG. 8 is a schematic structural diagram of a food volume detection device provided by an embodiment of the present disclosure.
- FIG. 9 is a schematic structural diagram of a food volume detection device provided by an embodiment of the present disclosure.
- FIG. 10 is a schematic structural diagram of a food volume detection device provided by an embodiment of the present disclosure.
- the image of the working area of the kitchen appliance is deep learning through the convolutional neural network model to obtain the volume of the food placed in the kitchen appliance. In this way, the quality of the food can be determined, and the food can be further processed. Baking control.
- Fig. 1 is a schematic flow chart of a method for detecting volume of foodstuffs provided by an embodiment of the present disclosure. As shown in Figure 1, the process of food volume detection includes:
- Step 101 Obtain an image of the working area of the kitchen electrical equipment where the ingredients are placed.
- the kitchen electrical equipment includes: ovens, microwave ovens, air fryer and other equipment with baking function.
- the kitchen appliance can be equipped with an image acquisition device, so that the image of the working area of the kitchen appliance on which the food is placed can be obtained through the image acquisition device.
- Step 102 Train the image through the convolutional neural network model to determine the image information of the position of the food in the kitchen appliance and the segmented image information of the set surface of the food.
- CNN Convolutional Neural Network
- different neural network models can be used to train the images respectively to obtain the image information of the position of the food in the kitchen appliance and the segmented image information of the set surface of the food.
- the neural compression network model can be used to train the image to obtain the grid mapping matrix corresponding to the hierarchical position of the food in the kitchen appliance; the network model can also be segmented by examples to train the image to obtain the set surface of the food The binarized mask matrix.
- Kitchen appliances such as ovens, air fryer, etc.
- the ingredients can be placed on different grills. Therefore, the image can be trained through the neural compression network model.
- the food material By obtaining the level position of the food material in the kitchen electrical equipment, the food material can be determined on which level of the grill.
- MobileNet is a lightweight neural compression network model, which has the advantages of small size and high accuracy. It can reduce response delay while ensuring accuracy, and can achieve rapid convergence. Use this lightweight Network training can also reduce the amount of calculation, thereby reducing the requirements for hardware equipment.
- the image is trained through the MobileNet model to obtain the grid mapping matrix corresponding to the hierarchical position of the food in the kitchen appliance.
- the image is trained through the MobileNet model to obtain the grid mapping matrix corresponding to the hierarchical position of the food in the kitchen appliance.
- other types of neural compression network models can also train images to obtain the image information of the location of the food in the kitchen appliance.
- Fig. 2 is a schematic diagram of a MobileNet model training provided by an embodiment of the present disclosure.
- input into the MobileNet model for training to determine the level position of the ingredients in the oven.
- the first level that is, Layer 1
- Figure 3 shows the grid mapping matrix.
- the grid mapping matrix is a two-dimensional matrix with the same size as the picture taken by the camera, and the value of each element represents the actual area corresponding to the corresponding position.
- Mask-RCNN is an instance segmentation network model. Through the Mask-RCNN algorithm model, segmentation tasks, positioning and classification tasks can be completed at the same time, and the mask image corresponding to the image can be obtained. In this embodiment, when the Mask-RCNN is trained, only the setting surface of the food material is marked. In some embodiments, the setting surface may be a visible surface on the image of the food material.
- the Mask-RCNN obtained by training can obtain the corresponding binarization mask matrix and the type recognition of the food.
- the image is trained to obtain the binarized mask matrix of the set surface of the food material and the type identification of the food material.
- FIG. 4 is a schematic diagram of a binary mask matrix provided by an embodiment of the present disclosure.
- the obtained image is input into the Mask-RCNN model, and the instance segmentation is performed according to each pixel in the food image, and the information is accurately labeled, so that the type judgment and precise location segmentation of the food can be realized, and the binarization mask shown in Figure 4 is obtained.
- Code matrix is a schematic diagram of a binary mask matrix provided by an embodiment of the present disclosure.
- segmentation network models can also be trained on the image to obtain segmented image information of the set surface of the food material.
- Step 103 Determine the area information of the set surface of the food material according to the position image information and the segmented image information.
- the position image information and segmented image information of the food material have been known, so that the area information of the set surface of the food material can be determined, which may include: the position image information and the segmented image information.
- the position image information and the segmented image information corresponding to the same pixel are performed. Operational processing, such as addition, multiplication, or weighted addition, etc.
- the information that meets the set conditions after processing is retained, and based on the retained information, the area information of the set surface of the food material is obtained. For example: retain information that is not equal to 0, or information that is greater than a set value, and sum the retained information, calculate the difference, or weighted addition, etc., to obtain the area information of the set surface of the food material.
- the position image information and segmented image information corresponding to the same pixel are subjected to the first calculation process, and the information that meets the set conditions after the processing is retained, and the second calculation process is performed according to the retained information to obtain the area information of the set surface of the food material .
- the grid mapping matrix and the binarization mask matrix can be multiplied by bit to obtain the area matrix information of the setting surface of the food; the element values in the area matrix information can be summed, Obtain the area information of the set surface of the food.
- the MobileNet model determine the level position of the ingredients in the kitchen appliance, read the grid mapping matrix T i of the corresponding layer as shown in Figure 3 to obtain the value of each element of the grid mapping matrix, and then use The grid mapping matrix T i is multiplied by the binarization mask matrix M j obtained by Mask-RCNN segmentation to obtain the result matrix, and the background area is shielded and only the corresponding area on the upper surface of the food is retained.
- the mask image of the ingredients as shown in FIG. 5 is obtained.
- the result matrix is the area matrix information of the setting surface of the food material.
- Step 104 Determine the volume of the food material corresponding to the area information according to the corresponding relationship between the area of the food material stored and the volume of the food material.
- the mapping relationship between the reference area S of the ingredient and the actual volume V of the ingredient namely (S, V)
- the fitting function is obtained by function fitting these mapping relationship pairs, That is, the corresponding relationship between the area of the food and the volume of the food corresponding to each food is pre-configured and saved, and then, using the fitting function obtained by fitting, the food volume V'is estimated according to the food area S, so as to determine the area information corresponding The volume of ingredients.
- Fig. 6 is a schematic diagram of a fitting function mapping provided by an embodiment of the present disclosure. As shown in Figure 6, if it is determined that the area in the area information is 15, the corresponding food material volume is 60.
- the corresponding relationship between the area of the different ingredients and the volume of the ingredients can be pre-configured and saved, so that the corresponding relationship can be saved when the information of the ingredients is determined by the Mask-RCNN model In the matching relationship, the corresponding relationship with the food type information is determined, and then, in the matching relationship, the food volume corresponding to the area information is determined.
- the image of the working area of the kitchen appliance is deep learning through the convolutional neural network model to obtain the volume information of the food placed in the kitchen appliance.
- the volume information of the food can be obtained according to the volume information of the food.
- the type and volume of the food can be identified, which reduces the manufacturing resources of kitchen appliances, and does not rely on the performance of the weight measurement hardware equipment, which improves the accuracy of determining the volume of the food.
- the food materials are put into the kitchen electrical equipment for baking or roasting the food materials. Therefore, after determining the volume of the food material corresponding to the area information, it also includes: controlling the baking of the food material according to the volume of the food material.
- specific parameters in the baking process can be determined according to the quality of the food material, such as baking time, baking temperature, etc., so that by running the baking process with the determined parameters, the baking control of the food material put in the food material can be realized.
- the corresponding baking time, baking temperature and other baking parameters can also be determined according to the volume of the food material, and the baking control can be directly performed.
- the type of food material can also be obtained in other ways, for example, the type of food material is determined according to the instruction information input by the user.
- the baking control of the food material can be further carried out. In this way, there is no need to add any hardware equipment.
- the volume and quality of the food can also be determined, which reduces the manufacturing resources of kitchen electrical equipment, and does not rely on the performance of the weight measurement hardware equipment to improve the determination. The accuracy of the specification information of the ingredients.
- the corresponding relationship between the area of the food material and the volume of the food material that matches the type of the food material is stored in the oven, which may include the fitting function mapping relationship as shown in FIG. 6.
- FIG. 7 is a schematic flowchart of a method for detecting the volume of food materials provided by an embodiment of the present disclosure. As shown in Figure 7, the food volume detection process includes:
- Step 701 Obtain an image of the working area of the oven where the ingredients are placed.
- a camera is built into the oven, so that an image of the working area of the oven can be obtained through the camera.
- Step 702 Train the image through the MobileNet model to obtain the grid mapping matrix corresponding to the hierarchical position of the food in the kitchen appliance.
- Step 703 Train the image through the Mask-RCNN model to obtain the binarized mask matrix of the set surface of the food material and the type information of the food material.
- step 702 and step 703 is not limited, and can be performed simultaneously or sequentially.
- Step 704 Perform a bitwise multiplication process on the grid mapping matrix and the binarized mask matrix to obtain the area matrix information of the set surface of the food material.
- Step 705 Perform a summation process on the element values in the area matrix information to obtain area information of the set surface of the food material.
- Step 706 Determine the corresponding relationship between the area of the food material matching the type information of the food material and the volume of the food material, and determine the food material volume corresponding to the area information according to the matched corresponding relationship.
- the image of the working area in the oven can be deep learning through the convolutional neural network model to obtain the volume information of the food.
- the volume of the food can be determined while identifying the type of food, so that there is no need to Adding any hardware device can determine the volume of the food material, reducing the manufacturing resources of the oven, and not relying on the performance of the weight measurement hardware device, improving the accuracy of determining the volume of the food material.
- a device for food volume detection can be constructed.
- FIG. 8 is a schematic structural diagram of a food volume detection device provided by an embodiment of the present disclosure. As shown in FIG. 8, the food volume detection device includes: an image acquisition module 810, a model training module 820, an area determination module 830, and a volume determination module 840.
- the image acquisition module 810 is configured to acquire an image of the working area of the kitchen appliance on which the food is placed.
- the model training module 820 is configured to train the image through the convolutional neural network model to determine the image information of the location of the food in the kitchen appliance and the segmented image information of the set surface of the food.
- the area determination module 830 is configured to determine the area information of the set surface of the food material according to the position image information and the segmented image information.
- the volume determining module 840 is configured to determine the volume of the food material corresponding to the area information according to the corresponding relationship between the area of the food material stored and the volume of the food material.
- the model training module 820 is specifically configured to train the image through the neural compression network model to obtain the grid mapping matrix corresponding to the hierarchical position of the food in the kitchen appliance; segment the network model by instance, The image is trained to obtain the binarized mask matrix of the set surface of the food material.
- the area determination module 830 is specifically configured to multiply the grid mapping matrix and the binarization mask matrix to obtain the area matrix information of the set surface of the food; in the area matrix information The element values of are summed to obtain the area information of the set surface of the food material.
- it further includes: a baking control module configured to control the baking of the food according to the volume of the food.
- the following is an example to illustrate the food volume detection process performed by the food volume detection device provided by the embodiment of the present disclosure.
- FIG. 9 is a schematic structural diagram of a food volume detection device provided by an embodiment of the present disclosure.
- the food volume detection device includes: an image acquisition module 810, a model training module 820, an area determination module 830, and a volume determination module 840, and may also include a baking control module 850.
- the image acquisition module 810 can acquire an image of the working area of the kitchen appliance on which the food is placed.
- the model training module 820 can train the image through the MobileNet model to obtain the grid mapping matrix corresponding to the hierarchical position of the food in the kitchen appliance, and train the image through the Mask-RCNN model to obtain the setting of the food.
- the binarization mask matrix of the surface and the type of food information can be used to train the image through the MobileNet model to obtain the grid mapping matrix corresponding to the hierarchical position of the food in the kitchen appliance.
- the area determination module 830 can perform the bitwise multiplication processing on the grid mapping matrix and the binarized mask matrix to obtain the area matrix information of the setting surface of the food material, and perform the summation processing on the element values in the area matrix information , Get the area information of the set surface of the food material.
- the volume determination module 840 can determine the corresponding relationship between the area of the food material matching the category information of the food material and the food material volume, and determine the food material volume corresponding to the area information according to the matched corresponding relationship.
- the baking process corresponding to the type of food information can be determined, and the instruction information of the food can be determined according to the density corresponding to the type of food information and the volume information of the food.
- specific parameters in the baking process can be determined according to the quality information of the food material, such as baking time, baking temperature, etc., so that the baking control module 850 can perform baking control on the food material according to the baking process for which the parameters are determined.
- the food volume detection device acquires the image of the working area in the kitchen appliance, it performs deep learning through the convolutional neural network model to obtain the volume information of the food in the kitchen appliance.
- the food can be determined
- the volume information of the food material is determined, so as to further control the baking of the food material.
- the volume of the food can also be determined, which reduces the manufacturing resources of kitchen electrical equipment, and does not rely on the performance of the weight measurement hardware equipment to improve the determination of the volume information of the food. accuracy.
- the embodiment of the present disclosure provides a device for detecting the volume of food ingredients, the structure of which is shown in FIG. 10, including:
- a processor (processor) 100 and a memory (memory) 101 may also include a communication interface (Communication Interface) 102 and a bus 103. Among them, the processor 100, the communication interface 102, and the memory 101 can communicate with each other through the bus 103. The communication interface 102 can be used for information transmission.
- the processor 100 can call the logic instructions in the memory 101 to execute the method for detecting the volume of food in the foregoing embodiment.
- logic instructions in the memory 101 can be implemented in the form of software functional units and when sold or used as independent products, they can be stored in a computer readable storage medium.
- the memory 101 can be used to store software programs and computer-executable programs, such as program instructions/modules corresponding to the methods in the embodiments of the present disclosure.
- the processor 100 executes functional applications and data processing by running the program instructions/modules stored in the memory 101, that is, realizes the food volume detection method in the foregoing method embodiment.
- the memory 101 may include a program storage area and a data storage area.
- the program storage area may store an operating system and an application program required by at least one function; the data storage area may store data created according to the use of a terminal device, and the like.
- the memory 101 may include a high-speed random access memory, and may also include a non-volatile memory.
- the embodiment of the present disclosure provides a kitchen appliance, which includes the above-mentioned food volume detection device.
- the embodiment of the present disclosure provides a computer-readable storage medium that stores computer-executable instructions, and the computer-executable instructions are configured to execute the aforementioned food volume detection method.
- the embodiments of the present disclosure provide a computer program product, the computer program product includes a computer program stored on a computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a computer, the computer program The computer executes the above food volume detection method.
- the aforementioned computer-readable storage medium may be a transitory computer-readable storage medium or a non-transitory computer-readable storage medium.
- the technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product.
- the computer software product is stored in a storage medium and includes one or more instructions to enable a computer device (which can be a personal computer, a server, or a network). Equipment, etc.) execute all or part of the steps of the method described in the embodiments of the present disclosure.
- the aforementioned storage medium may be a non-transitory storage medium, including: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks, etc.
- the first element can be called the second element, and likewise, the second element can be called the first element, as long as all occurrences of the "first element” are renamed consistently and all occurrences "Second component” can be renamed consistently.
- the first element and the second element are both elements, but they may not be the same element.
- the terms used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates, the singular forms "a” (a), “an” (an) and “the” (the) are intended to also include plural forms .
- the term “and/or” as used in this application refers to any and all possible combinations of one or more of the associated lists.
- the term “comprise” (comprise) and its variants “comprises” and/or including (comprising) and the like refer to the stated features, wholes, steps, operations, elements, and/or The existence of components does not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components, and/or groups of these. If there are no more restrictions, the element defined by the sentence “including one" does not exclude the existence of other same elements in the process, method, or device that includes the element.
- each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other.
- the methods, products, etc. disclosed in the embodiments if they correspond to the method parts disclosed in the embodiments, then the related parts can be referred to the description of the method parts.
- the disclosed methods and products may be implemented in other ways.
- the device embodiments described above are only illustrative.
- the division of the units may only be a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined. Or it can be integrated into another system, or some features can be ignored or not implemented.
- the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
- the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs.
- the functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
- each block in the flowchart or block diagram may represent a module, program segment, or part of the code, and the module, program segment, or part of the code contains one or more functions for realizing the specified logic function.
- Executable instructions may also occur in a different order than the order marked in the drawings. For example, two consecutive blocks can actually be executed in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Image Analysis (AREA)
Abstract
一种食材体积检测的方法及装置、厨电设备。该方法包括:获取放置了食材的厨电设备的工作区域的图像;通过卷积神经网络模型,对`图像进行训练,确定食材在厨电设备中的位置图像信息,以及食材的设定表面的分割图像信息;根据位置图像信息和分割图像信息,确定食材的设定表面的面积信息;根据保存的食材面积与食材体积的对应关系,确定面积信息对应的食材体积。这样,提高了确定食材体积信息的准确性。
Description
本申请基于申请号为201911044335.2、申请日为2019年10月30日的中国专利申请提出,并要求该中国专利申请的优先权,该中国专利申请的全部内容在此引入本申请作为参考。
本申请涉及智能设备技术领域,例如涉及食材体积检测的方法及装置、厨电设备。
目前,厨电设备,例如:烤箱,微波炉,空气炸锅等等都具有烤制食物的功能,一般,可通过识别食材的种类,以及通过重量检测装置确定食材的尺寸以及重量,从而,确定厨电设备烤制食物的设定温度以及设定时间,达到烤制食材的目的。
但是,在厨电设备中增加重量检测装置,例如重力传感器,会增加厨电设备的制造成本,并且,重量检测装置的性能、使用寿命等等都会影响食材的尺寸以及重量,导致食材的尺寸以及重量不够准确。
发明内容
为了对披露的实施例的一些方面有基本的理解,下面给出了简单的概括。所述概括不是泛泛评述,也不是要确定关键/重要组成元素或描绘这些实施例的保护范围,而是作为后面的详细说明的序言。
本公开实施例提供了一种食材体积检测的方法、装置和厨电设备,以解决厨电设备确定食材体积信息不够准确的技术问题。
在一些实施例中,所述方法包括:
获取放置了食材的厨电设备的工作区域的图像;
通过卷积神经网络模型,对所述图像进行训练,确定所述食材在所述厨电设备中的位置图像信息,以及,所述食材的设定表面的分割图像信息;
根据所述位置图像信息和所述分割图像信息,确定所述食材的设定表面的面积信息;
根据保存的食材面积与食材体积的对应关系,确定所述面积信息对应的食材体积。
在一些实施例中,所述装置包括:
图像获取模块,被配置为获取放置了食材的厨电设备的工作区域的图像;
模型训练模块,被配置为通过卷积神经网络模型,对所述图像进行训练,确定所述食材在所述厨电设备中的位置图像信息,以及,所述食材的设定表面的分割图像信息;
面积确定模块,被配置为根据所述位置图像信息和所述分割图像信息,确定所述食材的设定表面的面积信息;
体积确定模块,被配置为根据保存的食材面积与食材体积的对应关系,确定所述面积信息对应的食材体积。
在一些实施例中,所述食材体积检测的装置,包括处理器和存储有程序指令的存储器,所述处理器被配置为在执行所述程序指令时,执行上述食材体积检测方法
在一些实施例中,所述厨电设备包括:上述食材体积检测的装置。
本公开实施例提供的食材体积检测的方法、装置和厨电设备,可以实现以下技术效果:
对厨电设备的工作区域的图像,通过卷积神经网络模型,进行深度学习,得到厨电设备中放置食材的食材的体积信息,这样,不需增加任何硬件设备,在识别食材类别的同时,即可确定食材的体积进而确定食材的质量,减少了厨电设备的制造资源,并且,不依赖重量测量硬件设备的性能,提高了确定食材体积信息的准确性。
以上的总体描述和下文中的描述仅是示例性和解释性的,不用于限制本申请。
一个或多个实施例通过与之对应的附图进行示例性说明,这些示例性说明和附图并不构成对实施例的限定,附图中具有相同参考数字标号的元件示为类似的元件,附图不构成比例限制,并且其中:
图1是本公开实施例提供的一种食材体积检测方法的流程示意图;、
图2是本公开实施例提供的一种MobileNet模型训练的示意图;
图3是本公开实施例提供的一种网格映射矩阵的示意图;
图4是本公开实施例提供的一种二值化掩码矩阵的示意图;
图5是本公开实施例提供的一种食材掩码图像的示意图;
图6是本公开实施例提供的一种拟合函数映射示意图;
图7是本公开实施例提供的一种食材体积检测方法的流程示意图;
图8是本公开实施例提供的一种食材体积检测装置的结构示意图;
图9是本公开实施例提供的一种食材体积检测装置的结构示意图;
图10是本公开实施例提供的一种食材体积检测装置的结构示意图。
为了能够更加详尽地了解本公开实施例的特点与技术内容,下面结合附图对本公开实施例的实现进行详细阐述,所附附图仅供参考说明之用,并非用来限定本公开实施例。在以下的技术描述中,为方便解释起见,通过多个细节以提供对所披露实施例的充分理解。然而,在没有这些细节的情况下,一个或多个实施例仍然可以实施。在其它情况下,为简化附图,熟知的结构和装置可以简化展示。
本公开实施例中,对厨电设备的工作区域的图像,通过卷积神经网络模型,进行深度学习,得到厨电设备中放置食材的食材体积,这样,可确定食材的质量,进一步进行食材 的烘焙控制。
图1是本公开实施例提供的一种食材体积检测方法的流程示意图。如图1所示,食材体积检测的过程包括:
步骤101:获取放置了食材的厨电设备的工作区域的图像。
本公开实施例中,厨电设备包括:烤箱、微波炉、空气炸锅等等具有烤制功能的设备。厨电设备中可配置有图像采集装置,从而,可通过图像采集装置,获取到放置了食材的厨电设备的工作区域的图像。
步骤102:通过卷积神经网络模型,对图像进行训练,确定食材在厨电设备中的位置图像信息,以及,食材的设定表面的分割图像信息。
卷积神经网络(CNN)已经普遍应用在计算机图像处理领域,并且已经取得了不错的效果。目前,为了追求分类准确度,模型深度越来越深,模型复杂度也越来越高。本实施例中可采用不同的神经网络模型,分别对图像进行训练,得到食材在厨电设备中的位置图像信息,以及,食材的设定表面的分割图像信息。其中,可通过神经压缩网络模型,对图像进行训练,得到食材在厨电设备中的层次位置对应的网格映射矩阵;还可通过实例分割网络模型,对图像进行训练,得到食材的设定表面的二值化掩码矩阵。
厨电设备,例如:烤箱、空气炸锅等等,可能有一层、两层或多层烤架,食材可以放入不同的烤架上,因此,可以通过神经压缩网络模型,对图像进行训练,得到食材在厨电设备中的层次位置,即可确定食材在那一层烤架上。
其中,MobileNet是一种轻量级神经压缩网络模型,具有体积小、精度高的优点,能够在保证精度的同时降低响应时延,并且,能够达到快速收敛的效果,用这种轻量级的网络训练同时能够降低计算量,从而降低了对硬件设备的要求。
可选地,通过MobileNet模型,对图像进行训练,得到食材在厨电设备中的层次位置对应的网格映射矩阵。当然,在其他一些实施例中,其他类型的神经压缩网络模型,也可对图像进行训练,得到食材在厨电设备中的位置图像信息。
图2是本公开实施例提供的一种MobileNet模型训练的示意图。获取到烤箱工作区域的图像后,输入(Input)到MobileNet模型中,进行训练,可确定食材在烤箱中的层次位置,如图2所示,可确定第一层次,即Layer1,并可得到如图3所示的网格映射矩阵。网格映射矩阵是一个与相机拍摄图片同等大小的二维矩阵,每个元素的值代表相应位置实际对应的面积。
Mask-RCNN是一种实例分割网络模型,通过Mask-RCNN算法模型可同时完成分割任务、定位以及分类任务,得到图像对应的掩码图像。本实施例,在训练Mask-RCNN时仅对食材设定表面进行标注,在一些实施例中,设定表面可为可食材图像上可见表面。通过训练得到的Mask-RCNN可获得对应二值化掩码矩阵以及食材的种类识别。
可选地,通过Mask-RCNN模型,对图像进行训练,得到食材的设定表面的二值化掩码矩阵及食材的种类识别。
图4是本公开实施例提供的一种二值化掩码矩阵的示意图。获取的图像输入Mask-RCNN模型中,根据食材图像中每个像素进行实例分割,并精确进行信息标注,从而可实现食材的种类判断与精确位置分割,得到如图4所示的二值化掩码矩阵。
当然,在其他一些实施例中,其他类型的实例分割网络模型,也可对图像进行训练,得到食材的设定表面的分割图像信息。
步骤103:根据位置图像信息和分割图像信息,确定食材的设定表面的面积信息。
已经获知了食材的位置图像信息和分割图像信息,从而,可确定食材的设定表面的面积信息,可包括:位置图像信息和分割图像信息中,同一像素对应的位置图像信息和分割图像信息进行运算处理,例如:相加、相乘,或加权相加等等。然后保留处理后满足设定条件的信息,并根据保留的信息,得到食材的设定表面的面积信息。例如:保留不等于0的信息,或大于设定值的信息,并将保留的信息求和,求差,或加权相加等等,得到食材的设定表面的面积信息。即将同一像素对应的位置图像信息和分割图像信息进行第一运算处理,并保留处理后满足设定条件的信息,并根据保留的信息,进行第二运算处理,得到食材的设定表面的面积信息。
在一些实施例中,可将网格映射矩阵和二值化掩码矩阵进行对位相乘处理,得到食材的设定表面的区域矩阵信息;对区域矩阵信息中的元素值进行求和处理,得到食材的设定表面的面积信息。
例如:通过MobileNet模型,确定了食材在厨电设备中的层次位置,读取相应层的如图3所示的网格映射矩阵T
i,获得网格映射矩阵每个元素的值,然后,用网格映射矩阵T
i与Mask-RCNN分割得到的二值化掩码矩阵M
j对位相乘,得到结果矩阵,并屏蔽掉背景区域仅保留食材上表面对应区域
从而得到如图5所示的食材掩码图像。其中,结果矩阵为食材的设定表面的区域矩阵信息。
然后,对区域矩阵信息中的元素值进行求和处理,得到食材的设定表面的面积信息。即已知结果居中每个元素值S
i,从而,得到的食材的设定表面的面积S=∑
iS
i。
步骤104:根据保存的食材面积与食材体积的对应关系,确定面积信息对应的食材体积。
对于每种食材,可预先获取或保存若干食材参考面积S与食材真实体积V之间的映射关系即(S,V),然后,通过对这些映射关系对进行函数拟合,得到拟合函数,即预先配置并保存了与每种食材对应的食材面积与食材体积的对应关系,然后,利用拟合得到的拟合函数,根据食材面积S对食材体积V'进行估算,从而,确定面积信息对应的食材体积。
图6是本公开实施例提供的一种拟合函数映射示意图。如图6所示,若确定面积信息中面积为15,则对应的食材体积为60。
在一些实施例中,针对不同种类的食材,可以预先配置保存不同的食材面积与食材体积的对应关系,从而,在通过Mask-RCNN模型确定了食材种类信息的情况下,可保存的 对应的关系中,确定与食材种类信息匹配的对应的关系,然后,在匹配的对应关系中,确定面积信息对应的食材体积。
可见,本公开实施例中,对厨电设备的工作区域的图像,通过卷积神经网络模型,进行深度学习,得到厨电设备中放置食材的食材的体积信息,这样,可根据食材的体积信息,进一步进行食材的烘焙控制。不需增加任何硬件设备,即可识别食材的种类以及体积,减少了厨电设备的制造资源,并且,不依赖重量测量硬件设备的性能,提高了确定食材体积信息的准确性。
厨电设备中放入食材是为了对食材进行烘焙或者烤制,因此,确定面积信息对应的食材体积之后,还包括:根据食材的食材体积,对食材进行烘焙控制。
可选地,在通过Mask-RCNN模型确定了食材种类信息的情况下,即可确定食材种类信息对应的烘焙工艺,并且,还可食材的食材体积,以及食材种类,确定食材的质量,即确定的体积V与该类食材的密度ρ进行估算,即可确定食材质量M=V*ρ。这样,根据食材质量,可确定烘焙工艺中具体参数,例如:烘焙时间,烘焙温度等等,从而,运行已确定参数的烘焙工艺,即可实现对放入食材的食材的烘焙控制。当然,也可根据食材的食材体积确定对应的烘焙时间,烘焙温度等等烘焙参数,直接进行烘焙控制。
当然,食材的类型也可通过其他的方式获得,例如,根据用户输入的指令信息,确定食材的类型。
可见,确定食材的体积信息,可进一步进行食材的烘焙控制。这样,不需增加任何硬件设备,在进行食材种类的识别过程中,也可确定食材的体积以及质量,减少了厨电设备的制造资源,并且,不依赖重量测量硬件设备的性能,提高了确定食材规格信息的准确性。
下面将操作流程集合到具体实施例中,举例说明本发明实施例提供的食材体积检测过程。
本公开一实施例中,烤箱中保存了与食材种类匹配的,食材面积与食材体积的对应关系,其中,可包括如图6所示的拟合函数映射关系。
图7是本公开实施例提供的一种食材体积检测方法的流程示意图。如图7所示,食材体积检测过程包括:
步骤701:获取放置了食材的烤箱的工作区域的图像。
烤箱中内置了摄像头,从而,可通过摄像头获取烤箱的工作区域的图像。
步骤702:通过MobileNet模型,对图像进行训练,得到食材在厨电设备中的层次位置对应的网格映射矩阵。
步骤703:通过Mask-RCNN模型,对图像进行训练,得到食材的设定表面的二值化掩码矩阵以及食材的种类信息。
步骤702与步骤703的先后顺序不做限定,可同时进行,也可先后进行。
步骤704:将网格映射矩阵和二值化掩码矩阵进行对位相乘处理,得到食材的设定表面的区域矩阵信息。
步骤705:对区域矩阵信息中的元素值进行求和处理,得到食材的设定表面的面积信息。
步骤706:确定与食材的种类信息匹配的食材面积与食材体积的对应关系,并根据匹配的对应关系确定与面积信息对应的食材体积。
可见,本实施例中,可烤箱中工作区域的图像,通过卷积神经网络模型,进行深度学习,得到食材的体积信息,这样,可在识别食材种类的同时确定食材的体积,从而,不需增加任何硬件设备,即可确定食材的体积,减少了烤箱的制造资源,并且,不依赖重量测量硬件设备的性能,提高了确定食材体积信息的准确性。
根据上述食材体积检测的过程,可构建一种食材体积检测的装置。
图8是本公开实施例提供的一种食材体积检测装置的结构示意图。如图8所示,食材体积检测装置包括:图像获取模块810、模型训练模块820、面积确定模块830以及体积确定模块840。
图像获取模块810,被配置为获取放置了食材的厨电设备的工作区域的图像。
模型训练模块820,被配置为通过卷积神经网络模型,对图像进行训练,确定食材在厨电设备中的位置图像信息,以及,食材的设定表面的分割图像信息。
面积确定模块830,被配置为根据位置图像信息和分割图像信息,确定食材的设定表面的面积信息。
体积确定模块840,被配置为根据保存的食材面积与食材体积的对应关系,确定面积信息对应的食材体积。
在一些实施例中,模型训练模块820,具体被配置为通过神经压缩网络模型,对图像进行训练,得到食材在厨电设备中的层次位置对应的网格映射矩阵;通过实例分割网络模型,对图像进行训练,得到食材的设定表面的二值化掩码矩阵。
在一些实施例中,面积确定模块830,具体被配置为将网格映射矩阵和二值化掩码矩阵进行对位相乘处理,得到食材的设定表面的区域矩阵信息;对区域矩阵信息中的元素值进行求和处理,得到食材的设定表面的面积信息。
在一些实施例中,还包括:烘焙控制模块,被配置为根据食材体积,对食材进行烘焙控制。
下面举例说明本公开实施例提供的食材体积检测装置进行食材体积检测过程。
图9是本公开实施例提供的一种食材体积检测装置的结构示意图。如图9所示,食材体积检测装置包括:图像获取模块810、模型训练模块820、面积确定模块830以及体积确定模块840,还可包括:烘焙控制模块850。
其中,图像获取模块810可获取放置了食材的厨电设备的工作区域的图像。
这样,模型训练模块820可通过MobileNet模型,对图像进行训练,得到食材在厨电设备中的层次位置对应的网格映射矩阵,以及通过Mask-RCNN模型,对图像进行训练,得到食材的设定表面的二值化掩码矩阵以及食材的种类信息。
从而,面积确定模块830可将网格映射矩阵和二值化掩码矩阵进行对位相乘处理,得到食材的设定表面的区域矩阵信息,并对区域矩阵信息中的元素值进行求和处理,得到食材的设定表面的面积信息。
已预先保存了与食材种类匹配的,食材面积与食材体积的对应关系。从而,体积确定模块840可确定与食材的种类信息匹配的食材面积与食材体积的对应关系,并根据匹配的对应关系确定与面积信息对应的食材体积。
由于在通过Mask-RCNN模型确定了食材种类信息的情况下,即可确定食材种类信息对应的烘焙工艺,并且,还可根据食材种类信息对应的密度,与食材的体积信息确定食材的指令信息,从而,可根据食材质量信息确定烘焙工艺中具体参数,例如:烘焙时间,烘焙温度等等,从而,烘焙控制模块850可根据确定了参数的烘焙工艺,对食材进行烘焙控制。
可见,本实施例中,食材体积检测装置获取厨电设备内的工作区域的图像后,通过卷积神经网络模型,进行深度学习,得到厨电设备中食材的体积信息,这样,可在确定食材种类信息的同时确定食材的体积信息,从而进一步进行食材的烘焙控制。不需增加任何硬件设备,在进行食材种类的识别过程中,也可确定食材的体积,减少了厨电设备的制造资源,并且,不依赖重量测量硬件设备的性能,提高了确定食材体积信息的准确性。
本公开实施例提供了一种食材体积检测的装置,其结构如图10所示,包括:
处理器(processor)100和存储器(memory)101,还可以包括通信接口(Communication Interface)102和总线103。其中,处理器100、通信接口102、存储器101可以通过总线103完成相互间的通信。通信接口102可以用于信息传输。处理器100可以调用存储器101中的逻辑指令,以执行上述实施例的食材体积检测的方法。
此外,上述的存储器101中的逻辑指令可以通过软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。
存储器101作为一种计算机可读存储介质,可用于存储软件程序、计算机可执行程序,如本公开实施例中的方法对应的程序指令/模块。处理器100通过运行存储在存储器101中的程序指令/模块,从而执行功能应用以及数据处理,即实现上述方法实施例中的食材体积检测的方法。
存储器101可包括存储程序区和存储数据区,其中,存储程序区可存储操作系统、至少一个功能所需的应用程序;存储数据区可存储根据终端设备的使用所创建的数据等。此外,存储器101可以包括高速随机存取存储器,还可以包括非易失性存储器。
本公开实施例提供了一种厨电设备,包含上述的食材体积检测装置。
本公开实施例提供了一种计算机可读存储介质,存储有计算机可执行指令,所述计算机可执行指令设置为执行上述食材体积检测方法。
本公开实施例提供了一种计算机程序产品,所述计算机程序产品包括存储在计算机可读存储介质上的计算机程序,所述计算机程序包括程序指令,当所述程序指令被计算机执 行时,使所述计算机执行上述食材体积检测方法。
上述的计算机可读存储介质可以是暂态计算机可读存储介质,也可以是非暂态计算机可读存储介质。
本公开实施例的技术方案可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括一个或多个指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本公开实施例所述方法的全部或部分步骤。而前述的存储介质可以是非暂态存储介质,包括:U盘、移动硬盘、只读存储器(ROM,Read-Only Memory)、随机存取存储器(RAM,Random Access Memory)、磁碟或者光盘等多种可以存储程序代码的介质,也可以是暂态存储介质。
以上描述和附图充分地示出了本公开的实施例,以使本领域的技术人员能够实践它们。其他实施例可以包括结构的、逻辑的、电气的、过程的以及其他的改变。实施例仅代表可能的变化。除非明确要求,否则单独的部件和功能是可选的,并且操作的顺序可以变化。一些实施例的部分和特征可以被包括在或替换其他实施例的部分和特征。本公开实施例的范围包括权利要求书的整个范围,以及权利要求书的所有可获得的等同物。当用于本申请中时,虽然术语“第一”、“第二”等可能会在本申请中使用以描述各元件,但这些元件不应受到这些术语的限制。这些术语仅用于将一个元件与另一个元件区别开。比如,在不改变描述的含义的情况下,第一元件可以叫做第二元件,并且同样第,第二元件可以叫做第一元件,只要所有出现的“第一元件”一致重命名并且所有出现的“第二元件”一致重命名即可。第一元件和第二元件都是元件,但可以不是相同的元件。而且,本申请中使用的用词仅用于描述实施例并且不用于限制权利要求。如在实施例以及权利要求的描述中使用的,除非上下文清楚地表明,否则单数形式的“一个”(a)、“一个”(an)和“所述”(the)旨在同样包括复数形式。类似地,如在本申请中所使用的术语“和/或”是指包含一个或一个以上相关联的列出的任何以及所有可能的组合。另外,当用于本申请中时,术语“包括”(comprise)及其变型“包括”(comprises)和/或包括(comprising)等指陈述的特征、整体、步骤、操作、元素,和/或组件的存在,但不排除一个或一个以上其它特征、整体、步骤、操作、元素、组件和/或这些的分组的存在或添加。在没有更多限制的情况下,由语句“包括一个…”限定的要素,并不排除在包括所述要素的过程、方法或者设备中还存在另外的相同要素。本文中,每个实施例重点说明的可以是与其他实施例的不同之处,各个实施例之间相同相似部分可以互相参见。对于实施例公开的方法、产品等而言,如果其与实施例公开的方法部分相对应,那么相关之处可以参见方法部分的描述。
本领域技术人员可以意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行,可以取决于技术方案的特定应用和设计约束条件。所述技术人员可以对每个特定的应用来使用不同方法以实现所描述的功能,但是这种实现不应认为超出本公开实施例的范围。所述技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的 系统、装置和单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。
本文所披露的实施例中,所揭露的方法、产品(包括但不限于装置、设备等),可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述单元的划分,可以仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另外,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例。另外,在本公开实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。
附图中的流程图和框图显示了根据本公开实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上,流程图或框图中的每个方框可以代表一个模块、程序段或代码的一部分,所述模块、程序段或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中,方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如,两个连续的方框实际上可以基本并行地执行,它们有时也可以按相反的顺序执行,这可以依所涉及的功能而定。在附图中的流程图和框图所对应的描述中,不同的方框所对应的操作或步骤也可以以不同于描述中所披露的顺序发生,有时不同的操作或步骤之间不存在特定的顺序。例如,两个连续的操作或步骤实际上可以基本并行地执行,它们有时也可以按相反的顺序执行,这可以依所涉及的功能而定。框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合,可以用执行规定的功能或动作的专用的基于硬件的系统来实现,或者可以用专用硬件与计算机指令的组合来实现。
Claims (10)
- 一种食材体积检测的方法,其特征在于,包括:获取放置了食材的厨电设备的工作区域的图像;通过卷积神经网络模型,对所述图像进行训练,确定所述食材在所述厨电设备中的位置图像信息,以及,所述食材的设定表面的分割图像信息;根据所述位置图像信息和所述分割图像信息,确定所述食材的设定表面的面积信息;根据保存的食材面积与食材体积的对应关系,确定所述面积信息对应的食材体积。
- 根据权利要求1所述的方法,其特征在于,所述确定所述食材在所述厨电设备中的位置图像信息,以及,所述食材的设定表面的分割图像信息包括:通过神经压缩网络模型,对所述图像进行训练,得到所述食材在所述厨电设备中的层次位置对应的网格映射矩阵;通过实例分割网络模型,对所述图像进行训练,得到所述食材的设定表面的二值化掩码矩阵。
- 根据权利要求2所述的方法,其特征在于,所述确定所述食材的设定表面的面积信息包括:将所述网格映射矩阵和所述二值化掩码矩阵进行对位相乘处理,得到所述食材的设定表面的区域矩阵信息;对所述区域矩阵信息中的元素值进行求和处理,得到所述食材的设定表面的面积信息。
- 根据权利要求1所述的方法,其特征在于,所述确定所述面积信息对应的食材体积之后,还包括:根据所述食材体积,对所述食材进行烘焙控制。
- 一种食材体积检测的装置,其特征在于,包括:图像获取模块,被配置为获取放置了食材的厨电设备的工作区域的图像;模型训练模块,被配置为通过卷积神经网络模型,对所述图像进行训练,确定所述食材在所述厨电设备中的位置图像信息,以及,所述食材的设定表面的分割图像信息;面积确定模块,被配置为根据所述位置图像信息和所述分割图像信息,确定所述食材的设定表面的面积信息;体积确定模块,被配置为根据保存的食材面积与食材体积的对应关系,确定所述面积信息对应的食材体积。
- 根据权利要求5所述的装置,其特征在于,所述模型训练模块,具体被配置为通过神经压缩网络模型,对所述图像进行训练,得到所述食材在所述厨电设备中的层次位置对应的网格映射矩阵;通过实例分割网络模型,对所述图像进行训练,得到所述食材的设定表面的二值化掩码矩阵。
- 根据权利要求6所述的装置,其特征在于,所述面积确定模块,具体被配置为将所述网格映射矩阵和所述二值化掩码矩阵进行对位相乘处理,得到所述食材的设定表面的区域矩阵信息;对所述区域矩阵信息中的元素值进行求和处理,得到所述食材的设定表面的面积信息。
- 根据权利要求5所述的装置,其特征在于,还包括:烘焙控制模块,被配置为根据所述食材体积,对所述食材进行烘焙控制。
- 一种食材体积检测的装置,包括处理器和存储有程序指令的存储器,其特征在于,所述处理器被配置为在执行所述程序指令时,执行如权利要求1至4任一项所述的方法。
- 一种厨电设备,其特征在于,包括如权利要求5或9所述的装置。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911044335.2 | 2019-10-30 | ||
CN201911044335.2A CN112750158A (zh) | 2019-10-30 | 2019-10-30 | 食材体积检测的方法及装置、厨电设备 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021082285A1 true WO2021082285A1 (zh) | 2021-05-06 |
Family
ID=75641755
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/071725 WO2021082285A1 (zh) | 2019-10-30 | 2020-01-13 | 食材体积检测的方法及装置、厨电设备 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN112750158A (zh) |
WO (1) | WO2021082285A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115112211A (zh) * | 2022-08-02 | 2022-09-27 | 佛山市小熊厨房电器有限公司 | 基于智能厨电的食材重量智能确定方法及装置 |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114565840A (zh) * | 2022-02-18 | 2022-05-31 | 珠海格力电器股份有限公司 | 调味品推荐方法、装置、存储介质、电子设备及油烟机 |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108038879A (zh) * | 2017-12-12 | 2018-05-15 | 众安信息技术服务有限公司 | 一种食品体积估计方法及其装置 |
CN108846314A (zh) * | 2018-05-08 | 2018-11-20 | 天津大学 | 一种基于深度学习的食材辨识系统和食材辨识方法 |
CN109064509A (zh) * | 2018-06-29 | 2018-12-21 | 广州雅特智能科技有限公司 | 食物体积和食物热量的识别方法、装置和系统 |
CN110084244A (zh) * | 2019-03-14 | 2019-08-02 | 上海达显智能科技有限公司 | 基于图像识别物体的方法、智能设备及应用 |
US10424121B1 (en) * | 2016-11-06 | 2019-09-24 | Oded Melinek | Generated offering exposure |
CN110287207A (zh) * | 2019-06-30 | 2019-09-27 | 北京健康有益科技有限公司 | 一种基于密度表的食物质量估测方法 |
-
2019
- 2019-10-30 CN CN201911044335.2A patent/CN112750158A/zh active Pending
-
2020
- 2020-01-13 WO PCT/CN2020/071725 patent/WO2021082285A1/zh active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10424121B1 (en) * | 2016-11-06 | 2019-09-24 | Oded Melinek | Generated offering exposure |
CN108038879A (zh) * | 2017-12-12 | 2018-05-15 | 众安信息技术服务有限公司 | 一种食品体积估计方法及其装置 |
CN108846314A (zh) * | 2018-05-08 | 2018-11-20 | 天津大学 | 一种基于深度学习的食材辨识系统和食材辨识方法 |
CN109064509A (zh) * | 2018-06-29 | 2018-12-21 | 广州雅特智能科技有限公司 | 食物体积和食物热量的识别方法、装置和系统 |
CN110084244A (zh) * | 2019-03-14 | 2019-08-02 | 上海达显智能科技有限公司 | 基于图像识别物体的方法、智能设备及应用 |
CN110287207A (zh) * | 2019-06-30 | 2019-09-27 | 北京健康有益科技有限公司 | 一种基于密度表的食物质量估测方法 |
Non-Patent Citations (1)
Title |
---|
FORBES K.A., TATTERSFIELD G.M.: "Estimating fruit volume from digital images", AFRICON, 1999 IEEE CAPE TOWN, SOUTH AFRICA 28 SEPT.-1 OCT. 1999, PISCATAWAY, NJ, USA,IEEE, US, vol. 1, 28 September 1999 (1999-09-28) - 1 October 1999 (1999-10-01), US, pages 107 - 112, XP010367153, ISBN: 978-0-7803-5546-0, DOI: 10.1109/AFRCON.1999.820775 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115112211A (zh) * | 2022-08-02 | 2022-09-27 | 佛山市小熊厨房电器有限公司 | 基于智能厨电的食材重量智能确定方法及装置 |
CN115112211B (zh) * | 2022-08-02 | 2024-06-11 | 佛山市小熊厨房电器有限公司 | 基于智能厨电的食材重量智能确定方法及装置 |
Also Published As
Publication number | Publication date |
---|---|
CN112750158A (zh) | 2021-05-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10803315B2 (en) | Electronic device and method for processing information associated with food | |
US8811734B2 (en) | Color determination device, color determination system, color determination method, information recording medium, and program | |
CN104887183B (zh) | 基于光学的肌肤健康监测和预诊断智能方法 | |
US8891860B2 (en) | Color name determination device, color name determination method, information recording medium, and program | |
Ando et al. | Depthcaloriecam: A mobile application for volume-based foodcalorie estimation using depth cameras | |
CN108416902A (zh) | 基于差异识别的实时物体识别方法和装置 | |
WO2021082285A1 (zh) | 食材体积检测的方法及装置、厨电设备 | |
US20180211139A1 (en) | System and method for providing a recipe | |
CN107590420A (zh) | 视频分析中的场景关键帧提取方法及装置 | |
Severa et al. | Shape and size variability of roasted Arabica coffee beans | |
CN108985190A (zh) | 目标识别方法和装置、电子设备、存储介质、程序产品 | |
US11669566B2 (en) | Multi-resolution color-based image search | |
CN110956217A (zh) | 食物成熟度识别方法、装置以及计算机存储介质 | |
WO2014168177A1 (ja) | 画像処理装置 | |
CN107590460A (zh) | 人脸分类方法、装置及智能终端 | |
CN112784640A (zh) | 一种菜谱制作方法、装置及炒菜机 | |
WO2021082284A1 (zh) | 烘焙模具规格检测的方法及装置、厨电设备 | |
JP2018049584A (ja) | 食事量推定プログラム、食事量推定方法及び食事量推定装置 | |
Adachi et al. | DepthGrillCam: A mobile application for real-time eating action recording using RGB-D images | |
Shao et al. | Semi-automatic crowdsourcing tool for online food image collection and annotation | |
CN113610832B (zh) | 一种徽标缺陷检测方法、装置、设备和存储介质 | |
CN113066024B (zh) | 图像模糊检测模型的训练方法、图像模糊检测方法及装置 | |
JP6934001B2 (ja) | 画像処理装置、画像処理方法、プログラムおよび記録媒体 | |
CN104298707B (zh) | 一种信息处理方法及电子设备 | |
CN116452881B (zh) | 食品营养价值检测方法、装置、设备及存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20882707 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20882707 Country of ref document: EP Kind code of ref document: A1 |